Revisiting LLM Adaptation for 3D CT Report Generation: A Study of Scaling and Diagnostic Priors
This study investigates parameter-efficient adaptation strategies for volumetric CT report generation and introduces RAD3D-Prefix, a lightweight diagnostic-prior conditioning framework. By freezing the LLM and training only projection layers, the method minimizes trainable parameters and overfitting. Systematic experiments across LLMs from 96.1M to 1.6B parameters reveal that fine-tuning benefits smaller LLMs, while freezing larger ones (1B+) offers a superior trade-off. RAD3D-Prefix outperforms baselines on automatic metrics and clinical reader studies with strong out-of-domain generalization.
[2606.17213] Revisiting LLM Adaptation for 3D CT Report Generation: A Study of Scaling and Diagnostic Priors
[Submitted on 15 Jun 2026]
Title:Revisiting LLM Adaptation for 3D CT Report Generation: A Study of Scaling and Diagnostic Priors
View a PDF of the paper titled Revisiting LLM Adaptation for 3D CT Report Generation: A Study of Scaling and Diagnostic Priors, by Vanshali Sharma and 5 other authors
View PDF HTML (experimental)
Abstract:Recent advances in multimodal learning, including large language models (LLMs) and vision-language models (VLMs), have demonstrated strong adaptability to natural images. However, extending their use to the medical domain, particularly for volumetric (3D) images, is challenging due to high computational complexity, volumetric dependencies and the semantic gap between visual features and clinical terminology. Naively fine-tuning LLMs on limited medical data often leads to overfitting and clinical hallucination, where linguistic fluency is prioritized over clinical factuality. In this study, we investigate parameter-efficient adaptation strategies for volumetric CT report generation and introduce RAD3D-Prefix, a lightweight diagnostic-prior conditioning framework that minimizes the need for extensive parameter training. This module integrates image embeddings with multi-label diagnostic classification logits, preserving critical clinical details while bridging the semantic gap. By keeping the LLM frozen, our method requires minimal trainable parameters and mitigates the risk of overfitting on small, domain-specific datasets. Through a systematic study spanning LLMs from 96.1M to 1.6B parameters, we find that fine-tuning is most beneficial for smaller LLMs, whereas freezing larger (~1B+ LLMs and training only lightweight projection layers provides a superior trade-off between performance, generalization, and computational efficiency. Across multiple automatic metrics and a clinical reader study, RAD3D-Prefix outperforms comparable parameter-efficient baselines and demonstrates strong out-of-domain generalization while using substantially fewer trainable parameters than fully fine-tuned alternatives.
Subjects:
Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)
Cite as: arXiv:2606.17213 [cs.CL]
(or arXiv:2606.17213v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2606.17213
arXiv-issued DOI via DataCite (pending registration)
Submission history
From: Vanshali Sharma [view email] [v1] Mon, 15 Jun 2026 18:51:31 UTC (5,494 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled Revisiting LLM Adaptation for 3D CT Report Generation: A Study of Scaling and Diagnostic Priors, by Vanshali Sharma and 5 other authors
View PDF
HTML (experimental)
TeX Source
view license
Current browse context:
cs.CL
new | recent | 2026-06
Change to browse by:
cs cs.CV
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
Loading...
Data provided by:
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)