Advancing In-Context Learning for Efficient and Stable Medical Report Generation
Li, Mingjie Liu, Rui Shi, Zeyi Han, Mingfei Yao, Lina Li, Zhihui Chang, Xiaojun Pohl, Kilian M Islam, Tauhidul Xing, Lei
Li, Mingjie
Liu, Rui
Shi, Zeyi
Han, Mingfei
Yao, Lina
Li, Zhihui
Chang, Xiaojun
Pohl, Kilian M
Islam, Tauhidul
Xing, Lei
Supervisor
Department
Computer Vision
Embargo End Date
Type
Journal article
Date
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Vision-language models (VLMs) have shown strong generalization across multimodal tasks, but adapting them to medical report generation (MRG) often demands extensive paired image-text data that are limited due to data privacy and annotation cost. In-context learning (ICL) offers a promising training-free alternative, yet standard ICL approaches rely on long demonstration prompts that are computationally inefficient and often yield inconsistent or clinically inaccurate descriptions. To address these challenges, we propose Principal In-Context Vectors (PCVs), a compact latent-guidance framework that distills multimodal demonstrations into stable semantic representations. By extracting hidden states from auto-regressive VLMs and applying principal component analysis (PCA), we identify robust semantic directions that remain stable under input perturbations. These PCVs are then injected into new queries to steer generation toward accurate and clinically meaningful outputs without any model tuning. Extensive experiments on four MRG benchmark datasets show that our approach can enhance both zero-shot and fully supervised generation quality across diverse settings, including cross-center, cross-disease, and longitudinal scenarios. This work provides a lightweight and scalable approach to adapt pre-trained VLMs for practical clinical deployment.
Citation
M. Li, R. Liu, Z. Shi, M. Han, L. Yao, Z. Li , et al., "Advancing In-Context Learning for Efficient and Stable Medical Report Generation," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. PP, no. 99, pp. 1-18, 2026, https://doi.org/10.1109/tpami.2026.3689780.
Source
IEEE Transactions on Pattern Analysis and Machine Intelligence
Conference
Keywords
46 Information and Computing Sciences, 4611 Machine Learning
Subjects
Source
Publisher
Institute of Electrical and Electronics Engineers