Counterfactual Contrastive Learning for Enhanced Radiology Report Generation
Lin, Haokun
Lin, Haokun
Author
Supervisor
Department
Computer Vision
Embargo End Date
2025-05-30
Type
Thesis
Date
2025
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
This paper presents CoFE, a contrastive learning framework using counterfactual explanations to address false correlations in medical report generation caused by dataset bias. Due to similar anatomical content in medical images and reports, traditional models often learn spurious features. CoFE constructs contrastive pairs between factual and counterfactual images to improve model robustness. It introduces a patch-based strategy that swaps regions between semantically similar images with different diagnoses to locate key lesion areas. A learnable prompt integrates both types of information to fine-tune a pretrained language model, enhancing report coherence and accuracy. Experiments on IU-Xray and MIMIC-CXR show CoFE outperforms prior methods in both descriptive quality and clinical relevance.
Citation
Haokun Lin, “Counterfactual Contrastive Learning for Enhanced Radiology Report Generation,” Master of Science thesis, Computer Vision, MBZUAI, 2025.
Source
Conference
Keywords
Radiology Report Generation, Contrastive Learning, Counterfactual