DuPLUS: Dual-Prompt Vision-Language Framework for Universal Medical Image Segmentation and Prognosis
Saeed, Numan Saleem, Tausifa Jan Maani, Fadillah Ridzuan, Muhammad Wang, Hu Yaqub, Mohammad
Saeed, Numan
Saleem, Tausifa Jan
Maani, Fadillah
Ridzuan, Muhammad
Wang, Hu
Yaqub, Mohammad
Supervisor
Department
Computer Vision
Embargo End Date
Type
Conference proceeding
Date
License
Language
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Deep learning for medical imaging is hampered by task-specific models that lack generalizability and prognostic capabilities, while existing ’universal’ approaches suffer from simplistic conditioning and poor medical semantic understanding. To address these limitations, we introduce DuPLUS, a deep learning framework for efficient multimodal medical image analysis. DuPLUS introduces a novel vision-language framework that leverages hierarchical semantic prompts for fine-grained control over the analysis task, a capability absent in prior universal models. To enable extensibility to other medical tasks, it includes a hierarchical, text-controlled architecture driven by a unique dual-prompt mechanism. For segmentation, DuPLUS is able to generalize across three imaging modalities, ten different anatomically various medical datasets, encompassing more than 30 organs and tumor types. It outperforms the state-of-the-art task-specific and universal models on 8 out of 10 datasets. We demonstrate extensibility of its text-controlled architecture by seamless integration of electronic health record (EHR) data for prognosis prediction, and on a head and neck cancer dataset, DuPLUS achieved a Concordance Index (CI) of 0.69. Parameter-efficient fine-tuning enables rapid adaptation to new tasks and modalities from varying centers, establishing DuPLUS as a versatile and clinically relevant solution for medical image analysis. The code for this work is made available at: Code
Citation
N. Saeed, T.J. Saleem, F. Maani, M. Ridzuan, H. Wang, M. Yaqub, "DuPLUS: Dual-Prompt Vision-Language Framework for Universal Medical Image Segmentation and Prognosis," 2026, pp. 8386-8395.
Source
2026 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)
Conference
2026 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)
Keywords
46 Information and Computing Sciences, 4611 Machine Learning
Subjects
Source
2026 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)
Publisher
IEEE