MAFM3: Modular Adaptation of Foundation Models for Multi-Modal Medical AI
Qazi, Mohammad Areeb Nwadike, Munachiso S Almakky, Ibrahim Yaqub, Mohammad Saeed, Numan
Qazi, Mohammad Areeb
Nwadike, Munachiso S
Almakky, Ibrahim
Yaqub, Mohammad
Saeed, Numan
Supervisor
Department
Computer Vision
Embargo End Date
Type
Conference proceeding
Date
License
Language
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Foundational models are trained on extensive datasets to capture the general trends of a domain. However, in medical imaging, the scarcity of data makes pre-training for every domain, modality, or task challenging. Instead of building separate models, we propose MAFM3 (Modular Adaptation of Foundation Models for Multi-Modal Medical AI), a framework that enables a single foundation model to expand into diverse domains, tasks, and modalities through lightweight modular components. These components serve as specialized skill sets that allow the system to flexibly activate the appropriate capability at the inference time, depending on the input type or clinical objective. Unlike conventional adaptation methods that treat each new task or modality in isolation, MAFM3 provides a unified and expandable framework for efficient multitask and multimodality adaptation. Empirically, we validate our approach by adapting a chest CT foundation model initially trained for classification into prognosis and segmentation modules. Our results show improved performance on both tasks. Furthermore, by incorporating PET scans, MAFM3 achieved an improvement in the Dice score 5% compared to the respective baselines. These findings establish that foundation models, when equipped with modular components, are not inherently constrained to their initial training scope but can evolve into multitask, multimodality systems for medical imaging. The code implementation of this work will be made available upon acceptance. The code implementation of this work can be found at Code
Citation
M.A. Qazi, M.S. Nwadike, I. Almakky, M. Yaqub, N. Saeed, "MAFM3: Modular Adaptation of Foundation Models for Multi-Modal Medical AI," 2026, pp. 3494-3503.
Source
2026 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)
Conference
2026 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)
Keywords
46 Information and Computing Sciences, 4611 Machine Learning
Subjects
Source
2026 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)
Publisher
IEEE