PG-SAM: A Fine-Grained Prior-Guided SAM Framework for Prompt-Free Medical Image Segmentation
Zhong, Yiheng Luo, Zihong Liu, Chengzhi Tang, Feilong Hu, Yingzhen Peng, Zelin Hu, Ming Su, Jionglong Ge, Zongyuan Razzak, Imran
Zhong, Yiheng
Luo, Zihong
Liu, Chengzhi
Tang, Feilong
Hu, Yingzhen
Peng, Zelin
Hu, Ming
Su, Jionglong
Ge, Zongyuan
Razzak, Imran
Supervisor
Department
Computational Biology
Embargo End Date
Type
Conference proceeding
Date
License
Language
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Segment Anything Model (SAM) demonstrates powerful zero-shot capabilities; however, its accuracy and robustness significantly decrease when applied to medical image segmentation. Existing methods address this issue through modality fusion, integrating textual and image information to provide more detailed priors. In this study, we argue that the granularity of text and the domain gap affect the accuracy of the priors. Furthermore, the discrepancy between high-level abstract semantics and pixel-level boundary details in images can introduce noise into the fusion process. To address this, we propose Prior-Guided SAM (PG-SAM), which employs a fine-grained modality prior aligner to leverage specialized medical knowledge for better modality alignment. The core of our method lies in efficiently addressing the domain gap with fine-grained text from a medical large language model (LLM). Meanwhile, it also enhances the priors' quality after modality alignment, ensuring more accurate segmentation. In addition, our decoder enhances the model's expressive capabilities through multi-level feature fusion and iterative mask optimizer operations, supporting unprompted learning. We also propose a unified pipeline that effectively supplies high-quality semantic information to SAM. Extensive experiments on the datasets demonstrate that the proposed PG-SAM achieves state-of-the-art performance. Our anonymous code is released at https://github.com/logan-0623/PG-SAM.
Citation
Y. Zhong, Z. Luo, C. Liu, F. Tang, Y. Hu, Z. Peng, M. Hu, J. Su, Z. Ge, I. Razzak, "PG-SAM: A Fine-Grained Prior-Guided SAM Framework for Prompt-Free Medical Image Segmentation," 2026, pp. 3369-3376.
Source
Conference
2025 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)
Keywords
46 Information and Computing Sciences, 4603 Computer Vision and Multimedia Computation, 4605 Data Management and Data Science
Subjects
Source
2025 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)
Publisher
IEEE