Enhancing Novel Object Detection via Cooperative Foundational Models
Salman Khan Fahad Shahbaz Khan
Salman Khan
Fahad Shahbaz Khan
Author
Supervisor
Department
Computer Vision
Embargo End Date
Type
Conference proceeding
Date
2025
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
In this work, we address the challenging and emergent problem of novel object detection (NOD), focusing on the accurate detection of both known and novel object categories during inference. Traditional object detection algorithms are inherently closed-set, limiting their capability to handle NOD. We present a novel approach to transform existing closed-set detectors into open-set detectors. This transformation is achieved by leveraging the complementary strengths of pre-trained foundational models, specifically CLIP and SAM, through our cooperative mechanism. Furthermore, by integrating this mechanism with state-of-the-art open-set detectors such as GDINO, we establish new benchmarks in object detection performance. Our method achieves 17.42 mAP in novel object detection and 42.08 mAP for known objects on the challenging LVIS dataset. Adapting our approach to the COCO OVD split, we obtain an impressive result of 49.6 Novel AP50, which outperforms existing SOTA methods with similar backbone. Our code is available at: https://rohit901.github.io/coop-foundation-models/.
Co-author(s)
Bharadwaj, R., Naseer, M., Khan, S., Khan, F.S.
Citation
R. Bharadwaj, M. Naseer, S. Khan and F. S. Khan, "Enhancing Novel Object Detection via Cooperative Foundational Models," 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Tucson, AZ, USA, 2025, pp. 9043-9052, doi: 10.1109/WACV61041.2025.00876
Source
Proceedings - 2025 IEEE Winter Conference on Applications of Computer Vision, WACV 2025
Conference
2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)
Keywords
Novel object detection, Open vocabulary object detection, Zero-shot object detection, Foundational models, CLIP, SAM, Object detection
Subjects
Source
2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)
Publisher
IEEE