Semi-supervised Open-World Object Detection
Mullappilly, Sahal Shaji
Mullappilly, Sahal Shaji
Author
Supervisor
Department
Computer Vision
Embargo End Date
Type
Thesis
Date
2023
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
However, the current OWOD formulation heavily relies on the external human oracle for knowledge input during the incremental learning stages. Such reliance on run-time makes this formulation less realistic in a real-world deployment. To address this, we introduce a more realistic formulation, named semi-supervised open-world detection (SS-OWOD), that reduces the annotation cost by casting the incremental learning stages of OWOD in a semi-supervised manner. We demonstrate that the performance of the state-of-the-art OWOD detector dramatically deteriorates in the proposed SS-OWOD setting. Therefore, we introduce a novel SS-OWOD detector, named SS-OWFormer, that utilizes a feature-alignment scheme to better align the object query representations between the original and augmented images to leverage the large unlabeled and few labeled data. We further introduce a pseudo-labeling scheme for unknown detection that exploits the inherent capability of decoder object queries to capture object-specific information. On the COCO dataset, our SS-OWFormer using only 50\% of the labeled data achieves detection performance that is on par to the state-of-the-art (SOTA) OWOD detector using all the 100\% of labeled data. Further, our SS-OWFormer achieves an absolute gain of 4.8\% in unknown recall over the SOTA OWOD detector. Lastly, we demonstrate the effectiveness of our SS-OWOD problem setting and approach for remote sensing object detection on DOTA splits with performance comparisons to the baseline.
Citation
S.S. Mullappilly, "Semi-supervised Open-World Object Detection", M.S. Thesis, Computer Vision, MBZUAI, Abu Dhabi, UAE, 2023.