OSLOPROMPT: Bridging Low-Supervision Challenges and Open-Set Domain Generalization in CLIP
Hassan, Mohamad N. C. Gupta, Divyam Singha, Mainak Rongali, Sai Bhargav Jha, Ankit Khan, Muhammad Haris Banerjee, Biplab
Hassan, Mohamad N. C.
Gupta, Divyam
Singha, Mainak
Rongali, Sai Bhargav
Jha, Ankit
Khan, Muhammad Haris
Banerjee, Biplab
Supervisor
Department
Computer Vision
Embargo End Date
Type
Conference proceeding
Date
2025
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
We introduce Low-Shot Open-Set Domain Generalization (LSOSDG), a novel paradigm unifying low-shot learning with open-set domain generalization (ODG). While prompt-based methods using models like CLIP have advanced DG, they falter in low-data regimes (e.g., 1-shot) and lack precision in detecting open-set samples with fine-grained semantics related to training classes. To address these challenges, we propose OSLOPROMPT, an advanced prompt-learning framework for CLIP with two core innovations. First, to manage limited supervision across source domains and improve DG, we introduce a domain-agnostic prompt-learning mechanism that integrates adaptable domain-specific cues and visually guided semantic attributes through a novel cross-attention module, besides being supported by learnable domain- and class-generic visual prompts to enhance cross-modal adaptability. Second, to improve outlier rejection during inference, we classify unfamiliar samples as unknown and train specialized prompts with systematically synthesized pseudo-open samples that maintain fine-grained relationships to known classes, generated through a targeted query strategy with off-the-shelf foundation models. This strategy enhances feature learning, enabling our model to detect open samples with varied granularity more effectively. Extensive evaluations across five benchmarks demonstrate that OSLOPROMPT establishes a new state-of-the-art in LSOSDG, significantly outperforming existing methods.
Citation
M. H. N C et al., "OSLoPrompt: Bridging Low-Supervision Challenges and Open-Set Domain Generalization in CLIP," 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, 2025, pp. 10110-10120, doi: 10.1109/CVPR52734.2025.00945
Source
2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Conference
2025 Conference on Computer Vision and Pattern Recognition-CVPR-Annual
Keywords
Openset recognition, Low-shot learning, Prompt learning, Domain generalization
Subjects
Source
2025 Conference on Computer Vision and Pattern Recognition-CVPR-Annual
Publisher
IEEE