Loading...
HCVP: Leveraging Hierarchical Contrastive Visual Prompt for Domain Generalization
Zhou, Guanglin Han, Zhongyi Chen, Shiming Huang, Biwei Zhu, Liming Liu, Tongliang Yao, Lina Zhang, Kun
Zhou, Guanglin
Han, Zhongyi
Chen, Shiming
Huang, Biwei
Zhu, Liming
Liu, Tongliang
Yao, Lina
Zhang, Kun
Supervisor
Department
Computer Vision
Embargo End Date
Type
Journal article
Date
2025
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Domain Generalization (DG) endeavors to create machine learning models that excel in unseen scenarios by learning invariant features. In DG, the prevalent practice of constraining models to a fixed structure or uniform parameterization to encapsulate invariant features can inadvertently blend specific aspects. Such an approach struggles with nuanced differentiation of inter-domain variations and may exhibit bias towards certain domains, hindering the precise learning of domain-invariant features. Recognizing this, we introduce a novel method designed to supplement the model with domain-level and task-specific characteristics. This approach aims to guide the model in more effectively separating invariant features from specific characteristics, thereby boosting the generalization. Building on the emerging trend of visual prompts in the DG paradigm, our work introduces the novel Hierarchical Contrastive Visual Prompt (HCVP) methodology. This represents a significant advancement in the field, setting itself apart with a unique generative approach to prompts, alongside an explicit model structure and specialized loss functions. Differing from traditional visual prompts that are often shared across entire datasets, HCVP utilizes a hierarchical prompt generation network enhanced by prompt contrastive learning. These generative prompts are instance-dependent, catering to the unique characteristics inherent to different domains and tasks. Additionally, we devise a prompt modulation network that serves as a bridge, effectively incorporating the generated visual prompts into the vision transformer backbone. Experiments conducted on five DG datasets demonstrate the effectiveness of HCVP, outperforming both established DG algorithms and adaptation protocols.
Citation
G. Zhou et al., “HCVP: Leveraging Hierarchical Contrastive Visual Prompt for Domain Generalization,” IEEE Trans Multimedia, 2024, doi: 10.1109/TMM.2024.3521719.
Source
IEEE Transactions on Multimedia
Conference
Keywords
contrastive learning, Domain generalization, visual prompt
Subjects
Source
Publisher
IEEE