Hierarchical contrastive regularization with progressive self-distillation for few-shot learning
Zhou, Jun Li, Hanhui Huang, Jiehui Li, Xuejiao Tang, Zhenchao Liang, Xiaodan
Zhou, Jun
Li, Hanhui
Huang, Jiehui
Li, Xuejiao
Tang, Zhenchao
Liang, Xiaodan
Supervisor
Department
Computer Vision
Embargo End Date
Type
Journal article
Date
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Few-shot learning (FSL) with a generalizable embedding model trained on samples of seen classes is promising, as it enables the model to extract reliable representations for novel classes with only a few samples. The primary objective of FSL is to mitigate the model’s inductive bias toward the training classes, and most existing methods attempt to achieve this by modeling relationships at the instance level. However, such a paradigm struggles to thoroughly exploit the semantic information embedded within regions and pixels. To address this limitation and enhance feature generalizability, we propose a novel Hierarchical Contrastive Regularization with Progressive Self-Distillation (HCRD) framework in this paper. Our HCRD framework not only focuses on the intrinsic structures of training samples but also explores correspondences among instances at the regional and pixel levels. Specifically, HCRD consists of a Hierarchical Contrastive Enhancement (HCE) module and a Self-Distillation Embedding (SDE) module. The HCE module promotes intra-class compaction within the feature space across different semantic levels, while the SDE module progressively facilitates the transfer of useful knowledge in both the label space and the feature space. By efficiently leveraging hierarchical semantic information, the HCRD framework consistently outperforms state-of-the-art methods on four public benchmarks. Our code is available at https://github.com/zjgans/HCRD
Citation
J. Zhou, H. Li, J. Huang, X. Li, Z. Tang, X. Liang, "Hierarchical contrastive regularization with progressive self-distillation for few-shot learning," Neurocomputing, vol. 687, pp. 133744-133744, 2026, https://doi.org/10.1016/j.neucom.2026.133744.
Source
Neurocomputing
Conference
Keywords
46 Information and Computing Sciences, 4611 Machine Learning
Subjects
Source
Publisher
Elsevier