CoreKD: A Context-Aware Local Region Structural Contrastive Knowledge Distillation Framework for Object Detection
Yi, Junfei Mao, Jianxu Wang, Yaonan Liu, Tengfei Li, Mingjie Zeng, Kai Zhang, Hui Chang, Xiaojun
Yi, Junfei
Mao, Jianxu
Wang, Yaonan
Liu, Tengfei
Li, Mingjie
Zeng, Kai
Zhang, Hui
Chang, Xiaojun
Supervisor
Department
Computer Vision
Embargo End Date
Type
Journal article
Date
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Knowledge distillation (KD) aims to transfer knowledge from a cumbersome teacher to a lightweight student, thereby reducing overall model complexity without sacrificing performance. Current methods tend to focus excessively on pixel-level knowledge transfer while overlooking localized and contextual information. To address this, we propose a novel context-aware local region structural contrastive knowledge distillation framework (CoreKD) for object detection tasks. Specifically, we introduce a patch-based semantic structural distillation (PSD) method, facilitating efficient localized valuable knowledge transfer. This method guides the student model in learning both consistency and diverse local knowledge from the teacher model by integrating the semantic and structural information. We also propose the intra-region contextual (Ita-RC) and inter-region contextual (Ite-RC) constraints for the PSD framework. These constraints could transfer the region-wise contextual information between the student and teacher, significantly enhancing the student detector's capacity to model contextual dependencies. To validate the efficacy of CoreKD, we conduct extensive experiments on the public MS COCO 2017 and PASCAL VOC datasets. These results demonstrate that CoreKD markedly improves the performance of students in object detection tasks, indicating its ability to transfer valuable knowledge.
Citation
J. Yi, J. Mao, Y. Wang, T. Liu, M. Li, K. Zeng , et al., "CoreKD: A Context-Aware Local Region Structural Contrastive Knowledge Distillation Framework for Object Detection," IEEE Transactions on Neural Networks and Learning Systems, vol. PP, no. 99, pp. 1-15, 2026, https://doi.org/10.1109/tnnls.2026.3672967.
Source
IEEE Transactions on Neural Networks and Learning Systems
Conference
Keywords
46 Information and Computing Sciences, 4605 Data Management and Data Science
Subjects
Source
Publisher
IEEE