DViN: Dynamic Visual Routing Network for Weakly Supervised Referring Expression Comprehension
Chen, Xiaofu Luo, Yaxin Luo, Gen Ji, Jiayi Ding, Henghui Zhou, Yiyi
Chen, Xiaofu
Luo, Yaxin
Luo, Gen
Ji, Jiayi
Ding, Henghui
Zhou, Yiyi
Author
Supervisor
Department
Natural Language Processing
Embargo End Date
Type
Conference proceeding
Date
2025
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
In this paper, we focus on weakly supervised referring expression comprehension (REC), and identify that the lack of fine-grained visual capability greatly limits the upper performance bound of existing methods. To address this issue, we propose a novel framework for weakly supervised REC, namely Dynamic Visual routing Network (DViN), which overcomes the visual shortcomings from the perspective of feature combination and alignment. In particular, DViN is equipped with a novel sparse routing mechanism to efficiently combine features of multiple visual encoders in a dynamic manner, thus improving the visual descriptive power. Besides, we further propose an innovative weakly supervised objective, namely Routing-based Feature Alignment (RFA), which facilitates the visual understanding of routed features through the intra-modal and inter-modal alignment. To validate DViN, we conduct extensive experiments on four REC benchmark datasets. Experiments demonstrate that DViN achieves state-of-the-art results on four benchmarks while maintaining competitive inference efficiency. Besides, the strong generalization ability of DViN is also validated on weakly supervised referring expression segmentation. Source codes are anonymously released at: https://github.com/XxFChen/DViN.
Citation
X. Chen, Y. Luo, G. Luo, J. Ji, H. Ding and Y. Zhou, "DViN: Dynamic Visual Routing Network for Weakly Supervised Referring Expression Comprehension," 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, 2025, pp. 14347-14357, doi: 10.1109/CVPR52734.2025.01338.
Source
Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
Conference
2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2025
Keywords
Referring Expression Comprehension, Weakly Supervised
Subjects
Source
2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2025
Publisher
IEEE