Visual–Semantic Graph Matching Net for Zero-Shot Learning
Duan, Bowen Chen, Shiming Guo, Yufei Xie, Guo-Sen Ding, Weiping Wang, Yisong
Duan, Bowen
Chen, Shiming
Guo, Yufei
Xie, Guo-Sen
Ding, Weiping
Wang, Yisong
Supervisor
Department
Computer Vision
Embargo End Date
Type
Journal article
Date
2025
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Zero-shot learning (ZSL) aims to leverage additional semantic information to recognize unseen classes. To transfer knowledge from seen to unseen classes, most ZSL methods often learn a shared embedding space by simply aligning visual embeddings with semantic prototypes. However, methods trained under this paradigm often struggle to learn robust embedding space because they align the two modalities in an isolated manner among classes, which ignore the crucial class relationship during the alignment process. To address the aforementioned challenges, this article proposes a visual-semantic graph matching net (VSGMN), which leverages semantic relationships among classes to aid in visual-semantic embedding. VSGMN uses a graph build net (GBN) and a graph matching net (GMN) to achieve two-stage visual-semantic alignment. Specifically, GBN first uses an embedding-based approach to build visual and semantic graphs in the semantic space and align the embedding with its prototype for first-stage alignment. In addition, to supplement unseen class relationships in these graphs, GBN also builds the unseen class nodes based on semantic relationships. In the second stage, GMN continuously integrates neighbor and cross-graph information into the constructed graph nodes and aligns the node relationships between the two graphs under the class relationship constraint. Extensive experiments on three benchmark datasets demonstrate that VSGMN achieves superior performance in both conventional and generalized ZSL (GZSL) scenarios. The implementation of our VSGMN and experimental results are available at github: https://github.com/dbwfd/VSGMN.
Citation
B. Duan, S. Chen, Y. Guo, G. -S. Xie, W. Ding and Y. Wang, "Visual–Semantic Graph Matching Net for Zero-Shot Learning," in IEEE Transactions on Neural Networks and Learning Systems, vol. 36, no. 6, pp. 10171-10185, June 2025, doi: 10.1109/TNNLS.2024.3499377
Source
IEEE Transactions on Neural Networks and Learning Systems
Conference
Keywords
Graph match, graph neural network (GNN), semantic-visual alignment, zero-shot learning (ZSL)
Subjects
Source
Publisher
IEEE