Fast Partial-modal Online Cross-Modal Hashing
Li, Fengling Sun, Yang Wang, Tianshi Zhu, Lei Chang, Xiaojun
Li, Fengling
Sun, Yang
Wang, Tianshi
Zhu, Lei
Chang, Xiaojun
Supervisor
Department
Computer Vision
Embargo End Date
Type
Journal article
Date
2025
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Cross-Modal Hashing (CMH) has become a powerful technique for large-scale cross-modal retrieval, offering benefits like fast computation and efficient storage. However, most CMH models struggle to adapt to streaming multimodal data in real-time once deployed. Although recent online CMH studies have made progress in this area, they often overlook two key challenges: 1) learning effectively from streaming partial-modal multimodal data, and 2) avoiding the high costs associated with frequent hash function re-training and large-scale updates to database hash codes. To address these issues, we propose Fast Partial-modal Online Cross-Modal Hashing (FPO-CMH), the first approach to tackle online cross-modal hash learning with partial-modal data. This marks a significant shift from previous methods that rely on fully-available multimodal data. Specifically, our approach introduces a multimodal dual-tier anchor bank, initialized using offline training data, which allows offline-trained CMH models to adapt seamlessly to partial-modal data while progressively updating the anchor bank. By leveraging gradient accumulation and asynchronous optimization, FPO-CMH facilitates efficient online cross-modal hash learning. Additionally, an initial-anchor rehearsal strategy is employed to prevent model catastrophic forgetting during online optimization, ensuring the code invariance of database hash codes and eliminating the need for frequent hash function re-training. Extensive experiments validate the superiority of FPO-CMH, especially in handling streaming partial-modal multimodal data, a more realistic scenario. The source codes and datasets are available at https://github.com/DandelionWow/FPO-CMH
Citation
F. Li, Y. Sun, T. Wang, L. Zhu and X. Chang, "Fast Partial-Modal Online Cross-Modal Hashing," in IEEE Transactions on Image Processing, vol. 34, pp. 4440-4455, 2025, doi: 10.1109/TIP.2025.3586504
Source
IEEE Transactions on Image Processing, 2025
Conference
Keywords
Cross-modal hashing, online learning, partial-modal data, dual-tier anchor bank
Subjects
Source
Publisher
IEEE