MDD-Net: Multimodal Depression Detection through Mutual Transformer
Haque, Rezwanul Islam, Milon Raju, SM Taslim Uddin Altaheri, Hamdi Nassar, Lobna Karray, Fakhri
Haque, Rezwanul
Islam, Milon
Raju, SM Taslim Uddin
Altaheri, Hamdi
Nassar, Lobna
Karray, Fakhri
Supervisor
Department
Machine Learning
Embargo End Date
Type
Conference proceeding
Date
License
Language
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Depression is a major mental health condition that severely impacts the emotional and physical well-being of individuals. The simple nature of data collection from social media platforms has attracted significant interest in properly utilizing this information for mental health research. A Multimodal Depression Detection Network (MDD-Net), utilizing acoustic and visual data obtained from social media networks, is proposed in this work where mutual transformers are exploited to efficiently extract and fuse multimodal features for efficient depression detection. The MDD-Net consists of four core modules: an acoustic feature extraction module for retrieving relevant acoustic attributes, a visual feature extraction module for extracting significant high-level patterns, a mutual transformer for computing the correlations among the generated features and fusing these features from multiple modalities, and a detection layer for detecting depression using the fused feature representations. The extensive experiments are performed using the multimodal D-Vlog dataset, and the findings reveal that the developed multimodal depression detection network surpasses the state-of-the-art by up to 17.37% for F1-Score, demonstrating the greater performance of the proposed system. The source code is accessible at https://github.com/rezwanh001/Multimodal-Depression-Detection.
Citation
R. Haque, M. Islam, S.M.T.U. Raju, H. Altaheri, L. Nassar, F. Karray, "MDD-Net: Multimodal Depression Detection through Mutual Transformer," 2026, pp. 6592-6599.
Source
Conference
2025 IEEE International Conference on Systems, Man, and Cybernetics (SMC)
Keywords
46 Information and Computing Sciences, 4608 Human-Centred Computing, 3 Good Health and Well Being
Subjects
Source
2025 IEEE International Conference on Systems, Man, and Cybernetics (SMC)
Publisher
IEEE