Pulmofusion: Advancing Pulmonary Health with Efficient Multi-Modal Fusion
Sharshar, Ahmed Attia, Yasser Yaqub, Mohammad Guizani, Mohsen
Sharshar, Ahmed
Attia, Yasser
Yaqub, Mohammad
Guizani, Mohsen
Supervisor
Department
Computer Vision
Embargo End Date
Type
Conference proceeding
Date
2025
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Traditional remote spirometry lacks the precision required for effective pulmonary monitoring. We present a novel, non-invasive approach using multimodal predictive models that integrate RGB or thermal video data with patient meta-data. Our method leverages energy-efficient Spiking Neural Networks (SNNs) for the regression of Peak Expiratory Flow (PEF) and classification of Forced Expiratory Volume (FEVl) and Forced Vital Capacity (FVC), using lightweight CNNs to overcome SNN limitations in regression tasks. Multimodal data integration is improved with a Multi-Head Attention Layer, and we employ K-Fold validation and en-semble learning to boost robustness. Using thermal data, our SNN models achieve 92% ± 2% accuracy on a breathing-cycle basis and 99.5% ± 0.5% patient-wise. PEF regression models attain Relative RMSEs of 0.11 ± 0.05 (thermal) and 0.26 ± 0.07 (RGB), with an MAE of 4.52% for FEV1/FVC predictions, establishing state-of-the-art performance.11Code and dataset can be found on https://github.comJahmed-sharshar/RespiroDynamics.git.
Citation
A. Sharshar, Y. Attia, M. Yaqub and M. Guizani, "Pulmofusion: Advancing Pulmonary Health with Efficient Multi-Modal Fusion," 2025 IEEE 22nd International Symposium on Biomedical Imaging (ISBI), Houston, TX, USA, 2025, pp. 1-5, doi: 10.1109/ISBI60581.2025.10981233
Source
Proceedings - International Symposium on Biomedical Imaging
Conference
22nd IEEE International Symposium on Biomedical Imaging, ISBI 2025
Keywords
Lung Health, Multi-Modal, Remote Spirometry, Smart Healthcare, Spiking Neural Networks (SNN)
Subjects
Source
22nd IEEE International Symposium on Biomedical Imaging, ISBI 2025
Publisher
IEEE