Towards Evaluating the Robustness of Visual State Space Models
Malik, Hashmat Shadab Shamshad, Fahad Naseer, Muhammad Muzammal Nandakumar, Karthik Khan, Fahad Shahbaz Khan, Salman Hameed
Malik, Hashmat Shadab
Shamshad, Fahad
Naseer, Muhammad Muzammal
Nandakumar, Karthik
Khan, Fahad Shahbaz
Khan, Salman Hameed
Supervisor
Department
Computer Vision
Embargo End Date
Type
Conference proceeding
Date
2025
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Vision State Space Models (VSSMs), a novel architecture that combines the strengths of recurrent neural networks and latent variable models, have demonstrated remarkable performance in visual perception tasks by efficiently capturing long-range dependencies and modeling complex visual dynamics. However, their robustness under natural and adversarial perturbations remains a critical concern. In this work, we present a comprehensive evaluation of VSSMs' robustness under various perturbation scenarios, including occlusions, image structure, common corruptions, and adversarial attacks, and compare their performance to well-established architectures such as transformers and Convolutional Neural Networks. Furthermore, we investigate the resilience of VSSMs to object-background compositional changes on sophisticated benchmarks designed to test model performance in complex visual scenes. We also assess their robustness on object detection and segmentation tasks using corrupted datasets that mimic real-world scenarios. To gain a deeper understanding of VSSMs' adversarial robustness, we conduct a frequency-based analysis of adversarial attacks, evaluating their per formance against low-frequency and high-frequency perturbations. Our findings highlight the strengths and limitations of VSSMs in handling complex visual corruptions, offering valuable insights for future research. Our code and models are available on GitHub.
Citation
H. S. Malik, F. Shamshad, M. Naseer, K. Nandakumar, F. S. Khan and S. Khan, "Towards Evaluating the Robustness of Visual State Space Models," 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Nashville, TN, USA, 2025, pp. 3544-3553, doi: 10.1109/CVPRW67362.2025.00339.
Source
IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops
Conference
2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2025
Keywords
Adversarial Attacks, Robustness, VSSMs
Subjects
Source
2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2025
Publisher
IEEE