Towards Robust Training via Gradient-Diversified Backpropagation
He, Xilin Luo, Cheng Lin, Qinliang Xie, Weicheng Khan, Muhammad Haris Song, Siyang Shen, Linlin
He, Xilin
Luo, Cheng
Lin, Qinliang
Xie, Weicheng
Khan, Muhammad Haris
Song, Siyang
Shen, Linlin
Supervisor
Department
Computer Vision
Embargo End Date
Type
Conference proceeding
Date
2025
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Neural networks are prone to be vulnerable to adversarial attacks and domain shifts. Adversarial-driven methods including adversarial training and adversarial augmentation, have been frequently proposed to improve the model's robustness against adversarial attacks and distribution-shifted samples. Nonetheless, recent research on adversarial attacks has cast a spotlight on the robustness lacuna against attacks targeted at deep semantic layers. Our analysis reveals that previous adversarial-driven methods tend to generate overpowering perturbations in deep semantic layers, leading to distortion of the training for these layers. This can be primarily attributed to the exclusive utilization of loss functions on the output layer for adversarial gradient generation. This inherent practice projects an excessive adversarial impact on the deep semantic layers, elevating the difficulty of training such layers. Therefore, from the standing point of relaxing the excessive perturbations in the deep semantic layer and diversifying the adversarial gradients to ensure robust training for deep semantic layers, this paper proposes a novel Stochastic Loss Integration Method (SLIM), which can be instantiated into the existing adversarial-driven methods in a plug-and-play manner. Experimental results across diverse tasks, including classification and segmentation, as well as various areas such as adversarial robustness and domain generalization, validate the effectiveness of our proposed method. Furthermore, we provide an in-depth analysis to offer a comprehensive understanding of layer-wise training involving various loss terms.
Citation
X. He et al., "Towards Robust Training via Gradient-Diversified Backpropagation," 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Tucson, AZ, USA, 2025, pp. 7847-7856, doi: 10.1109/WACV61041.2025.00762.
Source
Proceedings - 2025 IEEE Winter Conference on Applications of Computer Vision, WACV 2025
Conference
2025 IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2025
Keywords
Adversarial training, Domain generalization
Subjects
Source
2025 IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2025
Publisher
IEEE