Adversarial Pixel Restoration as a Pretext Task for Transferable Perturbations
Malik, Hashmat Shadab
Malik, Hashmat Shadab
Author
Supervisor
Department
Computer Vision
Embargo End Date
Type
Thesis
Date
2022
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Transferable adversarial attacks optimize adversaries from a pretrained surrogate model and known label space to fool the unknown black-box models. Therefore, these attacks are restricted by the availability of an effective surrogate model. In this work, we relax this assumption and propose Adversarial Pixel Restoration as a self-supervised alternative to train an effective surrogate model from scratch under the condition of no labels and few data samples. Our training approach is based on a min-max scheme which reduces overfitting via an adversarial objective and thus optimizes for a more generalizable surrogate model. Our proposed attack is complimentary to the adversarial pixel restoration and is independent of any task specific objective as it can be launched in a self-supervised manner. We successfully demonstrate the adversarial transferability of our approach to Vision Transformers as well as Convolutional Neural Networks for the tasks of classification, object detection, and video segmentation. Our training approach improves the transferability of the baseline unsupervised training method by 18.9% on the selected 5000 images from the ImageNet validation set. Furthermore, we also consider the practical scenario of availability of large unlabelled dataset for training of surrogate models. We scale our training approach under this setting and construct cross-domain and cross-task adversarial examples. Our method significantly improves the cross-domain transferability of adversarial examples, with significant gain over the baseline method. Our codes & pre-trained surrogate models are available at: https://github.com/HashmatShadab/APR.
Citation
H.S. Malik, "Adversarial Pixel Restoration as a Pretext Task for Transferable Perturbations", M.S. Thesis, Computer Vision, MBZUAI, Abu Dhabi, UAE, 2022.