4 research outputs found
Testing Robustness Against Unforeseen Adversaries
Most existing adversarial defenses only measure robustness to L_p adversarial
attacks. Not only are adversaries unlikely to exclusively create small L_p
perturbations, adversaries are unlikely to remain fixed. Adversaries adapt and
evolve their attacks; hence adversarial defenses must be robust to a broad
range of unforeseen attacks. We address this discrepancy between research and
reality by proposing a new evaluation framework called ImageNet-UA. Our
framework enables the research community to test ImageNet model robustness
against attacks not encountered during training. To create ImageNet-UA's
diverse attack suite, we introduce a total of four novel adversarial attacks.
We also demonstrate that, in comparison to ImageNet-UA, prevailing L_inf
robustness assessments give a narrow account of model robustness. By evaluating
current defenses with ImageNet-UA, we find they provide little robustness to
unforeseen attacks. We hope the greater variety and realism of ImageNet-UA
enables development of more robust defenses which can generalize beyond attacks
seen during training
Understanding and mitigating universal adversarial perturbations for computer vision neural networks
Deep neural networks (DNNs) have become the algorithm of choice for many computer vision applications. They are able to achieve human level performance in many computer vision tasks, and enable the automation and large-scale deployment of applications such as object tracking, autonomous vehicles, and medical imaging. However, DNNs expose software applications to systemic vulnerabilities in the form of Universal Adversarial Perturbations (UAPs): input perturbation attacks that can cause DNNs to make classification errors on large sets of inputs.
Our aim is to improve the robustness of computer vision DNNs to UAPs without sacrificing the models' predictive performance. To this end, we increase our understanding of these vulnerabilities by investigating the visual structures and patterns commonly appearing in UAPs. We demonstrate the efficacy and pervasiveness of UAPs by showing how Procedural Noise patterns can be used to generate efficient zero-knowledge attacks for different computer vision models and tasks at minimal cost to the attacker. We then evaluate the UAP robustness of various shape and texture-biased models, and found that applying them in ensembles provides marginal improvement to robustness.
To mitigate UAP attacks, we develop two novel approaches. First, we propose the Jacobian of DNNs to measure the sensitivity of computer vision DNNs. We derive theoretical bounds and provide empirical evidence that shows how a combination of Jacobian regularisation and ensemble methods allow for increased model robustness against UAPs without degrading the predictive performance of computer vision DNNs. Our results evince a robustness-accuracy trade-off against UAPs that is better than those of models trained in conventional ways. Finally, we design a detection method that analyses the hidden layer activation values to identify a variety of UAP attacks in real-time with low-latency. We show that our work outperforms existing defences under realistic time and computation constraints.Open Acces
Sensitivity of Deep Convolutional Networks to Gabor Noise
Deep Convolutional Networks (DCNs) have been shown to be sensitive to Universal Adversarial Perturbations (UAPs): input-agnostic perturbations that fool a model on large portions of a dataset. These UAPs exhibit interesting visual patterns, but this phenomena is, as yet, poorly understood. Our work shows that visually similar procedural noise patterns also act as UAPs. In particular, we demonstrate that different DCN architectures are sensitive to Gabor noise patterns. This behaviour, its causes, and implications deserve further in-depth study