4,838 research outputs found
On the Robustness of Bayesian Neural Networks to Adversarial Attacks
Vulnerability to adversarial attacks is one of the principal hurdles to the
adoption of deep learning in safety-critical applications. Despite significant
efforts, both practical and theoretical, training deep learning models robust
to adversarial attacks is still an open problem. In this paper, we analyse the
geometry of adversarial attacks in the large-data, overparameterized limit for
Bayesian Neural Networks (BNNs). We show that, in the limit, vulnerability to
gradient-based attacks arises as a result of degeneracy in the data
distribution, i.e., when the data lies on a lower-dimensional submanifold of
the ambient space. As a direct consequence, we demonstrate that in this limit
BNN posteriors are robust to gradient-based adversarial attacks. Crucially, we
prove that the expected gradient of the loss with respect to the BNN posterior
distribution is vanishing, even when each neural network sampled from the
posterior is vulnerable to gradient-based attacks. Experimental results on the
MNIST, Fashion MNIST, and half moons datasets, representing the finite data
regime, with BNNs trained with Hamiltonian Monte Carlo and Variational
Inference, support this line of arguments, showing that BNNs can display both
high accuracy on clean data and robustness to both gradient-based and
gradient-free based adversarial attacks.Comment: arXiv admin note: text overlap with arXiv:2002.0435
MTDeep: Boosting the Security of Deep Neural Nets Against Adversarial Attacks with Moving Target Defense
Present attack methods can make state-of-the-art classification systems based
on deep neural networks misclassify every adversarially modified test example.
The design of general defense strategies against a wide range of such attacks
still remains a challenging problem. In this paper, we draw inspiration from
the fields of cybersecurity and multi-agent systems and propose to leverage the
concept of Moving Target Defense (MTD) in designing a meta-defense for
'boosting' the robustness of an ensemble of deep neural networks (DNNs) for
visual classification tasks against such adversarial attacks. To classify an
input image, a trained network is picked randomly from this set of networks by
formulating the interaction between a Defender (who hosts the classification
networks) and their (Legitimate and Malicious) users as a Bayesian Stackelberg
Game (BSG). We empirically show that this approach, MTDeep, reduces
misclassification on perturbed images in various datasets such as MNIST,
FashionMNIST, and ImageNet while maintaining high classification accuracy on
legitimate test images. We then demonstrate that our framework, being the first
meta-defense technique, can be used in conjunction with any existing defense
mechanism to provide more resilience against adversarial attacks that can be
afforded by these defense mechanisms. Lastly, to quantify the increase in
robustness of an ensemble-based classification system when we use MTDeep, we
analyze the properties of a set of DNNs and introduce the concept of
differential immunity that formalizes the notion of attack transferability.Comment: Accepted to the Conference on Decision and Game Theory for Security
(GameSec), 201
Optimization and Abstraction: A Synergistic Approach for Analyzing Neural Network Robustness
In recent years, the notion of local robustness (or robustness for short) has
emerged as a desirable property of deep neural networks. Intuitively,
robustness means that small perturbations to an input do not cause the network
to perform misclassifications. In this paper, we present a novel algorithm for
verifying robustness properties of neural networks. Our method synergistically
combines gradient-based optimization methods for counterexample search with
abstraction-based proof search to obtain a sound and ({\delta}-)complete
decision procedure. Our method also employs a data-driven approach to learn a
verification policy that guides abstract interpretation during proof search. We
have implemented the proposed approach in a tool called Charon and
experimentally evaluated it on hundreds of benchmarks. Our experiments show
that the proposed approach significantly outperforms three state-of-the-art
tools, namely AI^2 , Reluplex, and Reluval
Detecting adversarial manipulation using inductive Venn-ABERS predictors
Inductive Venn-ABERS predictors (IVAPs) are a type of probabilistic predictors with the theoretical guarantee that their predictions are perfectly calibrated. In this paper, we propose to exploit this calibration property for the detection of adversarial examples in binary classification tasks. By rejecting predictions if the uncertainty of the IVAP is too high, we obtain an algorithm that is both accurate on the original test set and resistant to adversarial examples. This robustness is observed on adversarials for the underlying model as well as adversarials that were generated by taking the IVAP into account. The method appears to offer competitive robustness compared to the state-of-the-art in adversarial defense yet it is computationally much more tractable
- …