42 research outputs found
Theoretical Foundations of Adversarially Robust Learning
Despite extraordinary progress, current machine learning systems have been
shown to be brittle against adversarial examples: seemingly innocuous but
carefully crafted perturbations of test examples that cause machine learning
predictors to misclassify. Can we learn predictors robust to adversarial
examples? and how? There has been much empirical interest in this contemporary
challenge in machine learning, and in this thesis, we address it from a
theoretical perspective.
In this thesis, we explore what robustness properties can we hope to
guarantee against adversarial examples and develop an understanding of how to
algorithmically guarantee them. We illustrate the need to go beyond traditional
approaches and principles such as empirical risk minimization and uniform
convergence, and make contributions that can be categorized as follows: (1)
introducing problem formulations capturing aspects of emerging practical
challenges in robust learning, (2) designing new learning algorithms with
provable robustness guarantees, and (3) characterizing the complexity of robust
learning and fundamental limitations on the performance of any algorithm.Comment: PhD Thesi
Guarantees on learning depth-2 neural networks under a data-poisoning attack
In recent times many state-of-the-art machine learning models have been shown
to be fragile to adversarial attacks. In this work we attempt to build our
theoretical understanding of adversarially robust learning with neural nets. We
demonstrate a specific class of neural networks of finite size and a
non-gradient stochastic algorithm which tries to recover the weights of the net
generating the realizable true labels in the presence of an oracle doing a
bounded amount of malicious additive distortion to the labels. We prove (nearly
optimal) trade-offs among the magnitude of the adversarial attack, the accuracy
and the confidence achieved by the proposed algorithm.Comment: 11 page
Sample complexity of robust learning against evasion attacks
It is becoming increasingly important to understand the vulnerability of machine learning models to adversarial attacks. One of the fundamental problems in adversarial machine learning is to quantify how much training data is needed in the presence of so-called evasion attacks, where data is corrupted at test time. In this thesis, we work with the exact-in-the-ball notion of robustness and study the feasibility of adversarially robust learning from the perspective of learning theory, considering sample complexity.
We start with two negative results. We show that no non-trivial concept class can be robustly learned in the distribution-free setting against an adversary who can perturb just a single input bit. We then exhibit a sample-complexity lower bound: the class of monotone conjunctions and any superclass on the boolean hypercube has sample complexity at least exponential in the adversary's budget (that is, the maximum number of bits it can perturb on each input). This implies, in particular, that these classes cannot be robustly learned under the uniform distribution against an adversary who can perturb bits of the input.
As a first route to obtaining robust learning guarantees, we consider restricting the class of distributions over which training and testing data are drawn. We focus on learning problems with probability distributions on the input data that satisfy a Lipschitz condition: nearby points have similar probability. We show that, if the adversary is restricted to perturbing bits, then one can robustly learn the class of monotone conjunctions with respect to the class of log-Lipschitz distributions. We then extend this result to show the learnability of 1-decision lists, 2-decision lists and monotone k-decision lists in the same distributional and adversarial setting. We finish by showing that for every fixed k the class of k-decision lists has polynomial sample complexity against a log(n)-bounded adversary. The advantage of considering intermediate subclasses of k-decision lists is that we are able to obtain improved sample complexity bounds for these cases.
As a second route, we study learning models where the learner is given more power through the use of local queries. The first learning model we consider uses local membership queries (LMQ), where the learner can query the label of points near the training sample. We show that, under the uniform distribution, the exponential dependence on the adversary's budget to robustly learn conjunctions and any superclass remains inevitable even when the learner is given access to LMQs in addition to random examples. Faced with this negative result, we introduce a local equivalence, query oracle, which returns whether the hypothesis and target concept agree in a given region around a point in the training sample, as well as a counterexample if it exists. We show a separation result: on the one hand, if the query radius λ is strictly smaller than the adversary's perturbation budget ρ, then distribution free robust learning is impossible for a wide variety of concept classes; on the other hand, the setting λ = ρ allows us to develop robust empirical risk minimization algorithms in the distribution-free setting. We then bound the query complexity of these algorithms based on online learning guarantees and further improve these bounds for the special case of conjunctions. We follow by giving a robust learning algorithm for halfspaces on {0,1}n. Finally, since the query complexity for halfspaces on Rn is unbounded, we instead consider adversaries with bounded precision and give query complexity upper bounds in this setting as well
Two Heads are Better than One: Towards Better Adversarial Robustness by Combining Transduction and Rejection
Both transduction and rejection have emerged as important techniques for
defending against adversarial perturbations. A recent work by Tram\`er showed
that, in the rejection-only case (no transduction), a strong rejection-solution
can be turned into a strong (but computationally inefficient) non-rejection
solution. This detector-to-classifier reduction has been mostly applied to give
evidence that certain claims of strong selective-model solutions are
susceptible, leaving the benefits of rejection unclear. On the other hand, a
recent work by Goldwasser et al. showed that rejection combined with
transduction can give provable guarantees (for certain problems) that cannot be
achieved otherwise. Nevertheless, under recent strong adversarial attacks
(GMSA, which has been shown to be much more effective than AutoAttack against
transduction), Goldwasser et al.'s work was shown to have low performance in a
practical deep-learning setting. In this paper, we take a step towards
realizing the promise of transduction+rejection in more realistic scenarios.
Theoretically, we show that a novel application of Tram\`er's
classifier-to-detector technique in the transductive setting can give
significantly improved sample-complexity for robust generalization. While our
theoretical construction is computationally inefficient, it guides us to
identify an efficient transductive algorithm to learn a selective model.
Extensive experiments using state of the art attacks (AutoAttack, GMSA) show
that our solutions provide significantly better robust accuracy