2,454 research outputs found
Adversarial Learning Guarantees for Linear Hypotheses and Neural Networks
Adversarial or test time robustness measures the susceptibility of a
classifier to perturbations to the test input. While there has been a flurry of
recent work on designing defenses against such perturbations, the theory of
adversarial robustness is not well understood. In order to make progress on
this, we focus on the problem of understanding generalization in adversarial
settings, via the lens of Rademacher complexity. We give upper and lower bounds
for the adversarial empirical Rademacher complexity of linear hypotheses with
adversarial perturbations measured in -norm for an arbitrary .
This generalizes the recent result of [Yin et al.'19] that studies the case of
, and provides a finer analysis of the dependence on the input
dimensionality as compared to the recent work of [Khim and Loh'19] on linear
hypothesis classes.
We then extend our analysis to provide Rademacher complexity lower and upper
bounds for a single ReLU unit. Finally, we give adversarial Rademacher
complexity bounds for feed-forward neural networks with one hidden layer.
Unlike previous works we directly provide bounds on the adversarial Rademacher
complexity of the given network, as opposed to a bound on a surrogate. A
by-product of our analysis also leads to tighter bounds for the Rademacher
complexity of linear hypotheses, for which we give a detailed analysis and
present a comparison with existing bounds
Formal methods and software engineering for DL. Security, safety and productivity for DL systems development
Deep Learning (DL) techniques are now widespread and being integrated into
many important systems. Their classification and recognition abilities ensure
their relevance for multiple application domains. As machine-learning that
relies on training instead of algorithm programming, they offer a high degree
of productivity. But they can be vulnerable to attacks and the verification of
their correctness is only just emerging as a scientific and engineering
possibility. This paper is a major update of a previously-published survey,
attempting to cover all recent publications in this area. It also covers an
even more recent trend, namely the design of domain-specific languages for
producing and training neural nets.Comment: Submitted to IEEE-CCECE201
Thwarting Adversarial Examples: An -RobustSparse Fourier Transform
We give a new algorithm for approximating the Discrete Fourier transform of
an approximately sparse signal that has been corrupted by worst-case
noise, namely a bounded number of coordinates of the signal have been corrupted
arbitrarily. Our techniques generalize to a wide range of linear
transformations that are used in data analysis such as the Discrete Cosine and
Sine transforms, the Hadamard transform, and their high-dimensional analogs. We
use our algorithm to successfully defend against well known adversaries
in the setting of image classification. We give experimental results on the
Jacobian-based Saliency Map Attack (JSMA) and the Carlini Wagner (CW)
attack on the MNIST and Fashion-MNIST datasets as well as the Adversarial Patch
on the ImageNet dataset.Comment: Accepted at 32nd Conference on Neural Information Processing Systems
(NeurIPS 2018), Montr\'eal, Canad
Bridging Theory and Algorithm for Domain Adaptation
This paper addresses the problem of unsupervised domain adaption from
theoretical and algorithmic perspectives. Existing domain adaptation theories
naturally imply minimax optimization algorithms, which connect well with the
domain adaptation methods based on adversarial learning. However, several
disconnections still exist and form the gap between theory and algorithm. We
extend previous theories (Mansour et al., 2009c; Ben-David et al., 2010) to
multiclass classification in domain adaptation, where classifiers based on the
scoring functions and margin loss are standard choices in algorithm design. We
introduce Margin Disparity Discrepancy, a novel measurement with rigorous
generalization bounds, tailored to the distribution comparison with the
asymmetric margin loss, and to the minimax optimization for easier training.
Our theory can be seamlessly transformed into an adversarial learning algorithm
for domain adaptation, successfully bridging the gap between theory and
algorithm. A series of empirical studies show that our algorithm achieves the
state of the art accuracies on challenging domain adaptation tasks.Comment: Proceedings of the 36th International Conference on Machine Learning,
Long Beach, California, PMLR 97, 201
Certified Adversarial Robustness via Randomized Smoothing
We show how to turn any classifier that classifies well under Gaussian noise
into a new classifier that is certifiably robust to adversarial perturbations
under the norm. This "randomized smoothing" technique has been
proposed recently in the literature, but existing guarantees are loose. We
prove a tight robustness guarantee in norm for smoothing with Gaussian
noise. We use randomized smoothing to obtain an ImageNet classifier with e.g. a
certified top-1 accuracy of 49% under adversarial perturbations with
norm less than 0.5 (=127/255). No certified defense has been shown feasible on
ImageNet except for smoothing. On smaller-scale datasets where competing
approaches to certified robustness are viable, smoothing delivers
higher certified accuracies. Our strong empirical results suggest that
randomized smoothing is a promising direction for future research into
adversarially robust classification. Code and models are available at
http://github.com/locuslab/smoothing.Comment: ICML 201
Ensemble Robustness and Generalization of Stochastic Deep Learning Algorithms
The question why deep learning algorithms generalize so well has attracted
increasing research interest. However, most of the well-established approaches,
such as hypothesis capacity, stability or sparseness, have not provided
complete explanations (Zhang et al., 2016; Kawaguchi et al., 2017). In this
work, we focus on the robustness approach (Xu & Mannor, 2012), i.e., if the
error of a hypothesis will not change much due to perturbations of its training
examples, then it will also generalize well. As most deep learning algorithms
are stochastic (e.g., Stochastic Gradient Descent, Dropout, and
Bayes-by-backprop), we revisit the robustness arguments of Xu & Mannor, and
introduce a new approach, ensemble robustness, that concerns the robustness of
a population of hypotheses. Through the lens of ensemble robustness, we reveal
that a stochastic learning algorithm can generalize well as long as its
sensitiveness to adversarial perturbations is bounded in average over training
examples. Moreover, an algorithm may be sensitive to some adversarial examples
(Goodfellow et al., 2015) but still generalize well. To support our claims, we
provide extensive simulations for different deep learning algorithms and
different network architectures exhibiting a strong correlation between
ensemble robustness and the ability to generalize.Comment: 16 pages, 2 figure
Synthetic Data Generators: Sequential and Private
We study the sample complexity of private synthetic data generation over an
unbounded sized class of statistical queries, and show that any class that is
privately proper PAC learnable admits a private synthetic data generator
(perhaps non-efficient). Previous work on synthetic data generators focused on
the case that the query class is finite and obtained sample
complexity bounds that scale logarithmically with the size .
Here we construct a private synthetic data generator whose sample complexity is
independent of the domain size, and we replace finiteness with the assumption
that is privately PAC learnable (a formally weaker task, hence we
obtain equivalence between the two tasks)
VC Classes are Adversarially Robustly Learnable, but Only Improperly
We study the question of learning an adversarially robust predictor. We show
that any hypothesis class with finite VC dimension is robustly
PAC learnable with an improper learning rule. The requirement of being improper
is necessary as we exhibit examples of hypothesis classes with
finite VC dimension that are not robustly PAC learnable with any proper
learning rule.Comment: COLT 2019 Camera Read
Constrained Deep Learning using Conditional Gradient and Applications in Computer Vision
A number of results have recently demonstrated the benefits of incorporating
various constraints when training deep architectures in vision and machine
learning. The advantages range from guarantees for statistical generalization
to better accuracy to compression. But support for general constraints within
widely used libraries remains scarce and their broader deployment within many
applications that can benefit from them remains under-explored. Part of the
reason is that Stochastic gradient descent (SGD), the workhorse for training
deep neural networks, does not natively deal with constraints with global scope
very well. In this paper, we revisit a classical first order scheme from
numerical optimization, Conditional Gradients (CG), that has, thus far had
limited applicability in training deep models. We show via rigorous analysis
how various constraints can be naturally handled by modifications of this
algorithm. We provide convergence guarantees and show a suite of immediate
benefits that are possible -- from training ResNets with fewer layers but
better accuracy simply by substituting in our version of CG to faster training
of GANs with 50% fewer epochs in image inpainting applications to provably
better generalization guarantees using efficiently implementable forms of
recently proposed regularizers
An Overview of Privacy in Machine Learning
Over the past few years, providers such as Google, Microsoft, and Amazon have
started to provide customers with access to software interfaces allowing them
to easily embed machine learning tasks into their applications. Overall,
organizations can now use Machine Learning as a Service (MLaaS) engines to
outsource complex tasks, e.g., training classifiers, performing predictions,
clustering, etc. They can also let others query models trained on their data.
Naturally, this approach can also be used (and is often advocated) in other
contexts, including government collaborations, citizen science projects, and
business-to-business partnerships. However, if malicious users were able to
recover data used to train these models, the resulting information leakage
would create serious issues. Likewise, if the inner parameters of the model are
considered proprietary information, then access to the model should not allow
an adversary to learn such parameters. In this document, we set to review
privacy challenges in this space, providing a systematic review of the relevant
research literature, also exploring possible countermeasures. More
specifically, we provide ample background information on relevant concepts
around machine learning and privacy. Then, we discuss possible adversarial
models and settings, cover a wide range of attacks that relate to private
and/or sensitive information leakage, and review recent results attempting to
defend against such attacks. Finally, we conclude with a list of open problems
that require more work, including the need for better evaluations, more
targeted defenses, and the study of the relation to policy and data protection
efforts
- β¦