4 research outputs found
Certified Defenses for Data Poisoning Attacks
Machine learning systems trained on user-provided data are susceptible to
data poisoning attacks, whereby malicious users inject false training data with
the aim of corrupting the learned model. While recent work has proposed a
number of attacks and defenses, little is understood about the worst-case loss
of a defense in the face of a determined attacker. We address this by
constructing approximate upper bounds on the loss across a broad family of
attacks, for defenders that first perform outlier removal followed by empirical
risk minimization. Our approximation relies on two assumptions: (1) that the
dataset is large enough for statistical concentration between train and test
error to hold, and (2) that outliers within the clean (non-poisoned) data do
not have a strong effect on the model. Our bound comes paired with a candidate
attack that often nearly matches the upper bound, giving us a powerful tool for
quickly assessing defenses on a given dataset. Empirically, we find that even
under a simple defense, the MNIST-1-7 and Dogfish datasets are resilient to
attack, while in contrast the IMDB sentiment dataset can be driven from 12% to
23% test error by adding only 3% poisoned data.Comment: Appeared at NIPS 201
A General Family of Trimmed Estimators for Robust High-dimensional Data Analysis
We consider the problem of robustifying high-dimensional structured
estimation. Robust techniques are key in real-world applications which often
involve outliers and data corruption. We focus on trimmed versions of
structurally regularized M-estimators in the high-dimensional setting,
including the popular Least Trimmed Squares estimator, as well as analogous
estimators for generalized linear models and graphical models, using possibly
non-convex loss functions. We present a general analysis of their statistical
convergence rates and consistency, and then take a closer look at the trimmed
versions of the Lasso and Graphical Lasso estimators as special cases. On the
optimization side, we show how to extend algorithms for M-estimators to fit
trimmed variants and provide guarantees on their numerical convergence. The
generality and competitive performance of high-dimensional trimmed estimators
are illustrated numerically on both simulated and real-world genomics data.Comment: 39 pages, 6 figure
Manipulating Machine Learning: Poisoning Attacks and Countermeasures for Regression Learning
As machine learning becomes widely used for automated decisions, attackers
have strong incentives to manipulate the results and models generated by
machine learning algorithms. In this paper, we perform the first systematic
study of poisoning attacks and their countermeasures for linear regression
models. In poisoning attacks, attackers deliberately influence the training
data to manipulate the results of a predictive model. We propose a
theoretically-grounded optimization framework specifically designed for linear
regression and demonstrate its effectiveness on a range of datasets and models.
We also introduce a fast statistical attack that requires limited knowledge of
the training process. Finally, we design a new principled defense method that
is highly resilient against all poisoning attacks. We provide formal guarantees
about its convergence and an upper bound on the effect of poisoning attacks
when the defense is deployed. We evaluate extensively our attacks and defenses
on three realistic datasets from health care, loan assessment, and real estate
domains.Comment: Preprint of the work accepted for publication at the 39th IEEE
Symposium on Security and Privacy, San Francisco, CA, USA, May 21-23, 201
Towards Privacy and Security of Deep Learning Systems: A Survey
Deep learning has gained tremendous success and great popularity in the past
few years. However, recent research found that it is suffering several inherent
weaknesses, which can threaten the security and privacy of the stackholders.
Deep learning's wide use further magnifies the caused consequences. To this
end, lots of research has been conducted with the purpose of exhaustively
identifying intrinsic weaknesses and subsequently proposing feasible
mitigation. Yet few is clear about how these weaknesses are incurred and how
effective are these attack approaches in assaulting deep learning. In order to
unveil the security weaknesses and aid in the development of a robust deep
learning system, we are devoted to undertaking a comprehensive investigation on
attacks towards deep learning, and extensively evaluating these attacks in
multiple views. In particular, we focus on four types of attacks associated
with security and privacy of deep learning: model extraction attack, model
inversion attack, poisoning attack and adversarial attack. For each type of
attack, we construct its essential workflow as well as adversary capabilities
and attack goals. Many pivot metrics are devised for evaluating the attack
approaches, by which we perform a quantitative and qualitative analysis. From
the analysis, we have identified significant and indispensable factors in an
attack vector, \eg, how to reduce queries to target models, what distance used
for measuring perturbation. We spot light on 17 findings covering these
approaches' merits and demerits, success probability, deployment complexity and
prospects. Moreover, we discuss other potential security weaknesses and
possible mitigation which can inspire relevant researchers in this area.Comment: 23 pages, 6 figure