51,533 research outputs found
The Curse of Concentration in Robust Learning: Evasion and Poisoning Attacks from Concentration of Measure
Many modern machine learning classifiers are shown to be vulnerable to
adversarial perturbations of the instances. Despite a massive amount of work
focusing on making classifiers robust, the task seems quite challenging. In
this work, through a theoretical study, we investigate the adversarial risk and
robustness of classifiers and draw a connection to the well-known phenomenon of
concentration of measure in metric measure spaces. We show that if the metric
probability space of the test instance is concentrated, any classifier with
some initial constant error is inherently vulnerable to adversarial
perturbations.
One class of concentrated metric probability spaces are the so-called Levy
families that include many natural distributions. In this special case, our
attacks only need to perturb the test instance by at most to make
it misclassified, where is the data dimension. Using our general result
about Levy instance spaces, we first recover as special case some of the
previously proved results about the existence of adversarial examples. However,
many more Levy families are known (e.g., product distribution under the Hamming
distance) for which we immediately obtain new attacks that find adversarial
examples of distance .
Finally, we show that concentration of measure for product spaces implies the
existence of forms of "poisoning" attacks in which the adversary tampers with
the training data with the goal of degrading the classifier. In particular, we
show that for any learning algorithm that uses training examples, there is
an adversary who can increase the probability of any "bad property" (e.g.,
failing on a particular test instance) that initially happens with
non-negligible probability to by substituting only of the examples with other (still correctly labeled) examples
Error Bounds and Holder Metric Subregularity
The Holder setting of the metric subregularity property of set-valued
mappings between general metric or Banach/Asplund spaces is investigated in the
framework of the theory of error bounds for extended real-valued functions of
two variables. A classification scheme for the general Holder metric
subregularity criteria is presented. The criteria are formulated in terms of
several kinds of primal and subdifferential slopes.Comment: 32 pages. arXiv admin note: substantial text overlap with
arXiv:1405.113
H\"older Error Bounds and H\"older Calmness with Applications to Convex Semi-Infinite Optimization
Using techniques of variational analysis, necessary and sufficient
subdifferential conditions for H\"older error bounds are investigated and some
new estimates for the corresponding modulus are obtained. As an application, we
consider the setting of convex semi-infinite optimization and give a
characterization of the H\"older calmness of the argmin mapping in terms of the
level set mapping (with respect to the objective function) and a special
supremum function. We also estimate the H\"older calmness modulus of the argmin
mapping in the framework of linear programming.Comment: 25 page
- …