6 research outputs found
Learning Geometric Concepts with Nasty Noise
We study the efficient learnability of geometric concept classes -
specifically, low-degree polynomial threshold functions (PTFs) and
intersections of halfspaces - when a fraction of the data is adversarially
corrupted. We give the first polynomial-time PAC learning algorithms for these
concept classes with dimension-independent error guarantees in the presence of
nasty noise under the Gaussian distribution. In the nasty noise model, an
omniscient adversary can arbitrarily corrupt a small fraction of both the
unlabeled data points and their labels. This model generalizes well-studied
noise models, including the malicious noise model and the agnostic (adversarial
label noise) model. Prior to our work, the only concept class for which
efficient malicious learning algorithms were known was the class of
origin-centered halfspaces.
Specifically, our robust learning algorithm for low-degree PTFs succeeds
under a number of tame distributions -- including the Gaussian distribution
and, more generally, any log-concave distribution with (approximately) known
low-degree moments. For LTFs under the Gaussian distribution, we give a
polynomial-time algorithm that achieves error , where
is the noise rate. At the core of our PAC learning results is an efficient
algorithm to approximate the low-degree Chow-parameters of any bounded function
in the presence of nasty noise. To achieve this, we employ an iterative
spectral method for outlier detection and removal, inspired by recent work in
robust unsupervised learning. Our aforementioned algorithm succeeds for a range
of distributions satisfying mild concentration bounds and moment assumptions.
The correctness of our robust learning algorithm for intersections of
halfspaces makes essential use of a novel robust inverse independence lemma
that may be of broader interest
On the Complexity of the Inverse Semivalue Problem for Weighted Voting Games
Weighted voting games are a family of cooperative games, typically used to
model voting situations where a number of agents (players) vote against or for
a proposal. In such games, a proposal is accepted if an appropriately weighted
sum of the votes exceeds a prespecified threshold. As the influence of a player
over the voting outcome is not in general proportional to her assigned weight,
various power indices have been proposed to measure each player's influence.
The inverse power index problem is the problem of designing a weighted voting
game that achieves a set of target influences according to a predefined power
index. In this work, we study the computational complexity of the inverse
problem when the power index belongs to the class of semivalues. We prove that
the inverse problem is computationally intractable for a broad family of
semivalues, including all regular semivalues. As a special case of our general
result, we establish computational hardness of the inverse problem for the
Banzhaf indices and the Shapley values, arguably the most popular power
indices.Comment: To appear in AAAI 201
Theoretical Foundations of Adversarially Robust Learning
Despite extraordinary progress, current machine learning systems have been
shown to be brittle against adversarial examples: seemingly innocuous but
carefully crafted perturbations of test examples that cause machine learning
predictors to misclassify. Can we learn predictors robust to adversarial
examples? and how? There has been much empirical interest in this contemporary
challenge in machine learning, and in this thesis, we address it from a
theoretical perspective.
In this thesis, we explore what robustness properties can we hope to
guarantee against adversarial examples and develop an understanding of how to
algorithmically guarantee them. We illustrate the need to go beyond traditional
approaches and principles such as empirical risk minimization and uniform
convergence, and make contributions that can be categorized as follows: (1)
introducing problem formulations capturing aspects of emerging practical
challenges in robust learning, (2) designing new learning algorithms with
provable robustness guarantees, and (3) characterizing the complexity of robust
learning and fundamental limitations on the performance of any algorithm.Comment: PhD Thesi