817 research outputs found
Consistency in Models for Distributed Learning under Communication Constraints
Motivated by sensor networks and other distributed settings, several models
for distributed learning are presented. The models differ from classical works
in statistical pattern recognition by allocating observations of an independent
and identically distributed (i.i.d.) sampling process amongst members of a
network of simple learning agents. The agents are limited in their ability to
communicate to a central fusion center and thus, the amount of information
available for use in classification or regression is constrained. For several
basic communication models in both the binary classification and regression
frameworks, we question the existence of agent decision rules and fusion rules
that result in a universally consistent ensemble. The answers to this question
present new issues to consider with regard to universal consistency. Insofar as
these models present a useful picture of distributed scenarios, this paper
addresses the issue of whether or not the guarantees provided by Stone's
Theorem in centralized environments hold in distributed settings.Comment: To appear in the IEEE Transactions on Information Theor
Active Learning from Imperfect Labelers
We study active learning where the labeler can not only return incorrect
labels but also abstain from labeling. We consider different noise and
abstention conditions of the labeler. We propose an algorithm which utilizes
abstention responses, and analyze its statistical consistency and query
complexity under fairly natural assumptions on the noise and abstention rate of
the labeler. This algorithm is adaptive in a sense that it can automatically
request less queries with a more informed or less noisy labeler. We couple our
algorithm with lower bounds to show that under some technical conditions, it
achieves nearly optimal query complexity.Comment: To appear in NIPS 201
An Adaptive Strategy for Active Learning with Smooth Decision Boundary
We present the first adaptive strategy for active learning in the setting of
classification with smooth decision boundary. The problem of adaptivity (to
unknown distributional parameters) has remained opened since the seminal work
of Castro and Nowak (2007), which first established (active learning) rates for
this setting. While some recent advances on this problem establish adaptive
rates in the case of univariate data, adaptivity in the more practical setting
of multivariate data has so far remained elusive. Combining insights from
various recent works, we show that, for the multivariate case, a careful
reduction to univariate-adaptive strategies yield near-optimal rates without
prior knowledge of distributional parameters
Towards optimally abstaining from prediction with OOD test examples
A common challenge across all areas of machine learning is that training data
is not distributed like test data, due to natural shifts, "blind spots," or
adversarial examples; such test examples are referred to as out-of-distribution
(OOD) test examples. We consider a model where one may abstain from predicting,
at a fixed cost. In particular, our transductive abstention algorithm takes
labeled training examples and unlabeled test examples as input, and provides
predictions with optimal prediction loss guarantees. The loss bounds match
standard generalization bounds when test examples are i.i.d. from the training
distribution, but add an additional term that is the cost of abstaining times
the statistical distance between the train and test distribution (or the
fraction of adversarial examples). For linear regression, we give a
polynomial-time algorithm based on Celis-Dennis-Tapia optimization algorithms.
For binary classification, we show how to efficiently implement it using a
proper agnostic learner (i.e., an Empirical Risk Minimizer) for the class of
interest. Our work builds on a recent abstention algorithm of Goldwasser,
Kalais, and Montasser (2020) for transductive binary classification.Comment: In NeurIPS 2021 (+spotlight), 24 page
- …