7 research outputs found
Learning with a Drifting Target Concept
We study the problem of learning in the presence of a drifting target
concept. Specifically, we provide bounds on the error rate at a given time,
given a learner with access to a history of independent samples labeled
according to a target concept that can change on each round. One of our main
contributions is a refinement of the best previous results for polynomial-time
algorithms for the space of linear separators under a uniform distribution. We
also provide general results for an algorithm capable of adapting to a variable
rate of drift of the target concept. Some of the results also describe an
active learning variant of this setting, and provide bounds on the number of
queries for the labels of points in the sequence sufficient to obtain the
stated bounds on the error rates
The Power of Localization for Efficiently Learning Linear Separators with Noise
We introduce a new approach for designing computationally efficient learning
algorithms that are tolerant to noise, and demonstrate its effectiveness by
designing algorithms with improved noise tolerance guarantees for learning
linear separators.
We consider both the malicious noise model and the adversarial label noise
model. For malicious noise, where the adversary can corrupt both the label and
the features, we provide a polynomial-time algorithm for learning linear
separators in under isotropic log-concave distributions that can
tolerate a nearly information-theoretically optimal noise rate of . For the adversarial label noise model, where the
distribution over the feature vectors is unchanged, and the overall probability
of a noisy label is constrained to be at most , we also give a
polynomial-time algorithm for learning linear separators in under
isotropic log-concave distributions that can handle a noise rate of .
We show that, in the active learning model, our algorithms achieve a label
complexity whose dependence on the error parameter is
polylogarithmic. This provides the first polynomial-time active learning
algorithm for learning linear separators in the presence of malicious noise or
adversarial label noise.Comment: Contains improved label complexity analysis communicated to us by
Steve Hannek
An Adaptive Algorithm for Learning with Unknown Distribution Drift
We develop and analyze a general technique for learning with an unknown
distribution drift. Given a sequence of independent observations from the last
steps of a drifting distribution, our algorithm agnostically learns a
family of functions with respect to the current distribution at time .
Unlike previous work, our technique does not require prior knowledge about the
magnitude of the drift. Instead, the algorithm adapts to the sample data.
Without explicitly estimating the drift, the algorithm learns a family of
functions with almost the same error as a learning algorithm that knows the
magnitude of the drift in advance. Furthermore, since our algorithm adapts to
the data, it can guarantee a better learning error than an algorithm that
relies on loose bounds on the drift.Comment: Fixed typos and references. Updated conclusio
A Stability Principle for Learning under Non-Stationarity
We develop a versatile framework for statistical learning in non-stationary
environments. In each time period, our approach applies a stability principle
to select a look-back window that maximizes the utilization of historical data
while keeping the cumulative bias within an acceptable range relative to the
stochastic error. Our theory showcases the adaptability of this approach to
unknown non-stationarity. The regret bound is minimax optimal up to logarithmic
factors when the population losses are strongly convex, or Lipschitz only. At
the heart of our analysis lie two novel components: a measure of similarity
between functions and a segmentation technique for dividing the non-stationary
data sequence into quasi-stationary pieces.Comment: 47 pages, 1 figur