27 research outputs found
Unconfused Ultraconservative Multiclass Algorithms
We tackle the problem of learning linear classifiers from noisy datasets in a
multiclass setting. The two-class version of this problem was studied a few
years ago by, e.g. Bylander (1994) and Blum et al. (1996): in these
contributions, the proposed approaches to fight the noise revolve around a
Perceptron learning scheme fed with peculiar examples computed through a
weighted average of points from the noisy training set. We propose to build
upon these approaches and we introduce a new algorithm called UMA (for
Unconfused Multiclass additive Algorithm) which may be seen as a generalization
to the multiclass setting of the previous approaches. In order to characterize
the noise we use the confusion matrix as a multiclass extension of the
classification noise studied in the aforementioned literature. Theoretically
well-founded, UMA furthermore displays very good empirical noise robustness, as
evidenced by numerical simulations conducted on both synthetic and real data.
Keywords: Multiclass classification, Perceptron, Noisy labels, Confusion MatrixComment: ACML, Australia (2013
Exploring Parity Challenges in Reinforcement Learning through Curriculum Learning with Noisy Labels
This paper delves into applying reinforcement learning (RL) in strategy
games, particularly those characterized by parity challenges, as seen in
specific positions of Go and Chess and a broader range of impartial games. We
propose a simulated learning process, structured within a curriculum learning
framework and augmented with noisy labels, to mirror the intricacies of
self-play learning scenarios. This approach thoroughly analyses how neural
networks (NNs) adapt and evolve from elementary to increasingly complex game
positions. Our empirical research indicates that even minimal label noise can
significantly impede NNs' ability to discern effective strategies, a difficulty
that intensifies with the growing complexity of the game positions. These
findings underscore the urgent need for advanced methodologies in RL training,
specifically tailored to counter the obstacles imposed by noisy evaluations.
The development of such methodologies is crucial not only for enhancing NN
proficiency in strategy games with significant parity elements but also for
broadening the resilience and efficiency of RL systems across diverse and
complex environments
Bayes classifier cannot be learned from noisy responses with unknown noise rates
Training a classifier with noisy labels typically requires the learner to
specify the distribution of label noise, which is often unknown in practice.
Although there have been some recent attempts to relax that requirement, we
show that the Bayes decision rule is unidentified in most classification
problems with noisy labels. This suggests it is generally not possible to
bypass/relax the requirement. In the special cases in which the Bayes decision
rule is identified, we develop a simple algorithm to learn the Bayes decision
rule, that does not require knowledge of the noise distribution.Comment: Invited to present in ICLR Tiny Paper 202
The Power of Localization for Efficiently Learning Linear Separators with Noise
We introduce a new approach for designing computationally efficient learning
algorithms that are tolerant to noise, and demonstrate its effectiveness by
designing algorithms with improved noise tolerance guarantees for learning
linear separators.
We consider both the malicious noise model and the adversarial label noise
model. For malicious noise, where the adversary can corrupt both the label and
the features, we provide a polynomial-time algorithm for learning linear
separators in under isotropic log-concave distributions that can
tolerate a nearly information-theoretically optimal noise rate of . For the adversarial label noise model, where the
distribution over the feature vectors is unchanged, and the overall probability
of a noisy label is constrained to be at most , we also give a
polynomial-time algorithm for learning linear separators in under
isotropic log-concave distributions that can handle a noise rate of .
We show that, in the active learning model, our algorithms achieve a label
complexity whose dependence on the error parameter is
polylogarithmic. This provides the first polynomial-time active learning
algorithm for learning linear separators in the presence of malicious noise or
adversarial label noise.Comment: Contains improved label complexity analysis communicated to us by
Steve Hannek
A Complete Characterization of Statistical Query Learning with Applications to Evolvability
Statistical query (SQ) learning model of Kearns (1993) is a natural
restriction of the PAC learning model in which a learning algorithm is allowed
to obtain estimates of statistical properties of the examples but cannot see
the examples themselves. We describe a new and simple characterization of the
query complexity of learning in the SQ learning model. Unlike the previously
known bounds on SQ learning our characterization preserves the accuracy and the
efficiency of learning. The preservation of accuracy implies that that our
characterization gives the first characterization of SQ learning in the
agnostic learning framework. The preservation of efficiency is achieved using a
new boosting technique and allows us to derive a new approach to the design of
evolutionary algorithms in Valiant's (2006) model of evolvability. We use this
approach to demonstrate the existence of a large class of monotone evolutionary
learning algorithms based on square loss performance estimation. These results
differ significantly from the few known evolutionary algorithms and give
evidence that evolvability in Valiant's model is a more versatile phenomenon
than there had been previous reason to suspect.Comment: Simplified Lemma 3.8 and it's application
Detecting Label Noise via Leave-One-Out Cross-Validation
We present a simple algorithm for identifying and correcting real-valued
noisy labels from a mixture of clean and corrupted sample points using Gaussian
process regression. A heteroscedastic noise model is employed, in which
additive Gaussian noise terms with independent variances are associated with
each and all of the observed labels. Optimizing the noise model using maximum
likelihood estimation leads to the containment of the GPR model's predictive
error by the posterior standard deviation in leave-one-out cross-validation. A
multiplicative update scheme is proposed for solving the maximum likelihood
estimation problem under non-negative constraints. While we provide proof of
convergence for certain special cases, the multiplicative scheme has
empirically demonstrated monotonic convergence behavior in virtually all our
numerical experiments. We show that the presented method can pinpoint corrupted
sample points and lead to better regression models when trained on synthetic
and real-world scientific data sets