277 research outputs found
Class Proportion Estimation with Application to Multiclass Anomaly Rejection
This work addresses two classification problems that fall under the heading
of domain adaptation, wherein the distributions of training and testing
examples differ. The first problem studied is that of class proportion
estimation, which is the problem of estimating the class proportions in an
unlabeled testing data set given labeled examples of each class. Compared to
previous work on this problem, our approach has the novel feature that it does
not require labeled training data from one of the classes. This property allows
us to address the second domain adaptation problem, namely, multiclass anomaly
rejection. Here, the goal is to design a classifier that has the option of
assigning a "reject" label, indicating that the instance did not arise from a
class present in the training data. We establish consistent learning strategies
for both of these domain adaptation problems, which to our knowledge are the
first of their kind. We also implement the class proportion estimation
technique and demonstrate its performance on several benchmark data sets.Comment: Accepted to AISTATS 2014. 15 pages. 2 figure
Decontamination of Mutual Contamination Models
Many machine learning problems can be characterized by mutual contamination
models. In these problems, one observes several random samples from different
convex combinations of a set of unknown base distributions and the goal is to
infer these base distributions. This paper considers the general setting where
the base distributions are defined on arbitrary probability spaces. We examine
three popular machine learning problems that arise in this general setting:
multiclass classification with label noise, demixing of mixed membership
models, and classification with partial labels. In each case, we give
sufficient conditions for identifiability and present algorithms for the
infinite and finite sample settings, with associated performance guarantees.Comment: Published in JMLR. Subsumes arXiv:1602.0623
Machine Learning with a Reject Option: A survey
Machine learning models always make a prediction, even when it is likely to
be inaccurate. This behavior should be avoided in many decision support
applications, where mistakes can have severe consequences. Albeit already
studied in 1970, machine learning with rejection recently gained interest. This
machine learning subfield enables machine learning models to abstain from
making a prediction when likely to make a mistake.
This survey aims to provide an overview on machine learning with rejection.
We introduce the conditions leading to two types of rejection, ambiguity and
novelty rejection, which we carefully formalize. Moreover, we review and
categorize strategies to evaluate a model's predictive and rejective quality.
Additionally, we define the existing architectures for models with rejection
and describe the standard techniques for learning such models. Finally, we
provide examples of relevant application domains and show how machine learning
with rejection relates to other machine learning research areas
Mixture Proportion Estimation Beyond Irreducibility
The task of mixture proportion estimation (MPE) is to estimate the weight of
a component distribution in a mixture, given observations from both the
component and mixture. Previous work on MPE adopts the irreducibility
assumption, which ensures identifiablity of the mixture proportion. In this
paper, we propose a more general sufficient condition that accommodates several
settings of interest where irreducibility does not hold. We further present a
resampling-based meta-algorithm that takes any existing MPE algorithm designed
to work under irreducibility and adapts it to work under our more general
condition. Our approach empirically exhibits improved estimation performance
relative to baseline methods and to a recently proposed regrouping-based
algorithm
Open Set Domain Adaptation using Optimal Transport
We present a 2-step optimal transport approach that performs a mapping from a
source distribution to a target distribution. Here, the target has the
particularity to present new classes not present in the source domain. The
first step of the approach aims at rejecting the samples issued from these new
classes using an optimal transport plan. The second step solves the target
(class ratio) shift still as an optimal transport problem. We develop a dual
approach to solve the optimization problem involved at each step and we prove
that our results outperform recent state-of-the-art performances. We further
apply the approach to the setting where the source and target distributions
present both a label-shift and an increasing covariate (features) shift to show
its robustness.Comment: Accepted at ECML-PKDD 2020, Acknowledgements adde
- …