5 research outputs found
A Symmetric Loss Perspective of Reliable Machine Learning
When minimizing the empirical risk in binary classification, it is a common
practice to replace the zero-one loss with a surrogate loss to make the
learning objective feasible to optimize. Examples of well-known surrogate
losses for binary classification include the logistic loss, hinge loss, and
sigmoid loss. It is known that the choice of a surrogate loss can highly
influence the performance of the trained classifier and therefore it should be
carefully chosen. Recently, surrogate losses that satisfy a certain symmetric
condition (aka., symmetric losses) have demonstrated their usefulness in
learning from corrupted labels. In this article, we provide an overview of
symmetric losses and their applications. First, we review how a symmetric loss
can yield robust classification from corrupted labels in balanced error rate
(BER) minimization and area under the receiver operating characteristic curve
(AUC) maximization. Then, we demonstrate how the robust AUC maximization method
can benefit natural language processing in the problem where we want to learn
only from relevant keywords and unlabeled documents. Finally, we conclude this
article by discussing future directions, including potential applications of
symmetric losses for reliable machine learning and the design of non-symmetric
losses that can benefit from the symmetric condition.Comment: Preprint of an Invited Review Articl
Active Learning for Regression with Aggregated Outputs
Due to the privacy protection or the difficulty of data collection, we cannot
observe individual outputs for each instance, but we can observe aggregated
outputs that are summed over multiple instances in a set in some real-world
applications. To reduce the labeling cost for training regression models for
such aggregated data, we propose an active learning method that sequentially
selects sets to be labeled to improve the predictive performance with fewer
labeled sets. For the selection measurement, the proposed method uses the
mutual information, which quantifies the reduction of the uncertainty of the
model parameters by observing the aggregated output. With Bayesian linear basis
functions for modeling outputs given an input, which include approximated
Gaussian processes and neural networks, we can efficiently calculate the mutual
information in a closed form. With the experiments using various datasets, we
demonstrate that the proposed method achieves better predictive performance
with fewer labeled sets than existing methods
Imprecise Label Learning: A Unified Framework for Learning with Various Imprecise Label Configurations
Learning with reduced labeling standards, such as noisy label, partial label,
and multiple label candidates, which we generically refer to as
\textit{imprecise} labels, is a commonplace challenge in machine learning
tasks. Previous methods tend to propose specific designs for every emerging
imprecise label configuration, which is usually unsustainable when multiple
configurations of imprecision coexist. In this paper, we introduce imprecise
label learning (ILL), a framework for the unification of learning with various
imprecise label configurations. ILL leverages expectation-maximization (EM) for
modeling the imprecise label information, treating the precise labels as latent
variables.Instead of approximating the correct labels for training, it
considers the entire distribution of all possible labeling entailed by the
imprecise information. We demonstrate that ILL can seamlessly adapt to partial
label learning, semi-supervised learning, noisy label learning, and, more
importantly, a mixture of these settings. Notably, ILL surpasses the existing
specified techniques for handling imprecise labels, marking the first unified
framework with robust and effective performance across various challenging
settings. We hope our work will inspire further research on this topic,
unleashing the full potential of ILL in wider scenarios where precise labels
are expensive and complicated to obtain.Comment: 29 pages, 3 figures, 16 tables, preprin