178 research outputs found
Consistent Polyhedral Surrogates for Top- Classification and Variants
Top- classification is a generalization of multiclass classification used
widely in information retrieval, image classification, and other extreme
classification settings. Several hinge-like (piecewise-linear) surrogates have
been proposed for the problem, yet all are either non-convex or inconsistent.
For the proposed hinge-like surrogates that are convex (i.e., polyhedral), we
apply the recent embedding framework of Finocchiaro et al. (2019; 2022) to
determine the prediction problem for which the surrogate is consistent. These
problems can all be interpreted as variants of top- classification, which
may be better aligned with some applications. We leverage this analysis to
derive constraints on the conditional label distributions under which these
proposed surrogates become consistent for top-. It has been further
suggested that every convex hinge-like surrogate must be inconsistent for
top-. Yet, we use the same embedding framework to give the first consistent
polyhedral surrogate for this problem
Recommended from our members
Designing Consistent and Convex Surrogates for General Prediction Tasks
Supervised machine learning algorithms are often predicated on the minimization of loss functions which measure error of a given prediction against a ground truth label. The choice of loss function to minimize corresponds to a summary statistic of the underlying data distribution that is learned in this process. Historically, loss function design has often been ad-hoc, and often results in losses that are not actually statistically consistent with respect to the target prediction task. This work focuses on the design of losses that are simultaneously convex, consistent with respect to a target prediction task, and efficient in the dimension of the prediction space. We provide frameworks to construct such losses in both discrete prediction and continuous estimation settings, as well as tools to lower bound the prediction dimension for certain classes of consistent convex losses. We apply our results throughout to understand prediction tasks such as high-confidence classification, top-k prediction, variance estimation, conditional value at risk, and ratios of expectations
In Defense of Softmax Parametrization for Calibrated and Consistent Learning to Defer
Enabling machine learning classifiers to defer their decision to a downstream
expert when the expert is more accurate will ensure improved safety and
performance. This objective can be achieved with the learning-to-defer
framework which aims to jointly learn how to classify and how to defer to the
expert. In recent studies, it has been theoretically shown that popular
estimators for learning to defer parameterized with softmax provide unbounded
estimates for the likelihood of deferring which makes them uncalibrated.
However, it remains unknown whether this is due to the widely used softmax
parameterization and if we can find a softmax-based estimator that is both
statistically consistent and possesses a valid probability estimator. In this
work, we first show that the cause of the miscalibrated and unbounded estimator
in prior literature is due to the symmetric nature of the surrogate losses used
and not due to softmax. We then propose a novel statistically consistent
asymmetric softmax-based surrogate loss that can produce valid estimates
without the issue of unboundedness. We further analyze the non-asymptotic
properties of our method and empirically validate its performance and
calibration on benchmark datasets.Comment: NeurIPS 202
Supervised classification and mathematical optimization
Data Mining techniques often ask for the resolution of optimization problems. Supervised Classification, and, in particular, Support Vector Machines, can be seen as a paradigmatic instance. In this paper, some links between Mathematical Optimization methods and Supervised Classification are emphasized. It is shown that many different areas of Mathematical Optimization play a central role in off-the-shelf Supervised Classification methods. Moreover, Mathematical Optimization turns out to be extremely
useful to address important issues in Classification, such as identifying relevant variables, improving the interpretability of classifiers or dealing with vagueness/noise in the data.Ministerio de Ciencia e InnovaciónJunta de Andalucí
Supervised Classification and Mathematical Optimization
Data Mining techniques often ask for the resolution of optimization problems. Supervised Classification, and, in particular, Support Vector Machines, can be seen as a paradigmatic instance. In this paper, some links between Mathematical Optimization methods and Supervised Classification are emphasized. It is shown that many different areas of Mathematical Optimization play a central role in off-the-shelf Supervised Classification methods. Moreover, Mathematical Optimization turns out to be extremely useful to address important issues in Classification, such as identifying relevant variables, improving the interpretability of classifiers or dealing with vagueness/noise in the data
The Mathematics of Phylogenomics
The grand challenges in biology today are being shaped by powerful
high-throughput technologies that have revealed the genomes of many organisms,
global expression patterns of genes and detailed information about variation
within populations. We are therefore able to ask, for the first time,
fundamental questions about the evolution of genomes, the structure of genes
and their regulation, and the connections between genotypes and phenotypes of
individuals. The answers to these questions are all predicated on progress in a
variety of computational, statistical, and mathematical fields.
The rapid growth in the characterization of genomes has led to the
advancement of a new discipline called Phylogenomics. This discipline results
from the combination of two major fields in the life sciences: Genomics, i.e.,
the study of the function and structure of genes and genomes; and Molecular
Phylogenetics, i.e., the study of the hierarchical evolutionary relationships
among organisms and their genomes. The objective of this article is to offer
mathematicians a first introduction to this emerging field, and to discuss
specific mathematical problems and developments arising from phylogenomics.Comment: 41 pages, 4 figure
A learning-based approach to multi-agent decision-making
We propose a learning-based methodology to reconstruct private information
held by a population of interacting agents in order to predict an exact outcome
of the underlying multi-agent interaction process, here identified as a
stationary action profile. We envision a scenario where an external observer,
endowed with a learning procedure, is allowed to make queries and observe the
agents' reactions through private action-reaction mappings, whose collective
fixed point corresponds to a stationary profile. By adopting a smart query
process to iteratively collect sensible data and update parametric estimates,
we establish sufficient conditions to assess the asymptotic properties of the
proposed learning-based methodology so that, if convergence happens, it can
only be towards a stationary action profile. This fact yields two main
consequences: i) learning locally-exact surrogates of the action-reaction
mappings allows the external observer to succeed in its prediction task, and
ii) working with assumptions so general that a stationary profile is not even
guaranteed to exist, the established sufficient conditions hence act also as
certificates for the existence of such a desirable profile. Extensive numerical
simulations involving typical competitive multi-agent control and decision
making problems illustrate the practical effectiveness of the proposed
learning-based approach
- …