246 research outputs found
EC3: Combining Clustering and Classification for Ensemble Learning
Classification and clustering algorithms have been proved to be successful
individually in different contexts. Both of them have their own advantages and
limitations. For instance, although classification algorithms are more powerful
than clustering methods in predicting class labels of objects, they do not
perform well when there is a lack of sufficient manually labeled reliable data.
On the other hand, although clustering algorithms do not produce label
information for objects, they provide supplementary constraints (e.g., if two
objects are clustered together, it is more likely that the same label is
assigned to both of them) that one can leverage for label prediction of a set
of unknown objects. Therefore, systematic utilization of both these types of
algorithms together can lead to better prediction performance. In this paper,
We propose a novel algorithm, called EC3 that merges classification and
clustering together in order to support both binary and multi-class
classification. EC3 is based on a principled combination of multiple
classification and multiple clustering methods using an optimization function.
We theoretically show the convexity and optimality of the problem and solve it
by block coordinate descent method. We additionally propose iEC3, a variant of
EC3 that handles imbalanced training data. We perform an extensive experimental
analysis by comparing EC3 and iEC3 with 14 baseline methods (7 well-known
standalone classifiers, 5 ensemble classifiers, and 2 existing methods that
merge classification and clustering) on 13 standard benchmark datasets. We show
that our methods outperform other baselines for every single dataset, achieving
at most 10% higher AUC. Moreover our methods are faster (1.21 times faster than
the best baseline), more resilient to noise and class imbalance than the best
baseline method.Comment: 14 pages, 7 figures, 11 table
Optimum Bayesian thresholds for rebalanced classification problems using class-switching ensembles
Asymmetric label switching is an effective and principled method for creating a diverse ensemble of learners for imbalanced classification problems. This technique can be combined with other rebalancing mechanisms, such as those based on cost policies or class proportion modifications. In this study, and under the Bayesian theory framework, we specify the optimal decision thresholds for the combination of these mechanisms. In addition, we propose using a gating network to aggregate the learners contributions as an additional mechanism to improve the overall performance of the system.We thank the anonymous reviewers for their valuable suggestions and comments. This work is partially funded by Project PID2021-125652OB-I00 from the Ministerio de Ciencia e Innovación of Spain. Funding for APC: Universidad Carlos III de Madrid (Read & Publish Agreement CRUE-CSIC 2022). In memoriam: Prof. AnÃbal R. Figueiras-Vidal (1950-2022)
Revisiting Precision and Recall Definition for Generative Model Evaluation
In this article we revisit the definition of Precision-Recall (PR) curves for
generative models proposed by Sajjadi et al. (arXiv:1806.00035). Rather than
providing a scalar for generative quality, PR curves distinguish mode-collapse
(poor recall) and bad quality (poor precision). We first generalize their
formulation to arbitrary measures, hence removing any restriction to finite
support. We also expose a bridge between PR curves and type I and type II error
rates of likelihood ratio classifiers on the task of discriminating between
samples of the two distributions. Building upon this new perspective, we
propose a novel algorithm to approximate precision-recall curves, that shares
some interesting methodological properties with the hypothesis testing
technique from Lopez-Paz et al (arXiv:1610.06545). We demonstrate the interest
of the proposed formulation over the original approach on controlled
multi-modal datasets.Comment: ICML 201
GBM-based Bregman Proximal Algorithms for Constrained Learning
As the complexity of learning tasks surges, modern machine learning
encounters a new constrained learning paradigm characterized by more intricate
and data-driven function constraints. Prominent applications include
Neyman-Pearson classification (NPC) and fairness classification, which entail
specific risk constraints that render standard projection-based training
algorithms unsuitable. Gradient boosting machines (GBMs) are among the most
popular algorithms for supervised learning; however, they are generally limited
to unconstrained settings. In this paper, we adapt the GBM for constrained
learning tasks within the framework of Bregman proximal algorithms. We
introduce a new Bregman primal-dual method with a global optimality guarantee
when the learning objective and constraint functions are convex. In cases of
nonconvex functions, we demonstrate how our algorithm remains effective under a
Bregman proximal point framework. Distinct from existing constrained learning
algorithms, ours possess a unique advantage in their ability to seamlessly
integrate with publicly available GBM implementations such as XGBoost (Chen and
Guestrin, 2016) and LightGBM (Ke et al., 2017), exclusively relying on their
public interfaces. We provide substantial experimental evidence to showcase the
effectiveness of the Bregman algorithm framework. While our primary focus is on
NPC and fairness ML, our framework holds significant potential for a broader
range of constrained learning applications. The source code is currently freely
available at
https://github.com/zhenweilin/ConstrainedGBM}{https://github.com/zhenweilin/ConstrainedGBM
- …