6,615 research outputs found
CCE: An ensemble architecture based on coupled ANN for solving multiclass problems
The resolution of multiclass classification problems has been usually addressed by using a "divide and conquer" strategy that splits the original problem into several binary subproblems. This approach is mandatory when the learning algorithm has been designed to solve binary problems and a multiclass version cannot be devised. Artificial Neural Networks, ANN, are binary learning models whose extension to multiclass problems is rather straightforward by using the standard 1-out-of N codification of the classes. However, the use of a single ANN can be inefficient in terms of accuracy and computational complexity when the data set is large, or the number of classes is high. In this work, we exhaustively describe CCE, a new classifier ensemble based on ANN. Each member of this new ensemble is a couple of multiclass ANN's. Each ANN is trained using different subsets of the dataset ensuring these subsets to be disjoint. This new approach allows to combine the benefits of the divide and conquer methodology, with the use of multiclass ANNs and with the combination of individual classification modules that give a complete answer to the addressed problem. The combination of these elements results in a classifier ensemble in which the diversity of the base classifiers provides high accuracy values. Moreover, the use of couples of ANN proves to be tolerant to labeling noise and computationally efficient. The performance of CCE has been tested on various datasets and the results show the higher performance of this approach with respect to other used classification systems.This research was supported by the Spanish MINECO under projects TRA2016-78886-C3-1-R and RTI2018-096036-B-C22
Convex Optimization for Binary Classifier Aggregation in Multiclass Problems
Multiclass problems are often decomposed into multiple binary problems that
are solved by individual binary classifiers whose results are integrated into a
final answer. Various methods, including all-pairs (APs), one-versus-all (OVA),
and error correcting output code (ECOC), have been studied, to decompose
multiclass problems into binary problems. However, little study has been made
to optimally aggregate binary problems to determine a final answer to the
multiclass problem. In this paper we present a convex optimization method for
an optimal aggregation of binary classifiers to estimate class membership
probabilities in multiclass problems. We model the class membership probability
as a softmax function which takes a conic combination of discrepancies induced
by individual binary classifiers, as an input. With this model, we formulate
the regularized maximum likelihood estimation as a convex optimization problem,
which is solved by the primal-dual interior point method. Connections of our
method to large margin classifiers are presented, showing that the large margin
formulation can be considered as a limiting case of our convex formulation.
Numerical experiments on synthetic and real-world data sets demonstrate that
our method outperforms existing aggregation methods as well as direct methods,
in terms of the classification accuracy and the quality of class membership
probability estimates.Comment: Appeared in Proceedings of the 2014 SIAM International Conference on
Data Mining (SDM 2014
Totally Corrective Multiclass Boosting with Binary Weak Learners
In this work, we propose a new optimization framework for multiclass boosting
learning. In the literature, AdaBoost.MO and AdaBoost.ECC are the two
successful multiclass boosting algorithms, which can use binary weak learners.
We explicitly derive these two algorithms' Lagrange dual problems based on
their regularized loss functions. We show that the Lagrange dual formulations
enable us to design totally-corrective multiclass algorithms by using the
primal-dual optimization technique. Experiments on benchmark data sets suggest
that our multiclass boosting can achieve a comparable generalization capability
with state-of-the-art, but the convergence speed is much faster than stage-wise
gradient descent boosting. In other words, the new totally corrective
algorithms can maximize the margin more aggressively.Comment: 11 page
- …