23,699 research outputs found
EC3: Combining Clustering and Classification for Ensemble Learning
Classification and clustering algorithms have been proved to be successful
individually in different contexts. Both of them have their own advantages and
limitations. For instance, although classification algorithms are more powerful
than clustering methods in predicting class labels of objects, they do not
perform well when there is a lack of sufficient manually labeled reliable data.
On the other hand, although clustering algorithms do not produce label
information for objects, they provide supplementary constraints (e.g., if two
objects are clustered together, it is more likely that the same label is
assigned to both of them) that one can leverage for label prediction of a set
of unknown objects. Therefore, systematic utilization of both these types of
algorithms together can lead to better prediction performance. In this paper,
We propose a novel algorithm, called EC3 that merges classification and
clustering together in order to support both binary and multi-class
classification. EC3 is based on a principled combination of multiple
classification and multiple clustering methods using an optimization function.
We theoretically show the convexity and optimality of the problem and solve it
by block coordinate descent method. We additionally propose iEC3, a variant of
EC3 that handles imbalanced training data. We perform an extensive experimental
analysis by comparing EC3 and iEC3 with 14 baseline methods (7 well-known
standalone classifiers, 5 ensemble classifiers, and 2 existing methods that
merge classification and clustering) on 13 standard benchmark datasets. We show
that our methods outperform other baselines for every single dataset, achieving
at most 10% higher AUC. Moreover our methods are faster (1.21 times faster than
the best baseline), more resilient to noise and class imbalance than the best
baseline method.Comment: 14 pages, 7 figures, 11 table
KFHE-HOMER: A multi-label ensemble classification algorithm exploiting sensor fusion properties of the Kalman filter
Multi-label classification allows a datapoint to be labelled with more than
one class at the same time. In spite of their success in multi-class
classification problems, ensemble methods based on approaches other than
bagging have not been widely explored for multi-label classification problems.
The Kalman Filter-based Heuristic Ensemble (KFHE) is a recent ensemble method
that exploits the sensor fusion properties of the Kalman filter to combine
several classifier models, and that has been shown to be very effective. This
article proposes KFHE-HOMER, an extension of the KFHE ensemble approach to the
multi-label domain. KFHE-HOMER sequentially trains multiple HOMER multi-label
classifiers and aggregates their outputs using the sensor fusion properties of
the Kalman filter. Experiments described in this article show that KFHE-HOMER
performs consistently better than existing multi-label methods including
existing approaches based on ensembles.Comment: The paper is under consideration at Pattern Recognition Letters,
Elsevie
- …