Ensembles as a Sequence of Classifiers

Abstract

An ensemble is a classifier created by combining the predictions of multiple component classifiers. We present a new method for combining classifiers into an ensemble based on a simple estimation of each classifier's competence. The classifiers are grouped into an ordered list where each classifier has a corresponding threshold. To classify an example, the first classifier on the list is consulted and if that classifier's confidence for predicting the example is above the classifier's threshold, then that classifier's prediction is used. Otherwise, the next classifier and its threshold is consulted and so on. If none of the classifiers predicts the example above its confidence threshold then the class of the example is predicted by averaging all of the component classifier predictions. The key to this method is the selection of the confidence threshold for each classifier. We have implemented this method in a system called Sequel which has been applied to the task o..