Reducing the overconfidence of base classifiers when combining their decisions

Abstract

When the sample size is small, the optimistically biased outputs produced by expert classifiers create serious problems for the combiner rule designer. To overcome these problems, we derive analytical expressions for bias reduction for situations when the standard Gaussian density-based quadratic classifiers serve as experts and the decisions of the base experts are aggregated by the behavior-space-knowledge (BKS) method. These reduction terms diminish the experts\u2019 overconfidence and improve the multiple classification system\u2019s generalization ability. The bias-reduction approach is compared with the standard BKS, majority voting and stacked generalization fusion rules on two real-life datasets for which the different base expert aggregates comprise the multiple classification system.Peer reviewed: YesNRC publication: Ye