Skip to main content
Article thumbnail
Location of Repository

REDUCING THE OVERFITTING OF ADABOOST BY CONTROLLING ITS DATA DISTRIBUTION SKEWNESS

By Yijun Sun, Sinisa Todorovic and Jian Li

Abstract

∗sun@dsp.ufl.edu AdaBoost rarely suffers from overfitting problems in low noise data cases. However, recent studies with highly noisy patterns have clearly shown that overfitting can occur. A natural strategy to alleviate the problem is to penalize the data distribution skewness in the learning process to prevent several hardest examples from spoiling decision boundaries. In this paper, we pursue such a penalty scheme in the mathematical programming setting, which allows us to define a suitable classifier soft margin. By using two smooth convex penalty functions, based on Kullback–Leibler divergence (KL) and l2 norm, we derive two new regularized AdaBoost algorithms, referred to as AdaBoostKL and AdaBoostNorm2, respectively. We prove that our algorithms perform stage-wise gradient descent on a cost function, defined in the domain of their associated soft margins. We demonstrate the effectiveness of the proposed algorithms through experiments over a wide variety of data sets. Compared with other regularized AdaBoost algorithms, our methods achieve at least the same or better performance

Topics: Adaptive boosting (AdaBoost, minimax problem, margin, soft margin
Year: 2009
OAI identifier: oai:CiteSeerX.psu:10.1.1.134.4072
Provided by: CiteSeerX
Download PDF:
Sorry, we are unable to provide the full text but you may find it at the following location(s):
  • http://citeseerx.ist.psu.edu/v... (external link)
  • http://vision.ai.uiuc.edu/~sin... (external link)
  • Suggested articles


    To submit an update or takedown request for this paper, please submit an Update/Correction/Removal Request.