7,311 research outputs found
Comment: Boosting Algorithms: Regularization, Prediction and Model Fitting
The authors are doing the readers of Statistical Science a true service with
a well-written and up-to-date overview of boosting that originated with the
seminal algorithms of Freund and Schapire. Equally, we are grateful for
high-level software that will permit a larger readership to experiment with, or
simply apply, boosting-inspired model fitting. The authors show us a world of
methodology that illustrates how a fundamental innovation can penetrate every
nook and cranny of statistical thinking and practice. They introduce the reader
to one particular interpretation of boosting and then give a display of its
potential with extensions from classification (where it all started) to least
squares, exponential family models, survival analysis, to base-learners other
than trees such as smoothing splines, to degrees of freedom and regularization,
and to fascinating recent work in model selection. The uninitiated reader will
find that the authors did a nice job of presenting a certain coherent and
useful interpretation of boosting. The other reader, though, who has watched
the business of boosting for a while, may have quibbles with the authors over
details of the historic record and, more importantly, over their optimism about
the current state of theoretical knowledge. In fact, as much as ``the
statistical view'' has proven fruitful, it has also resulted in some ideas
about why boosting works that may be misconceived, and in some recommendations
that may be misguided. [arXiv:0804.2752]Comment: Published in at http://dx.doi.org/10.1214/07-STS242B the Statistical
Science (http://www.imstat.org/sts/) by the Institute of Mathematical
Statistics (http://www.imstat.org
ada: An R Package for Stochastic Boosting
Boosting is an iterative algorithm that combines simple classification rules with "mediocre" performance in terms of misclassification error rate to produce a highly accurate classification rule. Stochastic gradient boosting provides an enhancement which incorporates a random mechanism at each boosting step showing an improvement in performance and speed in generating the ensemble. ada is an R package that implements three popular variants of boosting, together with a version of stochastic gradient boosting. In addition, useful plots for data analytic purposes are provided along with an extension to the multi-class case. The algorithms are illustrated with synthetic and real data sets.
Studies of Boosted Decision Trees for MiniBooNE Particle Identification
Boosted decision trees are applied to particle identification in the
MiniBooNE experiment operated at Fermi National Accelerator Laboratory
(Fermilab) for neutrino oscillations. Numerous attempts are made to tune the
boosted decision trees, to compare performance of various boosting algorithms,
and to select input variables for optimal performance.Comment: 28 pages, 22 figures, submitted to Nucl. Inst & Meth.
Optimizing 0/1 Loss for Perceptrons by Random Coordinate Descent
The 0/1 loss is an important cost function for perceptrons. Nevertheless it cannot be easily minimized by most existing perceptron learning algorithms. In this paper, we propose a family of random coordinate descent algorithms to directly minimize the 0/1 loss for perceptrons, and prove their convergence. Our algorithms are computationally efficient, and usually achieve the lowest 0/1 loss compared with other algorithms. Such advantages make them favorable for nonseparable real-world problems. Experiments show that our algorithms are especially useful for ensemble learning, and could achieve the lowest test error for many complex data sets when coupled with AdaBoost
- …