High-Dimensional L2L_2Boosting: Rate of Convergence

Boosting is one of the most significant developments in machine learning. This paper studies the rate of convergence of $L_2$Boosting, which is tailored for regression, in a high-dimensional setting. Moreover, we introduce so-called \textquotedblleft post-Boosting\textquotedblright. This is a post-selection estimator which applies ordinary least squares to the variables selected in the first stage by $L_2$Boosting. Another variant is \textquotedblleft Orthogonal Boosting\textquotedblright\ where after each step an orthogonal projection is conducted. We show that both post-$L_2$Boosting and the orthogonal boosting achieve the same rate of convergence as LASSO in a sparse, high-dimensional setting. We show that the rate of convergence of the classical $L_2$Boosting depends on the design matrix described by a sparse eigenvalue constant. To show the latter results, we derive new approximation results for the pure greedy algorithm, based on analyzing the revisiting behavior of $L_2$Boosting. We also introduce feasible rules for early stopping, which can be easily implemented and used in applied work. Our results also allow a direct comparison between LASSO and boosting which has been missing from the literature. Finally, we present simulation studies and applications to illustrate the relevance of our theoretical results and to provide insights into the practical aspects of boosting. In these simulation studies, post-$L_2$Boosting clearly outperforms LASSO.Comment: 19 pages, 4 tables; AMS 2000 subject classifications: Primary 62J05, 62J07, 41A25; secondary 49M15, 68Q3

Boosting the Anatomy of Volatility

Risk and, thus, the volatility of financial asset prices plays a major role in financial decision making and financial regulation. Therefore, understanding and predicting the volatility of financial instruments, asset classes or financial markets in general is of utmost importance for individual and institutional investors as well as for central bankers and financial regulators. In this paper we investigate new strategies for understanding and predicting financial risk. Specifically, we use componentwise, gradient boosting techniques to identify factors that drive financial-market risk and to assess the specific nature with which these factors affect future volatility. Componentwise boosting is a sequential learning method, which has the advantages that it can handle a large number of predictors and that it-in contrast to other machine-learning techniques-preserves interpretation. Adopting an EGARCH framework and employing a wide range of potential risk drivers, we derive monthly volatility predictions for stock, bond, commodity, and foreign exchange markets. Comparisons with alternative benchmark models show that boosting techniques improve out-of-sample volatility forecasts, especially for medium- and long-run horizons. Another finding is that a number of risk drivers affect volatility in a nonlinear fashion