Tuning Parameter Selection for Penalized Estimation via R2

Abstract

The tuning parameter selection strategy for penalized estimation is crucial to identify a model that is both interpretable and predictive. However, popular strategies (e.g., minimizing average squared prediction error via cross-validation) tend to select models with more predictors than necessary. This paper proposes a simple, yet powerful cross-validation strategy based on maximizing squared correlations between the observed and predicted values, rather than minimizing squared error loss. The strategy can be applied to all penalized least-squares estimators and we show that, under certain conditions, the metric implicitly performs a bias adjustment. Specific attention is given to the lasso estimator, in which our strategy is closely related to the relaxed lasso estimator. We demonstrate our approach on a functional variable selection problem to identify optimal placement of surface electromyogram sensors to control a robotic hand prosthesis