Ordinary least squares (OLS) is the default method for fitting linear models,
but is not applicable for problems with dimensionality larger than the sample
size. For these problems, we advocate the use of a generalized version of OLS
motivated by ridge regression, and propose two novel three-step algorithms
involving least squares fitting and hard thresholding. The algorithms are
methodologically simple to understand intuitively, computationally easy to
implement efficiently, and theoretically appealing for choosing models
consistently. Numerical exercises comparing our methods with penalization-based
approaches in simulations and data analyses illustrate the great potential of
the proposed algorithms.Comment: Added results for non-sparse models; Added results for elliptical
distribution; Added simulations for adaptive lass
