3,275 research outputs found
Sparsity oracle inequalities for the Lasso
This paper studies oracle properties of -penalized least squares in
nonparametric regression setting with random design. We show that the penalized
least squares estimator satisfies sparsity oracle inequalities, i.e., bounds in
terms of the number of non-zero components of the oracle vector. The results
are valid even when the dimension of the model is (much) larger than the sample
size and the regression matrix is not positive definite. They can be applied to
high-dimensional linear regression, to nonparametric adaptive regression
estimation and to the problem of aggregation of arbitrary estimators.Comment: Published at http://dx.doi.org/10.1214/07-EJS008 in the Electronic
Journal of Statistics (http://www.i-journals.org/ejs/) by the Institute of
Mathematical Statistics (http://www.imstat.org
Theoretical Properties of the Overlapping Groups Lasso
We present two sets of theoretical results on the grouped lasso with overlap
of Jacob, Obozinski and Vert (2009) in the linear regression setting. This
method allows for joint selection of predictors in sparse regression, allowing
for complex structured sparsity over the predictors encoded as a set of groups.
This flexible framework suggests that arbitrarily complex structures can be
encoded with an intricate set of groups. Our results show that this strategy
results in unexpected theoretical consequences for the procedure. In
particular, we give two sets of results: (1) finite sample bounds on prediction
and estimation, and (2) asymptotic distribution and selection. Both sets of
results give insight into the consequences of choosing an increasingly complex
set of groups for the procedure, as well as what happens when the set of groups
cannot recover the true sparsity pattern. Additionally, these results
demonstrate the differences and similarities between the the grouped lasso
procedure with and without overlapping groups. Our analysis shows the set of
groups must be chosen with caution - an overly complex set of groups will
damage the analysis.Comment: 20 pages, submitted to Annals of Statistic
- β¦