2,450 research outputs found
The Dantzig selector: Statistical estimation when is much larger than
In many important statistical applications, the number of variables or
parameters is much larger than the number of observations . Suppose then
that we have observations , where is a
parameter vector of interest, is a data matrix with possibly far fewer rows
than columns, , and the 's are i.i.d. . Is it
possible to estimate reliably based on the noisy data ? To estimate
, we introduce a new estimator--we call it the Dantzig selector--which
is a solution to the -regularization problem \min_{\tilde{\b
eta}\in\mathbf{R}^p}\|\tilde{\beta}\|_{\ell_1}\quad subject to\quad
\|X^*r\|_{\ell_{\infty}}\leq(1+t^{-1})\sqrt{2\log p}\cdot\sigma, where is
the residual vector and is a positive scalar. We show
that if obeys a uniform uncertainty principle (with unit-normed columns)
and if the true parameter vector is sufficiently sparse (which here
roughly guarantees that the model is identifiable), then with very large
probability, Our results are
nonasymptotic and we give values for the constant . Even though may be
much smaller than , our estimator achieves a loss within a logarithmic
factor of the ideal mean squared error one would achieve with an oracle which
would supply perfect information about which coordinates are nonzero, and which
were above the noise level. In multivariate regression and from a model
selection viewpoint, our result says that it is possible nearly to select the
best subset of variables by solving a very simple convex program, which, in
fact, can easily be recast as a convenient linear program (LP).Comment: This paper discussed in: [arXiv:0803.3124], [arXiv:0803.3126],
[arXiv:0803.3127], [arXiv:0803.3130], [arXiv:0803.3134], [arXiv:0803.3135].
Rejoinder in [arXiv:0803.3136]. Published in at
http://dx.doi.org/10.1214/009053606000001523 the Annals of Statistics
(http://www.imstat.org/aos/) by the Institute of Mathematical Statistics
(http://www.imstat.org
- …