13,492 research outputs found
Optimal Estimation and Prediction for Dense Signals in High-Dimensional Linear Models
Estimation and prediction problems for dense signals are often framed in
terms of minimax problems over highly symmetric parameter spaces. In this
paper, we study minimax problems over l2-balls for high-dimensional linear
models with Gaussian predictors. We obtain sharp asymptotics for the minimax
risk that are applicable in any asymptotic setting where the number of
predictors diverges and prove that ridge regression is asymptotically minimax.
Adaptive asymptotic minimax ridge estimators are also identified. Orthogonal
invariance is heavily exploited throughout the paper and, beyond serving as a
technical tool, provides additional insight into the problems considered here.
Most of our results follow from an apparently novel analysis of an equivalent
non-Gaussian sequence model with orthogonally invariant errors. As with many
dense estimation and prediction problems, the minimax risk studied here has
rate d/n, where d is the number of predictors and n is the number of
observations; however, when d is roughly proportional to n the minimax risk is
influenced by the spectral distribution of the predictors and is notably
different from the linear minimax risk for the Gaussian sequence model
(Pinsker, 1980) that often appears in other dense estimation and prediction
problems.Comment: 29 pages, 0 figure
Adaptive circular deconvolution by model selection under unknown error distribution
We consider a circular deconvolution problem, in which the density of a
circular random variable must be estimated nonparametrically based on an
i.i.d. sample from a noisy observation of . The additive measurement
error is supposed to be independent of . The objective of this work was to
construct a fully data-driven estimation procedure when the error density
is unknown. We assume that in addition to the i.i.d. sample from ,
we have at our disposal an additional i.i.d. sample drawn independently from
the error distribution. We first develop a minimax theory in terms of both
sample sizes. We propose an orthogonal series estimator attaining the minimax
rates but requiring optimal choice of a dimension parameter depending on
certain characteristics of and , which are not known in practice.
The main issue addressed in this work is the adaptive choice of this dimension
parameter using a model selection approach. In a first step, we develop a
penalized minimum contrast estimator assuming that the error density is known.
We show that this partially adaptive estimator can attain the lower risk bound
up to a constant in both sample sizes and . Finally, by randomizing the
penalty and the collection of models, we modify the estimator such that it no
longer requires any previous knowledge of the error distribution. Even when
dispensing with any hypotheses on , this fully data-driven estimator
still preserves minimax optimality in almost the same cases as the partially
adaptive estimator. We illustrate our results by computing minimal rates under
classical smoothness assumptions.Comment: Published in at http://dx.doi.org/10.3150/12-BEJ422 the Bernoulli
(http://isi.cbs.nl/bernoulli/) by the International Statistical
Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm
- …