1,745 research outputs found
On the Distribution of the Adaptive LASSO Estimator
We study the distribution of the adaptive LASSO estimator (Zou (2006)) in
finite samples as well as in the large-sample limit. The large-sample
distributions are derived both for the case where the adaptive LASSO estimator
is tuned to perform conservative model selection as well as for the case where
the tuning results in consistent model selection. We show that the
finite-sample as well as the large-sample distributions are typically highly
non-normal, regardless of the choice of the tuning parameter. The uniform
convergence rate is also obtained, and is shown to be slower than in
case the estimator is tuned to perform consistent model selection. In
particular, these results question the statistical relevance of the `oracle'
property of the adaptive LASSO estimator established in Zou (2006). Moreover,
we also provide an impossibility result regarding the estimation of the
distribution function of the adaptive LASSO estimator.The theoretical results,
which are obtained for a regression model with orthogonal design, are
complemented by a Monte Carlo study using non-orthogonal regressors.Comment: revised version; minor changes and some material adde
Can one estimate the conditional distribution of post-model-selection estimators?
We consider the problem of estimating the conditional distribution of a
post-model-selection estimator where the conditioning is on the selected model.
The notion of a post-model-selection estimator here refers to the combined
procedure resulting from first selecting a model (e.g., by a model selection
criterion such as AIC or by a hypothesis testing procedure) and then estimating
the parameters in the selected model (e.g., by least-squares or maximum
likelihood), all based on the same data set. We show that it is impossible to
estimate this distribution with reasonable accuracy even asymptotically. In
particular, we show that no estimator for this distribution can be uniformly
consistent (not even locally). This follows as a corollary to (local) minimax
lower bounds on the performance of estimators for this distribution. Similar
impossibility results are also obtained for the conditional distribution of
linear functions (e.g., predictors) of the post-model-selection estimator.Comment: Published at http://dx.doi.org/10.1214/009053606000000821 in the
Annals of Statistics (http://www.imstat.org/aos/) by the Institute of
Mathematical Statistics (http://www.imstat.org
Testing in the Presence of Nuisance Parameters: Some Comments on Tests Post-Model-Selection and Random Critical Values
We point out that the ideas underlying some test procedures recently proposed
for testing post-model-selection (and for some other test problems) in the
econometrics literature have been around for quite some time in the statistics
literature. We also sharpen some of these results in the statistics literature.
Furthermore, we show that some intuitively appealing testing procedures, that
have found their way into the econometrics literature, lead to tests that do
not have desirable size properties, not even asymptotically.Comment: Minor revision. Some typos and errors corrected, some references
adde
Can One Estimate the Conditional Distribution of Post-Model-Selection Estimators?
We consider the problem of estimating the conditional distribution of a post-model-selection estimator where the conditioning is on the selected model. The notion of a post-model-selection estimator here refers to the combined procedure resulting from first selecting a model (e.g., by a model selection criterion like AIC or by a hypothesis testing procedure) and second estimating the parameters in the selected model (e.g., by least-squares or maximum likelihood), all based on the same data set. We show that it is impossible to estimate this distribution with reasonable accuracy even asymptotically. In particular, we show that no estimator for this distribution can be uniformly consistent (not even locally). This follows as a corollary to (local) minimax lower bounds on the performance of estimators for this distribution. Similar impossibility results are also obtained for the conditional distribution of linear functions (e.g., predictors) of the post-model-selection estimator.Inference after model selection, Post-model-selection estimator, Pre-test estimator, Selection of regressors, Akaikeis information criterion AIC, Model uncertainty, Consistency, Uniform consistency, Lower risk bound
Sparse Estimators and the Oracle Property, or the Return of Hodges' Estimator
We point out some pitfalls related to the concept of an oracle property as used in Fan and Li (2001, 2002, 2004) which are reminiscent of the well-known pitfalls related to Hodges’ estimator. The oracle property is often a consequence of sparsity of an estimator. We show that any estimator satisfying a sparsity property has maximal risk that converges to the supremum of the loss function; in particular, the maximal risk diverges to infinity when ever the loss function is unbounded. For ease of presentation the result is set in the framework of a linear regression model, but generalizes far beyond that setting. In a Monte Carlo study we also assess the extent of the problem infinite samples for the smoothly clipped absolute deviation (SCAD) estimator introduced in Fan and Li (2001). We find that this estimator can perform rather poorly infinite samples and that its worst-case performance relative to maximum likelihood deteriorates with increasing sample size when the estimator is tuned to sparsity.Oracle property, Sparsity, Penalized maximum likelihood, Penalized least squares, Hodges’ estimator, SCAD, Lasso, Bridge estimator, Hard-thresholding, Maximal risk, Maximal absolute bias, Non-uniform limits
On the Distribution of Penalized Maximum Likelihood Estimators: The LASSO, SCAD, and Thresholding
We study the distributions of the LASSO, SCAD, and thresholding estimators,
in finite samples and in the large-sample limit. The asymptotic distributions
are derived for both the case where the estimators are tuned to perform
consistent model selection and for the case where the estimators are tuned to
perform conservative model selection. Our findings complement those of Knight
and Fu (2000) and Fan and Li (2001). We show that the distributions are
typically highly nonnormal regardless of how the estimator is tuned, and that
this property persists in large samples. The uniform convergence rate of these
estimators is also obtained, and is shown to be slower than 1/root(n) in case
the estimator is tuned to perform consistent model selection. An impossibility
result regarding estimation of the estimators' distribution function is also
provided
3-manifolds with(out) metrics of nonpositive curvature
In the context of Thurstons geometrisation program we address the question
which compact aspherical 3-manifolds admit Riemannian metrics of nonpositive
curvature. We show that non-geometric Haken manifolds generically, but not
always, admit such metrics. More precisely, we prove that a Haken manifold
with, possibly empty, boundary of zero Euler characteristic admits metrics of
nonpositive curvature if the boundary is non-empty or if at least one atoroidal
component occurs in its canonical topological decomposition. Our arguments are
based on Thurstons Hyperbolisation Theorem. We give examples of closed
graph-manifolds with linear gluing graph and arbitrarily many Seifert
components which do not admit metrics of nonpositive curvature.Comment: 16 page
Sparse Estimators and the Oracle Property, or the Return of Hodges' Estimator
We point out some pitfalls related to the concept of an oracle property as
used in Fan and Li (2001, 2002, 2004) which are reminiscent of the well-known
pitfalls related to Hodges' estimator. The oracle property is often a
consequence of sparsity of an estimator. We show that any estimator satisfying
a sparsity property has maximal risk that converges to the supremum of the loss
function; in particular, the maximal risk diverges to infinity whenever the
loss function is unbounded. For ease of presentation the result is set in the
framework of a linear regression model, but generalizes far beyond that
setting. In a Monte Carlo study we also assess the extent of the problem in
finite samples for the smoothly clipped absolute deviation (SCAD) estimator
introduced in Fan and Li (2001). We find that this estimator can perform rather
poorly in finite samples and that its worst-case performance relative to
maximum likelihood deteriorates with increasing sample size when the estimator
is tuned to sparsity.Comment: 18 pages, 5 figure
- …