We point out some pitfalls related to the concept of an oracle property as
used in Fan and Li (2001, 2002, 2004) which are reminiscent of the well-known
pitfalls related to Hodges' estimator. The oracle property is often a
consequence of sparsity of an estimator. We show that any estimator satisfying
a sparsity property has maximal risk that converges to the supremum of the loss
function; in particular, the maximal risk diverges to infinity whenever the
loss function is unbounded. For ease of presentation the result is set in the
framework of a linear regression model, but generalizes far beyond that
setting. In a Monte Carlo study we also assess the extent of the problem in
finite samples for the smoothly clipped absolute deviation (SCAD) estimator
introduced in Fan and Li (2001). We find that this estimator can perform rather
poorly in finite samples and that its worst-case performance relative to
maximum likelihood deteriorates with increasing sample size when the estimator
is tuned to sparsity.Comment: 18 pages, 5 figure