94 research outputs found
Penalized maximum likelihood and semiparametric second-order efficiency
We consider the problem of estimation of a shift parameter of an unknown
symmetric function in Gaussian white noise. We introduce a notion of
semiparametric second-order efficiency and propose estimators that are
semiparametrically efficient and second-order efficient in our model. These
estimators are of a penalized maximum likelihood type with an appropriately
chosen penalty. We argue that second-order efficiency is crucial in
semiparametric problems since only the second-order terms in asymptotic
expansion for the risk account for the behavior of the ``nonparametric
component'' of a semiparametric procedure, and they are not dramatically
smaller than the first-order terms.Comment: Published at http://dx.doi.org/10.1214/009053605000000895 in the
Annals of Statistics (http://www.imstat.org/aos/) by the Institute of
Mathematical Statistics (http://www.imstat.org
Regularization of statistical inverse problems and the Bakushinskii veto
In the deterministic context Bakushinskii's theorem excludes the existence of
purely data driven convergent regularization for ill-posed problems. We will
prove in the present work that in the statistical setting we can either
construct a counter example or develop an equivalent formulation depending on
the considered class of probability distributions. Hence, Bakushinskii's
theorem does not generalize to the statistical context, although this has often
been assumed in the past. To arrive at this conclusion, we will deduce from the
classic theory new concepts for a general study of statistical inverse problems
and perform a systematic clarification of the key ideas of statistical
regularization.Comment: 20 page
Estimating Mutual Information
We present two classes of improved estimators for mutual information
, from samples of random points distributed according to some joint
probability density . In contrast to conventional estimators based on
binnings, they are based on entropy estimates from -nearest neighbour
distances. This means that they are data efficient (with we resolve
structures down to the smallest possible scales), adaptive (the resolution is
higher where data are more numerous), and have minimal bias. Indeed, the bias
of the underlying entropy estimates is mainly due to non-uniformity of the
density at the smallest resolved scale, giving typically systematic errors
which scale as functions of for points. Numerically, we find that
both families become {\it exact} for independent distributions, i.e. the
estimator vanishes (up to statistical fluctuations) if . This holds for all tested marginal distributions and for all
dimensions of and . In addition, we give estimators for redundancies
between more than 2 random variables. We compare our algorithms in detail with
existing algorithms. Finally, we demonstrate the usefulness of our estimators
for assessing the actual independence of components obtained from independent
component analysis (ICA), for improving ICA, and for estimating the reliability
of blind source separation.Comment: 16 pages, including 18 figure
Robust Matrix Completion
This paper considers the problem of recovery of a low-rank matrix in the
situation when most of its entries are not observed and a fraction of observed
entries are corrupted. The observations are noisy realizations of the sum of a
low rank matrix, which we wish to recover, with a second matrix having a
complementary sparse structure such as element-wise or column-wise sparsity. We
analyze a class of estimators obtained by solving a constrained convex
optimization problem that combines the nuclear norm and a convex relaxation for
a sparse constraint. Our results are obtained for the simultaneous presence of
random and deterministic patterns in the sampling scheme. We provide guarantees
for recovery of low-rank and sparse components from partial and corrupted
observations in the presence of noise and show that the obtained rates of
convergence are minimax optimal
Affine-Invariant Dictionaries
[12] P. Jacquet, “Random infinite trees and supercritical behavior of collision resolution algorithms, ” IEEE Trans. Inform. Theory, vol. 39, pp
Submitted to the Bernoulli Mirror averaging with sparsity priors
We consider the problem of aggregating the elements of a possibly infinite dictionary for building a decisionprocedurethataimsatminimizingagivencriterion.Alongwiththedictionary,anindependent identically distributed training sample is available, on which the performance of a given procedure can betested.Inafairlygeneralset-up,weestablishanoracleinequalityfortheMirrorAveragingaggregate with any prior distribution. By choosing an appropriate prior, we apply this oracle inequality in the context of prediction under sparsity assumption for the problems of regression with random design, density estimation and binary classification
- …