2,395 research outputs found
Selective machine learning of doubly robust functionals
While model selection is a well-studied topic in parametric and nonparametric
regression or density estimation, selection of possibly high-dimensional
nuisance parameters in semiparametric problems is far less developed. In this
paper, we propose a selective machine learning framework for making inferences
about a finite-dimensional functional defined on a semiparametric model, when
the latter admits a doubly robust estimating function and several candidate
machine learning algorithms are available for estimating the nuisance
parameters. We introduce two new selection criteria for bias reduction in
estimating the functional of interest, each based on a novel definition of
pseudo-risk for the functional that embodies the double robustness property and
thus is used to select the pair of learners that is nearest to fulfilling this
property. We establish an oracle property for a multi-fold cross-validation
version of the new selection criteria which states that our empirical criteria
perform nearly as well as an oracle with a priori knowledge of the pseudo-risk
for each pair of candidate learners. We also describe a smooth approximation to
the selection criteria which allows for valid post-selection inference.
Finally, we apply the approach to model selection of a semiparametric estimator
of average treatment effect given an ensemble of candidate machine learners to
account for confounding in an observational study
Adaptive estimation of High-Dimensional Signal-to-Noise Ratios
We consider the equivalent problems of estimating the residual variance, the
proportion of explained variance and the signal strength in a
high-dimensional linear regression model with Gaussian random design. Our aim
is to understand the impact of not knowing the sparsity of the regression
parameter and not knowing the distribution of the design on minimax estimation
rates of . Depending on the sparsity of the regression parameter,
optimal estimators of either rely on estimating the regression parameter
or are based on U-type statistics, and have minimax rates depending on . In
the important situation where is unknown, we build an adaptive procedure
whose convergence rate simultaneously achieves the minimax risk over all up
to a logarithmic loss which we prove to be non avoidable. Finally, the
knowledge of the design distribution is shown to play a critical role. When the
distribution of the design is unknown, consistent estimation of explained
variance is indeed possible in much narrower regimes than for known design
distribution
Optimal estimation of high-order missing masses, and the rare-type match problem
Consider a random sample from an unknown discrete
distribution on a countable alphabet
, and let be the empirical frequencies of
distinct symbols 's in the sample. We consider the problem of estimating
the -order missing mass, which is a discrete functional of defined as
This is
generalization of the missing mass whose estimation is a classical problem in
statistics, being the subject of numerous studies both in theory and methods.
First, we introduce a nonparametric estimator of
and a corresponding non-asymptotic confidence interval through concentration
properties of . Then, we investigate minimax
estimation of , which is the main contribution of
our work. We show that minimax estimation is not feasible over the class of all
discrete distributions on , and not even for distributions with
regularly varying tails, which only guarantee that our estimator is consistent
for . This leads to introduce the stronger
assumption of second-order regular variation for the tail behaviour of ,
which is proved to be sufficient for minimax estimation of
, making the proposed estimator an optimal minimax
estimator of . Our interest in the -order
missing mass arises from forensic statistics, where the estimation of the
-order missing mass appears in connection to the estimation of the
likelihood ratio
,
known as the "fundamental problem of forensic mathematics". We present
theoretical guarantees to nonparametric estimation of
- …