17,985 research outputs found
Aggregation for Gaussian regression
This paper studies statistical aggregation procedures in the regression
setting. A motivating factor is the existence of many different methods of
estimation, leading to possibly competing estimators. We consider here three
different types of aggregation: model selection (MS) aggregation, convex (C)
aggregation and linear (L) aggregation. The objective of (MS) is to select the
optimal single estimator from the list; that of (C) is to select the optimal
convex combination of the given estimators; and that of (L) is to select the
optimal linear combination of the given estimators. We are interested in
evaluating the rates of convergence of the excess risks of the estimators
obtained by these procedures. Our approach is motivated by recently published
minimax results [Nemirovski, A. (2000). Topics in non-parametric statistics.
Lectures on Probability Theory and Statistics (Saint-Flour, 1998). Lecture
Notes in Math. 1738 85--277. Springer, Berlin; Tsybakov, A. B. (2003). Optimal
rates of aggregation. Learning Theory and Kernel Machines. Lecture Notes in
Artificial Intelligence 2777 303--313. Springer, Heidelberg]. There exist
competing aggregation procedures achieving optimal convergence rates for each
of the (MS), (C) and (L) cases separately. Since these procedures are not
directly comparable with each other, we suggest an alternative solution. We
prove that all three optimal rates, as well as those for the newly introduced
(S) aggregation (subset selection), are nearly achieved via a single
``universal'' aggregation procedure. The procedure consists of mixing the
initial estimators with weights obtained by penalized least squares. Two
different penalties are considered: one of them is of the BIC type, the second
one is a data-dependent -type penalty.Comment: Published in at http://dx.doi.org/10.1214/009053606000001587 the
Annals of Statistics (http://www.imstat.org/aos/) by the Institute of
Mathematical Statistics (http://www.imstat.org
Second-Order Inference for the Mean of a Variable Missing at Random
We present a second-order estimator of the mean of a variable subject to
missingness, under the missing at random assumption. The estimator improves
upon existing methods by using an approximate second-order expansion of the
parameter functional, in addition to the first-order expansion employed by
standard doubly robust methods. This results in weaker assumptions about the
convergence rates necessary to establish consistency, local efficiency, and
asymptotic linearity. The general estimation strategy is developed under the
targeted minimum loss-based estimation (TMLE) framework. We present a
simulation comparing the sensitivity of the first and second order estimators
to the convergence rate of the initial estimators of the outcome regression and
missingness score. In our simulation, the second-order TMLE improved the
coverage probability of a confidence interval by up to 85%. In addition, we
present a first-order estimator inspired by a second-order expansion of the
parameter functional. This estimator only requires one-dimensional smoothing,
whereas implementation of the second-order TMLE generally requires kernel
smoothing on the covariate space. The first-order estimator proposed is
expected to have improved finite sample performance compared to existing
first-order estimators. In our simulations, the proposed first-order estimator
improved the coverage probability by up to 90%. We provide an illustration of
our methods using a publicly available dataset to determine the effect of an
anticoagulant on health outcomes of patients undergoing percutaneous coronary
intervention. We provide R code implementing the proposed estimator
- …