106 research outputs found
Adaptive Minimax Estimation over Sparse lq-Hulls
Given a dictionary of Mn initial estimates of the unknown true regression function, we aim to construct linearly aggregated estimators that target the best performance
among all the linear combinations under a sparse q-norm (0 ≤ q ≤ 1) constraint on the linear coefficients. Besides identifying the optimal rates of aggregation for these `q-aggregation
problems, our multi-directional (or universal) aggregation strategies by model mixing or model
selection achieve the optimal rates simultaneously over the full range of 0 ≤ q ≤ 1 for general Mn and upper bound tn of the q-norm. Both random and fixed designs, with known or
unknown error variance, are handled, and the `q-aggregations examined in this work cover
major types of aggregation problems previously studied in the literature. Consequences on
minimax-rate adaptive regression under `q-constrained true coefficients (0 ≤ q ≤ 1) are also
provided
Sharp Oracle Inequalities for Aggregation of Affine Estimators
We consider the problem of combining a (possibly uncountably infinite) set of
affine estimators in non-parametric regression model with heteroscedastic
Gaussian noise. Focusing on the exponentially weighted aggregate, we prove a
PAC-Bayesian type inequality that leads to sharp oracle inequalities in
discrete but also in continuous settings. The framework is general enough to
cover the combinations of various procedures such as least square regression,
kernel ridge regression, shrinking estimators and many other estimators used in
the literature on statistical inverse problems. As a consequence, we show that
the proposed aggregate provides an adaptive estimator in the exact minimax
sense without neither discretizing the range of tuning parameters nor splitting
the set of observations. We also illustrate numerically the good performance
achieved by the exponentially weighted aggregate
Kernel Estimation and Model Combination in a Bandit Problem with Covariates
Multi-armed bandit problem is an important optimization game that requires an exploration-exploitation tradeoff to achieve optimal total reward. Motivated from industrial applications such as online advertising and clinical research, we consider a setting where the rewards of bandit machines are associated with covariates, and the accurate estimation of the corresponding mean reward functions plays an important role in the performance of allocation rules. Under a flexible problem setup, we establish asymptotic strong consistency and perform a finite-time regret analysis for a sequential randomized allocation strategy based on kernel estimation. In addition, since many nonparametric and parametric methods in supervised learning may be applied to estimating the mean reward functions but guidance on how to choose among them is generally unavailable, we propose a model combining allocation strategy for adaptive performance. Simulations and a real data evaluation are conducted to illustrate the performance of the proposed allocation strategy
Minimax rates of convergence for nonparametric location-scale models
This paper studies minimax rates of convergence for nonparametric
location-scale models, which include mean, quantile and expectile regression
settings. Under Hellinger differentiability on the error distribution and other
mild conditions, we show that the minimax rate of convergence for estimating
the regression function under the squared loss is determined by the
metric entropy of the nonparametric function class. Different error
distributions, including asymmetric Laplace distribution, asymmetric connected
double truncated gamma distribution, connected normal-Laplace distribution,
Cauchy distribution and asymmetric normal distribution are studied as examples.
Applications on low order interaction models and multiple index models are also
given
- …