On adaptive wavelet estimation of a class of weighted densities

We investigate the estimation of a weighted density taking the form $g=w(F)f$, where $f$ denotes an unknown density, $F$ the associated distribution function and $w$ is a known (non-negative) weight. Such a class encompasses many examples, including those arising in order statistics or when $g$ is related to the maximum or the minimum of $N$ (random or fixed) independent and identically distributed (\iid) random variables. We here construct a new adaptive non-parametric estimator for $g$ based on a plug-in approach and the wavelets methodology. For a wide class of models, we prove that it attains fast rates of convergence under the $\mathbb{L}_p$ risk with $p\ge 1$ (not only for $p = 2$ corresponding to the mean integrated squared error) over Besov balls. The theoretical findings are illustrated through several simulations

Confidence bands in density estimation

Given a sample from some unknown continuous density $f:\mathbb{R}\to\mathbb{R}$, we construct adaptive confidence bands that are honest for all densities in a "generic" subset of the union of $t$-H\"older balls, $0<t\le r$, where $r$ is a fixed but arbitrary integer. The exceptional ("nongeneric") set of densities for which our results do not hold is shown to be nowhere dense in the relevant H\"older-norm topologies. In the course of the proofs we also obtain limit theorems for maxima of linear wavelet and kernel density estimators, which are of independent interest.Comment: Published in at http://dx.doi.org/10.1214/09-AOS738 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

MDL Denoising Revisited

We refine and extend an earlier MDL denoising criterion for wavelet-based denoising. We start by showing that the denoising problem can be reformulated as a clustering problem, where the goal is to obtain separate clusters for informative and non-informative wavelet coefficients, respectively. This suggests two refinements, adding a code-length for the model index, and extending the model in order to account for subband-dependent coefficient distributions. A third refinement is derivation of soft thresholding inspired by predictive universal coding with weighted mixtures. We propose a practical method incorporating all three refinements, which is shown to achieve good performance and robustness in denoising both artificial and natural signals.Comment: Submitted to IEEE Transactions on Information Theory, June 200

A note on an Adaptive Goodness-of-Fit test with Finite Sample Validity for Random Design Regression Models

Given an i.i.d. sample $\{(X_i,Y_i)\}_{i \in \{1 \ldots n\}}$ from the random design regression model $Y = f(X) + \epsilon$ with $(X,Y) \in [0,1] \times [-M,M]$, in this paper we consider the problem of testing the (simple) null hypothesis $f = f_0$, against the alternative $f \neq f_0$ for a fixed $f_0 \in L^2([0,1],G_X)$, where $G_X(\cdot)$ denotes the marginal distribution of the design variable $X$. The procedure proposed is an adaptation to the regression setting of a multiple testing technique introduced by Fromont and Laurent (2005), and it amounts to consider a suitable collection of unbiased estimators of the $L^2$--distance $d_2(f,f_0) = \int {[f(x) - f_0 (x)]^2 d\,G_X (x)}$, rejecting the null hypothesis when at least one of them is greater than its $(1-u_\alpha)$ quantile, with $u_\alpha$ calibrated to obtain a level--$\alpha$ test. To build these estimators, we will use the warped wavelet basis introduced by Picard and Kerkyacharian (2004). We do not assume that the errors are normally distributed, and we do not assume that $X$ and $\epsilon$ are independent but, mainly for technical reasons, we will assume, as in most part of the current literature in learning theory, that $|f(x) - y|$ is uniformly bounded (almost everywhere). We show that our test is adaptive over a particular collection of approximation spaces linked to the classical Besov spaces

Adaptive circular deconvolution by model selection under unknown error distribution

We consider a circular deconvolution problem, in which the density $f$ of a circular random variable $X$ must be estimated nonparametrically based on an i.i.d. sample from a noisy observation $Y$ of $X$. The additive measurement error is supposed to be independent of $X$. The objective of this work was to construct a fully data-driven estimation procedure when the error density $\varphi$ is unknown. We assume that in addition to the i.i.d. sample from $Y$, we have at our disposal an additional i.i.d. sample drawn independently from the error distribution. We first develop a minimax theory in terms of both sample sizes. We propose an orthogonal series estimator attaining the minimax rates but requiring optimal choice of a dimension parameter depending on certain characteristics of $f$ and $\varphi$, which are not known in practice. The main issue addressed in this work is the adaptive choice of this dimension parameter using a model selection approach. In a first step, we develop a penalized minimum contrast estimator assuming that the error density is known. We show that this partially adaptive estimator can attain the lower risk bound up to a constant in both sample sizes $n$ and $m$. Finally, by randomizing the penalty and the collection of models, we modify the estimator such that it no longer requires any previous knowledge of the error distribution. Even when dispensing with any hypotheses on $\varphi$, this fully data-driven estimator still preserves minimax optimality in almost the same cases as the partially adaptive estimator. We illustrate our results by computing minimal rates under classical smoothness assumptions.Comment: Published in at http://dx.doi.org/10.3150/12-BEJ422 the Bernoulli (http://isi.cbs.nl/bernoulli/) by the International Statistical Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm

Spatially-adaptive sensing in nonparametric regression

While adaptive sensing has provided improved rates of convergence in sparse regression and classification, results in nonparametric regression have so far been restricted to quite specific classes of functions. In this paper, we describe an adaptive-sensing algorithm which is applicable to general nonparametric-regression problems. The algorithm is spatially adaptive, and achieves improved rates of convergence over spatially inhomogeneous functions. Over standard function classes, it likewise retains the spatial adaptivity properties of a uniform design

Exact oracle inequality for a sharp adaptive kernel density estimator

In one-dimensional density estimation on i.i.d. observations we suggest an adaptive cross-validation technique for the selection of a kernel estimator. This estimator is both asymptotic MISE-efficient with respect to the monotone oracle, and sharp minimax-adaptive over the whole scale of Sobolev spaces with smoothness index greater than 1/2. The proof of the central concentration inequality avoids "chaining" and relies on an additive decomposition of the empirical processes involved