Thresholding in Learning Theory

In this paper we investigate the problem of learning an unknown bounded function. We be emphasize special cases where it is possible to provide very simple (in terms of computation) estimates enjoying in addition the property of being universal : their construction does not depend on a priori knowledge on regularity conditions on the unknown object and still they have almost optimal properties for a whole bunch of functions spaces. These estimates are constructed using a thresholding schema, which has proven in the last decade in statistics to have very good properties for recovering signals with inhomogeneous smoothness but has not been extensively developed in Learning Theory. We will basically consider two particular situations. In the first case, we consider the RKHS situation. In this case, we produce a new algorithm and investigate its performances in $L\_2(\hat\rho\_X)$. The exponential rates of convergences are proved to be almost optimal, and the regularity assumptions are expressed in simple terms. The second case considers a more specified situation where the $X\_i$'s are one dimensional and the estimator is a wavelet thresholding estimate. The results are comparable in this setting to those obtained in the RKHS situation as concern the critical value and the exponential rates. The advantage here is that we are able to state the results in the $L\_2(\rho\_X)$ norm and the regularity conditions are expressed in terms of standard H\"older spaces

Heat kernel generated frames in the setting of Dirichlet spaces

Wavelet bases and frames consisting of band limited functions of nearly exponential localization on Rd are a powerful tool in harmonic analysis by making various spaces of functions and distributions more accessible for study and utilization, and providing sparse representation of natural function spaces (e.g. Besov spaces) on Rd. Such frames are also available on the sphere and in more general homogeneous spaces, on the interval and ball. The purpose of this article is to develop band limited well-localized frames in the general setting of Dirichlet spaces with doubling measure and a local scale-invariant Poincar\'e inequality which lead to heat kernels with small time Gaussian bounds and H\"older continuity. As an application of this construction, band limited frames are developed in the context of Lie groups or homogeneous spaces with polynomial volume growth, complete Riemannian manifolds with Ricci curvature bounded from below and satisfying the volume doubling property, and other settings. The new frames are used for decomposition of Besov spaces in this general setting

Decomposition of Triebel-Lizorkin and Besov spaces in the context of Laguerre expansions

A pair of dual frames with almost exponentially localized elements (needlets) are constructed on \RR_+^d based on Laguerre functions. It is shown that the Triebel-Lizorkin and Besov spaces induced by Laguerre expansions can be characterized in terms of respective sequence spaces that involve the needlet coefficients.Comment: 42 page

Asymptotics for spherical needlets

We investigate invariant random fields on the sphere using a new type of spherical wavelets, called needlets. These are compactly supported in frequency and enjoy excellent localization properties in real space, with quasi-exponentially decaying tails. We show that, for random fields on the sphere, the needlet coefficients are asymptotically uncorrelated for any fixed angular distance. This property is used to derive CLT and functional CLT convergence results for polynomial functionals of the needlet coefficients: here the asymptotic theory is considered in the high-frequency sense. Our proposals emerge from strong empirical motivations, especially in connection with the analysis of cosmological data sets.Comment: Published in at http://dx.doi.org/10.1214/08-AOS601 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

Radon needlet thresholding

We provide a new algorithm for the treatment of the noisy inversion of the Radon transform using an appropriate thresholding technique adapted to a well-chosen new localized basis. We establish minimax results and prove their optimality. In particular, we prove that the procedures provided here are able to attain minimax bounds for any $\mathbb {L}_p$ loss. It s important to notice that most of the minimax bounds obtained here are new to our knowledge. It is also important to emphasize the adaptation properties of our procedures with respect to the regularity (sparsity) of the object to recover and to inhomogeneous smoothness. We perform a numerical study that is of importance since we especially have to discuss the cubature problems and propose an averaging procedure that is mostly in the spirit of the cycle spinning performed for periodic signals

High Frequency Asymptotics for Wavelet-Based Tests for Gaussianity and Isotropy on the Torus

We prove a CLT for skewness and kurtosis of the wavelets coefficients of a stationary field on the torus. The results are in the framework of the fixed-domain asymptotics, i.e. we refer to observations of a single field which is sampled at higher and higher frequencies. We consider also studentized statistics for the case of an unknown correlation structure. The results are motivated by the analysis of cosmological data or high-frequency financial data sets, with a particular interest towards testing for Gaussianity and isotropyComment: 33 pages, 3 figure

Subsampling needlet coefficients on the sphere

In a recent paper, we analyzed the properties of a new kind of spherical wavelets (called needlets) for statistical inference procedures on spherical random fields; the investigation was mainly motivated by applications to cosmological data. In the present work, we exploit the asymptotic uncorrelation of random needlet coefficients at fixed angular distances to construct subsampling statistics evaluated on Voronoi cells on the sphere. We illustrate how such statistics can be used for isotropy tests and for bootstrap estimation of nuisance parameters, even when a single realization of the spherical random field is observed. The asymptotic theory is developed in detail in the high resolution sense.Comment: Published in at http://dx.doi.org/10.3150/08-BEJ164 the Bernoulli (http://isi.cbs.nl/bernoulli/) by the International Statistical Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm

Localized spherical deconvolution

We provide a new algorithm for the treatment of the deconvolution problem on the sphere which combines the traditional SVD inversion with an appropriate thresholding technique in a well chosen new basis. We establish upper bounds for the behavior of our procedure for any $\mathbb {L}_p$ loss. It is important to emphasize the adaptation properties of our procedures with respect to the regularity (sparsity) of the object to recover as well as to inhomogeneous smoothness. We also perform a numerical study which proves that the procedure shows very promising properties in practice as well.Comment: Published in at http://dx.doi.org/10.1214/10-AOS858 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org