224 research outputs found
A chain rule for the expected suprema of Gaussian processes
The expected supremum of a Gaussian process indexed by the image of an index
set under a function class is bounded in terms of separate properties of the
index set and the function class. The bound is relevant to the estimation of
nonlinear transformations or the analysis of learning algorithms whenever
hypotheses are chosen from composite classes, as is the case for multi-layer
models
A Chain Rule for the Expected Suprema of Bernoulli Processes
We obtain an upper bound on the expected supremum of a Bernoulli process
indexed by the image of an index set under a uniformly Lipschitz function class
in terms of properties of the index set and the function class, extending an
earlier result of Maurer for Gaussian processes. The proof makes essential use
of recent results of Bednorz and Latala on the boundedness of Bernoulli
processes.Comment: 14 page
Optimal Rates for Random Fourier Features
Kernel methods represent one of the most powerful tools in machine learning
to tackle problems expressed in terms of function values and derivatives due to
their capability to represent and model complex relations. While these methods
show good versatility, they are computationally intensive and have poor
scalability to large data as they require operations on Gram matrices. In order
to mitigate this serious computational limitation, recently randomized
constructions have been proposed in the literature, which allow the application
of fast linear algorithms. Random Fourier features (RFF) are among the most
popular and widely applied constructions: they provide an easily computable,
low-dimensional feature representation for shift-invariant kernels. Despite the
popularity of RFFs, very little is understood theoretically about their
approximation quality. In this paper, we provide a detailed finite-sample
theoretical analysis about the approximation quality of RFFs by (i)
establishing optimal (in terms of the RFF dimension, and growing set size)
performance guarantees in uniform norm, and (ii) presenting guarantees in
() norms. We also propose an RFF approximation to derivatives of
a kernel with a theoretical study on its approximation quality.Comment: To appear at NIPS-201
Quantitative version of the Kipnis-Varadhan theorem and Monte Carlo approximation of homogenized coefficients
This article is devoted to the analysis of a Monte Carlo method to
approximate effective coefficients in stochastic homogenization of discrete
elliptic equations. We consider the case of independent and identically
distributed coefficients, and adopt the point of view of the random walk in a
random environment. Given some final time t>0, a natural approximation of the
homogenized coefficients is given by the empirical average of the final squared
positions re-scaled by t of n independent random walks in n independent
environments. Relying on a quantitative version of the Kipnis-Varadhan theorem
combined with estimates of spectral exponents obtained by an original
combination of PDE arguments and spectral theory, we first give a sharp
estimate of the error between the homogenized coefficients and the expectation
of the re-scaled final position of the random walk in terms of t. We then
complete the error analysis by quantifying the fluctuations of the empirical
average in terms of n and t, and prove a large-deviation estimate, as well as a
central limit theorem. Our estimates are optimal, up to a logarithmic
correction in dimension 2.Comment: Published in at http://dx.doi.org/10.1214/12-AAP880 the Annals of
Applied Probability (http://www.imstat.org/aap/) by the Institute of
Mathematical Statistics (http://www.imstat.org
- …