35 research outputs found
Learning with Square Loss: Localization through Offset Rademacher Complexity
We consider regression with square loss and general classes of functions
without the boundedness assumption. We introduce a notion of offset Rademacher
complexity that provides a transparent way to study localization both in
expectation and in high probability. For any (possibly non-convex) class, the
excess loss of a two-step estimator is shown to be upper bounded by this offset
complexity through a novel geometric inequality. In the convex case, the
estimator reduces to an empirical risk minimizer. The method recovers the
results of \citep{RakSriTsy15} for the bounded case while also providing
guarantees without the boundedness assumption.Comment: 21 pages, 1 figur
Fast rates for support vector machines using Gaussian kernels
For binary classification we establish learning rates up to the order of
for support vector machines (SVMs) with hinge loss and Gaussian RBF
kernels. These rates are in terms of two assumptions on the considered
distributions: Tsybakov's noise assumption to establish a small estimation
error, and a new geometric noise condition which is used to bound the
approximation error. Unlike previously proposed concepts for bounding the
approximation error, the geometric noise assumption does not employ any
smoothness assumption.Comment: Published at http://dx.doi.org/10.1214/009053606000001226 in the
Annals of Statistics (http://www.imstat.org/aos/) by the Institute of
Mathematical Statistics (http://www.imstat.org
Local Rademacher complexities
We propose new bounds on the error of learning algorithms in terms of a
data-dependent notion of complexity. The estimates we establish give optimal
rates and are based on a local and empirical version of Rademacher averages, in
the sense that the Rademacher averages are computed from the data, on a subset
of functions with small empirical error. We present some applications to
classification and prediction with convex function classes, and with kernel
classes in particular.Comment: Published at http://dx.doi.org/10.1214/009053605000000282 in the
Annals of Statistics (http://www.imstat.org/aos/) by the Institute of
Mathematical Statistics (http://www.imstat.org