1,690 research outputs found
Regularized principal manifolds
Many settings of unsupervised learning can be viewed as quantization problems - the minimization
of the expected quantization error subject to some restrictions. This allows the
use of tools such as regularization from the theory of (supervised) risk minimization for
unsupervised learning. This setting turns out to be closely related to principal curves, the
generative topographic map, and robust coding.
We explore this connection in two ways: (1) we propose an algorithm for nding principal
manifolds that can be regularized in a variety of ways; and (2) we derive uniform
convergence bounds and hence bounds on the learning rates of the algorithm. In particular,
we give bounds on the covering numbers which allows us to obtain nearly optimal
learning rates for certain types of regularization operators. Experimental results demonstrate
the feasibility of the approach
Learning Theory and Approximation
Learning theory studies data structures from samples and aims at understanding unknown function relations behind them. This leads to interesting theoretical problems which can be often attacked with methods from Approximation Theory. This workshop - the second one of this type at the MFO - has concentrated on the following recent topics: Learning of manifolds and the geometry of data; sparsity and dimension reduction; error analysis and algorithmic aspects, including kernel based methods for regression and classification; application of multiscale aspects and of refinement algorithms to learning
Regularization in kernel learning
Under mild assumptions on the kernel, we obtain the best known error rates in
a regularized learning scenario taking place in the corresponding reproducing
kernel Hilbert space (RKHS). The main novelty in the analysis is a proof that
one can use a regularization term that grows significantly slower than the
standard quadratic growth in the RKHS norm.Comment: Published in at http://dx.doi.org/10.1214/09-AOS728 the Annals of
Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical
Statistics (http://www.imstat.org
Why and When Can Deep -- but Not Shallow -- Networks Avoid the Curse of Dimensionality: a Review
The paper characterizes classes of functions for which deep learning can be
exponentially better than shallow learning. Deep convolutional networks are a
special case of these conditions, though weight sharing is not the main reason
for their exponential advantage
- …