113,605 research outputs found
End-to-End Kernel Learning with Supervised Convolutional Kernel Networks
In this paper, we introduce a new image representation based on a multilayer
kernel machine. Unlike traditional kernel methods where data representation is
decoupled from the prediction task, we learn how to shape the kernel with
supervision. We proceed by first proposing improvements of the
recently-introduced convolutional kernel networks (CKNs) in the context of
unsupervised learning; then, we derive backpropagation rules to take advantage
of labeled training data. The resulting model is a new type of convolutional
neural network, where optimizing the filters at each layer is equivalent to
learning a linear subspace in a reproducing kernel Hilbert space (RKHS). We
show that our method achieves reasonably competitive performance for image
classification on some standard "deep learning" datasets such as CIFAR-10 and
SVHN, and also for image super-resolution, demonstrating the applicability of
our approach to a large variety of image-related tasks.Comment: to appear in Advances in Neural Information Processing Systems (NIPS
Harmonic Exponential Families on Manifolds
In a range of fields including the geosciences, molecular biology, robotics
and computer vision, one encounters problems that involve random variables on
manifolds. Currently, there is a lack of flexible probabilistic models on
manifolds that are fast and easy to train. We define an extremely flexible
class of exponential family distributions on manifolds such as the torus,
sphere, and rotation groups, and show that for these distributions the gradient
of the log-likelihood can be computed efficiently using a non-commutative
generalization of the Fast Fourier Transform (FFT). We discuss applications to
Bayesian camera motion estimation (where harmonic exponential families serve as
conjugate priors), and modelling of the spatial distribution of earthquakes on
the surface of the earth. Our experimental results show that harmonic densities
yield a significantly higher likelihood than the best competing method, while
being orders of magnitude faster to train.Comment: fixed typ
Unsupervised machine learning for detection of phase transitions in off-lattice systems I. Foundations
We demonstrate the utility of an unsupervised machine learning tool for the
detection of phase transitions in off-lattice systems. We focus on the
application of principal component analysis (PCA) to detect the freezing
transitions of two-dimensional hard-disk and three-dimensional hard-sphere
systems as well as liquid-gas phase separation in a patchy colloid model. As we
demonstrate, PCA autonomously discovers order-parameter-like quantities that
report on phase transitions, mitigating the need for a priori construction or
identification of a suitable order parameter--thus streamlining the routine
analysis of phase behavior. In a companion paper, we further develop the method
established here to explore the detection of phase transitions in various model
systems controlled by compositional demixing, liquid crystalline ordering, and
non-equilibrium active forces
Sample Complexity Analysis for Learning Overcomplete Latent Variable Models through Tensor Methods
We provide guarantees for learning latent variable models emphasizing on the
overcomplete regime, where the dimensionality of the latent space can exceed
the observed dimensionality. In particular, we consider multiview mixtures,
spherical Gaussian mixtures, ICA, and sparse coding models. We provide tight
concentration bounds for empirical moments through novel covering arguments. We
analyze parameter recovery through a simple tensor power update algorithm. In
the semi-supervised setting, we exploit the label or prior information to get a
rough estimate of the model parameters, and then refine it using the tensor
method on unlabeled samples. We establish that learning is possible when the
number of components scales as , where is the observed
dimension, and is the order of the observed moment employed in the tensor
method. Our concentration bound analysis also leads to minimax sample
complexity for semi-supervised learning of spherical Gaussian mixtures. In the
unsupervised setting, we use a simple initialization algorithm based on SVD of
the tensor slices, and provide guarantees under the stricter condition that
(where constant can be larger than ), where the
tensor method recovers the components under a polynomial running time (and
exponential in ). Our analysis establishes that a wide range of
overcomplete latent variable models can be learned efficiently with low
computational and sample complexity through tensor decomposition methods.Comment: Title change
- …