3,470 research outputs found
Intrinsic Universal Measurements of Non-linear Embeddings
A basic problem in machine learning is to find a mapping from a low
dimensional latent space to a high dimensional observation space. Equipped with
the representation power of non-linearity, a learner can easily find a mapping
which perfectly fits all the observations. However such a mapping is often not
considered as good as it is not simple enough and over-fits. How to define
simplicity? This paper tries to make such a formal definition of the amount of
information imposed by a non-linear mapping. This definition is based on
information geometry and is independent of observations, nor specific
parametrizations. We prove these basic properties and discuss relationships
with parametric and non-parametric embeddings.Comment: work in progres
Nonlinear Supervised Dimensionality Reduction via Smooth Regular Embeddings
The recovery of the intrinsic geometric structures of data collections is an
important problem in data analysis. Supervised extensions of several manifold
learning approaches have been proposed in the recent years. Meanwhile, existing
methods primarily focus on the embedding of the training data, and the
generalization of the embedding to initially unseen test data is rather
ignored. In this work, we build on recent theoretical results on the
generalization performance of supervised manifold learning algorithms.
Motivated by these performance bounds, we propose a supervised manifold
learning method that computes a nonlinear embedding while constructing a smooth
and regular interpolation function that extends the embedding to the whole data
space in order to achieve satisfactory generalization. The embedding and the
interpolator are jointly learnt such that the Lipschitz regularity of the
interpolator is imposed while ensuring the separation between different
classes. Experimental results on several image data sets show that the proposed
method outperforms traditional classifiers and the supervised dimensionality
reduction algorithms in comparison in terms of classification accuracy in most
settings
- …