2 research outputs found
Learning Multi-Modal Nonlinear Embeddings: Performance Bounds and an Algorithm
While many approaches exist in the literature to learn low-dimensional
representations for data collections in multiple modalities, the
generalizability of multi-modal nonlinear embeddings to previously unseen data
is a rather overlooked subject. In this work, we first present a theoretical
analysis of learning multi-modal nonlinear embeddings in a supervised setting.
Our performance bounds indicate that for successful generalization in
multi-modal classification and retrieval problems, the regularity of the
interpolation functions extending the embedding to the whole data space is as
important as the between-class separation and cross-modal alignment criteria.
We then propose a multi-modal nonlinear representation learning algorithm that
is motivated by these theoretical findings, where the embeddings of the
training samples are optimized jointly with the Lipschitz regularity of the
interpolators. Experimental comparison to recent multi-modal and single-modal
learning algorithms suggests that the proposed method yields promising
performance in multi-modal image classification and cross-modal image-text
retrieval applications
GENERALIZABLE SUPERVISED MANIFOLD LEARNING VIA LIPSCHITZ CONTINUOUS INTERPOLATORS
Many supervised dimensionality reduction methods have been proposed in the recent years. Linear manifold learning methods often have limited flexibility in learning effective representations, whereas nonlinear methods mainly focus on the embedding of the training samples and do not consider the performance of the generalization of the embedding to initially unseen test samples. In this paper, we build on recent theoretical results on the generalization performance of supervised manifold learners, which state that in order to achieve good generalization performance, a trade-off needs to be sought between the separation of different classes in the embedding and the possibility of constructing out-of-sample interpolators with good Lipschitz regularity. In the light of these results, we propose a new supervised manifold learning algorithm that computes an embedding of the training samples along with a smooth interpolation function generalizing the embedding to the whole space. Our method is based on a learning objective that explicitly takes into account the generalization performance to novel test samples. Experimental results show that the proposed method achieves high classification accuracy in comparison with state-of-the-art supervised manifold learning algorithms