2 research outputs found

    Learning Multi-Modal Nonlinear Embeddings: Performance Bounds and an Algorithm

    Full text link
    While many approaches exist in the literature to learn low-dimensional representations for data collections in multiple modalities, the generalizability of multi-modal nonlinear embeddings to previously unseen data is a rather overlooked subject. In this work, we first present a theoretical analysis of learning multi-modal nonlinear embeddings in a supervised setting. Our performance bounds indicate that for successful generalization in multi-modal classification and retrieval problems, the regularity of the interpolation functions extending the embedding to the whole data space is as important as the between-class separation and cross-modal alignment criteria. We then propose a multi-modal nonlinear representation learning algorithm that is motivated by these theoretical findings, where the embeddings of the training samples are optimized jointly with the Lipschitz regularity of the interpolators. Experimental comparison to recent multi-modal and single-modal learning algorithms suggests that the proposed method yields promising performance in multi-modal image classification and cross-modal image-text retrieval applications

    GENERALIZABLE SUPERVISED MANIFOLD LEARNING VIA LIPSCHITZ CONTINUOUS INTERPOLATORS

    No full text
    Many supervised dimensionality reduction methods have been proposed in the recent years. Linear manifold learning methods often have limited flexibility in learning effective representations, whereas nonlinear methods mainly focus on the embedding of the training samples and do not consider the performance of the generalization of the embedding to initially unseen test samples. In this paper, we build on recent theoretical results on the generalization performance of supervised manifold learners, which state that in order to achieve good generalization performance, a trade-off needs to be sought between the separation of different classes in the embedding and the possibility of constructing out-of-sample interpolators with good Lipschitz regularity. In the light of these results, we propose a new supervised manifold learning algorithm that computes an embedding of the training samples along with a smooth interpolation function generalizing the embedding to the whole space. Our method is based on a learning objective that explicitly takes into account the generalization performance to novel test samples. Experimental results show that the proposed method achieves high classification accuracy in comparison with state-of-the-art supervised manifold learning algorithms
    corecore