Out-of-sample generalizations for supervised manifold learning for classification

Supervised manifold learning methods for data classification map data samples residing in a high-dimensional ambient space to a lower-dimensional domain in a structure-preserving way, while enhancing the separation between different classes in the learned embedding. Most nonlinear supervised manifold learning methods compute the embedding of the manifolds only at the initially available training points, while the generalization of the embedding to novel points, known as the out-of-sample extension problem in manifold learning, becomes especially important in classification applications. In this work, we propose a semi-supervised method for building an interpolation function that provides an out-of-sample extension for general supervised manifold learning algorithms studied in the context of classification. The proposed algorithm computes a radial basis function (RBF) interpolator that minimizes an objective function consisting of the total embedding error of unlabeled test samples, defined as their distance to the embeddings of the manifolds of their own class, as well as a regularization term that controls the smoothness of the interpolation function in a direction-dependent way. The class labels of test data and the interpolation function parameters are estimated jointly with a progressive procedure. Experimental results on face and object images demonstrate the potential of the proposed out-of-sample extension algorithm for the classification of manifold-modeled data sets

Input Design for System Identification via Convex Relaxation

This paper proposes a new framework for the optimization of excitation inputs for system identification. The optimization problem considered is to maximize a reduced Fisher information matrix in any of the classical D-, E-, or A-optimal senses. In contrast to the majority of published work on this topic, we consider the problem in the time domain and subject to constraints on the amplitude of the input signal. This optimization problem is nonconvex. The main result of the paper is a convex relaxation that gives an upper bound accurate to within $2/\pi$ of the true maximum. A randomized algorithm is presented for finding a feasible solution which, in a certain sense is expected to be at least $2/\pi$ as informative as the globally optimal input signal. In the case of a single constraint on input power, the proposed approach recovers the true global optimum exactly. Extensions to situations with both power and amplitude constraints on both inputs and outputs are given. A simple simulation example illustrates the technique.Comment: Preprint submitted for journal publication, extended version of a paper at 2010 IEEE Conference on Decision and Contro