21,613 research outputs found
Local Dimensionality Reduction for Non-Parametric Regression
Locally-weighted regression is a computationally-efficient technique for
non-linear regression. However, for high-dimensional data, this technique becomes numerically
brittle and computationally too expensive if many local models need to be maintained
simultaneously. Thus, local linear dimensionality reduction combined with locally-weighted
regression seems to be a promising solution. In this context, we review linear dimensionalityreduction
methods, compare their performance on non-parametric locally-linear regression,
and discuss their ability to extend to incremental learning. The considered methods belong to
the following three groups: (1) reducing dimensionality only on the input data, (2) modeling
the joint input-output data distribution, and (3) optimizing the correlation between projection
directions and output data. Group 1 contains principal component regression (PCR);
group 2 contains principal component analysis (PCA) in joint input and output space, factor
analysis, and probabilistic PCA; and group 3 contains reduced rank regression (RRR) and
partial least squares (PLS) regression. Among the tested methods, only group 3 managed
to achieve robust performance even for a non-optimal number of components (factors or
projection directions). In contrast, group 1 and 2 failed for fewer components since these
methods rely on the correct estimate of the true intrinsic dimensionality. In group 3, PLS is
the only method for which a computationally-efficient incremental implementation exists
LWPR: A Scalable Method for Incremental Online Learning in High Dimensions
Locally weighted projection regression (LWPR) is a new algorithm for incremental nonlinear func-
tion approximation in high dimensional spaces with redundant and irrelevant input dimensions. At
its core, it employs nonparametric regression with locally linear models. In order to stay computa-
tionally efficient and numerically robust, each local model performs the regression analysis with a
small number of univariate regressions in selected directions in input space in the spirit of partial
least squares regression. We discuss when and how local learning techniques can successfully work
in high dimensional spaces and compare various techniques for local dimensionality reduction before
finally deriving the LWPR algorithm. The properties of LWPR are that it i) learns rapidly with
second order learning methods based on incremental training, ii) uses statistically sound stochastic
leave-one-out cross validation for learning without the need to memorize training data, iii) adjusts
its weighting kernels based only on local information in order to minimize the danger of negative
interference of incremental learning, iv) has a computational complexity that is linear in the num-
ber of inputs, and v) can deal with a large number of - possibly redundant - inputs, as shown in
various empirical evaluations with up to 50 dimensional data sets. For a probabilistic interpreta-
tion, predictive variance and confidence intervals are derived. To our knowledge, LWPR is the first
truly incremental spatially localized learning method that can successfully and efficiently operate
in very high dimensional spaces
Dimensionality Reduction Mappings
A wealth of powerful dimensionality reduction methods has been established which can be used for data visualization and preprocessing. These are accompanied by formal evaluation schemes, which allow a quantitative evaluation along general principles and which even lead to further visualization schemes based on these objectives. Most methods, however, provide a mapping of a priorly given finite set of points only, requiring additional steps for out-of-sample extensions. We propose a general view on dimensionality reduction based on the concept of cost functions, and, based on this general principle, extend dimensionality reduction to explicit mappings of the data manifold. This offers simple out-of-sample extensions. Further, it opens a way towards a theory of data visualization taking the perspective of its generalization ability to new data points. We demonstrate the approach based on a simple global linear mapping as well as prototype-based local linear mappings.
Manifold Elastic Net: A Unified Framework for Sparse Dimension Reduction
It is difficult to find the optimal sparse solution of a manifold learning
based dimensionality reduction algorithm. The lasso or the elastic net
penalized manifold learning based dimensionality reduction is not directly a
lasso penalized least square problem and thus the least angle regression (LARS)
(Efron et al. \cite{LARS}), one of the most popular algorithms in sparse
learning, cannot be applied. Therefore, most current approaches take indirect
ways or have strict settings, which can be inconvenient for applications. In
this paper, we proposed the manifold elastic net or MEN for short. MEN
incorporates the merits of both the manifold learning based dimensionality
reduction and the sparse learning based dimensionality reduction. By using a
series of equivalent transformations, we show MEN is equivalent to the lasso
penalized least square problem and thus LARS is adopted to obtain the optimal
sparse solution of MEN. In particular, MEN has the following advantages for
subsequent classification: 1) the local geometry of samples is well preserved
for low dimensional data representation, 2) both the margin maximization and
the classification error minimization are considered for sparse projection
calculation, 3) the projection matrix of MEN improves the parsimony in
computation, 4) the elastic net penalty reduces the over-fitting problem, and
5) the projection matrix of MEN can be interpreted psychologically and
physiologically. Experimental evidence on face recognition over various popular
datasets suggests that MEN is superior to top level dimensionality reduction
algorithms.Comment: 33 pages, 12 figure
Multi-View Face Recognition From Single RGBD Models of the Faces
This work takes important steps towards solving the following problem of current interest: Assuming that each individual in a population can be modeled by a single frontal RGBD face image, is it possible to carry out face recognition for such a population using multiple 2D images captured from arbitrary viewpoints? Although the general problem as stated above is extremely challenging, it encompasses subproblems that can be addressed today. The subproblems addressed in this work relate to: (1) Generating a large set of viewpoint dependent face images from a single RGBD frontal image for each individual; (2) using hierarchical approaches based on view-partitioned subspaces to represent the training data; and (3) based on these hierarchical approaches, using a weighted voting algorithm to integrate the evidence collected from multiple images of the same face as recorded from different viewpoints. We evaluate our methods on three datasets: a dataset of 10 people that we created and two publicly available datasets which include a total of 48 people. In addition to providing important insights into the nature of this problem, our results show that we are able to successfully recognize faces with accuracies of 95% or higher, outperforming existing state-of-the-art face recognition approaches based on deep convolutional neural networks
- …