210 research outputs found
On the convergence of maximum variance unfolding
Maximum Variance Unfolding is one of the main methods for (nonlinear)
dimensionality reduction. We study its large sample limit, providing specific
rates of convergence under standard assumptions. We find that it is consistent
when the underlying submanifold is isometric to a convex subset, and we provide
some simple examples where it fails to be consistent
Subspace Least Squares Multidimensional Scaling
Multidimensional Scaling (MDS) is one of the most popular methods for
dimensionality reduction and visualization of high dimensional data. Apart from
these tasks, it also found applications in the field of geometry processing for
the analysis and reconstruction of non-rigid shapes. In this regard, MDS can be
thought of as a \textit{shape from metric} algorithm, consisting of finding a
configuration of points in the Euclidean space that realize, as isometrically
as possible, some given distance structure. In the present work we cast the
least squares variant of MDS (LS-MDS) in the spectral domain. This uncovers a
multiresolution property of distance scaling which speeds up the optimization
by a significant amount, while producing comparable, and sometimes even better,
embeddings.Comment: Scale Space and Variational Methods in Computer Vision: 6th
International Conference, SSVM 2017, Kolding, Denmark, June 4-8, 201
Nonlinear feature extraction through manifold learning in an electronic tongue classification task
A nonlinear feature extraction-based approach using manifold learning algorithms is developed in order to improve the classification accuracy in an electronic tongue sensor array. The developed signal processing methodology is composed of four stages: data unfolding, scaling, feature extraction, and classification. This study aims to compare seven manifold learning algorithms: Isomap, Laplacian Eigenmaps, Locally Linear Embedding (LLE), modified LLE, Hessian LLE, Local Tangent Space Alignment (LTSA), and t-Distributed Stochastic Neighbor Embedding (t-SNE) to find the best classification accuracy in a multifrequency large-amplitude pulse voltammetry electronic tongue. A sensitivity study of the parameters of each manifold learning algorithm is also included. A data set of seven different aqueous matrices is used to validate the proposed data processing methodology. A leave-one-out cross validation was employed in 63 samples. The best accuracy (96.83%) was obtained when the methodology uses Mean-Centered Group Scaling (MCGS) for data normalization, the t-SNE algorithm for feature extraction, and k-nearest neighbors (kNN) as classifier.Peer ReviewedPostprint (published version
Spectral Dimensionality Reduction
In this paper, we study and put under a common framework a number of non-linear dimensionality reduction methods, such as Locally Linear Embedding, Isomap, Laplacian Eigenmaps and kernel PCA, which are based on performing an eigen-decomposition (hence the name 'spectral'). That framework also includes classical methods such as PCA and metric multidimensional scaling (MDS). It also includes the data transformation step used in spectral clustering. We show that in all of these cases the learning algorithm estimates the principal eigenfunctions of an operator that depends on the unknown data density and on a kernel that is not necessarily positive semi-definite. This helps to generalize some of these algorithms so as to predict an embedding for out-of-sample examples without having to retrain the model. It also makes it more transparent what these algorithm are minimizing on the empirical data and gives a corresponding notion of generalization error. Dans cet article, nous étudions et développons un cadre unifié pour un certain nombre de méthodes non linéaires de réduction de dimensionalité, telles que LLE, Isomap, LE (Laplacian Eigenmap) et ACP à noyaux, qui font de la décomposition en valeurs propres (d'où le nom "spectral"). Ce cadre inclut également des méthodes classiques telles que l'ACP et l'échelonnage multidimensionnel métrique (MDS). Il inclut aussi l'étape de transformation de données utilisée dans l'agrégation spectrale. Nous montrons que, dans tous les cas, l'algorithme d'apprentissage estime les fonctions propres principales d'un opérateur qui dépend de la densité inconnue de données et d'un noyau qui n'est pas nécessairement positif semi-défini. Ce cadre aide à généraliser certains modèles pour prédire les coordonnées des exemples hors-échantillons sans avoir à réentraîner le modèle. Il aide également à rendre plus transparent ce que ces algorithmes minimisent sur les données empiriques et donne une notion correspondante d'erreur de généralisation.non-parametric models, non-linear dimensionality reduction, kernel models, modèles non paramétriques, réduction de dimensionalité non linéaire, modèles à noyau
A survey of dimensionality reduction techniques
Experimental life sciences like biology or chemistry have seen in the recent
decades an explosion of the data available from experiments. Laboratory
instruments become more and more complex and report hundreds or thousands
measurements for a single experiment and therefore the statistical methods face
challenging tasks when dealing with such high dimensional data. However, much
of the data is highly redundant and can be efficiently brought down to a much
smaller number of variables without a significant loss of information. The
mathematical procedures making possible this reduction are called
dimensionality reduction techniques; they have widely been developed by fields
like Statistics or Machine Learning, and are currently a hot research topic. In
this review we categorize the plethora of dimension reduction techniques
available and give the mathematical insight behind them
On landmark selection and sampling in high-dimensional data analysis
In recent years, the spectral analysis of appropriately defined kernel
matrices has emerged as a principled way to extract the low-dimensional
structure often prevalent in high-dimensional data. Here we provide an
introduction to spectral methods for linear and nonlinear dimension reduction,
emphasizing ways to overcome the computational limitations currently faced by
practitioners with massive datasets. In particular, a data subsampling or
landmark selection process is often employed to construct a kernel based on
partial information, followed by an approximate spectral analysis termed the
Nystrom extension. We provide a quantitative framework to analyse this
procedure, and use it to demonstrate algorithmic performance bounds on a range
of practical approaches designed to optimize the landmark selection process. We
compare the practical implications of these bounds by way of real-world
examples drawn from the field of computer vision, whereby low-dimensional
manifold structure is shown to emerge from high-dimensional video data streams.Comment: 18 pages, 6 figures, submitted for publicatio
Exploiting Non-Linear Structure in Astronomical Data for Improved Statistical Inference
Many estimation problems in astrophysics are highly complex, with
high-dimensional, non-standard data objects (e.g., images, spectra, entire
distributions, etc.) that are not amenable to formal statistical analysis. To
utilize such data and make accurate inferences, it is crucial to transform the
data into a simpler, reduced form. Spectral kernel methods are non-linear data
transformation methods that efficiently reveal the underlying geometry of
observable data. Here we focus on one particular technique: diffusion maps or
more generally spectral connectivity analysis (SCA). We give examples of
applications in astronomy; e.g., photometric redshift estimation, prototype
selection for estimation of star formation history, and supernova light curve
classification. We outline some computational and statistical challenges that
remain, and we discuss some promising future directions for astronomy and data
mining.Comment: Invited talk at SCMA V, Penn State University, June 2011, PA. To
appear in the Proceedings of "Statistical Challenges in Modern Astronomy V
Spectral Echolocation via the Wave Embedding
Spectral embedding uses eigenfunctions of the discrete Laplacian on a
weighted graph to obtain coordinates for an embedding of an abstract data set
into Euclidean space. We propose a new pre-processing step of first using the
eigenfunctions to simulate a low-frequency wave moving over the data and using
both position as well as change in time of the wave to obtain a refined metric
to which classical methods of dimensionality reduction can then applied. This
is motivated by the behavior of waves, symmetries of the wave equation and the
hunting technique of bats. It is shown to be effective in practice and also
works for other partial differential equations -- the method yields improved
results even for the classical heat equation
Applying Ricci Flow to High Dimensional Manifold Learning
Traditional manifold learning algorithms often bear an assumption that the
local neighborhood of any point on embedded manifold is roughly equal to the
tangent space at that point without considering the curvature. The curvature
indifferent way of manifold processing often makes traditional dimension
reduction poorly neighborhood preserving. To overcome this drawback we propose
a new algorithm called RF-ML to perform an operation on the manifold with help
of Ricci flow before reducing the dimension of manifold.Comment: 18 pages, 4 figur
Geodesic Distance Function Learning via Heat Flow on Vector Fields
Learning a distance function or metric on a given data manifold is of great
importance in machine learning and pattern recognition. Many of the previous
works first embed the manifold to Euclidean space and then learn the distance
function. However, such a scheme might not faithfully preserve the distance
function if the original manifold is not Euclidean. Note that the distance
function on a manifold can always be well-defined. In this paper, we propose to
learn the distance function directly on the manifold without embedding. We
first provide a theoretical characterization of the distance function by its
gradient field. Based on our theoretical analysis, we propose to first learn
the gradient field of the distance function and then learn the distance
function itself. Specifically, we set the gradient field of a local distance
function as an initial vector field. Then we transport it to the whole manifold
via heat flow on vector fields. Finally, the geodesic distance function can be
obtained by requiring its gradient field to be close to the normalized vector
field. Experimental results on both synthetic and real data demonstrate the
effectiveness of our proposed algorithm
- …