DIMAL: Deep Isometric Manifold Learning Using Sparse Geodesic Sampling

This paper explores a fully unsupervised deep learning approach for computing distance-preserving maps that generate low-dimensional embeddings for a certain class of manifolds. We use the Siamese configuration to train a neural network to solve the problem of least squares multidimensional scaling for generating maps that approximately preserve geodesic distances. By training with only a few landmarks, we show a significantly improved local and nonlocal generalization of the isometric mapping as compared to analogous non-parametric counterparts. Importantly, the combination of a deep-learning framework with a multidimensional scaling objective enables a numerical analysis of network architectures to aid in understanding their representation power. This provides a geometric perspective to the generalizability of deep learning.Comment: 10 pages, 11 Figure

Manifold Learning Approach for Chaos in the Dripping Faucet

Dripping water from a faucet is a typical example exhibiting rich nonlinear phenomena. For such a system, the time stamps at which water drops separate from the faucet can be directly observed in real experiments, and the time series of intervals \tau_n between drop separations becomes a subject of analysis. Even if the mass m_n of a drop at the onset of the n-th separation, which cannot be observed directly, exhibits perfectly deterministic dynamics, it sometimes fails to obtain important information from time series of \tau_n. This is because the return plot \tau_n-1 vs. \tau_n may become a multi-valued function, i.e., not a deterministic dynamical system. In this paper, we propose a method to construct a nonlinear coordinate which provides a "surrogate" of the internal state m_n from the time series of \tau_n. Here, a key of the proposed approach is to use ISOMAP, which is a well-known method of manifold learning. We first apply it to the time series of $\tau_n$ generated from the numerical simulation of a phenomenological mass-spring model for the dripping faucet system. It is shown that a clear one-dimensional map is obtained by the proposed approach, whose characteristic quantities such as the Lyapunov exponent, the topological entropy, and the time correlation function coincide with the original dripping faucet system. Furthermore, we also analyze data obtained from real dripping faucet experiments which also provides promising results.Comment: 9 pages, 10 figure

Outlier Mining Methods Based on Graph Structure Analysis

Outlier detection in high-dimensional datasets is a fundamental and challenging problem across disciplines that has also practical implications, as removing outliers from the training set improves the performance of machine learning algorithms. While many outlier mining algorithms have been proposed in the literature, they tend to be valid or efficient for specific types of datasets (time series, images, videos, etc.). Here we propose two methods that can be applied to generic datasets, as long as there is a meaningful measure of distance between pairs of elements of the dataset. Both methods start by defining a graph, where the nodes are the elements of the dataset, and the links have associated weights that are the distances between the nodes. Then, the first method assigns an outlier score based on the percolation (i.e., the fragmentation) of the graph. The second method uses the popular IsoMap non-linear dimensionality reduction algorithm, and assigns an outlier score by comparing the geodesic distances with the distances in the reduced space. We test these algorithms on real and synthetic datasets and show that they either outperform, or perform on par with other popular outlier detection methods. A main advantage of the percolation method is that is parameter free and therefore, it does not require any training; on the other hand, the IsoMap method has two integer number parameters, and when they are appropriately selected, the method performs similar to or better than all the other methods tested.Peer ReviewedPostprint (published version

A scale-based approach to finding effective dimensionality in manifold learning

The discovering of low-dimensional manifolds in high-dimensional data is one of the main goals in manifold learning. We propose a new approach to identify the effective dimension (intrinsic dimension) of low-dimensional manifolds. The scale space viewpoint is the key to our approach enabling us to meet the challenge of noisy data. Our approach finds the effective dimensionality of the data over all scale without any prior knowledge. It has better performance compared with other methods especially in the presence of relatively large noise and is computationally efficient.Comment: Published in at http://dx.doi.org/10.1214/07-EJS137 the Electronic Journal of Statistics (http://www.i-journals.org/ejs/) by the Institute of Mathematical Statistics (http://www.imstat.org

Construction of embedded fMRI resting state functional connectivity networks using manifold learning

We construct embedded functional connectivity networks (FCN) from benchmark resting-state functional magnetic resonance imaging (rsfMRI) data acquired from patients with schizophrenia and healthy controls based on linear and nonlinear manifold learning algorithms, namely, Multidimensional Scaling (MDS), Isometric Feature Mapping (ISOMAP) and Diffusion Maps. Furthermore, based on key global graph-theoretical properties of the embedded FCN, we compare their classification potential using machine learning techniques. We also assess the performance of two metrics that are widely used for the construction of FCN from fMRI, namely the Euclidean distance and the lagged cross-correlation metric. We show that the FCN constructed with Diffusion Maps and the lagged cross-correlation metric outperform the other combinations