29,792 research outputs found
How to Solve Classification and Regression Problems on High-Dimensional Data with a Supervised Extension of Slow Feature Analysis
Supervised learning from high-dimensional data, e.g., multimedia data, is a challenging task. We propose an extension of slow feature analysis (SFA) for supervised dimensionality reduction called graph-based SFA (GSFA). The algorithm extracts a label-predictive low-dimensional set of features that can be post-processed by typical supervised algorithms to generate the final label or class estimation. GSFA is trained with a so-called training graph, in which the vertices are the samples and the edges represent similarities of the corresponding labels. A new weighted SFA optimization problem is introduced, generalizing the notion of slowness from sequences of samples to such training graphs. We show that GSFA computes an optimal solution to this problem in the considered function space, and propose several types of training graphs. For classification, the most straightforward graph yields features equivalent to those of (nonlinear) Fisher discriminant analysis. Emphasis is on regression, where four different graphs were evaluated experimentally with a subproblem of face detection on photographs. The method proposed is promising particularly when linear models are insufficient, as well as when feature selection is difficult
Chebyshev Polynomial Approximation for Distributed Signal Processing
Unions of graph Fourier multipliers are an important class of linear
operators for processing signals defined on graphs. We present a novel method
to efficiently distribute the application of these operators to the
high-dimensional signals collected by sensor networks. The proposed method
features approximations of the graph Fourier multipliers by shifted Chebyshev
polynomials, whose recurrence relations make them readily amenable to
distributed computation. We demonstrate how the proposed method can be used in
a distributed denoising task, and show that the communication requirements of
the method scale gracefully with the size of the network.Comment: 8 pages, 5 figures, to appear in the Proceedings of the IEEE
International Conference on Distributed Computing in Sensor Systems (DCOSS),
June, 2011, Barcelona, Spai
Supervised estimation of Granger-based causality between time series
Brain effective connectivity aims to detect causal interactions between distinct brain units and it is typically studied through the analysis of direct measurements of the neural activity, e.g., magneto/electroencephalography (M/EEG) signals. The literature on methods for causal inference is vast. It includes model-based methods in which a generative model of the data is assumed and model-free methods that directly infer causality from the probability distribution of the underlying stochastic process. Here, we firstly focus on the model-based methods developed from the Granger criterion of causality, which assumes the autoregressive model of the data. Secondly, we introduce a new perspective, that looks at the problem in a way that is typical of the machine learning literature. Then, we formulate the problem of causality detection as a supervised learning task, by proposing a classification-based approach. A classifier is trained to identify causal interactions between time series for the chosen model and by means of a proposed feature space. In this paper, we are interested in comparing this classification-based approach with the standard Geweke measure of causality in the time domain, through simulation study. Thus, we customized our approach to the case of a MAR model and designed a feature space which contains causality measures based on the idea of precedence and predictability in time. Two variations of the supervised method are proposed and compared to a standard Granger causal analysis method. The results of the simulations show that the supervised method outperforms the standard approach, in particular it is more robust to noise. As evidence of the efficacy of the proposed method, we report the details of our submission to the causality detection competition of Biomag2014, where the proposed method reached the 2nd place. Moreover, as empirical application, we applied the supervised approach on a dataset of neural recordings of rats obtaining an important reduction in the false positive rate
Recovery Conditions and Sampling Strategies for Network Lasso
The network Lasso is a recently proposed convex optimization method for
machine learning from massive network structured datasets, i.e., big data over
networks. It is a variant of the well-known least absolute shrinkage and
selection operator (Lasso), which is underlying many methods in learning and
signal processing involving sparse models. Highly scalable implementations of
the network Lasso can be obtained by state-of-the art proximal methods, e.g.,
the alternating direction method of multipliers (ADMM). By generalizing the
concept of the compatibility condition put forward by van de Geer and Buehlmann
as a powerful tool for the analysis of plain Lasso, we derive a sufficient
condition, i.e., the network compatibility condition, on the underlying network
topology such that network Lasso accurately learns a clustered underlying graph
signal. This network compatibility condition relates the location of the
sampled nodes with the clustering structure of the network. In particular, the
NCC informs the choice of which nodes to sample, or in machine learning terms,
which data points provide most information if labeled.Comment: nominated as student paper award finalist at Asilomar 2017. arXiv
admin note: substantial text overlap with arXiv:1704.0210
- …