16,126 research outputs found
Kernel Multivariate Analysis Framework for Supervised Subspace Learning: A Tutorial on Linear and Kernel Multivariate Methods
Feature extraction and dimensionality reduction are important tasks in many
fields of science dealing with signal processing and analysis. The relevance of
these techniques is increasing as current sensory devices are developed with
ever higher resolution, and problems involving multimodal data sources become
more common. A plethora of feature extraction methods are available in the
literature collectively grouped under the field of Multivariate Analysis (MVA).
This paper provides a uniform treatment of several methods: Principal Component
Analysis (PCA), Partial Least Squares (PLS), Canonical Correlation Analysis
(CCA) and Orthonormalized PLS (OPLS), as well as their non-linear extensions
derived by means of the theory of reproducing kernel Hilbert spaces. We also
review their connections to other methods for classification and statistical
dependence estimation, and introduce some recent developments to deal with the
extreme cases of large-scale and low-sized problems. To illustrate the wide
applicability of these methods in both classification and regression problems,
we analyze their performance in a benchmark of publicly available data sets,
and pay special attention to specific real applications involving audio
processing for music genre prediction and hyperspectral satellite images for
Earth and climate monitoring
Spatial Filtering Pipeline Evaluation of Cortically Coupled Computer Vision System for Rapid Serial Visual Presentation
Rapid Serial Visual Presentation (RSVP) is a paradigm that supports the
application of cortically coupled computer vision to rapid image search. In
RSVP, images are presented to participants in a rapid serial sequence which can
evoke Event-related Potentials (ERPs) detectable in their Electroencephalogram
(EEG). The contemporary approach to this problem involves supervised spatial
filtering techniques which are applied for the purposes of enhancing the
discriminative information in the EEG data. In this paper we make two primary
contributions to that field: 1) We propose a novel spatial filtering method
which we call the Multiple Time Window LDA Beamformer (MTWLB) method; 2) we
provide a comprehensive comparison of nine spatial filtering pipelines using
three spatial filtering schemes namely, MTWLB, xDAWN, Common Spatial Pattern
(CSP) and three linear classification methods Linear Discriminant Analysis
(LDA), Bayesian Linear Regression (BLR) and Logistic Regression (LR). Three
pipelines without spatial filtering are used as baseline comparison. The Area
Under Curve (AUC) is used as an evaluation metric in this paper. The results
reveal that MTWLB and xDAWN spatial filtering techniques enhance the
classification performance of the pipeline but CSP does not. The results also
support the conclusion that LR can be effective for RSVP based BCI if
discriminative features are available
An Open Source C++ Implementation of Multi-Threaded Gaussian Mixture Models, k-Means and Expectation Maximisation
Modelling of multivariate densities is a core component in many signal
processing, pattern recognition and machine learning applications. The
modelling is often done via Gaussian mixture models (GMMs), which use
computationally expensive and potentially unstable training algorithms. We
provide an overview of a fast and robust implementation of GMMs in the C++
language, employing multi-threaded versions of the Expectation Maximisation
(EM) and k-means training algorithms. Multi-threading is achieved through
reformulation of the EM and k-means algorithms into a MapReduce-like framework.
Furthermore, the implementation uses several techniques to improve numerical
stability and modelling accuracy. We demonstrate that the multi-threaded
implementation achieves a speedup of an order of magnitude on a recent 16 core
machine, and that it can achieve higher modelling accuracy than a previously
well-established publically accessible implementation. The multi-threaded
implementation is included as a user-friendly class in recent releases of the
open source Armadillo C++ linear algebra library. The library is provided under
the permissive Apache~2.0 license, allowing unencumbered use in commercial
products
How to Solve Classification and Regression Problems on High-Dimensional Data with a Supervised Extension of Slow Feature Analysis
Supervised learning from high-dimensional data, e.g., multimedia data, is a challenging task. We propose an extension of slow feature analysis (SFA) for supervised dimensionality reduction called graph-based SFA (GSFA). The algorithm extracts a label-predictive low-dimensional set of features that can be post-processed by typical supervised algorithms to generate the final label or class estimation. GSFA is trained with a so-called training graph, in which the vertices are the samples and the edges represent similarities of the corresponding labels. A new weighted SFA optimization problem is introduced, generalizing the notion of slowness from sequences of samples to such training graphs. We show that GSFA computes an optimal solution to this problem in the considered function space, and propose several types of training graphs. For classification, the most straightforward graph yields features equivalent to those of (nonlinear) Fisher discriminant analysis. Emphasis is on regression, where four different graphs were evaluated experimentally with a subproblem of face detection on photographs. The method proposed is promising particularly when linear models are insufficient, as well as when feature selection is difficult
Graph Laplacian for Image Anomaly Detection
Reed-Xiaoli detector (RXD) is recognized as the benchmark algorithm for image
anomaly detection; however, it presents known limitations, namely the
dependence over the image following a multivariate Gaussian model, the
estimation and inversion of a high-dimensional covariance matrix, and the
inability to effectively include spatial awareness in its evaluation. In this
work, a novel graph-based solution to the image anomaly detection problem is
proposed; leveraging the graph Fourier transform, we are able to overcome some
of RXD's limitations while reducing computational cost at the same time. Tests
over both hyperspectral and medical images, using both synthetic and real
anomalies, prove the proposed technique is able to obtain significant gains
over performance by other algorithms in the state of the art.Comment: Published in Machine Vision and Applications (Springer
Masking Strategies for Image Manifolds
We consider the problem of selecting an optimal mask for an image manifold,
i.e., choosing a subset of the pixels of the image that preserves the
manifold's geometric structure present in the original data. Such masking
implements a form of compressive sensing through emerging imaging sensor
platforms for which the power expense grows with the number of pixels acquired.
Our goal is for the manifold learned from masked images to resemble its full
image counterpart as closely as possible. More precisely, we show that one can
indeed accurately learn an image manifold without having to consider a large
majority of the image pixels. In doing so, we consider two masking methods that
preserve the local and global geometric structure of the manifold,
respectively. In each case, the process of finding the optimal masking pattern
can be cast as a binary integer program, which is computationally expensive but
can be approximated by a fast greedy algorithm. Numerical experiments show that
the relevant manifold structure is preserved through the data-dependent masking
process, even for modest mask sizes
- …