Non-Redundant Spectral Dimensionality Reduction

Spectral dimensionality reduction algorithms are widely used in numerous domains, including for recognition, segmentation, tracking and visualization. However, despite their popularity, these algorithms suffer from a major limitation known as the "repeated Eigen-directions" phenomenon. That is, many of the embedding coordinates they produce typically capture the same direction along the data manifold. This leads to redundant and inefficient representations that do not reveal the true intrinsic dimensionality of the data. In this paper, we propose a general method for avoiding redundancy in spectral algorithms. Our approach relies on replacing the orthogonality constraints underlying those methods by unpredictability constraints. Specifically, we require that each embedding coordinate be unpredictable (in the statistical sense) from all previous ones. We prove that these constraints necessarily prevent redundancy, and provide a simple technique to incorporate them into existing methods. As we illustrate on challenging high-dimensional scenarios, our approach produces significantly more informative and compact representations, which improve visualization and classification tasks

Comparative study of unsupervised dimension reduction techniques for the visualization of microarray gene expression data

<p>Abstract</p> <p>Background</p> <p>Visualization of DNA microarray data in two or three dimensional spaces is an important exploratory analysis step in order to detect quality issues or to generate new hypotheses. Principal Component Analysis (PCA) is a widely used linear method to define the mapping between the high-dimensional data and its low-dimensional representation. During the last decade, many new nonlinear methods for dimension reduction have been proposed, but it is still unclear how well these methods capture the underlying structure of microarray gene expression data. In this study, we assessed the performance of the PCA approach and of six nonlinear dimension reduction methods, namely Kernel PCA, Locally Linear Embedding, Isomap, Diffusion Maps, Laplacian Eigenmaps and Maximum Variance Unfolding, in terms of visualization of microarray data.</p> <p>Results</p> <p>A systematic benchmark, consisting of Support Vector Machine classification, cluster validation and noise evaluations was applied to ten microarray and several simulated datasets. Significant differences between PCA and most of the nonlinear methods were observed in two and three dimensional target spaces. With an increasing number of dimensions and an increasing number of differentially expressed genes, all methods showed similar performance. PCA and Diffusion Maps responded less sensitive to noise than the other nonlinear methods.</p> <p>Conclusions</p> <p>Locally Linear Embedding and Isomap showed a superior performance on all datasets. In very low-dimensional representations and with few differentially expressed genes, these two methods preserve more of the underlying structure of the data than PCA, and thus are favorable alternatives for the visualization of microarray data.</p

Single View Reconstruction for Human Face and Motion with Priors

Single view reconstruction is fundamentally an under-constrained problem. We aim to develop new approaches to model human face and motion with model priors that restrict the space of possible solutions. First, we develop a novel approach to recover the 3D shape from a single view image under challenging conditions, such as large variations in illumination and pose. The problem is addressed by employing the techniques of non-linear manifold embedding and alignment. Specifically, the local image models for each patch of facial images and the local surface models for each patch of 3D shape are learned using a non-linear dimensionality reduction technique, and the correspondences between these local models are then learned by a manifold alignment method. Local models successfully remove the dependency of large training databases for human face modeling. By combining the local shapes, the global shape of a face can be reconstructed directly from a single linear system of equations via least square. Unfortunately, this learning-based approach cannot be successfully applied to the problem of human motion modeling due to the internal and external variations in single view video-based marker-less motion capture. Therefore, we introduce a new model-based approach for capturing human motion using a stream of depth images from a single depth sensor. While a depth sensor provides metric 3D information, using a single sensor, instead of a camera array, results in a view-dependent and incomplete measurement of object motion. We develop a novel two-stage template fitting algorithm that is invariant to subject size and view-point variations, and robust to occlusions. Starting from a known pose, our algorithm first estimates a body configuration through temporal registration, which is used to search the template motion database for a best match. The best match body configuration as well as its corresponding surface mesh model are deformed to fit the input depth map, filling in the part that is occluded from the input and compensating for differences in pose and body-size between the input image and the template. Our approach does not require any makers, user-interaction, or appearance-based tracking. Experiments show that our approaches can achieve good modeling results for human face and motion, and are capable of dealing with variety of challenges in single view reconstruction, e.g., occlusion

Interpretable Dimensionally-Consistent Feature Extraction from Electrical Network Sensors

International audienceElectrical power networks are heavily monitored systems, requiring operators to perform intricate information synthesis before understanding the underlying network state. Our study aims at helping this synthesis step by automatically creating features from the sensor data. We propose a supervised feature extraction approach using a grammar-guided evolution, which outputs interpretable and dimensionally consistent features. Operations restrictions on dimensions are introduced in the learning process through context-free grammars. They ensure coherence with physical laws, dimensional-consistency, and also introduce technical expertise in the created features. We compare our approach to other state-of-the-art feature extraction methods on a real dataset taken from the French electrical network sensors

Statistical pattern classifier in machine vision quality control

Konenäkölaadunvalvonnalla pyritään takaamaan valmistettavien tuotteiden ja tuotannon laatu. Automatisoitu virheidentunnistus ja luokittelu parantaa laadunvalvonnan luotettavuutta, vaikeiden ongelmien ratkaisukykyä ja prosessin seurantaa. Käyttämällä tilastollista luokittelijaa lisätään konenäkölaadunvalvontajärjestelmän käyttökelpoisuutta laaduntakaamisprosessin työkaluna. Virheen tyypin luotettava tunnistaminen on kuitenkin osoittautunut ongelmalliseksi, minkä vuoksi useat alan tutkimukset keskittyvät yhden tietyn tuotteen tarkastamisessa esiintyviin luokitteluongelmiin. Tässä työssä esitelty luokittelijaohjelmisto tarjoaa menetelmiä konenäkölaadunvalvonnassa esiintyvien luokittelutehtävien ratkaisemiseen. Työllä oli kolme tavoitetta. Ensimmäisenä tavoitteena oli esitellä konenäkölaadunvalvonnan ammattilaisille joukko käyttökelpoisia visuaalisten virheiden luokittelumenetelmiä. Toisena tavoitteena oli tutkia, miten nämä menetelmät soveltuvat erilaisiin konenäkölaadunvalvonnan luokittelutehtäviin. Kolmas tavoite oli tuottaa työn aikana tilaajan olemassa olevien järjestelmien kanssa yhteensopiva luokittelukirjasto. Toisen ja kolmannen tavoitteen saavuttamiseksi valittiin toteutettavaksi ne luokittimet, joiden ennakoitiin soveltuvan käytettäväksi konenäkölaadunvalvonnassa. Kirjallisuudesta valittuja testiaineistoja käytettiin toteutettujen luokittimien oikean toiminnan tarkistamiseen. Konenäkölaadunvalvonta-alan testiaineistolla testattiin luokittimien selviytymistä erilaisista laadunvalvontatehtävistä. Luokittimien erot etsittiin vertailemalla virheluokitusten yleisyyttä ja laskennallisia suoritusaikavaatimuksia. Saatujen tulosten avulla analysoitiin toteutettujen tilastollisten luokittimien soveltuvuutta erilaisiin konenäkölaadunvalvonnassa esiintyviin luokitteluongelmiin. Tämän työn taustalla oli visio kokonaisvaltaisesta laadunvalvontajärjestelmästä, johon luokitin olennaisena osana kuuluu. Luokitin tekee laadunvalvontajärjestelmästä voimakkaan laadun takaamisen työkalun, sillä visuaalisten virheiden luokittelutieto mahdollistaa laadunvalvonnan, tiedonkeruujärjestelmien, raportointijärjestelmien ja ennen kaikkea tuotantoprosessin kehittämisen. /Kir1

Enhancing brain-computer interfacing through advanced independent component analysis techniques

A Brain-computer interface (BCI) is a direct communication system between a brain and an external device in which messages or commands sent by an individual do not pass through the brain’s normal output pathways but is detected through brain signals. Some severe motor impairments, such as Amyothrophic Lateral Sclerosis, head trauma, spinal injuries and other diseases may cause the patients to lose their muscle control and become unable to communicate with the outside environment. Currently no effective cure or treatment has yet been found for these diseases. Therefore using a BCI system to rebuild the communication pathway becomes a possible alternative solution. Among different types of BCIs, an electroencephalogram (EEG) based BCI is becoming a popular system due to EEG’s fine temporal resolution, ease of use, portability and low set-up cost. However EEG’s susceptibility to noise is a major issue to develop a robust BCI. Signal processing techniques such as coherent averaging, filtering, FFT and AR modelling, etc. are used to reduce the noise and extract components of interest. However these methods process the data on the observed mixture domain which mixes components of interest and noise. Such a limitation means that extracted EEG signals possibly still contain the noise residue or coarsely that the removed noise also contains part of EEG signals embedded. Independent Component Analysis (ICA), a Blind Source Separation (BSS) technique, is able to extract relevant information within noisy signals and separate the fundamental sources into the independent components (ICs). The most common assumption of ICA method is that the source signals are unknown and statistically independent. Through this assumption, ICA is able to recover the source signals. Since the ICA concepts appeared in the fields of neural networks and signal processing in the 1980s, many ICA applications in telecommunications, biomedical data analysis, feature extraction, speech separation, time-series analysis and data mining have been reported in the literature. In this thesis several ICA techniques are proposed to optimize two major issues for BCI applications: reducing the recording time needed in order to speed up the signal processing and reducing the number of recording channels whilst improving the final classification performance or at least with it remaining the same as the current performance. These will make BCI a more practical prospect for everyday use. This thesis first defines BCI and the diverse BCI models based on different control patterns. After the general idea of ICA is introduced along with some modifications to ICA, several new ICA approaches are proposed. The practical work in this thesis starts with the preliminary analyses on the Southampton BCI pilot datasets starting with basic and then advanced signal processing techniques. The proposed ICA techniques are then presented using a multi-channel event related potential (ERP) based BCI. Next, the ICA algorithm is applied to a multi-channel spontaneous activity based BCI. The final ICA approach aims to examine the possibility of using ICA based on just one or a few channel recordings on an ERP based BCI. The novel ICA approaches for BCI systems presented in this thesis show that ICA is able to accurately and repeatedly extract the relevant information buried within noisy signals and the signal quality is enhanced so that even a simple classifier can achieve good classification accuracy. In the ERP based BCI application, after multichannel ICA the data just applied to eight averages/epochs can achieve 83.9% classification accuracy whilst the data by coherent averaging can reach only 32.3% accuracy. In the spontaneous activity based BCI, the use of the multi-channel ICA algorithm can effectively extract discriminatory information from two types of singletrial EEG data. The classification accuracy is improved by about 25%, on average, compared to the performance on the unpreprocessed data. The single channel ICA technique on the ERP based BCI produces much better results than results using the lowpass filter. Whereas the appropriate number of averages improves the signal to noise rate of P300 activities which helps to achieve a better classification. These advantages will lead to a reliable and practical BCI for use outside of the clinical laboratory

Visualización bidimensional de problemas de clasificación en alta dimensión

El objetivo de este proyecto es obtener buenas representaciones en dos dimensiones de problemas N dimensionales. Para ello se propone una función de coste que mida la similitud entre una representación en N dimensiones y la misma en 2 dimensiones. Esto permite que se pueda realizar una comparación de la eficacia y una clasificación de diferentes técnicas de reducción de dimensionalidad para descubrir en distintos conjuntos cuales de estas son las que mantienen un mayor grado de similitud entre ambos espacios dimensionales.The main objective of this Project is to get proper good-quality representations in two dimensions of N dimensional problems. In order to do that, a cost function is proposed to measure the similarity between a representation in N dimensions and the same in two dimensions. ; that allows us to be able to carry out an effectiveness comparison and a classification of different techniques to reduce ‘dimensionality’ so as to find out, in different data sets, which of those are the ones that keep an upper similarity level between both dimensional spaces.Ingeniería Técnica en Sonido e Image