FINDING EEG SPACE-TIME-SCALE LOCALIZED FEATURES USING MATRIX-BASED PENALIZED DISCRIMINANT ANALYSIS

International audienceThis paper proposes a new method for constructing and selecting of discriminant space-time-scale features for electroencephalogram (EEG) signal classification, suitable for Error Related Potentials (ErrP)detection in brain-computer interface (BCI). The method rests on a new variant of matrix-variate Linear Discriminant Analysis (LDA), and differs from previously proposed approaches in mainly three ways. First, a discrete wavelet expansion is introduced for mapping time-courses to time-scale coefficients, yielding time-scale localized features. Second, the matrix-variate LDA is modified in such a way that it yields an interesting duality property, that makes interpretation easier. Third, a space penalization is introduced using a surface Laplacian, so as to enforce spatial smoothness. The proposed approaches, termed D-MLDA and D-MPDA are tested on EEG signals, with the goal of detecting ErrP. Numerical results show that D-MPDA outperforms D-MLDA and other matrix-variate LDA techniques. In addition this method produces relevant features for interpretation in ErrP signals

A New Regularized Matrix Discriminant Analysis (R-MDA) Enabled Human-Centered EEG Monitoring Systems

The wider use of wearable devices for electroencephalogram (EEG) data capturing providesa very useful way for the monitoring and self-management of human health. However, the large volumesof data with high dimensions cause computational complexity in EEG data processing and pose a greatchallenge to the use of wearable EEG devices in healthcare. This paper proposes a new approach to extract thestructural information of EEG data and tackle the curse of dimensionality of the EEG data. A set of methodsfor dimensionality reduction (DR)-like linear discriminant analysis (LDA) and their improved methodshave been developed for EEG processing in the literature. However, the existing LDA-related methodssuffer from the singularity problem or expensive computational cost, and none of existing methods takeinto consideration the structure of the projection matrix, which is crucial for the extraction of the structuralinformation of the EEG data. In this paper, a new method called a regularized matrix discriminant analysis(R-MDA) is proposed for EEG feature representation and DR. In the R-MDA, the EEG data are representedas a data matrix, and projection vectors are reshaped to be a set of projection matrices stacking together. Byreformulating the LDA as a least-square formulation and imposing specified constraint on each projectionmatrix, the new R-MDA has been constructed to effectively reduce EEG dimensions and capturing thestructural information of the EEG data. Experimental results demonstrate that this new R-MDA outperformsthe existing LDA-related methods, including achieving improved accuracy with significant DR of the EEGdata. This offers an effective way to enable wearable EEG devices be applicable in human-centered healthmonitorin

Detecting single-trial EEG evoked potential using a wavelet domain linear mixed model: application to error potentials classification

Objective. The main goal of this work is to develop a model for multi-sensor signals such as MEG or EEG signals, that accounts for the inter-trial variability, suitable for corresponding binary classification problems. An important constraint is that the model be simple enough to handle small size and unbalanced datasets, as often encountered in BCI type experiments. Approach. The method involves linear mixed effects statistical model, wavelet transform and spatial filtering, and aims at the characterization of localized discriminant features in multi-sensor signals. After discrete wavelet transform and spatial filtering, a projection onto the relevant wavelet and spatial channels subspaces is used for dimension reduction. The projected signals are then decomposed as the sum of a signal of interest (i.e. discriminant) and background noise, using a very simple Gaussian linear mixed model. Main results. Thanks to the simplicity of the model, the corresponding parameter estimation problem is simplified. Robust estimates of class-covariance matrices are obtained from small sample sizes and an effective Bayes plug-in classifier is derived. The approach is applied to the detection of error potentials in multichannel EEG data, in a very unbalanced situation (detection of rare events). Classification results prove the relevance of the proposed approach in such a context. Significance. The combination of linear mixed model, wavelet transform and spatial filtering for EEG classification is, to the best of our knowledge, an original approach, which is proven to be effective. This paper improves on earlier results on similar problems, and the three main ingredients all play an important role

Analyse discriminante matricielle descriptive. Application a l'\'etude de signaux EEG

We focus on the descriptive approach to linear discriminant analysis for matrix-variate data in the binary case. Under a separability assumption on row and column variability, the most discriminant linear combinations of rows and columns are determined by the singular value decomposition of the difference of the class-averages with the Mahalanobis metric in the row and column spaces. This approach provides data representations of data in two-dimensional or three-dimensional plots and singles out discriminant components. An application to electroencephalographic multi-sensor signals illustrates the relevance of the method.Comment: in French, Journ{\'e}es de statistique de la SFDS, Jun 2015, Lille, Franc

Analyse discriminante matricielle descriptive. Application a l'étude de signaux EEG

National audienceWe focus on the descriptive approach to linear discriminant analysis for matrix-variate data in the binary case. Under a separability assumption on row and column variability, the most discriminant linear combinations of rows and columns are determined by the singular value decomposition of the difference of the class-averages with the Mahalanobis metric in the row and column spaces. This approach provides data representations of data in two-dimensional or three-dimensional plots and singles out discriminant components. An application to electroencephalographic multi-sensor signals illustrates the relevance of the method.Nous nous intéressons à l'approche descriptive de l'analyse discriminante linéaire de données matricielles dans le cas binaire. Sous l'hypothèse de séparabilité de la variabilité des lignes de celle des colonnes, les combinaisons linéaires des lignes et des colonnes les plus discriminantes sont déterminées par la décomposition en valeurs singulières de la différence des moyennes des deux classes en munissant les espaces des lignes et des colonnes de la métrique de Mahalanobis. Cette approche permet d'obtenir des représentations des données dans des plans factoriels et de dégager des composantes discriminantes. Une application a des signaux d'électroencéphalographie multi-capteurs illustre la pertinence de la méthode

Brain signal recognition using deep learning

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel UniversityBrain Computer Interface (BCI) has the potential to offer a new generation of applications independent of muscular activity and controlled by the human brain. Brain imaging technologies are used to transfer the cognitive tasks into control commands for a BCI system. The electroencephalography (EEG) technology serves as the best available non-invasive solution for extracting signals from the brain. On the other hand, speech is the primary means of communication, but for patients suffering from locked-in syndrome, there is no easy way to communicate. Therefore, an ideal communication system for locked-in patients is a thought-to-speech BCI system. This research aims to investigate methods for the recognition of imagined speech from EEG signals using deep learning techniques. In order to design an optimal imagined speech recognition BCI, variety of issues have been solved. These include 1) proposing new feature extraction and classification framework for recognition of imagined speech from EEG signals, 2) grammatical class recognition of imagined words from EEG signals, 3) discriminating different cognitive tasks associated with speech in the brain such as overt speech, covert speech, and visual imagery. In this work machine learning, deep learning methods were used to analyze EEG signals. For recognition of imagined speech from EEG signals, a new EEG database was collected while the participants mentally spoke (imagined speech) the presented words. Along with imagined speech, EEG data was recorded for visual imagery (imagining a scene or an image) and overt speech (verbal speech). Spectro-temporal and spatio-temporal domain features were investigated for the classification of imagined words from EEG signals. Further, a deep learning framework using the convolutional network and attention mechanism was implemented for learning features in the spatial, temporal, and spectral domains. The method achieved a recognition rate of 76.6% for three binary word pairs. These experiments show that deep learning algorithms are ideal for imagined speech recognition from EEG signals due to their ability to interpret features from non-linear and non-stationary signals. Grammatical classes of imagined words from EEG signals were also recognized using a multi-channel convolution network framework. This method was extended to a multi-level recognition system for multi-class classification of imagined words which achieved an accuracy of 52.9% for 10 words, which is much better in comparison to previous work. In order to investigate the difference between imagined speech with verbal speech and visual imagery from EEG signals, we used multivariate pattern analysis (MVPA). MVPA provided the time segments when the neural oscillation for the different cognitive tasks was linearly separable. Further, frequencies that result in most discrimination between the different cognitive tasks were also explored. A framework was proposed to discriminate two cognitive tasks based on the spatio-temporal patterns in EEG signals. The proposed method used the K-means clustering algorithm to find the best electrode combination and convolutional-attention network for feature extraction and classification. The proposed method achieved a high recognition rate of 82.9% and 77.7%. The results in this research suggest that a communication based BCI system can be designed using deep learning methods. Further, this work add knowledge to the existing work in the field of communication based BCI system

Decoding non-invasive brain activity with novel deep-learning approaches

This thesis delves into the world of non-invasive electrophysiological brain signals like electroencephalography (EEG) and magnetoencephalography (MEG), focusing on modelling and decoding such data. The research aims to investigate what happens in the brain when we perceive visual stimuli or engage in covert speech (inner speech) and enhance the decoding performance of such stimuli. The findings have significant implications for the development of brain-computer interfaces (BCIs), leading to assistive communication technologies for paralysed individuals. The thesis is divided into two main sections, methodological and experimental work. A central concern in both sections is the large variability present in electrophysiological recordings, whether it be within-subject or between-subject variability, and to a certain extent between-dataset variability. In the methodological sections, we explore the potential of deep learning for brain decoding. The research acknowledges the urgent need for more sophisticated models and larger datasets to improve the decoding and modelling of EEG and MEG signals. We present advancements in decoding visual stimuli using linear models at the individual subject level. We then explore how deep learning techniques can be employed for group decoding, introducing new methods to deal with between-subject variability. Finally, we also explores novel forecasting models of MEG data based on convolutional and Transformer-based architectures. In particular, Transformer-based models demonstrate superior capabilities in generating signals that closely match real brain data, thereby enhancing the accuracy and reliability of modelling the brain’s electrophysiology. In the experimental section, we present a unique dataset containing high-trial inner speech EEG, MEG, and preliminary optically pumped magnetometer (OPM) data. We highlight the limitations of current BCI systems used for communication, which are either invasive or extremely slow. While inner speech decoding from non-invasive brain signals has great promise, it has been a challenging goal in the field with limited decoding approaches, indicating a significant gap that needs to be addressed. Our aim is to investigate different types of inner speech and push decoding performance by collecting a high number of trials and sessions from a few participants. However, the decoding results are found to be mostly negative, underscoring the difficulty of decoding inner speech. In conclusion, this thesis provides valuable insight into the challenges and potential solutions in the field of electrophysiology, particularly in the decoding of visual stimuli and inner speech. The findings could pave the way for future research and advancements in the field, ultimately improving communication capabilities for paralysed individuals

Machine learning approaches to video activity recognition: from computer vision to signal processing

244 p.La investigación presentada se centra en técnicas de clasificación para dos tareas diferentes, aunque relacionadas, de tal forma que la segunda puede ser considerada parte de la primera: el reconocimiento de acciones humanas en vídeos y el reconocimiento de lengua de signos.En la primera parte, la hipótesis de partida es que la transformación de las señales de un vídeo mediante el algoritmo de Patrones Espaciales Comunes (CSP por sus siglas en inglés, comúnmente utilizado en sistemas de Electroencefalografía) puede dar lugar a nuevas características que serán útiles para la posterior clasificación de los vídeos mediante clasificadores supervisados. Se han realizado diferentes experimentos en varias bases de datos, incluyendo una creada durante esta investigación desde el punto de vista de un robot humanoide, con la intención de implementar el sistema de reconocimiento desarrollado para mejorar la interacción humano-robot.En la segunda parte, las técnicas desarrolladas anteriormente se han aplicado al reconocimiento de lengua de signos, pero además de ello se propone un método basado en la descomposición de los signos para realizar el reconocimiento de los mismos, añadiendo la posibilidad de una mejor explicabilidad. El objetivo final es desarrollar un tutor de lengua de signos capaz de guiar a los usuarios en el proceso de aprendizaje, dándoles a conocer los errores que cometen y el motivo de dichos errores