Single-trial analysis of EEG during rapid visual discrimination: enabling cortically-coupled computer vision

We describe our work using linear discrimination of multi-channel electroencephalography for single-trial detection of neural signatures of visual recognition events. We demonstrate the approach as a methodology for relating neural variability to response variability, describing studies for response accuracy and response latency during visual target detection. We then show how the approach can be utilized to construct a novel type of brain-computer interface, which we term cortically-coupled computer vision. In this application, a large database of images is triaged using the detected neural signatures. We show how ‘corticaltriaging’ improves image search over a strictly behavioral response

Integrating EEG and MEG signals to improve motor imagery classification in brain-computer interfaces

We propose a fusion approach that combines features from simultaneously recorded electroencephalographic (EEG) and magnetoencephalographic (MEG) signals to improve classification performances in motor imagery-based brain-computer interfaces (BCIs). We applied our approach to a group of 15 healthy subjects and found a significant classification performance enhancement as compared to standard single-modality approaches in the alpha and beta bands. Taken together, our findings demonstrate the advantage of considering multimodal approaches as complementary tools for improving the impact of non-invasive BCIs

MEG:hen perustuvan aivo-tietokone -käyttöliittymän kehitys

Brain–computer interfaces (BCI) have recently gained interest both in basic neuroscience and clinical interventions. The majority of noninvasive BCIs measure brain activity with electroencephalography (EEG). However, the real-time signal analysis and decoding of brain activity suffer from low signal-to-noise ratio and poor spatial resolution of EEG. These limitations could be overcome by using magnetoencephalography (MEG) as an alternative measurement modality. The aim of this thesis is to develop an MEG-based BCI for decoding hand motor imagery, which could eventually serve as a therapeutic method for patients recovering from e.g. cerebral stroke. Here, machine learning methods for decoding motor imagery -related brain activity are validated with healthy subjects’ MEG measurements. The first part of the thesis (Study I) involves a comparison of feature extraction methods for classifying left- vs right-hand motor imagery (MI), and MI vs rest. It was found that spatial filtering and further extraction of bandpower features yield better classification accuracy than time–frequency features extracted from parietal gradiometers. Furthermore, prior spatial filtering improved the discrimination capability of time–frequency features. The training data for a BCI is typically collected in the beginning of each measurement session. However, as this can be time-consuming and exhausting for the subject, the training data from other subjects’ measurements could be used as well. In the second part of the thesis (Study II), methods for across-subject classification of MI were compared. The results showed that a classifier based on multi-task learning with a l2,1-norm regularized logistic regression was the best method for across-subject decoding for both MEG and EEG. In Study II, we also compared the decoding results of simultaneously measured EEG and MEG data, and investigated whether the MEG responses to passive hand movements could be used to train a classifier to detect MI. MEG yielded altogether slightly, but not significantly, better results than EEG. Training the classifiers with subject’s own or other subjects’ passive movements did not result in high accuracy, which indicates that passive movements should not be used for calibrating an MI-BCI. The methods presented in this thesis are suitable for a real-time MEG-based BCI. The decoding results can be used as a benchmark when developing other classifiers specifically for motor imagery -related MEG data.Aivo-tietokone -käyttöliittymät (brain–computer interface; BCI) ovat viime aikoina herättäneet kiinnostusta niin neurotieteen perustutkimuksessa kuin kliinisissä interventioissakin. Suurin osa ei-invasiivisista BCI:stä mittaa aivotoimintaa elektroenkefalografialla (EEG). EEG:n matala signaali-kohinasuhde ja huono avaruudellinen resoluutio kuitenkin hankaloittavat reaaliaikais-ta signaalianalyysia ja aivotoiminnan luokittelua. Nämä rajoitteet voidaan kiertää käyttämällä magnetoenkefalografiaa (MEG) vaihtoehtoisena mittausmenetelmänä. Tämän työn tavoitteena on kehittää käden liikkeen kuvittelua luokitteleva, MEG:hen perustuva BCI, jota voidaan myöhemmin käyttää terapeuttisena menetelmänä esimerkiksi aivoinfarktista toipuvien potilaiden kuntoutuk-sessa. Tutkimuksessa validoidaan terveillä koehenkilöillä tehtyjen MEG-mittausten perusteella koneoppimismenetelmiä, joilla luokitellaan liikkeen kuvittelun aiheuttamaa aivotoimintaa. Ensimmäisessä osatyössä (Tutkimus I) vertailtiin piirteenirrotusmenetelmiä, joita käytetään erottamaan toisistaan vasemman ja oikean käden kuvittelu sekä liikkeen kuvittelu ja lepotila. Ha-vaittiin, että avaruudellisesti suodatettujen signaalien taajuuskaistan teho luokittelupiirteenä tuotti parempia luokittelutarkkuuksia kuin parietaalisista gradiometreistä mitatut aika-taajuuspiirteet. Lisäksi edeltävä avaruudellinen suodatus paransi aika-taajuuspiirteiden erottelukykyä luokittelu-tehtävissä.BCI:n opetusdata kerätään yleensä kunkin mittauskerran alussa. Koska tämä voi kuitenkin olla aikaavievää ja uuvuttavaa koehenkilölle, opetusdatana voidaan käyttää myös muilta koehenkilöiltä kerättyjä mittaussignaaleja. Toisessa osatyössä (Tutkimus II) vertailtiin koehenkilöiden väliseen luo-kitteluun soveltuvia menetelmiä. Tulosten perusteella monitehtäväoppimista ja l2,1-regularisoitua logistista regressiota käyttävä luokittelija oli paras menetelmä koehenkilöiden väliseen luokitteluun sekä MEG:llä että EEG:llä. Toisessa osatyössä vertailtiin myös samanaikaisesti mitattujen MEG:n ja EEG:n tuottamia luokit-telutuloksia, sekä tutkittiin voidaanko passiivisten kädenliikkeiden aikaansaamia MEG-vasteita käyttää liikkeen kuvittelua tunnistavien luokittelijoiden opetukseen. MEG tuotti hieman, muttei merkittävästi, parempia tuloksia kuin EEG. Luokittelijoiden opetus koehenkilöiden omilla tai mui-den koehenkilöiden passiiviliikkeillä ei tuottanut hyviä luokittelutarkkuuksia, mikä osoittaa että passiiviliikkeitä ei tulisi käyttää liikkeen kuvittelua tunnistavan BCI:n kalibrointiin. Työssä esitettyjä menetelmiä voidaan käyttää reaaliaikaisessa MEG-BCI:ssä. Luokittelutuloksia voidaan käyttää vertailukohtana kehitettäessä muita liikkeen kuvitteluun liittyvän MEG-datan luokittelijoita

Graph Neural Network-based EEG Classification:A Survey

Graph neural networks (GNN) are increasingly used to classify EEG for tasks such as emotion recognition, motor imagery and neurological diseases and disorders. A wide range of methods have been proposed to design GNN-based classifiers. Therefore, there is a need for a systematic review and categorisation of these approaches. We exhaustively search the published literature on this topic and derive several categories for comparison. These categories highlight the similarities and differences among the methods. The results suggest a prevalence of spectral graph convolutional layers over spatial. Additionally, we identify standard forms of node features, with the most popular being the raw EEG signal and differential entropy. Our results summarise the emerging trends in GNN-based approaches for EEG classification. Finally, we discuss several promising research directions, such as exploring the potential of transfer learning methods and appropriate modelling of cross-frequency interactions.</p

Spatio-spectral patterns based on stein kernel for EEG signal classification

El trastorno por déficit de atención con hiperactividad (TDAH) es un trastorno neurológico de inicio en la niñez que puede persistir en la adolescencia y la vida adulta, reduciendo la concentración, la memoria y la productividad. El principal inconveniente de las anomalías de la salud mental de este tipo es la técnica de diagnóstico tradicional, ya que se basa exclusivamente en una descripción sintomatológica sin considerar ningún dato biológico, lo que genera altas tasas de sobrediagnóstico. Para abordar el problema anterior, los investigadores clínicos están intentando extraer biomarcadores de TDAH a partir de señales electroencefalográficas (EEG) registradas. Entre los biomarcadores más comunes se encuentran la relación Theta / Beta y P300, de los cuales estudios recientes han demostrado una falta de importancia en las diferencias entre el TDAH y los sujetos de control. Además, otro gran desafío en el procesamiento del electroencefalograma viene dado por la sensibilidad de las señales, ya que pueden verse fácilmente afectadas por ruidos de fondo, artefactos musculares, movimientos de la cabeza y parpadeos que perjudican enormemente su calidad, lo que limita su introducción en aplicaciones del mundo real. Este trabajo propone una metodología de representación de señales de EEG para identificar discrepancias de respuestas inhibitorias en el sujeto, decodificar la estructura de datos y respaldar el diagnóstico de trastornos mentales. Para esto, primero desarrollamos un enfoque de extracción de características basado en los patrones espaciales comunes (CSP) de las señales de EEG para respaldar el diagnóstico de TDAH como se muestra en el capítulo 3. Luego, desarrollamos una metodología para la representación de señales de EEG que utiliza la similitud entre series de tiempo a través de sus matrices de covarianza en la variedad riemanniana de matrices semidefinitas positivas (PSD), utilizando la divergencia logdet de Jensen Bregman, el kernel de Stein y la alineación de kernel centrada (CKA) como una función de costo para realizar una optimización de filtros espaciales. Finalmente, en el capítulo 5 presentamos una metodología para el apoyo diagnóstico del TDAH. La propuesta implica el uso de los patrones espaciales óptimos desarrollados en el capítulo 4, una descomposición en los ritmos cerebrales y la decodificación discriminativa del capítulo 3. Las características subjetivas resultantes alimentaron un análisis discriminante lineal como herramienta de diagnóstico. La tasa de precisión alcanzada del 93% demuestra que el índice discriminativo basado en los patrones espaciales de stein supera a los biomarcadores convencionales en el diagnóstico de TDAH.Attention-Deficit/Hyperactivity Disorder (ADHD) is a childhood-onset neurological disorder that can persist in adolescence and adult life, reducing concentration, memory, and productivity. The main drawback with mental health abnormalities of this type is the traditional diagnostic technique. Since this is based exclusively on a symptomatological description without considering any biological data, leading to high overdiagnosis rates. To address the above problem, clinical researchers are attempting to extract ADHD biomarkers from recorded electroencephalographic (EEG) signals. Among the most common biomarkers are Theta/Beta Ratio and P300, of which recent studies have shown a lack of significance on the differences between ADHD and control subjects. Besides, another great challenge in EEG processing is given by the sensitivity of the signals, since they can be easily affected by background noise, muscle artifacts, head movements and flickering that greatly impair their quality, which limits its introduction into real world applications. This work proposes an EEG signal representation methodology for identifying subject-wise discrepancies of inhibitory responses, decoding the data structure, and supporting diagnosis of mental disorders. For this, first we develop a feature extraction approach based on the common spatial patterns (CSP) from EEG signals to support the ADHD diagnosis as show in chapter 3. Then, we develop a methodology for the representation of EEG signals that uses the similarity between time series through their covariance matrices in the Riemannian manifold of positive semidefinite matrices (PSD), using the logdet-divergence of Jensen Bregman, the Stein kernel, and Centered Kernel Alignment (CKA) as a cost function to perform a spatial filters optimization. Finally, in chapter 5 we present a methodology for the diagnostic support of ADHD. The proposal involves the use of the optimal spatial patterns developed in chapter 4, a decomposition in brain rhythms, and the discriminative decoding of chapter 3. The resulting subject-wise features fed a linear discriminant analysis as the supported-diagnosis tool. Achieved 93% accuracy rate proves that the discriminative index based on the stein spatial patterns outperforms conventional biomarkers in the ADHD diagnosis.MaestríaMagíster en Ingeniería EléctricaContents 1 List of Symbols and Abbreviations 6 1.1 Symbols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.2 Abbrevations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2 Introduction 8 2.1 Problem statement . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2.2 Justification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.3 State of the art . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.4 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.4.1 General objective . . . . . . . . . . . . . . . . . . . . . . . . 12 2.4.2 Specific objectives . . . . . . . . . . . . . . . . . . . . . . . 12 3 CSP-based discriminative capacity index from EEG 13 3.1 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 3.1.1 Common Spatial Patterns . . . . . . . . . . . . . . . . . . . . 13 3.1.2 Discriminative decoding of CSP . . . . . . . . . . . . . . . . 14 3.2 Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 3.2.1 Synthetic EEG records . . . . . . . . . . . . . . . . . . . . . 15 3.2.2 Real EEG records . . . . . . . . . . . . . . . . . . . . . . . . 16 3.2.3 Proposed scheme for feature extraction . . . . . . . . . . . . 19 3.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.3.1 Discriminative decoding on simulated data . . . . . . . . . . 19 3.3.2 Feature extraction by discriminative decoding . . . . . . . . . 21 3.3.3 Diagnostic support of ADHD . . . . . . . . . . . . . . . . . 21 4 Multiple Kernel Stein Spatial Patterns 24 4.1 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 4.1.1 EEG Decomposition . . . . . . . . . . . . . . . . . . . . . . 24 4.1.2 Time-Series Similarity through the Stein Kernel for PSD Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 4.1.3 Spatial Filter Optimization Using Centered Kernel Alignment 27 4.1.4 Assembling of Multiple Kernel Representations . . . . . . . . 27 4.2 Experimental Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 4.2.1 Dataset IIa from BCI Competition IV (BCICIV2a) . . . . . . 28 4.2.2 Proposed BCI Methodology . . . . . . . . . . . . . . . . . . 29 4.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 4.3.1 Performance Results . . . . . . . . . . . . . . . . . . . . . . 30 4.3.2 Model Interpretability . . . . . . . . . . . . . . . . . . . . . 33 5 SSP-based discriminative capacity index from EEG supporting ADHD di agnosis 37 5.1 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 5.1.1 Brain rhythms EEG decomposition . . . . . . . . . . . . . . 38 5.1.2 Stein Spatial Patterns (SSP) . . . . . . . . . . . . . . . . . . 39 5.1.3 Discriminative decoding of SSP . . . . . . . . . . . . . . . . 39 5.1.4 Generative-supervised feature relevance . . . . . . . . . . . . 40 5.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 6 Conclusions 45 6.1 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Improved Motor Imagery Classification Using Adaptive Spatial Filters Based on Particle Swarm Optimization Algorithm

As a typical self-paced brain-computer interface (BCI) system, the motor imagery (MI) BCI has been widely applied in fields such as robot control, stroke rehabilitation, and assistance for patients with stroke or spinal cord injury. Many studies have focused on the traditional spatial filters obtained through the common spatial pattern (CSP) method. However, the CSP method can only obtain fixed spatial filters for specific input signals. Besides, CSP method only focuses on the variance difference of two types of electroencephalogram (EEG) signals, so the decoding ability of EEG signals is limited. To obtain more effective spatial filters for better extraction of spatial features that can improve classification to MI-EEG, this paper proposes an adaptive spatial filter solving method based on particle swarm optimization algorithm (PSO). A training and testing framework based on filter bank and spatial filters (FBCSP-ASP) is designed for MI EEG signal classification. Comparative experiments are conducted on two public datasets (2a and 2b) from BCI competition IV, which show the outstanding average recognition accuracy of FBCSP-ASP. The proposed method has achieved significant performance improvement on MI-BCI. The classification accuracy of the proposed method has reached 74.61% and 81.19% on datasets 2a and 2b, respectively. Compared with the baseline algorithm (FBCSP), the proposed algorithm improves 11.44% and 7.11% on two datasets respectively. Furthermore, the analysis based on mutual information, t-SNE and Shapley values further proves that ASP features have excellent decoding ability for MI-EEG signals, and explains the improvement of classification performance by the introduction of ASP features.Comment: 25 pages, 8 figure

Towards Zero Training for Brain-Computer Interfacing

Electroencephalogram (EEG) signals are highly subject-specific and vary considerably even between recording sessions of the same user within the same experimental paradigm. This challenges a stable operation of Brain-Computer Interface (BCI) systems. The classical approach is to train users by neurofeedback to produce fixed stereotypical patterns of brain activity. In the machine learning approach, a widely adapted method for dealing with those variances is to record a so called calibration measurement on the beginning of each session in order to optimize spatial filters and classifiers specifically for each subject and each day. This adaptation of the system to the individual brain signature of each user relieves from the need of extensive user training. In this paper we suggest a new method that overcomes the requirement of these time-consuming calibration recordings for long-term BCI users. The method takes advantage of knowledge collected in previous sessions: By a novel technique, prototypical spatial filters are determined which have better generalization properties compared to single-session filters. In particular, they can be used in follow-up sessions without the need to recalibrate the system. This way the calibration periods can be dramatically shortened or even completely omitted for these ‘experienced’ BCI users. The feasibility of our novel approach is demonstrated with a series of online BCI experiments. Although performed without any calibration measurement at all, no loss of classification performance was observed