Subject-Independent Deep Architecture for EEG-based Motor Imagery Classification

Motor imagery (MI) classification based on electroencephalogram (EEG) is a widely-used technique in non-invasive brain-computer interface (BCI) systems. Since EEG recordings suffer from heterogeneity across subjects and labeled data insufficiency, designing a classifier that performs the MI independently from the subject with limited labeled samples would be desirable. To overcome these limitations, we propose a novel subject-independent semi-supervised deep architecture (SSDA). The proposed SSDA consists of two parts: an unsupervised and a supervised element. The training set contains both labeled and unlabeled data samples from multiple subjects. First, the unsupervised part, known as the columnar spatiotemporal auto-encoder (CST-AE), extracts latent features from all the training samples by maximizing the similarity between the original and reconstructed data. A dimensional scaling approach is employed to reduce the dimensionality of the representations while preserving their discriminability. Second, a supervised part learns a classifier based on the labeled training samples using the latent features acquired in the unsupervised part. Moreover, we employ center loss in the supervised part to minimize the embedding space distance of each point in a class to its center. The model optimizes both parts of the network in an end-to-end fashion. The performance of the proposed SSDA is evaluated on test subjects who were not seen by the model during the training phase. To assess the performance, we use two benchmark EEG-based MI task datasets. The results demonstrate that SSDA outperforms state-of-the-art methods and that a small number of labeled training samples can be sufficient for strong classification performance

Improved Motor Imagery Classification Using Adaptive Spatial Filters Based on Particle Swarm Optimization Algorithm

As a typical self-paced brain-computer interface (BCI) system, the motor imagery (MI) BCI has been widely applied in fields such as robot control, stroke rehabilitation, and assistance for patients with stroke or spinal cord injury. Many studies have focused on the traditional spatial filters obtained through the common spatial pattern (CSP) method. However, the CSP method can only obtain fixed spatial filters for specific input signals. Besides, CSP method only focuses on the variance difference of two types of electroencephalogram (EEG) signals, so the decoding ability of EEG signals is limited. To obtain more effective spatial filters for better extraction of spatial features that can improve classification to MI-EEG, this paper proposes an adaptive spatial filter solving method based on particle swarm optimization algorithm (PSO). A training and testing framework based on filter bank and spatial filters (FBCSP-ASP) is designed for MI EEG signal classification. Comparative experiments are conducted on two public datasets (2a and 2b) from BCI competition IV, which show the outstanding average recognition accuracy of FBCSP-ASP. The proposed method has achieved significant performance improvement on MI-BCI. The classification accuracy of the proposed method has reached 74.61% and 81.19% on datasets 2a and 2b, respectively. Compared with the baseline algorithm (FBCSP), the proposed algorithm improves 11.44% and 7.11% on two datasets respectively. Furthermore, the analysis based on mutual information, t-SNE and Shapley values further proves that ASP features have excellent decoding ability for MI-EEG signals, and explains the improvement of classification performance by the introduction of ASP features.Comment: 25 pages, 8 figure

Analysis of Human Gait Using Hybrid EEG-fNIRS-Based BCI System: A Review

Human gait is a complex activity that requires high coordination between the central nervous system, the limb, and the musculoskeletal system. More research is needed to understand the latter coordination\u27s complexity in designing better and more effective rehabilitation strategies for gait disorders. Electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS) are among the most used technologies for monitoring brain activities due to portability, non-invasiveness, and relatively low cost compared to others. Fusing EEG and fNIRS is a well-known and established methodology proven to enhance brain–computer interface (BCI) performance in terms of classification accuracy, number of control commands, and response time. Although there has been significant research exploring hybrid BCI (hBCI) involving both EEG and fNIRS for different types of tasks and human activities, human gait remains still underinvestigated. In this article, we aim to shed light on the recent development in the analysis of human gait using a hybrid EEG-fNIRS-based BCI system. The current review has followed guidelines of preferred reporting items for systematic reviews and meta-Analyses (PRISMA) during the data collection and selection phase. In this review, we put a particular focus on the commonly used signal processing and machine learning algorithms, as well as survey the potential applications of gait analysis. We distill some of the critical findings of this survey as follows. First, hardware specifications and experimental paradigms should be carefully considered because of their direct impact on the quality of gait assessment. Second, since both modalities, EEG and fNIRS, are sensitive to motion artifacts, instrumental, and physiological noises, there is a quest for more robust and sophisticated signal processing algorithms. Third, hybrid temporal and spatial features, obtained by virtue of fusing EEG and fNIRS and associated with cortical activation, can help better identify the correlation between brain activation and gait. In conclusion, hBCI (EEG + fNIRS) system is not yet much explored for the lower limb due to its complexity compared to the higher limb. Existing BCI systems for gait monitoring tend to only focus on one modality. We foresee a vast potential in adopting hBCI in gait analysis. Imminent technical breakthroughs are expected using hybrid EEG-fNIRS-based BCI for gait to control assistive devices and Monitor neuro-plasticity in neuro-rehabilitation. However, although those hybrid systems perform well in a controlled experimental environment when it comes to adopting them as a certified medical device in real-life clinical applications, there is still a long way to go

Deep recurrent–convolutional neural network for classification of simultaneous EEG–fNIRS signals

Brain–computer interface (BCI) is a powerful system for communicating between the brain and outside world. Traditional BCI systems work based on electroencephalogram (EEG) signals only. Recently, researchers have used a combination of EEG signals with other signals to improve the performance of BCI systems. Among these signals, the combination of EEG with functional near-infrared spectroscopy (fNIRS) has achieved favourable results. In most studies, only EEGs or fNIRs have been considered as chain-like sequences, and do not consider complex correlations between adjacent signals, neither in time nor channel location. In this study, a deep neural network model has been introduced to identify the exact objectives of the human brain by introducing temporal and spatial features. The proposed model incorporates the spatial relationship between EEG and fNIRS signals. This could be implemented by transforming the sequences of these chain-like signals into hierarchical three-rank tensors. The tests show that the proposed model has a precision of 99.6%

EEG-Based Brain-Computer Interfacing via Motor-Imagery: Practical Implementation and Feature Analysis

The human brain is the most intriguing and complex signal processing unit ever known to us. A unique characteristic of our brain is its plasticity property, i.e., the ability of neurons to modify their behavior (structure and functionality) in response to environmental diversity. The plasticity property of brain has motivated design of brain-computer interfaces (BCI) to develop an alternative form of communication channel between brain signals and the external world. The BCI systems have several therapeutic applications of significant importance including but not limited to rehabilitation/ assistive systems, rehabilitation robotics, and neuro-prosthesis control. Despite recent advancements in BCIs, such systems are still far from being reliably incorporated within humanmachine inference networks. In this regard, the thesis focuses on Motor Imagery (MI)-based BCI systems with the objective of tackling some key challenges observed in existing solutions. The MI is defined as a cognitive process in which a person imagines performing a movement without peripheral (muscle) activation. At one hand, the thesis focuses on feature extraction, which is one of the most crucial steps for the development of an effective BCI system. In this regard, the thesis proposes a subject-specific filtering framework, referred to as the regularized double-band Bayesian (R-B2B) spectral filtering. The proposed R-B2B framework couples three main feature extraction categories, namely filter-bank solutions, regularized techniques, and optimized Bayesian mechanisms to enhance the overall classification accuracy of the BCI. To further evaluate the effects of deploying optimized subject-specific spectra-spatial filters, it is vital to examine and investigate different aspects of data collection and in particular, effects of the stimuli provided to subjects to trigger MI tasks. The second main initiative of the thesis is to propose an element of experimental design dealing with MI-based BCI systems. In this regard, we have implemented an EEG-based BCI system and constructed a benchmark dataset associated with 10 healthy subjects performing actual movement and MI tasks. To investigate effects of stimulus on the overall achievable performance, four different protocols are designed and implemented via introduction of visual and voice stimuli. Finally, the work investigates effects of adaptive trimming of EEG epochs resulting in an adaptive and subject-specific solution

A systematic review on artifact removal and classification techniques for enhanced MEG-based BCI systems

Neurological disease victims may be completely paralyzed and unable to move, but they may still be able to think. Their brain activity is the only means by which they can interact with their environment. Brain-Computer Interface (BCI) research attempts to create tools that support subjects with disabilities. Furthermore, BCI research has expanded rapidly over the past few decades as a result of the interest in creating a new kind of human-to-machine communication. As magnetoencephalography (MEG) has superior spatial and temporal resolution than other approaches, it is being utilized to measure brain activity non-invasively. The recorded signal includes signals related to brain activity as well as noise and artifacts from numerous sources. MEG can have a low signal-to-noise ratio because the magnetic fields generated by cortical activity are small compared to other artifacts and noise. By using the right techniques for noise and artifact detection and removal, the signal-to-noise ratio can be increased. This article analyses various methods for removing artifacts as well as classification strategies. Additionally, this offers a study of the influence of Deep Learning models on the BCI system. Furthermore, the various challenges in collecting and analyzing MEG signals as well as possible study fields in MEG-based BCI are examined

A Hybrid End-to-End Spatio-Temporal Attention Neural Network with Graph-Smooth Signals for EEG Emotion Recognition

Recently, physiological data such as electroencephalography (EEG) signals have attracted significant attention in affective computing. In this context, the main goal is to design an automated model that can assess emotional states. Lately, deep neural networks have shown promising performance in emotion recognition tasks. However, designing a deep architecture that can extract practical information from raw data is still a challenge. Here, we introduce a deep neural network that acquires interpretable physiological representations by a hybrid structure of spatio-temporal encoding and recurrent attention network blocks. Furthermore, a preprocessing step is applied to the raw data using graph signal processing tools to perform graph smoothing in the spatial domain. We demonstrate that our proposed architecture exceeds state-of-the-art results for emotion classification on the publicly available DEAP dataset. To explore the generality of the learned model, we also evaluate the performance of our architecture towards transfer learning (TL) by transferring the model parameters from a specific source to other target domains. Using DEAP as the source dataset, we demonstrate the effectiveness of our model in performing cross-modality TL and improving emotion classification accuracy on DREAMER and the Emotional English Word (EEWD) datasets, which involve EEG-based emotion classification tasks with different stimuli

Metoda raspodijeljenog zajedničkog prostornog uzorka za klasifikaciju EEG signala sučelja mozak-računalo u jednoj procjeni

Common spatial pattern (CSP) method is highly successful in calculating spatial filters for motor imagery-based brain-computer interfaces (BCIs). However, conventional CSP algorithm is based on a single wide frequency band with a poor frequency selectivity which will lead to poor recognition accuracy. To solve this problem, a novel Partitioned CSP (PCSP) algorithm is proposed to find the most relevant spatial frequency distribution with motor imaginary, so that the algorithm has flexible frequency selectivity. Firstly, we partition the dataset into frequency components using a constant-bandwidth filters bank. Then, a features selection method based on the Bhattacharyya distance is adopted for PCSP features ranking, selection and evaluation. Subsequently, the PCSP features are used to obtain scores which reflect the classification capability and being used for EEG signal classification. The experimental results on 4 subjects showed that the PCSP method significantly outperforms the other two existing approaches based on conventional CSP and Common Spatio-Spectral Pattern (CSSP).Metoda zajedničkog prostornog uzorka (eng. common spatial pattern, CSP) je vrlo uspješna u izračunu prostornih filtara za sučelja mozak-računalo zasnovana na motoričkoj predodžbi (eng. brain-computer interface, BCI). Međutim, konvencionalni CSP algoritam je zasnovan na jednom širokom pojasu frekvencija s lošom selektivnosti frekvencija što rezultira manjom točnošću prepoznavanja. Za rješavanje navedenog problema u ovom radu je predložen novi raspodijeljeni CSP algoritam za pronalaženje najznačajnije prostorno frekvencijske distribucije s motoričkom predodžbom, sa svojstvima fleksibilne selektivnosti frekvencije. Početna faza metode je podjela podataka na frekvencijske komponente korištenjem filtarskog sloga s konstantnom širinom pojasa. Potom, prilagođena je metoda odabira svojstava zasnovana na Bhattacharyya udaljenosti za rangiranje, odabir i evaluaciju PCSP svojstava. Zatim, PCSP svojstva se koriste za dobivanje ocjena koje reflektiraju mogućnosti klasifikacije te za klasifikaciju EEG signala. Eksperimentalni rezultati na 4 ispitanika pokazali su da PCSP metoda po performansama značajno nadmašuje druga dva postojeća pristupa zasnovana na konvencionalnom CSP-u i zajedničkom prostor-spektralnom uzorku (eng. common spatio-spectral pattern, CSSP)

Adaptive feature extraction in EEG-based motor imagery BCI: tracking mental fatigue

Objective: Electroencephelogram (EEG) signals are non-stationary. This could be due to the internal fluctuation of brain states such as fatigue, frustration, etc. This necessitates the development of adaptive Brain-Computer Interfaces (BCI) whose performance does not deteriorate significantly with the adversary change in the cognitive state. In this paper, we put forward an unsupervised adaptive scheme to adapt the feature extractor of motor imagery (MI) BCIs by tracking the fatigue level of the user. Approach: Eleven subjects participated in the study during which they accomplished MI tasks while self-reporting their perceived levels of mental fatigue. Out of the 11 subjects, only 6 completed the whole experiment, while the others quit in the middle because of experiencing high fatigue. The adaptive feature extractor is attained through the adaptation of the common spatial patterns (CSP), one of the most popular feature extraction algorithms in EEG-based BCIs. The proposed method was analyzed in two ways: oine and in near real-time. The separability of the MI EEG features extracted by the proposed adaptive CSP (ADCSP) has been compared with that by the conventional CSP (C-CSP) and another CSP based adaptive method (ACSP) in terms of: Davies Bouldin Index (DBI), Fisher Score (FS) and Dunn's Index (DI). Results: Experimental results show significant improvement in the separability of MI EEG features extracted by ADCSP as compared to that by C-CSP and ACSP. Signficance: Collectively, the results of the experiments in this study suggest that adapting CSP based on mental fatigue can improve the class separability of MI EEG features