Enhancing Motor Imagery Decoding in Brain Computer Interfaces using Riemann Tangent Space Mapping and Cross Frequency Coupling

Objective: Motor Imagery (MI) serves as a crucial experimental paradigm within the realm of Brain Computer Interfaces (BCIs), aiming to decoding motor intentions from electroencephalogram (EEG) signals. Method: Drawing inspiration from Riemannian geometry and Cross-Frequency Coupling (CFC), this paper introduces a novel approach termed Riemann Tangent Space Mapping using Dichotomous Filter Bank with Convolutional Neural Network (DFBRTS) to enhance the representation quality and decoding capability pertaining to MI features. DFBRTS first initiates the process by meticulously filtering EEG signals through a Dichotomous Filter Bank, structured in the fashion of a complete binary tree. Subsequently, it employs Riemann Tangent Space Mapping to extract salient EEG signal features within each sub-band. Finally, a lightweight convolutional neural network is employed for further feature extraction and classification, operating under the joint supervision of cross-entropy and center loss. To validate the efficacy, extensive experiments were conducted using DFBRTS on two well-established benchmark datasets: the BCI competition IV 2a (BCIC-IV-2a) dataset and the OpenBMI dataset. The performance of DFBRTS was benchmarked against several state-of-the-art MI decoding methods, alongside other Riemannian geometry-based MI decoding approaches. Results: DFBRTS significantly outperforms other MI decoding algorithms on both datasets, achieving a remarkable classification accuracy of 78.16% for four-class and 71.58% for two-class hold-out classification, as compared to the existing benchmarks.Comment: 22 pages, 7 figure

Support matrix machine: A review

Support vector machine (SVM) is one of the most studied paradigms in the realm of machine learning for classification and regression problems. It relies on vectorized input data. However, a significant portion of the real-world data exists in matrix format, which is given as input to SVM by reshaping the matrices into vectors. The process of reshaping disrupts the spatial correlations inherent in the matrix data. Also, converting matrices into vectors results in input data with a high dimensionality, which introduces significant computational complexity. To overcome these issues in classifying matrix input data, support matrix machine (SMM) is proposed. It represents one of the emerging methodologies tailored for handling matrix input data. The SMM method preserves the structural information of the matrix data by using the spectral elastic net property which is a combination of the nuclear norm and Frobenius norm. This article provides the first in-depth analysis of the development of the SMM model, which can be used as a thorough summary by both novices and experts. We discuss numerous SMM variants, such as robust, sparse, class imbalance, and multi-class classification models. We also analyze the applications of the SMM model and conclude the article by outlining potential future research avenues and possibilities that may motivate academics to advance the SMM algorithm

A new feature and associated optimal spatial filter for EEG signal classification: Waveform Length

International audienceIn this paper, we introduce Waveform Length (WL), a new feature for ElectroEncephaloGraphy (EEG) signal classification which measures the signal complexity. We also propose the Waveformlength Optimal Spatial Filter (WOSF), an optimal spatial filter to classify EEG signals based on WL features. Evaluations on 15 subjects suggested that WOSF with WL features provide performances that are competitive with that of Common Spatial Patterns (CSP) with Band Power (BP) features, CSP being the optimal spatial filter for BP features. More interestingly, our results suggested that combining WOSF with CSP features leads to classification performances that are significantly better than that of CSP alone (80% versus 77% average accuracy respectively)

The influence of graphical user interface on motion onset brain-computer interface performance and the effect of data augmentation on motor imagery brain-computer interface

Motor Imagery Brain Computer Interface (MI BCI) is one of the most frequently used BCI modalities, due to the versatility of its applications. However, it still has unresolved issues like time-consuming calibration, low information transfer rate, and inconsistent performance across individuals. Combining MI BCI with Motion Onset Visual Evoked Potential (mVEP) BCI in a hybrid structure may solve some of these problems. Combining MI BCI with more robust mVEP BCI, would increase the degrees of freedom thereby increasing the information transfer rate, and would also indirectly improve intrasubject consistency in performance by replacing some MI-based tasks with mVEP. Unfortunately, due to Covid -19 pandemic experimental research on hybrid BCI was not possible, therefore this thesis focuses on two BCI separately. Chapter 1 provides an overview of different BCIs modalities and the underlying neurophysiological principles, followed by the objectives of the thesis. The research contributions are also highlighted. Finally, the thesis outlines are presented at the end of this chapter. Chapter 2 presents a comprehensive state of the art to the thesis, drawing on a wide range of literature in relevant fields. Specifically, it delves into MI BCI, mVEP BCI, Deep Learning, Transfer Learning (TL), Data Augmentation (DA) and Generative Adversarial Networks (GANs). Chapter 3 investigates the effect of graphical elements, in online and offline experiments. In the offline experiment, graphical elements such as the color, size, position, and layout were explored. Replacing a default red moving bar with a green and blue bar, changing the background color from white to gray, and using smaller visual angles did not lead to statistically significant improvement in accuracy. However, the effect size of η2 (0.085) indicated a moderate effect for these changes of graphical factors. Similarly, no statistically significant difference was found for the two different layouts in online experiments. Overall, the mVEP BCI has achieved a classification accuracy of approximately 80%, and it is relatively impervious to changes in graphical interface parameters. This suggests that mVEP is a promising candidate for a hybrid BCI system combined with MI, that requires dynamic, versatile graphical design features. In Chapter 4, various DA methods are explored, including Segmentation and Recombination in Time Domain, Segmentation and Recombination in Time-Frequency Domain, and Spatial Analogy. These methods are evaluated based on three feature extraction approaches: Common Spatial Patterns, Time Domain Parameters (TDP), and Band Power. The evaluation was conducted using a validated BCI set, namely the BCI Competition IV dataset 2a, as well as a dataset obtained from our research group. The methods are effective when a small dataset of single subject are available. All three DA methods significantly affect the performance of the TDP feature extraction method. Chapter 5 explored the use of GANs for DA in combination with TL and cropped training strategies using ShallowFBCSP classifier. It also used the same validated dataset (BCI competition IV dataset 2a) as in Chapter 4. In contrast to DA method explored in Chapter 4, this DA is suitable for larger datasets and for generalizing training based on other people’s data. Applying GAN-based DA to the dataset resulted on average in a 2% improvement in average accuracy (from 68.2% to 70.7%). This study provides a novel method to enable MI GAN training with only 40 trials per participant with the rest 8 people’s data for TL, addressing the data insufficiency issue for GANs. The evaluation of generated artificial trials revealed the importance of inter-class differences in MI patterns, which can be easily identified by GANs. Overall the thesis addressed the main practical issues of both mVEP and MI BCI paving the way for their successful combination in future experiments