Robust Brain-computer interface for virtual Keyboard (RoBIK): project results

Special issue : ANR TECSAN : Technologies for Health and AutonomyNational audienceBrain-ComputerInterface (BCI)is a technology that translatesthe brain electrical activity into a command for a device such as a robotic arm, a wheelchair or a spelling device. BCIs have long been described as an assistive technology forseverely disabled patients because they completely bypass the need for muscular activity. The clinical reality is however dramatically different and most patients who use BCIs today are doing so as part of constraining clinical trials. To achieve the technological transfer from bench to bedside, BCI must gain ease of use and robustness of bothmeasure (electroencephalography [EEG]) and interface (signal processing and applications).TheRobustBrain-computerInterface for virtual Keyboard (RoBIK) project aimed atthe development of aBCIsystemfor communication that could be used on a daily basis by patientswithoutthe help of a trained teamofresearchers.To guide further developments cliniciansfirst assessed patients' needs.The prototype subsequently developed consisted in a 14 felt-pad electrodes EEG headsetsampling at 256Hz by an electronic component capable of transmitting signals wirelessly. The application was a virtual keyboard generating a novelstimulation paradigm to elicit P300 Evoked Related Potentials(ERPs) for communication. Raw EEG signals were treated with OpenViBE open-source software including novelsignal processing and stimulation techniques

Enhancing Motor Imagery Decoding in Brain Computer Interfaces using Riemann Tangent Space Mapping and Cross Frequency Coupling

Objective: Motor Imagery (MI) serves as a crucial experimental paradigm within the realm of Brain Computer Interfaces (BCIs), aiming to decoding motor intentions from electroencephalogram (EEG) signals. Method: Drawing inspiration from Riemannian geometry and Cross-Frequency Coupling (CFC), this paper introduces a novel approach termed Riemann Tangent Space Mapping using Dichotomous Filter Bank with Convolutional Neural Network (DFBRTS) to enhance the representation quality and decoding capability pertaining to MI features. DFBRTS first initiates the process by meticulously filtering EEG signals through a Dichotomous Filter Bank, structured in the fashion of a complete binary tree. Subsequently, it employs Riemann Tangent Space Mapping to extract salient EEG signal features within each sub-band. Finally, a lightweight convolutional neural network is employed for further feature extraction and classification, operating under the joint supervision of cross-entropy and center loss. To validate the efficacy, extensive experiments were conducted using DFBRTS on two well-established benchmark datasets: the BCI competition IV 2a (BCIC-IV-2a) dataset and the OpenBMI dataset. The performance of DFBRTS was benchmarked against several state-of-the-art MI decoding methods, alongside other Riemannian geometry-based MI decoding approaches. Results: DFBRTS significantly outperforms other MI decoding algorithms on both datasets, achieving a remarkable classification accuracy of 78.16% for four-class and 71.58% for two-class hold-out classification, as compared to the existing benchmarks.Comment: 22 pages, 7 figure

A Data Imputation Method for Matrices in the Symmetric Positive Definite Manifold

International audienc

Riemannian approaches in Brain-Computer Interfaces: a review

International audienceAlthough promising from numerous applications, current Brain-Computer Interfaces (BCIs) still suffer from a number of limitations. In particular, they are sensitive to noise, outliers and the non-stationarity of ElectroEncephaloGraphic (EEG) signals, they require long calibration times and are not reliable. Thus, new approaches and tools, notably at the EEG signal processing and classification level, are necessary to address these limitations. Riemannian approaches, spearheaded by the use of covariance matrices, are such a very promising tool slowly adopted by a growing number of researchers. This article, after a quick introduction to Riemannian geometry and a presentation of the BCI-relevant manifolds, reviews how these approaches have been used for EEG-based BCI, in particular for feature representation and learning, classifier design and calibration time reduction. Finally, relevant challenges and promising research directions for EEG signal classification in BCIs are identified, such as feature tracking on manifold or multi-task learning

On Riemannian tools for classification improvement in Brain-Computer Interfaces

A Brain Computer Interface (BCI) or Brain Machine Interface (BMI) is a device that allows the exchange of information between the brain of a person and a computer without the need of physical interaction. This technology promises to change the way in which we interact with machines, but it is not yet affordable, robust or quick enough to substitute other classic human to machine interfaces for the general public. This being said, the lack of need of interaction makes them a very promising solution that would provide people with severe motor disabilities with a new way of interacting with their surroundings, improving their quality of life. The most extended method of extracting information about brain activity and the one used for this project is the Electroencefalogram (EEG). This device consists of multiple electrodes mounted on a helmet-like structure that is placed on the user’s scalp. The electrodes detect the sum of action potentials from large populations of neurons on the brain’s cortex. The main advantages of this technique are the relative low cost of the device, portability, and the high temporal resolution and ease of use of a non invasive technique. This is not free of disadvantages, as the method suffers from a low signal to noise ratio, low robustness to interference, low spatial resolution and the effects of inter and intra session drift, that is, the movement of the electrodes during and between sessions produce variations on the acquisition of the signal. There are also multiple paradigms in the field of BCI, each one of them focusing on a different brain signal. This work is centered around the Motor Imagery Brain Computer Interface (MI-BCI), which differs from other BCIs in the fact that it directly decodes the intention of the user without the need of inducing a specific response in the brain by presenting an stimulus. This approach is considered to be more natural and can be more comfortable, but also requires a higher level of mental effort and proficiency from part of the user. The MI-BCI is based on a signal of unknown origin that is produced on the sensorymotor cortex, responsible for voluntary movements and touch among others, the Sensorimotor Rhythms (SMR). This signal is atenuated when the person performs or thinks about performing a movement, which is called an Event Related Desynchronization (ERD) and amplified when going back to the idling state, an Event Related Synchronization (ERS). As the brain is a distributed system, the origin of these events can be estimated and is related to the movement that the person imagined. In an implementation, these movements are limited to a discrete set of posibilities and each one is mapped to a computer instruction, allowing the unidirectional transfer of information between brain and machine. The classical machine learning approach to this problem has been to use very specific signal processing techniques to extract relevant features for this problem that can then be fed to a general classification algorithm. The main tecnique is known as Common Spatial Patterns (CSP) followed by classification with Linear Discriminant Analysis (LDA) or Support Vector Machine (SVM). This has some advantages such as a relative low requirement of training samples, but also lacks the capability of generalisation, and a system fine tuned for one user cannot be used for other users or even for another session from the same user reliably. In this work we study an alternative framework that uses the covariance matrices of the EEG signals as observations and exploits the Riemannian geometry of Symmetric Positive Definite (SPD) matrices to classify them in their natural space. This is not only a more general signal processing approach that has been used in other fields of research, but also opens the possibility of transfering some information between users and sessions, which may result in a more robust system or in a system that requires less data for training. This is crucial for the usability of MI-BCI because recording a training session before each use of the system is mentally exhausting and time consuming.Universidad de Sevilla. Máster Universitario en Ingeniería de Telecomunicació

The classification of wink-based eeg signals by means of transfer learning models

Stroke is one of the dominant causes of impairme nt. An estimation of half post-stroke survivors suffer from a severe motor or cognitive deterioration, that affects the functionality of the affected parts of the body, which in turn, prevents the patients from carrying out Activities of Daily Living (ADL). EEG signals which contains information on the activities carried out by a human that is widely used in many applications of BCI technologies which offers a means of controlling exoskeletons or automated orthosis to facilitate their ADL. Although motor imagery signals have been used in assisting the hand grasping motion amongst others motions, nonetheless, such signals are often difficult to be generated. It is non-trivial to note that EEG-based signals for instance, winking could mitigate the aforesaid issue. Nevertheless, extracting and attaining significant features from EEG signals are also somewhat challenging. The utilization of deep learning, particularly Transfer Learning (TL), have been demonstrated in the literature to b e able to provide seamless extraction of such signals in a myria d of various applications. Hitherto, limited studies have investigated the classification of wink-based EEG signals through TL accompanied by classical Machine Learning (ML) pipelines. This study aimed to explore the performance of different pre-processing methods, namely Fast Fourier Transform, Short-Time Fourier Transform, Discrete Wavelet Transform, and Continuous Wavelet Transform (CWT) that could allow TL models to extract features from the images generated and classify through selected classical ML algorithms . These pre-processing methods were utilized to convert the digital signals into respective images of all the right and left winking EEG signals along with no winking signals that were collected from ten (6 males and 4 females, aged between 22 and 29) subjects. The implementation of pre-processing algorithms has been demonstrated to be able to mitigate the signal noises that arises from the winking signals without the need for the use signal filtering algorithms. A new form of input which consists of scalogram and spectrogram images that represents both time and frequency domains , are then introduced in the classification of wink-based EEG signals. Different TL models were exploited to extract features from the transformed EEG signals. The features extracted were then classified through three classical ML models, namely Support Vector Machine, k -Nearest Neighbour (k-NN) and Random Forest to determine the best pipeline for wink -based EEG signals. The hyperparameters of the ML models were tuned through a 5-fold crossvalidation technique via an exhaustive grid search approach. The training, validation and testing of the models were split with a stratified ratio of 60:20:20, respectively. The results obtained from the TL-ML pipelines were evaluated in terms of classification accuracy, Precision, Recall, F1-Score and confusion matrix. It was demonstrated from the simulation investigation that the CWT model could yield a better signal transformation amongst the preprocessing algorithms. In addition, amongst the eighteen TL models evaluated based on the CWT transformation, fourteen was f ound to be able to extract the features reasonable, i.e., VGG16, VGG19, ResNet101, ResNet101 V2, ResNet152, ResNet152 V2, Inception V3, Inception ResNet V2, Xception, MobileNetV2, DenseNet 121, DenseNet 169, NasNetMobile and NasNetLarge. Whilst it was observed that the optimized k-NN model based on the aforesaid pipeline could achieve a classification accuracy of 100% for the training, validation, and tes t data. Nonetheless, upon carrying out a robustness test on new data, it was demonstrated that the CWT-NasNetMobile-kNN pipeline yielded the best performance. Therefore, it could be concluded that the proposed CWT-NasNetMobile-k-NN pipeline is suitable to be adopted to classify -winkbased EEG signals for BCI applications,for instance a grasping exoskeleton