A Transformer-based deep neural network model for SSVEP classification

Steady-state visual evoked potential (SSVEP) is one of the most commonly used control signal in the brain-computer interface (BCI) systems. However, the conventional spatial filtering methods for SSVEP classification highly depend on the subject-specific calibration data. The need for the methods that can alleviate the demand for the calibration data become urgent. In recent years, developing the methods that can work in inter-subject classification scenario has become a promising new direction. As the popular deep learning model nowadays, Transformer has excellent performance and has been used in EEG signal classification tasks. Therefore, in this study, we propose a deep learning model for SSVEP classification based on Transformer structure in inter-subject classification scenario, termed as SSVEPformer, which is the first application of the transformer to the classification of SSVEP. Inspired by previous studies, the model adopts the frequency spectrum of SSVEP data as input, and explores the spectral and spatial domain information for classification. Furthermore, to fully utilize the harmonic information, an extended SSVEPformer based on the filter bank technology (FB-SSVEPformer) is proposed to further improve the classification performance. Experiments were conducted using two open datasets (Dataset 1: 10 subjects, 12-class task; Dataset 2: 35 subjects, 40-class task) in the inter-subject classification scenario. The experimental results show that the proposed models could achieve better results in terms of classification accuracy and information transfer rate, compared with other baseline methods. The proposed model validates the feasibility of deep learning models based on Transformer structure for SSVEP classification task, and could serve as a potential model to alleviate the calibration procedure in the practical application of SSVEP-based BCI systems

Robust asynchronous control of ERP-Based brain-Computer interfaces using deep learning

Producción CientíficaBackground and Objective. Brain-computer interfaces (BCI) based on event-related potentials (ERP) are a promising technology for alternative and augmented communication in an assistive context. However, most approaches to date are synchronous, requiring the intervention of a supervisor when the user wishes to turn his attention away from the BCI system. In order to bring these BCIs into real-life applications, a robust asynchronous control of the system is required through monitoring of user attention. Despite the great importance of this limitation, which prevents the deployment of these systems outside the laboratory, it is often overlooked in research articles. This study was aimed to propose a novel method to solve this problem, taking advantage of deep learning for the first time in this context to overcome the limitations of previous strategies based on hand-crafted features. Methods. The proposed method, based on EEG-Inception, a novel deep convolutional neural network, divides the problem in 2 stages to achieve the asynchronous control: (i) the model detects user’s control state, and (ii) decodes the command only if the user is attending to the stimuli. Additionally, we used transfer learning to reduce the calibration time, even exploring a calibration-less approach. Results. Our method was evaluated with 22 healthy subjects, analyzing the impact of the calibration time and number of stimulation sequences on the system’s performance. For the control state detection stage, we report average accuracies above 91% using only 1 sequence of stimulation and 30 calibration trials, reaching a maximum of 96.95% with 15 sequences. Moreover, our calibration-less approach also achieved suitable results, with a maximum accuracy of 89.36%, showing the benefits of transfer learning. As for the overall asynchronous system, which includes both stages, the maximum information transfer rate was 35.54 bpm, a suitable value for high-speed communication. Conclusions. The proposed strategy achieved higher performance with less calibration trials and stimulation sequences than former approaches, representing a promising step forward that paves the way for more practical applications of ERP-based spellers.Ministerio de Ciencia, Innovación y Universidades - Agencia Estatal de Investigación (grants PID2020-115468RB-I00 and RTC2019-007350-1)Comisión Europea - Fondo Europeo de Desarrollo Regional (cooperation programme Interreg V-A Spain-Portugal POCTEP 2014–2020

Cross-subject dual-domain fusion network with task-related and task-discriminant component analysis enhancing one-shot SSVEP classification

This study addresses the significant challenge of developing efficient decoding algorithms for classifying steady-state visual evoked potentials (SSVEPs) in scenarios characterized by extreme scarcity of calibration data, where only one calibration is available for each stimulus target. To tackle this problem, we introduce a novel cross-subject dual-domain fusion network (CSDuDoFN) incorporating task-related and task-discriminant component analysis (TRCA and TDCA) for one-shot SSVEP classification. The CSDuDoFN framework is designed to comprehensively transfer information from source subjects, while TRCA and TDCA are employed to exploit the single available calibration of the target subject. Specifically, we develop multi-reference least-squares transformation (MLST) to map data from both source subjects and the target subject into the domain of sine-cosine templates, thereby mitigating inter-individual variability and benefiting transfer learning. Subsequently, the transformed data in the sine-cosine templates domain and the original domain data are separately utilized to train a convolutional neural network (CNN) model, with the adequate fusion of their feature maps occurring at distinct network layers. To further capitalize on the calibration of the target subject, source aliasing matrix estimation (SAME) data augmentation is incorporated into the training process of the ensemble TRCA (eTRCA) and TDCA models. Ultimately, the outputs of the CSDuDoFN, eTRCA, and TDCA are combined for SSVEP classification. The effectiveness of our proposed approach is comprehensively evaluated on three publicly available SSVEP datasets, achieving the best performance on two datasets and competitive performance on one. This underscores the potential for integrating brain-computer interface (BCI) into daily life.Comment: 10 pages,6 figures, and 3 table

BCI applications based on artificial intelligence oriented to deep learning techniques

A Brain-Computer Interface, BCI, can decode the brain signals corresponding to the intentions of individuals who have lost neuromuscular connection, to reestablish communication to control external devices. To this aim, BCI acquires brain signals as Electroencephalography (EEG) or Electrocorticography (ECoG), uses signal processing techniques and extracts features to train classifiers for providing proper control instructions. BCI development has increased in the last decades, improving its performance through the use of different signal processing techniques for feature extraction and artificial intelligence approaches for classification, such as deep learning-oriented classifiers. All of these can assure more accurate assistive systems but also can enable an analysis of the learning process of signal characteristics for the classification task. Initially, this work proposes the use of a priori knowledge and a correlation measure to select the most discriminative ECoG signal electrodes. Then, signals are processed using spatial filtering and three different types of temporal filtering, followed by a classifier made of stacked autoencoders and a softmax layer to discriminate between ECoG signals from two types of visual stimuli. Results show that the average accuracy obtained is 97% (+/- 0.02%), which is similar to state-of-the-art techniques, nevertheless, this method uses minimal prior physiological and an automated statistical technique to select some electrodes to train the classifier. Also, this work presents classifier analysis, figuring out which are the most relevant signal features useful for visual stimuli classification. The features and physiological information such as the brain areas involved are compared. Finally, this research uses Convolutional Neural Networks (CNN) or Convnets to classify 5 categories of motor tasks EEG signals. Movement-related cortical potentials (MRCPs) are used as a priori information to improve the processing of time-frequency representation of EEG signals. Results show an increase of more than 25% in average accuracy compared to a state-of-the-art method that uses the same database. In addition, an analysis of CNN or ConvNets filters and feature maps is done to and the most relevant signal characteristics that can help classify the five types of motor tasks.DoctoradoDoctor en Ingeniería Eléctrica y Electrónic

Classification of Frequency and Phase Encoded Steady State Visual Evoked Potentials for Brain Computer Interface Speller Applications using Convolutional Neural Networks

Over the past decade there have been substantial improvements in vision based Brain-Computer Interface (BCI) spellers for quadriplegic patient populations. This thesis contains a review of the numerous bio-signals available to BCI researchers, as well as a brief chronology of foremost decoding methodologies used to date. Recent advances in classification accuracy and information transfer rate can be primarily attributed to time consuming patient specific parameter optimization procedures. The aim of the current study was to develop analysis software with potential ‘plug-in-and-play’ functionality. To this end, convolutional neural networks, presently established as state of the art analytical techniques for image processing, were utilized. The thesis herein defines deep convolutional neural network architecture for the offline classification of phase and frequency encoded SSVEP bio-signals. Networks were trained using an extensive 35 participant open source Electroencephalographic (EEG) benchmark dataset (Department of Bio-medical Engineering, Tsinghua University, Beijing). Average classification accuracies of 82.24% and information transfer rates of 22.22 bpm were achieved on a BCI naïve participant dataset for a 40 target alphanumeric display, in absence of any patient specific parameter optimization

Enhancing the Decoding Performance of Steady-State Visual Evoked Potentials based Brain-Computer Interface

Non-invasive Brain-Computer Interfaces (BCIs) based on steady-state visual evoked potential (SSVEP) responses are the most widely used BCI. SSVEP are responses elicited in the visual cortex when a user gazes at an object flickering at a certain frequency. In this thesis, we investigate different BCI system design parameters for enhancing the detection of SSVEP such as change in inter-stimulus distance (ISD), EEG channels, detection algorithms and training methodologies. Closely placed SSVEP stimuli compete for neural representations. This influences the performance and limits the flexibility of the stimulus interface. In this thesis, we study the influence of changing ISD on the decoding performance of an SSVEP BCI. We propose: (i) a user-specific channel selection method and (ii) using complex spectrum features as input to a convolutional neural network (C-CNN) to overcome this challenge. We also evaluate the proposed C-CNN method in a user-independent (UI) training scenario as this will lead to a minimal calibration system and provide the ability to run inference in a plug-and-play mode. The proposed methods were evaluated on a 7-class SSVEP dataset collected on 21 healthy participants (Dataset 1). The UI method was also assessed on a publicly available 12-class dataset collected on 10 healthy participants (Dataset 2). We compared the proposed methods with canonical correlation analysis (CCA) and CNN classification using magnitude spectrum features (M-CNN). We demonstrated that the user-specific channel set (UC) is robust to change in ISD (viewing angles of 5.24ᵒ, 8.53ᵒ, and 12.23ᵒ) compared to the classic 3-channel set (3C - O1, O2, Oz) and 6-channel set (6C - PO3, PO4, POz, O1, O2, Oz). A significant improvement in accuracy of over 5% (p=0.001) and a reduction in variation of 56% (p=0.035) was achieved across ISDs using the UC set compared to the 3C set and 6C set. Secondly, the proposed C-CNN method obtained a significantly higher classification accuracy across ISDs and window lengths compared to M-CNN and CCA. The average accuracy of the C-CNN increased by over 12.8% compared to CCA and an increase of over 6.5% compared to the M-CNN for the closest ISD across all window lengths was achieved. Thirdly, the C-CNN method achieved the highest accuracy in both UD and UI training scenarios on both 7-class and 12-class SSVEP Datasets. The overall accuracies of the different methods for 1 s window length for Dataset 1 were: CCA: 69.1±10.8%, UI-M-CNN: 73.5±16.1%, UI-C-CNN: 81.6±12.3%, UD-M-CNN: 87.8±7.6% and UD-C-CNN: 92.5±5%. And for Dataset 2 were: CCA: 62.7±21.5%, UI-M-CNN: 70.5±22%, UI-C-CNN: 81.6±18%, UD-M-CNN: 82.8±16.7%, and UD-C-CNN: 92.3±11.1%. In summary, using the complex spectrum features, the C-CNN likely learned to use both frequency and phase related information to classify SSVEP responses. Therefore, the CNN can be trained independent of the ISD resulting in a model that generalizes to other ISDs. This suggests that the proposed methods are robust to changes in inter-stimulus distance for SSVEP detection and provides increased flexibility for user interface design of SSVEP BCIs for commercial applications. Finally, the UI method provides a virtually calibration free approach to SSVEP BCI