Integrating dynamic stopping, transfer learning and language models in an adaptive zero-training ERP speller

Objective. Most BCIs have to undergo a calibration session in which data is recorded to train decoders with machine learning. Only recently zero-training methods have become a subject of study. This work proposes a probabilistic framework for BCI applications which exploit event-related potentials (ERPs). For the example of a visual P300 speller we show how the framework harvests the structure suitable to solve the decoding task by (a) transfer learning, (b) unsupervised adaptation, (c) language model and (d) dynamic stopping. Approach. A simulation study compares the proposed probabilistic zero framework (using transfer learning and task structure) to a state-of-the-art supervised model on n = 22 subjects. The individual influence of the involved components (a)–(d) are investigated. Main results. Without any need for a calibration session, the probabilistic zero-training framework with inter-subject transfer learning shows excellent performance—competitive to a state-of-the-art supervised method using calibration. Its decoding quality is carried mainly by the effect of transfer learning in combination with continuous unsupervised adaptation. Significance. A high-performing zero-training BCI is within reach for one of the most popular BCI paradigms: ERP spelling. Recording calibration data for a supervised BCI would require valuable time which is lost for spelling. The time spent on calibration would allow a novel user to spell 29 symbols with our unsupervised approach. It could be of use for various clinical and non-clinical ERP-applications of BCI

Dynamic stopping in a calibration-less P300 speller

Even though the P300 based speller has proved to be usable by real patients, it is not a user-friendly system. The necesarry calibration session and slow spelling make the system tedious. We present a machine learning approach to P300 spelling that enables us to remove the calibration session. We achieve this by a combination of unsupervised training, transfer learning across subjects and language models. On top of that, we can increase the spelling speed by incorporating a dynamic stopping approach. This yields a P300 speller that works instantly and with high accuracy and spelling speed, even for unseen subjects

Optimizing P300-speller sequences by RIP-ping groups apart

International audienceSo far P300-speller design has put very little emphasis on the design of optimized flash patterns, a surprising fact given the importance of the sequence of flashes on the selection outcome. Previous work in this domain has consisted in studying consecutive flashes, to prevent the same letter or its neighbors from flashing consecutively. To this effect, the flashing letters form more random groups than the original row-column sequences for the P300 paradigm, but the groups remain fixed across repetitions. This has several important consequences, among which a lack of discrepancy between the scores of the different letters. The new approach proposed in this paper accumulates evidence for individual elements, and optimizes the sequences by relaxing the constraint that letters should belong to fixed groups across repetitions. The method is inspired by the theory of Restricted Isometry Property matrices in Compressed Sensing, and it can be applied to any display grid size, and for any target flash frequency. This leads to P300 sequences which are shown here to perform significantly better than the state of the art, in simulations and online tests

True zero-training brain-computer interfacing: an online study

Despite several approaches to realize subject-to-subject transfer of pre-trained classifiers, the full performance of a Brain-Computer Interface (BCI) for a novel user can only be reached by presenting the BCI system with data from the novel user. In typical state-of-the-art BCI systems with a supervised classifier, the labeled data is collected during a calibration recording, in which the user is asked to perform a specific task. Based on the known labels of this recording, the BCI's classifier can learn to decode the individual's brain signals. Unfortunately, this calibration recording consumes valuable time. Furthermore, it is unproductive with respect to the final BCI application, e.g. text entry. Therefore, the calibration period must be reduced to a minimum, which is especially important for patients with a limited concentration ability. The main contribution of this manuscript is an online study on unsupervised learning in an auditory event-related potential (ERP) paradigm. Our results demonstrate that the calibration recording can be bypassed by utilizing an unsupervised trained classifier, that is initialized randomly and updated during usage. Initially, the unsupervised classifier tends to make decoding mistakes, as the classifier might not have seen enough data to build a reliable model. Using a constant re-analysis of the previously spelled symbols, these initially misspelled symbols can be rectified posthoc when the classifier has learned to decode the signals. We compare the spelling performance of our unsupervised approach and of the unsupervised posthoc approach to the standard supervised calibration-based dogma for n = 10 healthy users. To assess the learning behavior of our approach, it is unsupervised trained from scratch three times per user. Even with the relatively low SNR of an auditory ERP paradigm, the results show that after a limited number of trials (30 trials), the unsupervised approach performs comparably to a classic supervised model

Subject-independent P300 BCI using ensemble classifier, dynamic stopping and adaptive learning

© 2017 IEEE. Brain-computer interfaces (BCIs) are used to assist people, especially those with verbal or physical disabilities, communicate with the computer to indicate their selections, control a device or answer questions only by their mere thoughts. Due to the noisy nature of brain signals, the required time for each experimental session must be lengthened to reach satisfactory accuracy. This is the trade-off between the speed and the precision of a BCI system. In this paper, we propose a unified method which is the integration of ensemble classifier, dynamic stopping, and adaptive learning. We are able to both increase the accuracy, as well as to reduce the spelling time of the P300-Speller. Another merit of our study is that it does not require the training phase for any new subject, hence eliminates the extensively time-consuming process for learning purposes. Experimental results show that we achieve the averaged bit rate boost up of 182% on 15 subjects. Our best achieved accuracy is 95.95% by using 7.49 flashing iterations and our best achieved bit rate is 40.87 bits/min with 83.99% accuracy and 3.64 iterations. To the best of our knowledge, these results outperformed most of the related P300-based BCI studies

Improving Brain-Computer Interface Performance By Using Dynamic Methods Based on Analysis of Cognitive State

Communication for individuals with severe motor and speech impairments can be very difficult and they find the need for the assistance of augmentative and alternative communication (AAC) systems. Common commercialized AAC systems require some amount of voluntary control and are unusable by individuals with severe disabilities. Non-invasive brain-computer interfaces (BCIs) are an emerging means of communication for people with severe motor and speech impairments. BCIs allow the user to make selections on the computer just using their brain signals, electroencephalogram (EEG). However, although they are revolutionary for individuals that cannot control other available AAC systems, BCIs have several limitations. Two major limitations of BCIs are: 1) BCIs are static/synchronous in nature; 2) BCIs are susceptible to changes in user attention. Since people in the populations that need BCI technology the most (e.g. amyotrophic lateral sclerosis ;ALS) may experience attention impairments, incorporating attention-monitoring features into the BCI would improve BCI performance by reducing errors in these populations. This research presents two dynamic methods developed to help the BCI become more user-aware and allow users to control the BCI at their own pace. Using a well-established negative correlation between alpha band power in the EEG and attention, the first method used alpha band analysis to detect losses in user attention and abstained selections that were unattended to reduce potential errors. The second method, called P300-Certainty, abstained selections that do not reach a specified confidence level. To test both methods, off-line analysis was performed on recorded EEG from 30 subjects using the BCI for spelling. Subjects selected 9 sentences and at least 23 characters per sentence with additional corrections. Alpha band analysis and P300-Certainty were tested off-line, separately and together, on this dataset to determine their efficacy at increasing BCI accuracy by abstaining potential errors. In addition, P300-Certainty was implemented in a BCI-facilitated cognitive assessment to reduce potential errors, as well as, only choosing selections when they reach a specified confidence level. The on-line performance of P300-Certainty was calculated from this data. Alpha band analysis was performed off-line on this on-line data to determine its efficacy at increasing P300-Certainty on-line BCI accuracy. Alpha band power was shown to be significant between correct and incorrect character selections with a significance of p = 0.01004. Using this significance, alpha band analysis was used to classify selections as correct or incorrect based on the EEG, however it was only improved accuracy for a subset of subjects (subjects exhibiting high alpha variance). Off-line analysis of P300-Certainty was shown to increase accuracy from 82.01±12.59% to 88.82±8.85% by abstaining potential errors, with a statistical significance of p = 0.038. Furthermore, P300-Certainty and alpha band analysis used together, improved BCI accuracy, over all subjects, more than either method did alone. The increase was statistically significant (p = 0.008) when compared to the raw BCI accuracy. The on-line accuracy of P300-Certainty was 83.62 ± 9.14%. Alpha band analysis and P300-Certainty abstain potential errors using different, yet orthogonal, methods of measuring attention. Each method abstains potential errors that the other may have not detected. In conclusion, this research has introduced two methods that quantify attention in orthogonal ways that increase BCI accuracy by abstaining potential errors more than either method alone. Using these methods together allows the BCI to be more user-aware and allows the user to type at their own pace.PHDBiomedical EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/144120/1/awaref_1.pd

Improving P300 Speller performance by means of optimization and machine learning

Brain-Computer Interfaces (BCIs) are systems allowing people to interact with the environment bypassing the natural neuromuscular and hormonal outputs of the peripheral nervous system (PNS). These interfaces record a user’s brain activity and translate it into control commands for external devices, thus providing the PNS with additional artificial outputs. In this framework, the BCIs based on the P300 Event-Related Potentials (ERP), which represent the electrical responses recorded from the brain after specific events or stimuli, have proven to be particularly successful and robust. The presence or the absence of a P300 evoked potential within the EEG features is determined through a classification algorithm. Linear classifiers such as stepwise linear discriminant analysis and support vector machine (SVM) are the most used discriminant algorithms for ERPs’ classification. Due to the low signal-to-noise ratio of the EEG signals, multiple stimulation sequences (a.k.a. iterations) are carried out and then averaged before the signals being classified. However, while augmenting the number of iterations improves the Signal-to-Noise Ratio, it also slows down the process. In the early studies, the number of iterations was fixed (no stopping environment), but recently several early stopping strategies have been proposed in the literature to dynamically interrupt the stimulation sequence when a certain criterion is met in order to enhance the communication rate. In this work, we explore how to improve the classification performances in P300 based BCIs by combining optimization and machine learning. First, we propose a new decision function that aims at improving classification performances in terms of accuracy and Information Transfer Rate both in a no stopping and early stopping environment. Then, we propose a new SVM training problem that aims to facilitate the target-detection process. Our approach proves to be effective on several publicly available datasets

Improving P300 Speller performance by means of optimization and machine learning

Brain-Computer Interfaces (BCIs) are systems allowing people to interact with the environment bypassing the natural neuromuscular and hormonal outputs of the peripheral nervous system (PNS). These interfaces record a user's brain activity and translate it into control commands for external devices, thus providing the PNS with additional artificial outputs. In this framework, the BCIs based on the P300 Event-Related Potentials (ERP), which represent the electrical responses recorded from the brain after specific events or stimuli, have proven to be particularly successful and robust. The presence or the absence of a P300 evoked potential within the EEG features is determined through a classification algorithm. Linear classifiers such as SWLDA and SVM are the most used for ERPs' classification. Due to the low signal-to-noise ratio of the EEG signals, multiple stimulation sequences (a.k.a. iterations) are carried out and then averaged before the signals being classified. However, while augmenting the number of iterations improves the Signal-to-Noise Ratio (SNR), it also slows down the process. In the early studies, the number of iterations was fixed (no stopping), but recently, several early stopping strategies have been proposed in the literature to dynamically interrupt the stimulation sequence when a certain criterion is met to enhance the communication rate. In this work, we explore how to improve the classification performances in P300 based BCIs by combining optimization and machine learning. First, we propose a new decision function that aims at improving classification performances in terms of accuracy and Information Transfer Rate both in a no stopping and early stopping environment. Then, we propose a new SVM training problem that aims to facilitate the target-detection process. Our approach proves to be effective on several publicly available datasets.Comment: 32 pages, research articl

How Visual Stimuli Evoked P300 is Transforming the Brain–Computer Interface Landscape: A PRISMA Compliant Systematic Review

Non-invasive Visual Stimuli evoked-EEGbased P300 BCIs have gained immense attention in recent years due to their ability to help patients with disability using BCI-controlled assistive devices and applications. In addition to the medical field, P300 BCI has applications in entertainment, robotics, and education. The current article systematically reviews 147 articles that were published between 2006-2021*. Articles that pass the pre-defined criteria are included in the study. Further, classification based on their primary focus, including article orientation, participants’ age groups, tasks given, databases, the EEG devices used in the studies, classification models, and application domain, is performed. The application-based classification considers a vast horizon, including medical assessment, assistance, diagnosis, applications, robotics, entertainment, etc. The analysis highlights an increasing potential for P300 detection using visual stimuli as a prominent and legitimate research area and demonstrates a significant growth in the research interest in the field of BCI spellers utilizing P300. This expansion was largely driven by the spread of wireless EEG devices, advances in computational intelligence methods, machine learning, neural networks and deep learning