An uncued brain-computer interface using reservoir computing

Brain-Computer Interfaces are an important and promising avenue for possible next-generation assistive devices. In this article, we show how Reservoir Comput- ing – a computationally efficient way of training recurrent neural networks – com- bined with a novel feature selection algorithm based on Common Spatial Patterns can be used to drastically improve performance in an uncued motor imagery based Brain-Computer Interface (BCI). The objective of this BCI is to label each sample of EEG data as either motor imagery class 1 (e.g. left hand), motor imagery class 2 (e.g. right hand) or a rest state (i.e., no motor imagery). When comparing the re- sults of the proposed method with the results from the BCI Competition IV (where this dataset was introduced), it turns out that the proposed method outperforms the winner of the competition

Switching characters between stimuli improves P300 speller accuracy

In this paper, an alternative stimulus presentation paradigm for the P300 speller is introduced. Similar to the checkerboard paradigm it minimizes the occurrence of the two most common causes of spelling errors: adjacency distraction and double flashes. Moreover, in contrast to the checkerboard paradigm, this new stimulus sequence does not increase the time required per stimulus iteration. Our new paradigm is compared to the basic row-column paradigm and the results indicate that, on average, the accuracy is improved

Dynamic stopping in a calibration-less P300 speller

Even though the P300 based speller has proved to be usable by real patients, it is not a user-friendly system. The necesarry calibration session and slow spelling make the system tedious. We present a machine learning approach to P300 spelling that enables us to remove the calibration session. We achieve this by a combination of unsupervised training, transfer learning across subjects and language models. On top of that, we can increase the spelling speed by incorporating a dynamic stopping approach. This yields a P300 speller that works instantly and with high accuracy and spelling speed, even for unseen subjects

Distance dependent extensions of the Chinese restaurant process

In this paper we consider the clustering of text documents using the Chinese Restau- rant Process (CRP) and extensions that take time-correlations into account. To this pur- pose, we implement and test the Distance Dependent Chinese Restaurant Process (DD- CRP) for mixture models on both generated and real-world data. We also propose and im- plement a novel clustering algorithm, the Av- eraged Distance Dependent Chinese Restau- rant Process (ADDCRP), to model time- correlations, that is faster per iteration and attains similar performance as the fully dis- tance dependent CRP

Reducing BCI calibration time with transfer learning: a shrinkage approach

Introduction: A brain-computer interface system (BCI) allows subjects to make use of neural control signals to drive a computer application. Therefor a BCI is generally equipped with a decoder to differentiate between types of responses recorded in the brain. For example, an application giving feedback to the user can benefit from recognizing the presence or absence of a so-called error potential (Errp), elicited in the brain of the user when this feedback is perceived as being ‘wrong’, a mistake of the system. Due to the high inter- and intra- subject variability in these response signals, calibration data needs to be recorded to train the decoder. This calibration session is exhausting and demotivating for the subject. Transfer Learning is a general name for techniques in which data from previous subjects is used as additional information to train a decoder for a new subject, thereby reducing the amount of subject specific data that needs to be recorded during calibration. In this work we apply transfer learning to an Errp detection task by applying single-target shrinkage to Linear Discriminant Analysis (LDA), a method originally proposed by Höhne et. al. to improve accuracy by compensating for inter-stimuli differences in an ERP-speller [1]. Material, Methods and Results: For our study we used the error potential dataset recorded by Perrin et al. in [2]. For 26 subjects each, 340 Errp/nonErrp responses were recorded with a #Errp to #nonErrp ratio of 0.41 to 0.94. 272 responses were available for training the decoder and the remaining 68 responses were left out for testing. For every subject separately we built three different decoders. First, a subject specific LDA decoder was built solely making use of the subject’s own train data. Second, we added the train data of the other 25 subjects to train a global LDA decoder, naively ignoring the difference between subjects. Finally, the single-target-shrinkage method (STS) [1] is used to regularize the parameters of the subject specific decoder towards those of the global decoder. Making use of cross validation this method assigns an optimal weight to the subject specific data and data from previous subjects to be used for training. Figure 1 shows the performance of the three decoders on the test data in terms of AUC as a function of the amount of subject specific calibration data used. Discussion: The subject specific decoder in Figure 1 shows how sensitive the decoding performance is to the amount of calibration data provided. Using data from previously recorded subjects the amount of calibration data, and as such the calibration time, can be reduced as shown by the global decoder. A certain amount of quality is however sacrificed. Making an optimal compromise between the subject specific and global decoder, the single-target-shrinkage decoder allows the calibration time to be reduced by 20% without any change in decoder quality (confirmed by a paired sample t-test giving p=0.72). Significance: This work serves as a first proof of concept in the use of shrinkage LDA as a transfer learning method. More specific, the error potential decoder built with reduced calibration time boosts the opportunity for error correcting methods in BCI

A Bayesian machine learning framework for true zero-training brain-computer interfaces

Brain-Computer Interfaces (BCI) are developed to allow the user to take control of a computer (e.g. a spelling application) or a device (e.g. a robotic arm) by using just his brain signals. The concept of BCI was introduced in 1973 by Jacques Vidal. The early types of BCI relied on tedious user training to enable them to modulate their brain signals such that they can take control over the computer. Since then, training has shifted from the user to the computer. Hence, modern BCI systems rely on a calibration session, during which the user is instructed to perform specific tasks. The result of this calibration recording is a labelled data-set that can be used to train the (supervised) machine learning algorithm. Such a calibration recording is, however, of no direct use for the end user. Hence, it is especially important for patients to limit this tedious process. For this reason, the BCI community has invested a lot of effort in reducing the dependency on calibration data. Nevertheless, despite these efforts, true zero-training BCIs are rather rare. Event-Related Potential based spellers One of the most common types of BCI is the Event-Related Potentials (ERP) based BCI, which was invented by Farwell and Donchin in 1988. In the ERP-BCI, actions, such as spelling a letter, are coupled to specific stimuli. The computer continuously presents these stimuli to the user. By attending a specific stimulus, the user is able to select an action. More concretely, in the original ERP-BCI, these stimuli were the intensifications of rows and column in a matrix of symbols on a computer screen. By detecting which row and which column elicit an ERP response, the computer can infer which symbol the user wants to spell. Initially, the ERP-BCI was aimed at restoring communication, but novel applications have been proposed too. Examples are web browsing, gaming, navigation and painting. Additionally, current BCIs are not limited to using visual stimuli, but variations using auditory or tactile stimuli have been developed as well. In their quest to improve decoding performance in the ERP-BCI, the BCI community has developed increasingly more complex machine learning algorithms. However, nearly all of them rely on intensive subject-specific fine-tuning. The current generation of decoders has gone beyond a standard ERP classifier and they incorporate language models, which are similar to a spelling corrector on a computer, and extensions to speed up the communication, commonly referred to as dynamic stopping. Typically, all these different components are separate entities that have to be tied together by heuristics. This introduces an additional layer of complexity and the result is that these state of the art methods are difficult to optimise due to the large number of free parameters. We have proposed a single unified probabilistic model that integrates language models and a natural dynamic stopping strategy. This coherent model is able to achieve state of the art performance, while at the same time, minimising the complexity of subject-specific tuning on labelled data. A second and major contribution of this thesis is the development of the first unsupervised decoder for ERP spellers. Recall that typical decoders have to be tuned on labelled data for each user individually. Moreover, recording this labelled data is a tedious process, which has no direct use for the end user. The unsupervised approach, which is an extension of our unified probabilistic model, is able to learn how to decode a novel user’s brain signals without requiring such a labelled dataset. Instead, the user starts using the system and in the meantime the decoder is learning how to decode the brain signals. This method has been evaluated extensively, both in an online and offline setting. Our offline validation was executed on three different datasets of visual ERP data in the standard matrix speller. Combined, these datasets contain 25 different subjects. Additionally, we present the results of an offline evaluation on auditory ERP data from 21 subjects. Due to a less clear signal, this auditory ERP data present an even greater challenge than visual ERP data. On top of that we present the results from an online study on auditory ERP, which was conducted in cooperation with Michael Tangermann, Martijn Schreuder and Klaus-Robert Müller at the TU-Berlin. Our simulations indicate that when enough unlabelled data is available, the unsupervised method can compete with state of the art supervised approaches. Furthermore, when non-stationarity is present in the EEG recordings, e.g. due to fatigue during longer experiments, then the unsupervised approach can outperform supervised methods by adapting to these changes in the data. However, the limitation of the unsupervised method lies in the fact that while labelled data is not required, a substantial amount of unlabelled data must be processed before a reliable model can be found. Hence, during online experiments the model suffers from a warm-up period. During this warm-up period, the output is unreliable, but the mistakes made during this warm-up period can be corrected automatically when enough data is processed. To maximise the usability of ERP-BCI, the warm-up of the unsupervised method has to be minimised. For this reason, we propose one of the first transfer learning methods for ERP-BCI. The idea behind transfer learning is to share information on how to decode the brain signals between users. The concept of transfer learning stands in stark contrast with the strong tradition of subject-specific decoders commonly used by the BCI community. Nevertheless, by extending our unified model with inter-subject transfer learning, we are able to build a decoder that can decode the brain signals of novel users without any subject-specific training. Unfortunately, basic transfer learning models do perform as well as subject-specific (supervised models). For this reason, we have combined our transfer learning approach with our unsupervised learning approach to adapt it during usage to a highly accurate subject-specific model. Analogous to our unsupervised model, we have performed an extensive evaluation of transfer learning with unsupervised adaptation. We tested the model offline on visual ERP data from 22 subjects and on auditory ERP data from 21 subjects. Additionally, we present the results from an online study, which was also performed at the TUBerlin, where we evaluate transfer learning online on the auditory AMUSE paradigm. From these experiments, we can conclude that transfer learning in combination with unsupervised adaptation results in a true zero training BCI, that can compete with state of the art supervised models, without needing a single data point from a calibration recording. This method allows us to build a BCI that works out of the box

Integrating dynamic stopping, transfer learning and language models in an adaptive zero-training ERP speller

Objective. Most BCIs have to undergo a calibration session in which data is recorded to train decoders with machine learning. Only recently zero-training methods have become a subject of study. This work proposes a probabilistic framework for BCI applications which exploit event-related potentials (ERPs). For the example of a visual P300 speller we show how the framework harvests the structure suitable to solve the decoding task by (a) transfer learning, (b) unsupervised adaptation, (c) language model and (d) dynamic stopping. Approach. A simulation study compares the proposed probabilistic zero framework (using transfer learning and task structure) to a state-of-the-art supervised model on n = 22 subjects. The individual influence of the involved components (a)–(d) are investigated. Main results. Without any need for a calibration session, the probabilistic zero-training framework with inter-subject transfer learning shows excellent performance—competitive to a state-of-the-art supervised method using calibration. Its decoding quality is carried mainly by the effect of transfer learning in combination with continuous unsupervised adaptation. Significance. A high-performing zero-training BCI is within reach for one of the most popular BCI paradigms: ERP spelling. Recording calibration data for a supervised BCI would require valuable time which is lost for spelling. The time spent on calibration would allow a novel user to spell 29 symbols with our unsupervised approach. It could be of use for various clinical and non-clinical ERP-applications of BCI

A case study demonstrating the pitfalls during evaluation of a predictive seizure detector

Epilepsy is a neurological disorder characterized by recurring epileptic seizures that can occur at any given time. A system predicting these seizures could give a patient sufficient time to bring himself to safety and to apply a fast-working anti-epileptic treatment to suppress the upcoming seizure. Many seizure detection techniques claim to be able to detect seizures before the marked seizure onset on the EEG. In this work we study the predictions of such a seizure detection system. Materials: For the experiments the MIT Scalp EEG dataset was used, which contains at least 20 hours of EEG and 3 seizures for 24 pediatric patients [1]. Methods: The data is preprocessed using a filter-bank of 8 Butterworth filters of 3 Hz wide between 0.5 and 24.5 Hz [1]. Next the energy is determined for windows of 2 seconds wide with 1 second overlap. This data is presented as input for the machine learning component based on Reservoir Computing (RC) [1]. RC uses a randomly created recurrent artificial neural network, the reservoir, to map the input to a higher dimensional space. The system is trained using a linear readout of the reservoir. After this readout a simple thresholding technique is applied for classification [1]. Experiments and results: For each patient, the system is trained on the data of the 23 other patients. During training, the 2 minutes of EEG prior or following a seizure is not used. Next the system is evaluated on the data of the considered patient. Detections which occurred 10 minutes before the marked seizure onset were considered as true positives. This resulted in a system that was able to detect 75% of the seizures with about 6 false positives per correctly detected seizure. For 11 out of 24 patients some seizures were detected before the marked seizure onset. Furthermore, in 4 of these patients at least half of the seizures were detected before the marked onset, and in a single patient all seizures were detected before the marked onset. Discussion: However, in retrospect, 65% of the early detections are caused by EEG artifacts. Most others can be attributed to inter-ictal spike and wave discharges in the EEG preceding the seizure. Only 3% of the early detections have currently an unknown cause and could be actual early detections. Although nearly all early detections can be considered as false positives. However such false positives have a significantly greater occurrence right before marked seizure onsets, but further research is needed to analyze the cause of this correlation. It might be that these artifacts contain predictive information or for example that the selection criteria for adding EEG sections to the dataset were less strict for EEG sections containing a seizure. These pitfalls call for common guidelines and datasets to evaluate early seizure detection methods. References: [1] Buteneers, P. (2012). Detection of epileptic seizures: the reservoir computing approach (Doctoral dissertation, Ghent University)