Improved inference and autotyping in EEG-based BCI typing systems

ABSTRACT The RSVP Keyboard TM is a brain-computer interface (BCI)-based typing system for people with severe physical disabilities, specifically those with locked-in syndrome (LIS). It uses signals from an electroencephalogram (EEG) combined with information from an n-gram language model to select letters to be typed. One characteristic of the system as currently configured is that it does not keep track of past EEG observations, i.e., observations of user intent made while the user was in a different part of a typed message. We present a principled approach for taking all past observations into account, and show that this method results in a 20% increase in simulated typing speed under a variety of conditions on realistic stimuli. We also show that this method allows for a principled and improved estimate of the probability of the backspace symbol, by which mis-typed symbols are corrected. Finally, we demonstrate the utility of automatically typing likely letters in certain contexts, a technique that achieves increased typing speed under our new method, though not under the baseline approach

A review of rapid serial visual presentation-based brain-computer interfaces

International audienceRapid serial visual presentation (RSVP) combined with the detection of event related brain responses facilitates the selection of relevant information contained in a stream of images presented rapidly to a human. Event related potentials (ERPs) measured non-invasively with electroencephalography (EEG) can be associated with infrequent targets amongst a stream of images. Human-machine symbiosis may be augmented by enabling human interaction with a computer, without overt movement, and/or enable optimization of image/information sorting processes involving humans. Features of the human visual system impact on the success of the RSVP paradigm, but pre-attentive processing supports the identification of target information post presentation of the information by assessing the co-occurrence or time-locked EEG potentials. This paper presents a comprehensive review and evaluation of the limited but significant literature on research in RSVP-based brain-computer interfaces (BCIs). Applications that use RSVP-based BCIs are categorized based on display mode and protocol design, whilst a range of factors influencing ERP evocation and detection are analyzed. Guidelines for using the RSVP-based BCI paradigms are recommended, with a view to further standardizing methods and enhancing the inter-relatability of experimental design to support future research and the use of RSVP-based BCIs in practice

Incorporation of a language model into a brain computer interface based speller

Brain computer interface (BCI) research deals with the problem of establishing direct communication pathways between the brain and external devices. The primary motivation is to enable patients with limited or no muscular control to use external devices by automatically interpreting their intent based on brain electrical activity, measured by, e.g., electroencephalography (EEG). The P300 speller is a widely practised BCI set up that involves having subjects type letters based on P300 signals generated by their brains in response to visual stimuli. Because of the low signal-to-noise ratio (SNR) and variability of EEG signals, existing typing systems use many repetitions of the visual stimuli in order to increase accuracy at the cost of speed. The main motivation for the work in this thesis comes from the observation that the prior information provided by both neighbouring and current letters within words in a particular language can assist letter estimation with the aim of developing a system that achieves higher accuracy and speed simultaneously. Based on this observation, in this thesis, we present an approach for incorporation of such information into a BCI-based speller through Hidden Markov Models (HMM) trained by a language model. We then describe filtering and smoothing algorithms in conjunction with n-gram language models for inference over such a model. We have designed data collection experiments for offline and online decision-making which demonstrate that incorporation of the language model in this manner results in significant improvements in letter estimation and typing speed

The first brain-computer interface utilizing a Turkish language model (Türkçe dil modeli kullanan ilk beyin-bilgisayar arayüzü)

One of the widely studied electroencephalography (EEG) based Brain-Computer Interface (BCI) set ups involves having subjects type letters based on so-called P300 signals generated by their brains in response to unpredictable stimuli. Due to the low signal-to-noise ratio (SNR) of EEG signals, current BCI typing systems need several stimulus repetitions to obtain acceptable accuracy, resulting in low typing speed. However, in the context of typing letters within words in a particular language, neighboring letters would provide information about the current letter as well. Based on this observation, we propose an approach for incorporation of such information into a BCI-based speller through a Hidden Markov Model (HMM) trained by a Turkish language model. We describe smoothing and Viterbi algorithms for inference over such a model. Experiments on real EEG data collected in our laboratory demonstrate that incorporation of the language model in this manner leads to significant improvements in classification accuracy and bit rate

Disjunctive normal unsupervised LDA for P300-based brain-computer interfaces

Can people use text-entry based brain-computer interface (BCI) systems and start a free spelling mode without any calibration session? Brain activities differ largely between people and across sessions for the same user. Thus, how can the text-entry system classify the target character among the other characters in the P300-based BCI speller matrix? In this thesis, we introduce a new unsupervised classifier for a P300-based BCI speller, which uses a disjunctive normal form representation to de ne an energy function involving a logistic sigmoid function for classification. Our proposed classifier updates the initialized random weights performing classification for the P300 signals from the recorded data exploiting the knowledge of the sequence of row/column highlights. To verify the effectiveness of the proposed method, we performed an experimental analysis on data from 7 healthy subjects, collected in our laboratory and used public BCI competition datasets. We compare the proposed unsupervised method to a baseline supervised linear discriminant analysis (LDA) classifier and Bayesian linear discriminant analysis (BLDA) and demonstrate its performance. Our analysis shows that the proposed approach facilitates unsupervised learning from unlabelled test data

Applications of non-invasive brain-computer interfaces for communication and affect recognition

Doctor of PhilosophyDepartment of Electrical and Computer EngineeringDavid E. ThompsonVarious assistive technologies are available for people with communication disorders. While these technologies are quite useful for moderate to severe movement impairments, certain progressive diseases can cause a total locked-in state (TLIS). These conditions include amyotrophic lateral sclerosis (ALS), neuromuscular disease (NMD), and several other disorders that can cause impairment between the neural pathways and the muscles. For people in a locked-in state (LIS), brain-computer interfaces (BCIs) may be the only possible solution. BCIs could help to restore communication to these people, with the help of external devices and neural recordings. The present dissertation investigates the role of latency jitter on BCIs system performance and, at the same time, the possibility of affect recognition using BCIs. BCIs that can recognize human affect are referred to as affective brain-computer interfaces (aBCIs). These aBCIs are a relatively new area of research in affective computing. Estimation of affective states can improve human-computer interaction as well as improve the care of people with severe disabilities. The present work used a publicly available dataset as well as a dataset collected at the Brain and Body Sensing Lab at K-State to assess the effectiveness of EEG recordings in recognizing affective states. This work proposed an extended classifier-based latency estimation (CBLE) method using sparse autoencoders (SAE) to investigate the role of latency jitter on BCI system performance. The recent emergence of autoencoders motivated the present work to develop an SAE based CBLE method. Here, the newly-developed SAE-based CBLE method is applied to a newly-collected dataset. Results from our data showed a significant (p < 0.001) negative correlation between BCI accuracy and estimated latency jitter. Furthermore, the SAE-based CBLE method is also able to predict BCI accuracy. In the aBCI-related investigation, this work explored the effectiveness of different features extracted from EEG to identify the affect of a user who was experiencing affective stimuli. Furthermore, this dissertation reviewed articles that used the Database for Emotion Analysis Using Physiological Signals (DEAP) (i.e., a publicly available affective database) and found that a significant number of studies did not consider the presence of the class imbalance in the dataset. Failing to consider class imbalance creates misleading results. Furthermore, ignoring class imbalance makes comparing results between studies impossible, since different datasets will have different class imbalances. Class imbalance also shifts the chance level. Hence, it is vital to consider class bias while determining if the results are above chance. This dissertation suggests the use of balanced accuracy as a performance metric and its posterior distribution for computing confidence intervals to account for the effect of class imbalance

Evaluating a Novel Brain-Computer Interface and EEG Biomarkers For Cognitive Assessment in Children With Cerebral Palsy

Standardized neuropsychological assessments and research instruments are typically administered with verbal queries, pictures and manipulatives that require verbal or motor responses. Thus, they are often inaccessible to people with physical and communicative impairments. The goal of this dissertation was to investigate alternative approaches that do not require any motor or speech input to assess cognitive capacity of an individual. The first approach involved using a brain-computer interface (BCI) that was adapted to facilitate the administration of a Peabody Picture Vocabulary Test (PPVT-IV). Which is a receptive vocabulary assessment than can be used as a proxy for intelligence. The second approach was to use brain dynamics such as functional connectivity and bandpass analysis to assess cognitive capacity of an individual. We then tested these two approaches on typically developing (TD) individuals (N=11) people with cerebral palsy (CP) (N=18). Our results suggest that children with cerebral palsy show signs of lower intelligence than typically developing children when using functional connectivity and power band analysis, however, they performed equally well in the PPVT-IV. We believe this is due to the neural compensation resulting from the subjects’ pathology. Thus, the preferred method for assessing cognitive measures in an individual with severe motoric impairments is a BCI. By using a BCI, a user can respond to standardized cognitive assessments that already have well established norms. However, it is important to make sure that when designing these systems, the changes made to adapt the cognitive assessment for the BCI do not alter the format and psychometrics of the test. Our BCI able to maintain the psychometrics of a PPVT-IV test and perform with an accuracy of 97.78 ± 4.06. In addition, scores on the BCI-facilitated PPVT-IV and the standard PPVT-IV were highly correlated (r = 0.95, p<0.001) with a mean difference of 2.0 ± 6.4 points, which is within the standard error of the PPVT-IV. Thus, our BCI-facilitated PPVT-IV provided comparable results to the standard PPVT-IV, suggesting that populations for whom standardized cognitive tests are not accessible could benefit from our BCI-facilitated approach.PHDNeuroscienceUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/136949/1/pharoram_1.pd

A machine learning approach to taking EEG-based brain-computer interfaces out of the lab

Despite being a subject of study for almost three decades, non-invasive brain- computer interfaces (BCIs) are still trapped in the laboratory. In order to move into more common use, it is necessary to have systems that can be reliably used over time with a minimum of retraining. My research focuses on machine learning methods to minimize necessary retraining, as well as a data science approach to validate processing pipelines more robustly. Via a probabilistic transfer learning method that scales well to large amounts of data in high dimensions it is possible to reduce the amount of calibration data needed for optimal performance. However, a good model still requires reliable features that are resistant to recording artifacts. To this end we have also investigated a novel feature of the electroencephalogram which is predictive of multiple types of brain-related activity. As cognitive neuroscience literature suggests, shifts in the peak frequency of a neural oscillation – hereafter referred to as frequency modulation – can be predictive of activity in standard BCI tasks, which we validate for the first time in multiple paradigms. Finally, in order to test the robustness of our techniques, we have built a codebase for reliable comparison of pipelines across over fifteen open access EEG datasets