Applications of non-invasive brain-computer interfaces for communication and affect recognition

Doctor of PhilosophyDepartment of Electrical and Computer EngineeringDavid E. ThompsonVarious assistive technologies are available for people with communication disorders. While these technologies are quite useful for moderate to severe movement impairments, certain progressive diseases can cause a total locked-in state (TLIS). These conditions include amyotrophic lateral sclerosis (ALS), neuromuscular disease (NMD), and several other disorders that can cause impairment between the neural pathways and the muscles. For people in a locked-in state (LIS), brain-computer interfaces (BCIs) may be the only possible solution. BCIs could help to restore communication to these people, with the help of external devices and neural recordings. The present dissertation investigates the role of latency jitter on BCIs system performance and, at the same time, the possibility of affect recognition using BCIs. BCIs that can recognize human affect are referred to as affective brain-computer interfaces (aBCIs). These aBCIs are a relatively new area of research in affective computing. Estimation of affective states can improve human-computer interaction as well as improve the care of people with severe disabilities. The present work used a publicly available dataset as well as a dataset collected at the Brain and Body Sensing Lab at K-State to assess the effectiveness of EEG recordings in recognizing affective states. This work proposed an extended classifier-based latency estimation (CBLE) method using sparse autoencoders (SAE) to investigate the role of latency jitter on BCI system performance. The recent emergence of autoencoders motivated the present work to develop an SAE based CBLE method. Here, the newly-developed SAE-based CBLE method is applied to a newly-collected dataset. Results from our data showed a significant (p < 0.001) negative correlation between BCI accuracy and estimated latency jitter. Furthermore, the SAE-based CBLE method is also able to predict BCI accuracy. In the aBCI-related investigation, this work explored the effectiveness of different features extracted from EEG to identify the affect of a user who was experiencing affective stimuli. Furthermore, this dissertation reviewed articles that used the Database for Emotion Analysis Using Physiological Signals (DEAP) (i.e., a publicly available affective database) and found that a significant number of studies did not consider the presence of the class imbalance in the dataset. Failing to consider class imbalance creates misleading results. Furthermore, ignoring class imbalance makes comparing results between studies impossible, since different datasets will have different class imbalances. Class imbalance also shifts the chance level. Hence, it is vital to consider class bias while determining if the results are above chance. This dissertation suggests the use of balanced accuracy as a performance metric and its posterior distribution for computing confidence intervals to account for the effect of class imbalance

SSL for Auditory ERP-Based BCI

A brain–computer interface (BCI) is a communication tool that analyzes neural activity and relays the translated commands to carry out actions. In recent years, semi-supervised learning (SSL) has attracted attention for visual event-related potential (ERP)-based BCIs and motor-imagery BCIs as an effective technique that can adapt to the variations in patterns among subjects and trials. The applications of the SSL techniques are expected to improve the performance of auditory ERP-based BCIs as well. However, there is no conclusive evidence supporting the positive effect of SSL techniques on auditory ERP-based BCIs. If the positive effect could be verified, it will be helpful for the BCI community. In this study, we assessed the effects of SSL techniques on two public auditory BCI datasets—AMUSE and PASS2D—using the following machine learning algorithms: step-wise linear discriminant analysis, shrinkage linear discriminant analysis, spatial temporal discriminant analysis, and least-squares support vector machine. These backbone classifiers were firstly trained by labeled data and incrementally updated by unlabeled data in every trial of testing data based on SSL approach. Although a few data of the datasets were negatively affected, most data were apparently improved by SSL in all cases. The overall accuracy was logarithmically increased with every additional unlabeled data. This study supports the positive effect of SSL techniques and encourages future researchers to apply them to auditory ERP-based BCIs

A review of rapid serial visual presentation-based brain-computer interfaces

International audienceRapid serial visual presentation (RSVP) combined with the detection of event related brain responses facilitates the selection of relevant information contained in a stream of images presented rapidly to a human. Event related potentials (ERPs) measured non-invasively with electroencephalography (EEG) can be associated with infrequent targets amongst a stream of images. Human-machine symbiosis may be augmented by enabling human interaction with a computer, without overt movement, and/or enable optimization of image/information sorting processes involving humans. Features of the human visual system impact on the success of the RSVP paradigm, but pre-attentive processing supports the identification of target information post presentation of the information by assessing the co-occurrence or time-locked EEG potentials. This paper presents a comprehensive review and evaluation of the limited but significant literature on research in RSVP-based brain-computer interfaces (BCIs). Applications that use RSVP-based BCIs are categorized based on display mode and protocol design, whilst a range of factors influencing ERP evocation and detection are analyzed. Guidelines for using the RSVP-based BCI paradigms are recommended, with a view to further standardizing methods and enhancing the inter-relatability of experimental design to support future research and the use of RSVP-based BCIs in practice