Disjunctive normal unsupervised LDA for P300-based brain-computer interfaces

Can people use text-entry based brain-computer interface (BCI) systems and start a free spelling mode without any calibration session? Brain activities differ largely across people and across sessions for the same user. Thus, how can the text-entry system classify the desired character among the other characters in the P300-based BCI speller matrix? In this paper, we introduce a new unsupervised classifier for a P300-based BCI speller, which uses a disjunctive normal form representation to define an energy function involving a logistic sigmoid function for classification. Our proposed classifier updates the initialized random weights performing classification for the P300 signals from the recorded data exploiting the knowledge of the sequence of row/column highlights. To verify the effectiveness of the proposed method, we performed an experimental analysis on data from 7 healthy subjects, collected in our laboratory. We compare the proposed unsupervised method to a baseline supervised linear discriminant analysis (LDA) classifier and demonstrate its effectiveness

Probabilistic graphical models for brain computer interfaces

Brain computer interfaces (BCI) are systems that aim to establish a new communication path for subjects who su er from motor disabilities, allowing interaction with the environment through computer systems. BCIs make use of a diverse group of physiological phenomena recorded using electrodes placed on the scalp (Electroencephalography, EEG) or electrodes placed directly over the brain cortex (Electrocorticography, ECoG). One commonly used phenomenon is the activity observed in specific areas of the brain in response to external events, called Event Related Potentials (ERP). Among those, a type of response called P300 is the most used phenomenon. The P300 has found application in spellers that make use of the brain's response to the presentation of a sequence of visual stimuli. Another commonly used phenomenon is the synchronization or desynchronization of brain rhythms during the execution or imagination of a motor task, which can be used to differentiate between two or more subject intentions. In the most basic scenario, a BCI system calculates the differences in the power of the EEG rhythms during execution of different tasks. Based on those differences, the BCI decides which task has been executed (e.g., motor imagination of left or right hand). Current approaches are mainly based on machine learning techniques that learn the distribution of the power values of the brain signals for each of the possible classes. In this thesis, making use of EEG and ECoG recording methods, we propose the use of probabilistic graphical models for brain computer interfaces. In the case of ERPs, in particular P300-based spellers, we propose the incorporation of language models at the level of words to increase significantly the performance of the spelling system. The proposed framework allows also the incorporation of different methods that take into account language models based on n-grams, all of this in an integrated structure whose parameters can be efficiently learned. In the context of execution or imagination of motor tasks, we propose techniques that take into account the temporal structure of the signals. Stochastic processes that model temporal dynamics of the brain signals in different frequency bands such as non-parametric Bayesian hidden Markov models are proposed in order to solve the problem of selection of the number of brain states during the execution of motor tasks as well as the selection of the number of components used to model the distribution of the brain signals. Following up on the same line of thought, hidden conditional random fields are proposed for classification of synchronous motor tasks. The combination of hidden states with the discriminative power of conditional random fields is shown to increase the classification performance of imaginary motor movements. In the context of asynchronous BCIs, we propose a method based on latent dynamic conditional random fields that is capable of modeling the internal temporal dynamics related to the generation of the brain signals, and external brain dynamics related to the execution of different mental tasks. Finally, in the context of asynchronous BCIs a model based on discriminative graphical models is presented for continuous classification of finger movements from ECoG data. We show that the incorporation of temporal dynamics of the brain signals in the classification stages increases significantly the classification accuracy of different mental states which can lead to a more effective interaction between the subject and the environment

Development of a concept-based EMG-based speller

Physiological computing is a paradigm of computing that treats users’ physiological data as input during computing tasks in an Ambient Assisted Living (AAL) environment. By monitoring, analyzing and responding to such inputs, Physiological Computing Systems (PCS) are able to respond to the users’ cognitive, emotional and physical states. A specific case of PCS is Neural Computer Interface (NCI), which uses electrical signals governing users’ muscular activity (EMG data) to establish a direct communication pathway between the user and a computer. We present taxonomy of speller application parameters, propose a model of PCS, and describe the development of the EMG-based speller as a benchmark application. We analyze and develop an EMG-based speller application with a traditional letter-based as well as visual concept-based interface. Finally, we evaluate the performance and usability of the developed speller using empirical (accuracy, information transfer speed, input speed) metrics

BrainBraille: Towards Passive Training in Brain-Computer Interfaces using fNIRS

Amyotrophic Lateral Sclerosis (ALS) is a debilitating movement disability that causes patients to gradually lose their ability to voluntarily control their muscles. In some cases, patients who are "locked-in" are unable to move any muscles, leaving them with no means of communicating with caregivers. Brain-computer interfaces (BCIs) attempt to create a means of communication directly through brain activity, removing the need for movement. BrainBraille is a novel interaction method for BCIs, enabling complex text-based communication using attempted movements with a six-region pseudo-binary encoding. In this dissertation, I explore a wearable BCI using functional near-infrared scanning (fNIRS) to make BrainBraille mobile. In an early study, I show that transitional gestures based on executed movements of two hands can be classified in two participants with up to 93% accuracy. I explore how transitional gestures can benefit BrainBraille by expanding the vocabulary and enabling faster responses. Finally, I evaluate future paths for integrating passive haptic training into BrainBraille to reduce the physical exertion needed to learn a BCI for ALS patients.Undergraduat

Probabilistic Graphical Models for ERP-Based Brain Computer Interfaces

An event related potential (ERP) is an electrical potential recorded from the nervous system of humans or other animals. An ERP is observed after the presentation of a stimulus. Some examples of the ERPs are P300, N400, among others. Although ERPs are used very often in neuroscience, its generation is not yet well understood and different theories have been proposed to explain the phenomena. ERPs could be generated due to changes in the alpha rhythm, an internal neural control that reset the ongoing oscillations in the brain, or separate and distinct additive neuronal phenomena. When different repetitions of the same stimuli are averaged, a coherence addition of the oscillations is obtained which explain the increase in amplitude in the signals. Two ERPs are mostly studied: N400 and P300. N400 signals arise when a subject tries to make semantic operations that support neural circuits for explicit memory. N400 potentials have been observed mostly in the rhinal cortex. P300 signals are related to attention and memory operations. When a new stimulus appears, a P300 ERP (named P3a) is generated in the frontal lobe. In contrast, when a subject perceives an expected stimulus, a P300 ERP (named P3b) is generated in the temporal – parietal areas. This implicates P3a and P3b are related, suggesting a circuit pathway between the frontal and temporal–parietal regions, whose existence has not been verified. Un potencial relacionado con un evento (ERP) es un potencial eléctrico registrado en el sistema nervioso de los seres humanos u otros animales. Un ERP se observa tras la presentación de un estímulo. Aunque los ERPs se utilizan muy a menudo en neurociencia, su generación aún no se entiende bien y se han propuesto diferentes teorías para explicar el fenómeno. Una interfaz cerebro-computador (BCI) es un sistema de comunicación en el que los mensajes o las órdenes que un sujeto envía al mundo exterior proceden de algunas señales cerebrales en lugar de los nervios y músculos periféricos. La BCI utiliza ritmos sensorimotores o señales ERP, por lo que se necesita un clasificador para distinguir entre los estímulos correctos y los incorrectos. En este trabajo, proponemos utilizar modelos probabilísticos gráficos para el modelado de la dinámica temporal y espacial de las señales cerebrales con aplicaciones a las BCIs. Los modelos gráficos han sido seleccionados por su flexibilidad y capacidad de incorporar información previa. Esta flexibilidad se ha utilizado anteriormente para modelar únicamente la dinámica temporal. Esperamos que el modelo refleje algunos aspectos del funcionamiento del cerebro relacionados con los ERPs, al incluir información espacial y temporal.DoctoradoDoctor en Ingeniería Eléctrica y Electrónic

Adaptation in p300 and motor imagery-based BCI systems

Brain Computer Interface (BCI) is an alternative communication tool between human and computer. Motivation of BCI is to create a non-muscular communication environment for the use of external devices. Electroencephalography (EEG) signals are analyzed for understanding the user's intent in BCI systems. The nonstationary behavior of brain electrical activity (such as EEG), caused by changes in subject brain activities, environment conditions and calibration issues, is one of the main challenges of BCI systems. Another set of challenges involves limited amount of training data and subject-dependent characteristics of EEG. In this thesis, we suggest a semi-supervised adaptation approach for P300 based BCI speller systems to address these types of problems. The proposed approach is applied on a P300 speller which also incorporates a language model using Hidden Markov Models (HMM). The estimated labels from the classifier are used to retrain the classifier for adaptation. We have analyzed the effects of this adaptation approach on BCI systems with non-stationary EEG data and small size of training data. We propose to solve both problems by updating the BCI system with labels obtained from the classifier. We have shown that such an adaptation approach would improve BCI performance around 30% for systems with limited amount of training data, and 40% for transferring the system subject-to-subject. Moreover, we have investigated the potential use of error related potential (ErrP) signals in the P300-based BCI systems. The detection and classification of ErrP signals in BCI setting are presented along with the experimental analysis of ErrP

Incorporation of a language model into a brain computer interface based speller

Brain computer interface (BCI) research deals with the problem of establishing direct communication pathways between the brain and external devices. The primary motivation is to enable patients with limited or no muscular control to use external devices by automatically interpreting their intent based on brain electrical activity, measured by, e.g., electroencephalography (EEG). The P300 speller is a widely practised BCI set up that involves having subjects type letters based on P300 signals generated by their brains in response to visual stimuli. Because of the low signal-to-noise ratio (SNR) and variability of EEG signals, existing typing systems use many repetitions of the visual stimuli in order to increase accuracy at the cost of speed. The main motivation for the work in this thesis comes from the observation that the prior information provided by both neighbouring and current letters within words in a particular language can assist letter estimation with the aim of developing a system that achieves higher accuracy and speed simultaneously. Based on this observation, in this thesis, we present an approach for incorporation of such information into a BCI-based speller through Hidden Markov Models (HMM) trained by a language model. We then describe filtering and smoothing algorithms in conjunction with n-gram language models for inference over such a model. We have designed data collection experiments for offline and online decision-making which demonstrate that incorporation of the language model in this manner results in significant improvements in letter estimation and typing speed

Silent Speech Interfaces for Speech Restoration: A Review

This work was supported in part by the Agencia Estatal de Investigacion (AEI) under Grant PID2019-108040RB-C22/AEI/10.13039/501100011033. The work of Jose A. Gonzalez-Lopez was supported in part by the Spanish Ministry of Science, Innovation and Universities under Juan de la Cierva-Incorporation Fellowship (IJCI-2017-32926).This review summarises the status of silent speech interface (SSI) research. SSIs rely on non-acoustic biosignals generated by the human body during speech production to enable communication whenever normal verbal communication is not possible or not desirable. In this review, we focus on the first case and present latest SSI research aimed at providing new alternative and augmentative communication methods for persons with severe speech disorders. SSIs can employ a variety of biosignals to enable silent communication, such as electrophysiological recordings of neural activity, electromyographic (EMG) recordings of vocal tract movements or the direct tracking of articulator movements using imaging techniques. Depending on the disorder, some sensing techniques may be better suited than others to capture speech-related information. For instance, EMG and imaging techniques are well suited for laryngectomised patients, whose vocal tract remains almost intact but are unable to speak after the removal of the vocal folds, but fail for severely paralysed individuals. From the biosignals, SSIs decode the intended message, using automatic speech recognition or speech synthesis algorithms. Despite considerable advances in recent years, most present-day SSIs have only been validated in laboratory settings for healthy users. Thus, as discussed in this paper, a number of challenges remain to be addressed in future research before SSIs can be promoted to real-world applications. If these issues can be addressed successfully, future SSIs will improve the lives of persons with severe speech impairments by restoring their communication capabilities.Agencia Estatal de Investigacion (AEI) PID2019-108040RB-C22/AEI/10.13039/501100011033Spanish Ministry of Science, Innovation and Universities under Juan de la Cierva-Incorporation Fellowship IJCI-2017-3292