A novel onset detection technique for brain?computer interfaces using sound-production related cognitive tasks in simulated-online system

Objective. Self-paced EEG-based BCIs (SP-BCIs) have traditionally been avoided due to two sources of uncertainty: (1) precisely when an intentional command is sent by the brain, i.e., the command onset detection problem, and (2) how different the intentional command is when compared to non-specific (or idle) states. Performance evaluation is also a problem and there are no suitable standard metrics available. In this paper we attempted to tackle these issues. Approach. Self-paced covert sound-production cognitive tasks (i.e., high pitch and siren-like sounds) were used to distinguish between intentional commands (IC) and idle states. The IC states were chosen for their ease of execution and negligible overlap with common cognitive states. Band power and a digital wavelet transform were used for feature extraction, and the Davies?Bouldin index was used for feature selection. Classification was performed using linear discriminant analysis. Main results. Performance was evaluated under offline and simulated-online conditions. For the latter, a performance score called true-false-positive (TFP) rate, ranging from 0 (poor) to 100 (perfect), was created to take into account both classification performance and onset timing errors. Averaging the results from the best performing IC task for all seven participants, an 77.7% true-positive (TP) rate was achieved in offline testing. For simulated-online analysis the best IC average TFP score was 76.67% (87.61% TP rate, 4.05% false-positive rate). Significance. Results were promising when compared to previous IC onset detection studies using motor imagery, in which best TP rates were reported as 72.0% and 79.7%, and which, crucially, did not take timing errors into account. Moreover, based on our literature review, there is no previous covert sound-production onset detection system for spBCIs. Results showed that the proposed onset detection technique and TFP performance metric have good potential for use in SP-BCIs

Discriminating between imagined speech and non-speech tasks using EEG

People who are severely disabled (e.g Locked-in patients) need a communication tool translating their thoughts using their brain signals. This technology should be intuitive and easy to use. To this line, this study investigates the possibility of discriminating between imagined speech and two types of non-speech tasks related to either a visual stimulus or relaxation. In comparison to previous studies, this work examines a variety of different words with only single imagination in each trial. Moreover, EEG data are recorded from a small number of electrodes using a low-cost portable EEG device. Thus, our experiment is closer to what we want to achieve in the future as communication tool for locked-in patients. However, this design makes the EEG classification more challenging due to a higher level of noise and variations in EEG signals. Spectral and temporal features, with and without common spatial filtering, were used for classifying every imagined word ( and for a group of words) against the non-speech tasks. The results show the potential for discriminating between each imagined word and non-speech tasks. Importantly, the results are different between subjects using different features showing the need for having subject specific features

Nonuniform Power Changes and Spatial, Temporal and Spectral Diversity in High Gamma Band (\u3e60 Hz) Signals in Human Electrocorticography

High-gamma band: \u3e60Hz) power changes in cortical electrophysiology are a reliable indicator of focal, event-related cortical activity. In spite of discoveries of oscillatory subthreshold and synchronous suprathreshold activity at the cellular level, there is an increasingly popular view that high-gamma band amplitude changes recorded from cellular ensembles are the result of asynchronous firing activity that yields wideband and uniform power increases. Others have demonstrated independence of power changes in the low- and high-gamma bands, but to date, no studies have shown evidence of any such independence above 60Hz. Based on non-uniformities in time-frequency analyses of electrocorticographic: ECoG) signals, we hypothesized that induced high-gamma band: 60-500Hz) power changes are more heterogeneous than currently understood. We quantified this spectral non-uniformity with two different approaches using single-word repetition tasks in human subjects. First, we showed that the functional responsiveness of different ECoG high-gamma sub-bands can discriminate cognitive tasks: e.g., hearing, reading, speaking) and cortical locations. Power changes in these sub-bands of the high-gamma range are consistently present within single trials and have statistically different time courses within the trial structure. Moreover, when consolidated across all subjects within three task-relevant anatomic regions: sensorimotor, Broca\u27s area, and superior temporal gyrus), these behavior- and location- dependent power changes evidenced nonuniform trends across the population of subjects. Second, we studied the dynamics of multiple frequency bands in order to quantify the diversity present in the ECoG signals. Using a matched filter construct and receiver operating characteristic: ROC) analysis we show that power modulations correlated with phonemic content in spoken and heard words are represented diffusely in space, time and frequency. Correlating power modulation in multiple frequency bands above 60 Hz over broad cortical areas, with time varying envelopes significantly improved performed area under the ROC curve scores in phoneme prediction experiments. Finally we show preliminary evidence supporting our hypothesis in microarray ECoG data. Taken together, the nonuniformity of high frequency power changes and the information content captured in the spatio-temporal dynamics of those frequencies suggests that a new approach to evaluating high-gamma band cortical activity is necessary. These findings show that in addition to time and location, frequency is another fundamental dimension of high-gamma dynamics

Brain Computer Interfaces for the Control of Robotic Swarms

abstract: A robotic swarm can be defined as a large group of inexpensive, interchangeable robots with limited sensing and/or actuating capabilities that cooperate (explicitly or implicitly) based on local communications and sensing in order to complete a mission. Its inherent redundancy provides flexibility and robustness to failures and environmental disturbances which guarantee the proper completion of the required task. At the same time, human intuition and cognition can prove very useful in extreme situations where a fast and reliable solution is needed. This idea led to the creation of the field of Human-Swarm Interfaces (HSI) which attempts to incorporate the human element into the control of robotic swarms for increased robustness and reliability. The aim of the present work is to extend the current state-of-the-art in HSI by applying ideas and principles from the field of Brain-Computer Interfaces (BCI), which has proven to be very useful for people with motor disabilities. At first, a preliminary investigation about the connection of brain activity and the observation of swarm collective behaviors is conducted. After showing that such a connection may exist, a hybrid BCI system is presented for the control of a swarm of quadrotors. The system is based on the combination of motor imagery and the input from a game controller, while its feasibility is proven through an extensive experimental process. Finally, speech imagery is proposed as an alternative mental task for BCI applications. This is done through a series of rigorous experiments and appropriate data analysis. This work suggests that the integration of BCI principles in HSI applications can be successful and it can potentially lead to systems that are more intuitive for the users than the current state-of-the-art. At the same time, it motivates further research in the area and sets the stepping stones for the potential development of the field of Brain-Swarm Interfaces (BSI).Dissertation/ThesisMasters Thesis Mechanical Engineering 201

A novel EEG based linguistic BCI

While a human being can think coherently, physical limitations no matter how severe, should never become disabling. Thinking and cognition are performed and expressed through language, which is the most natural form of human communication. The use of covert speech tasks for BCIs has been successfully achieved for invasive and non-invasive systems. In this work, by incorporating the most recent discoveries on the spatial, temporal, and spectral signatures of word production, a novel system is designed, which is custom-build for linguistic tasks. Other than paying attention and waiting for the onset cue, this BCI requires absolutely no cognitive effort from the user and operates using automatic linguistic functions of the brain in the first 312ms post onset, which is also completely out of the control of the user and immune from inconsistencies. With four classes, this online BCI achieves classification accuracy of 82.5%. Each word produces a signature as unique as its phonetic structure, and the number of covert speech tasks used in this work is limited by computational power. We demonstrated that this BCI can successfully use wireless dry electrode EEG systems, which are becoming as capable as traditional laboratory grade systems. This frees the potential user from the confounds of the lab, facilitating real-world application. Considering that the number of words used in daily life does not exceed 2000, the number of words used by this type of novel BCI may indeed reach this number in the future, with no need to change the current system design or experimental protocol. As a promising step towards noninvasive synthetic telepathy, this system has the potential to not only help those in desperate need, but to completely change the way we communicate with our computers in the future as covert speech is much easier than any form of manual communication and control

Caracterización de la sincronía de fase de EEG para su aplicación en Interfaces Cerebro-Computadora

Dentro de la línea de investigación de Interfaces-Cerebro Computadora (BCI, por sus siglas en inglés), uno de los retos en el diseño y construcción de estos sistemas es identificar la intención real del usuario de emplear el sistema, es decir, que el usuario tenga la libertad para decidir cuándo interactuar con el sistema y con qué frecuencia, para lo cual es necesario un paradigma que no dependa de estímulos externos y poder tomar decisiones binarias (por ejemplo, encender o apagar la BCI), sin que el usuario esté atado o sincronizado a estímulos externos. En otras palabras, una BCI independiente y asincrónica, que hasta la fecha es motivo de investigación en búsqueda de alternativas efectivas. En este trabajo se propone evaluar la imaginación musical como una opción para ser utilizada como tarea de control para una BCI de estas características, dada la extensa red neurológica que implica al abarcar diversas modalidades de imaginación, sea auditiva, visual, motora, entre otras. Aunado a esto, uno de los aportes principales de esta investigación es la metodología propuesta para analizar la imaginación musical, cuyo objetivo es caracterizar las relaciones de fase entre series de tiempo multivariadas de la actividad eléctrica cortical obtenida a partir del electroencefalograma (EEG), representando dichas relaciones mediante conglomerados (clusters) de electrodos altamente sincronizados. El método está basado en un nuevo algoritmo de agrupamiento propuesto en este trabajo, denominado Agrupamiento (clustering) por Sincronía de Fase de Series de Tiempo Multivariadas (mCPS, por sus siglas en inglés), donde la idea principal es generar conglomerados difusos (un solo dato puede tener distintos grados de membresía en los conglomerados) para cada muestra multivariada en el tiempo t, e iterativamente refinar la agrupación hasta obtener conglomerados duros (cada dato asignado a un solo conglomerado), de acuerdo a un umbral de varianza circular, que es una medida estadística para datos angulares. Los resultados del mCPS se representan en mapas TiempoFrecuencia-Topográficos, de esta manera se puede hacer un inspección visual de los patrones de sincronía representados por conglomerados de canales. En una primera etapa, la metodología propuesta se utilizó para evaluar un paradigma exógeno de BCI (P300). Los resultados permitieron caracterizar la información neurológica asociada al potencial cognitivo en cuestión, encontrando mediante mCPS conglomerados de señales que coinciden con el contenido tiempo-frecuencia de las épocas del EEG que contienen P300, caso contrario con aquellas épocas donde no se encuentra el potencial, permitiendo hacer un contraste de las diferencias de ambas condiciones. Como valor agregado, también se identificó un artefacto de estado estable (que podría ser información relevante si se tratase de un paradigma con potenciales de esta naturaleza). Dentro de la metodología se incorpora la distancia de Levenshtein como métrica para discernir las diferencias entre condiciones, como fue en el caso del paradigma exógeno evaluado. En una etapa posterior, se aplicó la metodología en un paradigma endógeno con imaginación musical (diseñado y propuesto para este trabajo), donde se aprecian diferencias entre la condición de estado basal y las condiciones de imaginación, quedando abierta la posibilidad de seguir explorando y hacer los ajustes necesarios para recuperar rasgos que no solo permitan distinguir entre estados mentales, sino que puedan asociarse directamente a una tarea en específico. La metodología propuesta provee de una herramienta novedosa para el análisis de sincronía de fase de señales de EEG y la caracterización de su variabilidad en el tiempo, dando un amplio panorama del comportamiento de la sincronización de fase en las bandas de EEG de interés y su localización temporal, tomando en cuenta algunos aspectos como la noestacionariedad de la frecuencia de sincronización y la flexibilidad para el uso de otras medidas de sincronización, además de la varianza circular. Este trabajo es entonces una ventana de observación de la actividad eléctrica cerebral tomada del EEG para evaluar la integración a gran escala de patrones de sincronización de fase instantánea que emergen durante un estado mental, abstrayendo dichos patrones en arreglos de conglomerados de sincronización de fase sobre la serie de tiempo de señales de EEG.A mayor challenge in Brain-Computer Interfaces (BCI) development is to identify whether the user really needs to interact with the system; thereby, the subject has free will to make binary decisions (for example, to choose when to turn on/off the BCI). Thus, an independent (of external stimuli) and asynchronous BCI paradigm is well suited to fulfill this issue, and effective solutions are still on demand. Music imagery involves different types of imagination, such as auditory, visual, motor, among others, which implies neural activity over a broad neurological network. In this work, assessment of music imagery as control task for BCI is proposed. In addition to this, within the main contributions is the method to perform music imagery analysis, which aims to characterize phase relationships between multivariate time series of cortical electrical activity obtained from the electroencephalogram (EEG), representing such relationships in clusters of highly synchronized multichannel data. The framework relies in a novel clustering algorithm, termed Multivariate Time Series Clustering by Phase Synchrony (mCPS), where the main idea is to generate fuzzy clusters (a data point might have different degrees of membership in each output cluster) for each multivalued time sample t, and thereupon obtain hard clusters (the data point is only assigned to one cluster) according to a circular variance threshold, which is a measure of circular spread of angular data. The method was used to evaluate an exogenous BCI paradigm (P300) and the endogenous paradigm involving musical imagination. The neural data associated with the P300 wave was successfully characterized. Regarding the music imagery mental task, differences between baseline and imagination mental states were found, with the possibility of making necessary adjustments for retrieving features that could be directly associated with the mental task in forthcoming research. This research provides a novel tool for analysis of phase synchrony of EEG signals and characterization of their variability over time