Exploring combinations of different color and facial expression stimuli for gaze-independent BCIs

Background: Some studies have proven that a conventional visual brain computer interface (BCI) based on overt attention cannot be used effectively when eye movement control is not possible. To solve this problem, a novel visual-based BCI system based on covert attention and feature attention has been proposed and was called the gaze-independent BCI. Color and shape difference between stimuli and backgrounds have generally been used in examples of gaze-independent BCIs. Recently, a new paradigm based on facial expression changes has been presented, and obtained high performance. However, some facial expressions were so similar that users couldn't tell them apart, especially when they were presented at the same position in a rapid serial visual presentation (RSVP) paradigm. Consequently, the performance of the BCI is reduced. New Method: In this paper, we combined facial expressions and colors to optimize the stimuli presentation in the gaze-independent BCI. This optimized paradigm was called the colored dummy face pattern. It is suggested that different colors and facial expressions could help users to locate the target and evoke larger event-related potentials (ERPs). In order to evaluate the performance of this new paradigm, two other paradigms were presented, called the gray dummy face pattern and the colored ball pattern. Comparison with Existing Method(s): The key point that determined the value of the colored dummy faces stimuli in BCI systems was whether the dummy face stimuli could obtain higher performance than gray faces or colored balls stimuli. Ten healthy participants (seven male, aged 21–26 years, mean 24.5 ± 1.25) participated in our experiment. Online and offline results of four different paradigms were obtained and comparatively analyzed. Results: The results showed that the colored dummy face pattern could evoke higher P300 and N400 ERP amplitudes, compared with the gray dummy face pattern and the colored ball pattern. Online results showed that the colored dummy face pattern had a significant advantage in terms of classification accuracy (p < 0.05) and information transfer rate (p < 0.05) compared to the other two patterns. Conclusions: The stimuli used in the colored dummy face paradigm combined color and facial expressions. This had a significant advantage in terms of the evoked P300 and N400 amplitudes and resulted in high classification accuracies and information transfer rates. It was compared with colored ball and gray dummy face stimuli

Toward a semi-self-paced EEG brain computer interface: decoding initiation state from non-initiation state in dedicated time slots.

Brain computer interfaces (BCIs) offer a broad class of neurologically impaired individuals an alternative means to interact with the environment. Many BCIs are "synchronous" systems, in which the system sets the timing of the interaction and tries to infer what control command the subject is issuing at each prompting. In contrast, in "asynchronous" BCIs subjects pace the interaction and the system must determine when the subject's control command occurs. In this paper we propose a new idea for BCI which draws upon the strengths of both approaches. The subjects are externally paced and the BCI is able to determine when control commands are issued by decoding the subject's intention for initiating control in dedicated time slots. A single task with randomly interleaved trials was designed to test whether it can be used as stimulus for inducing initiation and non-initiation states when the sensory and motor requirements for the two types of trials are very nearly identical. Further, the essential problem on the discrimination between initiation state and non-initiation state was studied. We tested the ability of EEG spectral power to distinguish between these two states. Among the four standard EEG frequency bands, beta band power recorded over parietal-occipital cortices provided the best performance, achieving an average accuracy of 86% for the correct classification of initiation and non-initiation states. Moreover, delta band power recorded over parietal and motor areas yielded a good performance and thus could also be used as an alternative feature to discriminate these two mental states. The results demonstrate the viability of our proposed idea for a BCI design based on conventional EEG features. Our proposal offers the potential to mitigate the signal detection challenges of fully asynchronous BCIs, while providing greater flexibility to the subject than traditional synchronous BCIs

Language Model Applications to Spelling with Brain-Computer Interfaces

Within the Ambient Assisted Living (AAL) community, Brain-Computer Interfaces (BCIs) have raised great hopes as they provide alternative communication means for persons with disabilities bypassing the need for speech and other motor activities. Although significant advancements have been realized in the last decade, applications of language models (e.g., word prediction, completion) have only recently started to appear in BCI systems. The main goal of this article is to review the language model applications that supplement non-invasive BCI-based communication systems by discussing their potential and limitations, and to discern future trends. First, a brief overview of the most prominent BCI spelling systems is given, followed by an in-depth discussion of the language models appli

Interfacce cervello-computer per la comunicazione aumentativa: algoritmi asincroni e adattativi e validazione con utenti finali

This thesis aimed at addressing some of the issues that, at the state of the art, avoid the P300-based brain computer interface (BCI) systems to move from research laboratories to end users’ home. An innovative asynchronous classifier has been defined and validated. It relies on the introduction of a set of thresholds in the classifier, and such thresholds have been assessed considering the distributions of score values relating to target, non-target stimuli and epochs of voluntary no-control. With the asynchronous classifier, a P300-based BCI system can adapt its speed to the current state of the user and can automatically suspend the control when the user diverts his attention from the stimulation interface. Since EEG signals are non-stationary and show inherent variability, in order to make long-term use of BCI possible, it is important to track changes in ongoing EEG activity and to adapt BCI model parameters accordingly. To this aim, the asynchronous classifier has been subsequently improved by introducing a self-calibration algorithm for the continuous and unsupervised recalibration of the subjective control parameters. Finally an index for the online monitoring of the EEG quality has been defined and validated in order to detect potential problems and system failures. This thesis ends with the description of a translational work involving end users (people with amyotrophic lateral sclerosis-ALS). Focusing on the concepts of the user centered design approach, the phases relating to the design, the development and the validation of an innovative assistive device have been described. The proposed assistive technology (AT) has been specifically designed to meet the needs of people with ALS during the different phases of the disease (i.e. the degree of motor abilities impairment). Indeed, the AT can be accessed with several input devices either conventional (mouse, touchscreen) or alterative (switches, headtracker) up to a P300-based BCI.Questa tesi affronta alcune delle problematiche che, allo stato dell'arte, limitano l'usabilità delle interfacce cervello computer (Brain Computer Interface - BCI) al di fuori del contesto sperimentale. E' stato inizialmente definito e validato un classificatore asincrono. Quest'ultimo basa il suo funzionamento sull'inserimento di un set di soglie all'interno del classificatore. Queste soglie vengono definite considerando le distribuzioni dei valori di score relativi agli stimoli target e non-target e alle epoche EEG in cui il soggetto non intendeva effettuare nessuna selezione (no-control). Con il classificatore asincrono, un BCI basato su potenziali P300 può adattare la sua velocità allo stato corrente dell'utente e sospendere automaticamente il controllo quando l'utente non presta attenzione alla stimolazione. Dal momento che i segnali EEG sono non-stazionari e mostrano una variabilità intrinseca, al fine di rendere possibile l'utilizzo dei sistemi BCI sul lungo periodo, è importante rilevare i cambiamenti dell'attività EEG e adattare di conseguenza i parametri del classificatore. A questo scopo, il classificatore asincrono è stato successivamente migliorato introducendo un algoritmo di autocalibrazione per la continua e non supervisionata ricalibrazione dei parametri di controllo soggettivi. Infine è stato definito e validato un indice per monitorare on-line la qualità del segnale EEG, in modo da rilevare potenziali problemi e malfunzionamenti del sistema. Questa tesi si conclude con la descrizione di un lavoro che ha coinvolto gli utenti finali (persone affette da sclerosi laterale amiotrofica-SLA). In particolare, basandosi sui principi dell’user-centered design, sono state descritte le fasi relative alla progettazione, sviluppo e validazione di una tecnologia assistiva (TA) innovativa. La TA è stata specificamente progettata per rispondere alla esigenze delle persone affetta da SLA durante le diverse fasi della malattia. Infatti, la TA proposta può essere utilizzata sia mediante dispositivi d’input tradizionali (mouse, tastiera) che alternativi (bottoni, headtracker) fino ad arrivare ad un BCI basato su potenziali P300

Classification for Single-Trial N170 During Responding to Facial Picture With Emotion

Whether an event-related potential (ERP), N170, related to facial recognition was modulated by emotion has always been a controversial issue. Some researchers considered the N170 to be independent of emotion, whereas a recent study has shown the opposite view. In the current study, electroencephalogram (EEG) recordings while responding to facial pictures with emotion were utilized to investigate whether the N170 was modulated by emotion. We found that there was a significant difference between ERP trials with positive and negative emotions of around 170 ms at the occipitotemporal electrodes (i.e., N170). Then, we further proposed the application of the single-trial N170 as a feature for the classification of facial emotion, which could avoid the fact that ERPs were obtained by averaging most of the time while ignoring the trial-to-trial variation. In order to find an optimal classifier for emotional classification with single-trial N170 as a feature, three types of classifiers, namely, linear discriminant analysis (LDA), L1-regularized logistic regression (L1LR), and support vector machine with radial basis function (RBF-SVM), were comparatively investigated. The results showed that the single-trial N170 could be used as a classification feature to successfully distinguish positive emotion from negative emotion. L1-regularized logistic regression classifiers showed a good generalization, whereas LDA showed a relatively poor generalization. Moreover, when compared with L1LR, the RBF-SVM required more time to optimize the parameters during the classification, which became an obstacle while applying it to the online operating system of brain-computer interfaces (BCIs). The findings suggested that face-related N170 could be affected by facial expression and that the single-trial N170 could be a biomarker used to monitor the emotional states of subjects for the BCI domain

ON THE INTERPLAY BETWEEN BRAIN-COMPUTER INTERFACES AND MACHINE LEARNING ALGORITHMS: A SYSTEMS PERSPECTIVE

Today, computer algorithms use traditional human-computer interfaces (e.g., keyboard, mouse, gestures, etc.), to interact with and extend human capabilities across all knowledge domains, allowing them to make complex decisions underpinned by massive datasets and machine learning. Machine learning has seen remarkable success in the past decade in obtaining deep insights and recognizing unknown patterns in complex data sets, in part by emulating how the brain performs certain computations. As we increase our understanding of the human brain, brain-computer interfaces can benefit from the power of machine learning, both as an underlying model of how the brain performs computations and as a tool for processing high-dimensional brain recordings. The technology (machine learning) has come full circle and is being applied back to understanding the brain and any electric residues of the brain activity over the scalp (EEG). Similarly, domains such as natural language processing, machine translation, and scene understanding remain beyond the scope of true machine learning algorithms and require human participation to be solved. In this work, we investigate the interplay between brain-computer interfaces and machine learning through the lens of end-user usability. Specifically, we propose the systems and algorithms to enable synergistic and user-friendly integration between computers (machine learning) and the human brain (brain-computer interfaces). In this context, we provide our research contributions in two interrelated aspects by, (i) applying machine learning to solve challenges with EEG-based BCIs, and (ii) enabling human-assisted machine learning with EEG-based human input and implicit feedback.Ph.D

Psychologische Prädiktoren der Brain-Computer Interface Steuerung

Theorie: Brain-Computer Interfaces (BCIs) stellen eine direkte Verbindung zwischen dem Gehirn und einem Computer dar. Mit Hilfe von BCIs ist es daher möglich, die elektrischen Signale des Gehirns in Steuersignale umzuwandeln, um damit ein Anwendungsprogramm (z.B. Kommunikationsprogramm oder Navigation eines Rollstuhls) zu steuern. Für schwerbeeinträchtigte Personen, z.B. im sog. Locked-in Zustand, stellt ein BCI eine der letzten Möglichkeiten dar, mit der Umwelt in Kontakt zu treten. Jedoch ist es einer bedeutenden Anzahl von BCI Anwendern (Gesunden wie Kranken) nicht möglich, eine hinreichend hohe Trefferquote bei der BCI Steuerung zu erzielen. Die Überwindung dieses sog. „BCI Ineffizienz Phänomens“ stellt auch nach über 40 Jahren BCI Forschung immer noch eine große Herausforderung dar. Das Ziel der dieser Zusammenfassung zugrundeliegenden Forschungsarbeiten war es, psychologische Variablen zu identifizieren, die als Prädiktoren die Leistung in einem BCI vorhersagen können. Methode: Für alle drei Forschungsarbeiten wurden gesunde Studienteilnehmer, überwiegend Studenten mit keinerlei Vorerfahrung in der BCI Steuerung, rekrutiert. Die psychologischen Tests wurden drei Untergruppen (Leistungstests, Persönlichkeitstests und klinische Tests) zugeteilt und alle in elektronischer Form präsentiert und bearbeitet. In den ersten beiden Studien wurden BCIs eingesetzt, die sensomotorische Rhythmen als Steuerungssignale nutzen. In der dritten Studie wurde ein BCI, das auf dem ereigniskorrelierten Potential P300 basiert, eingesetzt. Die Prädiktoranalyse erfolgte mit Hilfe linearer Regressionsanalysen. Ergebnisse: In der ersten Studie wurde das Berliner Brain-Computer Interface (BBCI) eingesetzt, das auf den Techniken des maschinellen Lernens basiert. Die visuomotorische Koordinationsfähigkeit (Variable „mittlere Fehlerdauer gesamt“, gemessen mit dem Zwei-Hand-Test) wurde mit 11,4% Varianzaufklärung als signifikanter Prädiktor identifiziert. Die Variable „Leistungsniveau“ (Maß für Konzentrationsfähigkeit) aus dem Test Arbeitshaltungen zeigte ebenfalls eine signifikante Korrelation mit der Leistung im BCI, verfehlte das Signifikanzniveau im Regressionsmodell jedoch knapp. Ziel der zweiten Studie war es, die Ergebnisse der ersten zu replizieren und auf Basis des ersten Regressionsmodells, die BCI Leistungen in der zweiten Studie, in der ein klassisches Neurofeedback SMR-BCI eingesetzt wurde, vorherzusagen. Die Variablen visuomotorische Koordinationsfähigkeit und „aufmerksamkeitsbasierte Impulsivität“ klärten hierbei fast 20% der Gesamtvarianz auf. Auf Basis des ersten Regressionsmodells war es möglich, die BCI Leistung mit einem durchschnittlichen Vorhersagefehler von M = 12.07% vorherzusagen. In der dritten Studie korrelierte der Persönlichkeitsfaktor „Emotionale Stabilität“ negativ und eine Leistungsvariable des Nonverbalen Lerntests, die die Lernfähigkeit eines Probanden erfasst, positiv mit der Trefferquote im visuellen P300-BCI. Beide Variablen klärten 24% an der Gesamtvarianz auf, wobei die Lernvariable mit 19% Varianzaufklärung als signifikanter Prädiktor identifiziert wurde. Die Variable „Emotionale Stabilität“ korrelierte ebenfalls negativ mit der Trefferquote im auditorischen P300-BCI, wurde jedoch nicht als signifikanter Prädiktor identifiziert. Schlussfolgerung: Die drei Forschungsarbeiten bestätigen einen moderaten Einfluss psychologischer Variablen auf die BCI Steuerung in unterschiedlichen BCI Paradigmen. Die Ergebnisse zeigen große Überschneidungen mit anderen Studienergebnissen. Basierend auf diesen Ergebnissen können weitere Studien entwickelt werden, mit dem Ziel, bestehende BCI Systeme zu adaptieren und Trainingsprogramme (z.B. für das Training visuomotorischer Koordinationsfähigkeit) für BCI Anwender zu entwickeln