Towards a home-use BCI: fast asynchronous control and robust non-control state detection

Eine Hirn-Computer Schnittstelle (engl. Brain-Computer Interface, BCI) erlaubt einem Nutzer einen Computer nur mittels Gehirn-Aktivität zu steuern. Der Hauptanwendungszweck ist die Wiederherstellung verschiedener Funktionen von motorisch eingeschränkten Menschen, zum Beispiel, die Wiederherstellung der Kommunikationsfähigkeit. Bisherige BCIs die auf visuell evozierten Potentialen (VEPs) basieren, erlauben bereits hohe Kommunikationsgeschwindigkeiten. VEPs sind Reaktionen, die im Gehirn durch visuelle Stimulation hervorgerufen werden. Allerdings werden bisherige BCIs hauptsächlich in der Forschung verwendet und sind nicht für reale Anwendungszwecke geeignet. Grund dafür ist, dass sie auf dem synchronen Steuerungsprinzip beruhen, dies bedeutet, dass Aktionen nur in vorgegebenen Zeitslots ausgeführt werden können. Dies bedeutet wiederum, dass der Nutzer keine Aktionen nach seinem Belieben ausführen kann, was für reale Anwendungszwecke ein Problem darstellt. Um dieses Problem zu lösen, müssen BCIs die Intention des Nutzers, das System zu steuern oder nicht, erkennen. Solche BCIs werden asynchron oder selbstbestimmt genannt. Bisherige asynchrone BCIs zeigen allerdings keine ausreichende Genauigkeit bei der Erkennung der Intention und haben zudem eine deutlich reduzierte Kommunikationsgeschwindigkeit im Vergleich zu synchronen BCIs. In dieser Doktorarbeit wird das erste asynchrone BCI vorgestellt, welches sowohl eine annäherungsweise perfekte Erkennung der Intention des Nutzers als auch eine ähnliche Kommunikationsgeschwindigkeit wie synchrone BCIs erzielt. Dies wurde durch die Entwicklung eines allgemeinen Modells für die Vorhersage von sensorischen Reizen erzielt. Dadurch können beliebige visuelle Stimulationsmuster basierend auf den gemessenen VEPs vorhergesagt werden. Das Modell wurde sowohl mit einem "traditionellen" maschinellen Lernverfahren als auch mit einer deep-learning Methode implementiert und evaluiert. Das resultierende asynchrone BCI übertrifft bisherige Methoden in mehreren Disziplinen um ein Vielfaches und ist ein wesentlicher Schritt, um BCI-Anwendungen aus dem Labor in die Praxis zu bringen. Durch weitere Optimierungen, die in dieser Arbeit diskutiert werden, könnte es sich zum allerersten geeigneten BCI für Endanwender entwickeln, da es effektiv (hohe Genauigkeit), effizient (schnelle Klassifizierungen), und einfach zu bedienen ist. Ein weiteres Alleinstellungsmerkmal ist, dass das entwickelte BCI für beliebige Szenarien verwendet werden kann, da es annähernd unendlich viele gleichzeitige Aktionsfelder erlaubt.Brain-Computer Interfaces (BCIs) enable users to control a computer by using pure brain activity. Their main purpose is to restore several functionalities of motor disabled people, for example, to restore the communication ability. Recent BCIs based on visual evoked potentials (VEPs), which are brain responses to visual stimuli, have shown to achieve high-speed communication. However, BCIs have not really found their way out of the lab yet. This is mainly because all recent high-speed BCIs are based on synchronous control, which means commands can only be executed in time slots controlled by the BCI. Therefore, the user is not able to select a command at his own convenience, which poses a problem in real-world applications. Furthermore, all those BCIs are based on stimulation paradigms which restrict the number of possible commands. To be suitable for real-world applications, a BCI should be asynchronous, or also called self-paced, and must be able to identify the user’s intent to control the system or not. Although there some asynchronous BCI approaches, none of them achieved suitable real-world performances. In this thesis, the first asynchronous high-speed BCI is proposed, which allows using a virtually unlimited number of commands. Furthermore, it achieved a nearly perfect distinction between intentional control (IC) and non-control (NC), which means commands are only executed if the user intends to. This was achieved by a completely different approach, compared to recent methods. Instead of using a classifier trained on specific stimulation patterns, the presented approach is based on a general model that predicts arbitrary stimulation patterns. The approach was evaluated with a "traditional" as well as a deep machine learning method. The resultant asynchronous BCI outperforms recent methods by a multi-fold in multiple disciplines and is an essential step for moving BCI applications out of the lab and into real life. With further optimization, discussed in this thesis, it could evolve to the very first end-user suitable BCI, as it is effective (high accuracy), efficient (fast classifications), ease of use, and allows to perform as many different tasks as desired

Control of transcranial brain stimulation by a brain-computer interface based loop

Tese de mestrado integrado em Engenharia Biomédica e Biofísica, apresentada à Universidade de Lisboa, através da Faculdade de Ciências, 2014A Estimulação Transcraniana com Corrente Alternada (tACS) é uma forma de estimulação que permite a influência direta das oscilações cerebrais. Entre outros efeitos, foi demonstrado que quando a aplicação de tACS se inicia em fase com o sinal cerebral de interesse, a perceção a estímulos visuais apresentados aumenta. Presentemente, a aplicação de tACS não obedece a nenhum tipo de sistema de feedback. A estimulação é simplesmente iniciada num qualquer momento, o que dificulta o controlo sobre o tipo de efeitos em estudo. Como tal, a presente tese propõe a utilização de ferramentas do foro de Brain-Computer Interfaces (BCIs) na criação de um sistema fechado de controlo dos parâmetros da estimulação, nomeadamente a fase inicial. Assim, três metodologias para deteção dos parâmetros a partir do electroencefalograma (EEG) espontâneo e ajuste dos parâmetros de estimulação são propostos e testados em conjuntos de dados artificiais. O melhor dos três métodos é aplicado a um conjunto de dados de EEG real. O resultado é supreendentemente instável (comparável ao obtido para um sinal de ruído Gaussiano). Assim, o foco da investigação muda para uma análise detalhada da estabilidade de sinais de EEG real, com base em metodologias inicialmente destinadas ao estudo de sincronia neuronal. Mostra-se que não há evidência para sustentar a assunção clássica de que a fase dos sinais cerebrais numa determinada banda de frequências é estável, pelo menos durante um curto intervalo de tempo. A presente tese representa um passo importante no sentido de compreender uma característica do EEG que muitas vezes se considera como bem conhecida e estudada, mas sobre a qual existem muitas dúvidas e lacunas de conhecimento. É ainda um avanço no problema de como aproximar um sistema de controlo fechado para tACS.Transcranial Alternating Current Stimulation (tACS) is a technique that enables the direct influence of ongoing brain oscillations. Among other after-effects, it has been shown to enhance perception to visual stimuli when started with the same phase as the ongoing brain oscillation of interest. Currently tACS is delivered using a feedforward paradigm, and thus it is very difficult to ensure that the parameters of stimulation meet the optimal requirements for studying a determined response. Thus, the present thesis proposes the use of Brain-Computer Interface (BCI) tools for devising a feedback loop to control the parameters of stimulation. Different methods are proposed and tested using artificial data. The best one is chosen, and tested on real Electroencephalogram (EEG) data. This yields surprisingly unstable results, that lead to a detailed investigation of the stability of the phase of real signals. Several datasets are analysed using systematic methodologies based on tools devised for the study of neuronal synchrony. The results for real signals are compared to artificially generated noise signals. It is shown that there is no evidence to support a claim of stability of phase behaviour along short time intervals, unlike what is assumed in classical EEG analysis. The present thesis presents an important step towards understanding a widely overlooked feature in EEG, and in tackling the problem of a feedback loop for tACS

Development of a Practical Visual-Evoked Potential-Based Brain-Computer Interface

There are many different neuromuscular disorders that disrupt the normal communication pathways between the brain and the rest of the body. These diseases often leave patients in a `locked-in state, rendering them unable to communicate with their environment despite having cognitively normal brain function. Brain-computer interfaces (BCIs) are augmentative communication devices that establish a direct link between the brain and a computer. Visual evoked potential (VEP)- based BCIs, which are dependent upon the use of salient visual stimuli, are amongst the fastest BCIs available and provide the highest communication rates compared to other BCI modalities. However. the majority of research focuses solely on improving the raw BCI performance; thus, most visual BCIs still suffer from a myriad of practical issues that make them impractical for everyday use. The focus of this dissertation is on the development of novel advancements and solutions that increase the practicality of VEP-based BCIs. The presented work shows the results of several studies that relate to characterizing and optimizing visual stimuli. improving ergonomic design. reducing visual irritation, and implementing a practical VEP-based BCI using an extensible software framework and mobile devices platforms

Non-binary m-sequences for more comfortable brain–computer interfaces based on c-VEPs

Producción CientíficaCode-modulated visual evoked potentials (c-VEPs) have marked a milestone in the scientific literature due to their ability to achieve reliable, high-speed brain–computer interfaces (BCIs) for communication and control. Generally, these expert systems rely on encoding each command with shifted versions of binary pseudorandom sequences, i.e., flashing black and white targets according to the shifted code. Despite the excellent results in terms of accuracy and selection time, these high-contrast stimuli cause eyestrain for some users. In this work, we propose the use of non-binary p-ary m-sequences, whose levels are encoded with different shades of gray, as a more pleasant alternative than traditional binary codes. The performance and visual fatigue of these p-ary m-sequences, as well as their ability to provide reliable c-VEP-based BCIs, are analyzed for the first time.Ministerio de Ciencia e Innovación/AEI- FEDER [TED2021-129915B-I00, RTC2019-007350-1 y PID2020-115468RB-I00

Deep Learning Techniques for Electroencephalography Analysis

In this thesis we design deep learning techniques for training deep neural networks on electroencephalography (EEG) data and in particular on two problems, namely EEG-based motor imagery decoding and EEG-based affect recognition, addressing challenges associated with them. Regarding the problem of motor imagery (MI) decoding, we first consider the various kinds of domain shifts in the EEG signals, caused by inter-individual differences (e.g. brain anatomy, personality and cognitive profile). These domain shifts render multi-subject training a challenging task and impede robust cross-subject generalization. We build a two-stage model ensemble architecture and propose two objectives to train it, combining the strengths of curriculum learning and collaborative training. Our subject-independent experiments on the large datasets of Physionet and OpenBMI, verify the effectiveness of our approach. Next, we explore the utilization of the spatial covariance of EEG signals through alignment techniques, with the goal of learning domain-invariant representations. We introduce a Riemannian framework that concurrently performs covariance-based signal alignment and data augmentation, while training a convolutional neural network (CNN) on EEG time-series. Experiments on the BCI IV-2a dataset show that our method performs superiorly over traditional alignment, by inducing regularization to the weights of the CNN. We also study the problem of EEG-based affect recognition, inspired by works suggesting that emotions can be expressed in relative terms, i.e. through ordinal comparisons between different affective state levels. We propose treating data samples in a pairwise manner to infer the ordinal relation between their corresponding affective state labels, as an auxiliary training objective. We incorporate our objective in a deep network architecture which we jointly train on the tasks of sample-wise classification and pairwise ordinal ranking. We evaluate our method on the affective datasets of DEAP and SEED and obtain performance improvements over deep networks trained without the additional ranking objective