Analogue mouse pointer control via an online steady state visual evoked potential (SSVEP) brain-computer interface

The steady state visual evoked protocol has recently become a popular paradigm in brain–computer interface (BCI) applications. Typically (regardless of function) these applications offer the user a binary selection of targets that perform correspondingly discrete actions. Such discrete control systems are appropriate for applications that are inherently isolated in nature, such as selecting numbers from a keypad to be dialled or letters from an alphabet to be spelled. However motivation exists for users to employ proportional control methods in intrinsically analogue tasks such as the movement of a mouse pointer. This paper introduces an online BCI in which control of a mouse pointer is directly proportional to a user's intent. Performance is measured over a series of pointer movement tasks and compared to the traditional discrete output approach. Analogue control allowed subjects to move the pointer faster to the cued target location compared to discrete output but suffers more undesired movements overall. Best performance is achieved when combining the threshold to movement of traditional discrete techniques with the range of movement offered by proportional control

Comparison of PSDA and CCA detection methods in a SSVEP-based BCI-system

Using steady-state visually evoked potential (SSVEP) in brain-computer interface (BCI) systems is the subject of a lot of research. One of the most popular and widely used detection method is using a power spectral density analysis (PSDA). Lately there have been some new methods emerging, one of them is using canonical correlation analysis (CCA) which seems to have some promising improvements and advantages compared to traditional SSVEP detection methods, like better signal-to-noise ratio (SNR), lower inter-subject variability and the possibility to use harmonic frequencies, i.e., a serie of frequencies which have the same fundamental frequency. In this research two different SSVEP detection methods, one using PSDA and one using CCA are compared. The results show that the CCA-based detection method performs significantly better than the PSDA-based detection method. The increase of performance can in particular be seen when using harmonic frequencies. While the PSDA-based detection method has difficulties detecting harmonic frequencies, the CCA-based detection method is able to detect harmonic frequencies

A Survey of Stimulation Methods Used in SSVEP-Based BCIs

Brain-computer interface (BCI) systems based on the steady-state visual evoked potential (SSVEP) provide higher information throughput and require shorter training than BCI systems using other brain signals. To elicit an SSVEP, a repetitive visual stimulus (RVS) has to be presented to the user. The RVS can be rendered on a computer screen by alternating graphical patterns, or with external light sources able to emit modulated light. The properties of an RVS (e.g., frequency, color) depend on the rendering device and influence the SSVEP characteristics. This affects the BCI information throughput and the levels of user safety and comfort. Literature on SSVEP-based BCIs does not generally provide reasons for the selection of the used rendering devices or RVS properties. In this paper, we review the literature on SSVEP-based BCIs and comprehensively report on the different RVS choices in terms of rendering devices, properties, and their potential influence on BCI performance, user safety and comfort

Towards an SSVEP Based BCI With High ITR

A brain-computer interface (BCI) provides the possibility to translate brain neural activity patterns into control commands without movement by the user. In recent years, there has been increasing interest in using steady-state visual evoked potential (SSVEP) in BCI systems; the SSVEP approach provides currently the fastest and most reliable communication paradigm for the implementation of a non-invasive BCI system. However, many aspects of current system realizations need improvement, specifically in relation to speed (in terms of information transfer rate as well as time needed to perform a single command), user variability and ease of use. With these improvements in mind, this paper presents the Bremen-BCI, an online multi-channel SSVEP-based BCI system that operates on a conventional computer making use of the minimum energy combination method for extraction of power information associated with the SSVEP responses. An additional advantage of the presented methodology is that it is fully online, i.e., no calibration data for noise estimation, feature extraction, or electrode selection is needed, the system is ready to use once the subject is prepared. The SSVEP-based Bremen-BCI system with five targets, an adaptive time segment length between 0.75s and 4s, and six EEG channel locations on the occipital area, was used for online testing on 27 subjects. ALL participants were able to successfully complete spelling tasks with a mean accuracy of 93.83% and an information transfer rate (ITR) of 49.93 bit/min

Electroencephalogram Signal Processing For Hybrid Brain Computer Interface Systems

The goal of this research was to evaluate and compare three types of brain computer interface (BCI) systems, P300, steady state visually evoked potentials (SSVEP) and Hybrid as virtual spelling paradigms. Hybrid BCI is an innovative approach to combine the P300 and SSVEP. However, it is challenging to process the resulting hybrid signals to extract both information simultaneously and effectively. The major step executed toward the advancement to modern BCI system was to move the BCI techniques from traditional LED system to electronic LCD monitor. Such a transition allows not only to develop the graphics of interest but also to generate objects flickering at different frequencies. There were pilot experiments performed for designing and tuning the parameters of the spelling paradigms including peak detection for different range of frequencies of SSVEP BCI, placement of objects on LCD monitor, design of the spelling keyboard, and window time for the SSVEP peak detection processing. All the experiments were devised to evaluate the performance in terms of the spelling accuracy, region error, and adjacency error among all of the paradigms: P300, SSVEP and Hybrid. Due to the different nature of P300 and SSVEP, designing a hybrid P300-SSVEP signal processing scheme demands significant amount of research work in this area. Eventually, two critical questions in hybrid BCl are: (1) which signal processing strategy can best measure the user\u27s intent and (2) what a suitable paradigm is to fuse these two techniques in a simple but effective way. In order to answer these questions, this project focused mainly on developing signal processing and classification technique for hybrid BCI. Hybrid BCI was implemented by extracting the specific information from brain signals, selecting optimum features which contain maximum discrimination information about the speller characters of our interest and by efficiently classifying the hybrid signals. The designed spellers were developed with the aim to improve quality of life of patients with disability by utilizing visually controlled BCI paradigms. The paradigms consist of electrodes to record electroencephalogram signal (EEG) during stimulation, a software to analyze the collected data, and a computing device where the subject’s EEG is the input to estimate the spelled character. Signal processing phase included preliminary tasks as preprocessing, feature extraction, and feature selection. Captured EEG data are usually a superposition of the signals of interest with other unwanted signals from muscles, and from non-biological artifacts. The accuracy of each trial and average accuracy for subjects were computed. Overall, the average accuracy of the P300 and SSVEP spelling paradigm was 84% and 68.5 %. P300 spelling paradigms have better accuracy than both the SSVEP and hybrid paradigm. Hybrid paradigm has the average accuracy of 79 %. However, hybrid system is faster in time and more soothing to look than other paradigms. This work is significant because it has great potential for improving the BCI research in design and application of clinically suitable speller paradigm

Development of a Practical Visual-Evoked Potential-Based Brain-Computer Interface

There are many different neuromuscular disorders that disrupt the normal communication pathways between the brain and the rest of the body. These diseases often leave patients in a `locked-in state, rendering them unable to communicate with their environment despite having cognitively normal brain function. Brain-computer interfaces (BCIs) are augmentative communication devices that establish a direct link between the brain and a computer. Visual evoked potential (VEP)- based BCIs, which are dependent upon the use of salient visual stimuli, are amongst the fastest BCIs available and provide the highest communication rates compared to other BCI modalities. However. the majority of research focuses solely on improving the raw BCI performance; thus, most visual BCIs still suffer from a myriad of practical issues that make them impractical for everyday use. The focus of this dissertation is on the development of novel advancements and solutions that increase the practicality of VEP-based BCIs. The presented work shows the results of several studies that relate to characterizing and optimizing visual stimuli. improving ergonomic design. reducing visual irritation, and implementing a practical VEP-based BCI using an extensible software framework and mobile devices platforms

Development of a practical and mobile brain-computer communication device for profoundly paralyzed individuals

Thesis (Ph.D.)--Boston UniversityBrain-computer interface (BCI) technology has seen tremendous growth over the past several decades, with numerous groundbreaking research studies demonstrating technical viability (Sellers et al., 2010; Silvoni et al., 2011). Despite this progress, BCIs have remained primarily in controlled laboratory settings. This dissertation proffers a blueprint for translating research-grade BCI systems into real-world applications that are noninvasive and fully portable, and that employ intelligent user interfaces for communication. The proposed architecture is designed to be used by severely motor-impaired individuals, such as those with locked-in syndrome, while reducing the effort and cognitive load needed to communicate. Such a system requires the merging of two primary research fields: 1) electroencephalography (EEG)-based BCIs and 2) intelligent user interface design. The EEG-based BCI portion of this dissertation provides a history of the field, details of our software and hardware implementation, and results from an experimental study aimed at verifying the utility of a BCI based on the steady-state visual evoked potential (SSVEP), a robust brain response to visual stimulation at controlled frequencies. The visual stimulation, feature extraction, and classification algorithms for the BCI were specially designed to achieve successful real-time performance on a laptop computer. Also, the BCI was developed in Python, an open-source programming language that combines programming ease with effective handling of hardware and software requirements. The result of this work was The Unlock Project app software for BCI development. Using it, a four-choice SSVEP BCI setup was implemented and tested with five severely motor-impaired and fourteen control participants. The system showed a wide range of usability across participants, with classification rates ranging from 25-95%. The second portion of the dissertation discusses the viability of intelligent user interface design as a method for obtaining a more user-focused vocal output communication aid tailored to motor-impaired individuals. A proposed blueprint of this communication "app" was developed in this dissertation. It would make use of readily available laptop sensors to perform facial recognition, speech-to-text decoding, and geo-location. The ultimate goal is to couple sensor information with natural language processing to construct an intelligent user interface that shapes communication in a practical SSVEP-based BCI

Kessel Run: towards emotion adaptation in a BCI multiplayer game

Tese de mestrado integrado, Engenharia Biomédica e Biofísica (Sinais e Imagens Médicas) Universidade de Lisboa, Faculdade de Ciências, 2017O objetivo original de uma Interface Cerebro-Máquina (BCI, do inglês Brain-Computer Interface) é o restauro de função a portadores de deficiências motoras, com aplicações que abrangem desde o mover de um cursor de computador ou de uma cadeira de rodas, a dispositivos complexos de soletração que substituem a fala. No entanto, com o recente aparecimento no mercado de aparelhos de BCI portáteis e económicos, as aplicações de BCI têm vindo a migrar lentamente para áreas fora do âmbito da saúde, como é o caso do entretenimento. Em particular, o desenvolvimento de videojogos em que os modos de interação tradicionais (teclado ou botões, por exemplo) são substituídos por controlos BCI é uma aposta frequente em vários grupos de investigação em neurociências. O uso de paradigmas de BCI como controladores de jogos tem a capacidade de não só possibilitar novos meios de interação mais intuitivos (como é o caso de apenas pensar em mover a personagem do jogo, em vez de pressionar o botão que a move), mas também de criar novos mecanismos de jogo que não são possíveis com dispositivos tradicionais. Para a criação destes novos mecanismos a Computação Afetiva é de relativo interesse, já que esta é a área de investigação encarregue de encontrar relações entre o estado emocional de um sujeito, através de BCIs, por exemplo, e utilizá-las para melhorar a interação com um computador (ou um jogo). Apesar de beneficiarem de um ligação direta ao cérebro, poucos são os videojogos BCI que a utilizam para adaptar o conteúdo do jogo ao estado emocional do jogador, em parte porque são poucas as relações conhecidas entre o eletroencefalograma (EEG) e o estado emocional do indivíduo, especialmente em condições pouco controladas e em cenários realistas. De facto, a maioria dos estudos em Computação Afetiva feitos com o objetivo de procurar correlações entre o estado emocional do sujeito e o seu EEG pecam por serem realizados sob condições pouco realistas, e, em particular, nunca durante uma situação de jogo. Por outro lado, apesar da frequente aposta no desenvolvimento de novos videojogos controlados por um paradigma de BCI, poucos têm em consideração as regras de um bom desenho de jogos, resultando muitas vezes num jogo que mesmo sendo funcional, é aborrecido. Com as perspetivas da aplicação de BCI e Computação Afetiva aos videojogos em mente, esta dissertação tem como objetivo o desenvolvimento de um jogo multiplayer controlado por BCI, que ao seguir as regras de bom desenho de jogos, é capaz de desencadear uma sensação de divertimento nos seus jogadores. Para além disso, o jogo também deve ser capaz de evocar um conjunto diversificado de estados emocionais nos seus jogadores, de forma a poder estudar-se as correlações entre o EEG e o estado emocional de cada indivíduo no espectro da frequência. Desta forma, poder-se-á comparar as correlações obtidas num cenário realístico de jogo com o estado-da-arte, frequentemente realizado em situações controladas, e assim contribuir para o avanço da adaptação emocional em videojogos BCI. Para concretizar estes objetivos, o videojogo Kessel Run foi desenvolvido. Kessel Run é um jogo 3D de uma corrida espacial para dois jogadores, em que ambos devem cooperar um com o outro de forma a direcionar uma nave espacial para longe de asteróides e assim conseguir finalizar uma corrida de 2 minutos com o mínimo de danos possível. Neste jogo, as regras básicas de desenho de jogos (Teoria de Flow e o Paradoxo de Controlo) foram aplicadas de forma a criar uma sensação de divertimento e de controlo no jogador. A sensação de controlo por parte do jogador é particularmente importante na criação de um jogo BCI, uma vez que a sua falta poderá levar a perda de imersão no jogo e, consequentemente, à diminuição do divertimento. Assim, de forma a garantir o bom controlo do jogo o paradigma SSVEP (do inglês Steady-State Visually Evoked Potential) foi escolhido como modo de interação BCI. De forma a evocarem-se um conjunto diversificado de estados emocionais nos jogadores, várias estratégias de elicitação foram aplicadas no jogo. Em primeiro lugar, este dispõe de dois níveis de dificuldade (um fácil e um difícil). O primeiro nível desafia as capacidades dos jogadores sem contudo ser demasiado difícil, pelo que se espera que evoque emoções mais positivas. Já o segundo nível aumenta bastante a dificuldade do jogo, tornando-se muito difícil batê-lo. Para além da dificuldade acrescida, o nível difícil do jogo foi programado de forma a que o controlo BCI falhe com frequência sem o conhecimento do jogador. Espera-se por isso que o segundo nível evoque níveis de frustração maiores, e estados emocionais mais negativos e excitados. O jogo Kessel Run foi colocado em prática ao desenvolver-se um protocolo experimental onde 12 participantes jogaram os dois níveis de dificuldade do jogo. A cada participante foi pedido a classificação do jogo em termos de experiência do utilizador, e de cada nível relativamente às emoções sentidas no decorrer do jogo, na forma de questionários. Foram também adquiridos os sinais de EEG de cada participante. De forma geral, o desempenho do paradigma BCI foi menor do o que esperado, conseguindo-se apenas um máximo de 79% classificações correctas. Este resultado deve-se essencialmente a dois factores: o grau deficiente de escuridão da sala laboratorial, responsável pela perda de desempenho na ordem dos 6%, e a deteção individual das frequências escolhidas para estímulo SSVEP (12 e 15 Hz). Neste último, os participantes tiveram maior facilidade em reconhecer o estímulo de 12 Hz, com um desempenho individual médio de 63%, face ao estímulo de 15 Hz com apenas 38%, o que comprometeu a performance geral do reconhecimento SSVEP. No entanto, apesar do desempenho fraco do paradigma, os participantes reportaram uma experiência bastante divertida (média de flow = 2:6 numa escala 0-5) e desafiante (média de challenge = 2:3 numa escala 0-5), com apenas um ligeiro aborrecimento (média de tension=annoyance = 1:1 numa escala 0-5), podendo-se concluir o sucesso do emprego das regras de bom desenho de jogos. As estratégias de elicitação de emoções foram apenas parcialmente bem sucedidas; não foram observadas diferenças significativas entre os níveis de dificuldade do jogo Kessel Run em termos de valência e excitação emocionais. No entanto conseguiu-se uma boa distribuição das avaliações emocionais dos participantes pelos quatro quadrantes das dimensões de valência e excitação, possibilitando o estudo de correlações entre o EEG dos participantes e as suas avaliações para cada nível de jogo em termos de oscilações no espectro da frequência e assimetrias na banda alfa. Encontraram-se correlações significativas na dimensão da valência que parecem contradizer a teoria da assimetria da banda alfa. Em particular, obteve-se uma correlação positiva significativa indicando uma relação de diminuição da activação hemisférica esquerda e consequente aumento da banda alfa. Esta contradição foi também confirmada pela obtenção de uma assimetria esquerda bastante significativa na banda alfa para o córtex frontal. Observou-se ainda uma diminuição da potência central da banda beta e um aumento occipital e temporal direito para a mesma banda relacionado com a dimensão da valência. Para a excitação encontrou-se uma correlação negativa significativa em regiões centrais e frontais na banda alfa, indicando uma activação destas regiões cerebrais aquando de estados mais excitados. Mais ainda, uma correlação significativa indicou uma assimetria direita na banda alfa para um par de eléctrodos fronto-centrais. Espera-se que este estudo possa contribuir para uma futura geração de videojogos com a capacidade de adaptação ao conteúdo emocional do seu jogador.Lately the field of (digital) game research is rapidly growing, with studies dedicated to capture game experience, adopting new technologies or exploring outside traditional input methods. Alongside, research in Brain-Computer Interfaces (BCI) has significantly increased in its applications for healthy users, such as games. BCIs benefit from access to brain activity which can bypass bodily mediation (e.g. controllers) and enable gamers to express themselves more naturally in a given game context. Moreover, BCI can provide significant insight into the user's emotional state. Recent research points to numerous correlates of emotion in brain signals. A complex challenge is to use BCI for access to the player's affective state in a real gaming context, improving and tailoring the user experience. The goal of this dissertation project is to introduce affective research to BCI games by creating a novel multiplayer Steady-State Visually Evoked Potential (SSVEP) BCI game, capable of providing a fun experience to its players and eliciting emotions for a study on EEG correlates of emotion. The multiplayer game Kessel Run was created, resulting in a space exploration game with a exible system that followed good game design rules with emotion elicitation strategies, controlled by the SSVEP paradigm. Twelve participants played Kessel Run using a 32-electrode EEG cap and rated the emotions felt during gameplay in a questionnaire. The SSVEP game performance achieved a maximum of 79% accuracy and an average of 55%. In addition, players reported that playing the game created a fun and immersive experience. A significant correlation with increased alpha power on the left hemisphere and positive valence led to the contradiction of the popular alpha asymmetry theory, which states that processing of positive information causes a decrease in alpha power on the left frontal hemisphere. Furthermore, correlates in the beta frequency range have been found for valence on right temporal and central sites. In the arousal dimension a significant central and frontal alpha power decrease was found, along with significant alpha asymmetry on fronto-central pairs for increased arousal