Electroencephalogram Signal Processing For Hybrid Brain Computer Interface Systems

The goal of this research was to evaluate and compare three types of brain computer interface (BCI) systems, P300, steady state visually evoked potentials (SSVEP) and Hybrid as virtual spelling paradigms. Hybrid BCI is an innovative approach to combine the P300 and SSVEP. However, it is challenging to process the resulting hybrid signals to extract both information simultaneously and effectively. The major step executed toward the advancement to modern BCI system was to move the BCI techniques from traditional LED system to electronic LCD monitor. Such a transition allows not only to develop the graphics of interest but also to generate objects flickering at different frequencies. There were pilot experiments performed for designing and tuning the parameters of the spelling paradigms including peak detection for different range of frequencies of SSVEP BCI, placement of objects on LCD monitor, design of the spelling keyboard, and window time for the SSVEP peak detection processing. All the experiments were devised to evaluate the performance in terms of the spelling accuracy, region error, and adjacency error among all of the paradigms: P300, SSVEP and Hybrid. Due to the different nature of P300 and SSVEP, designing a hybrid P300-SSVEP signal processing scheme demands significant amount of research work in this area. Eventually, two critical questions in hybrid BCl are: (1) which signal processing strategy can best measure the user\u27s intent and (2) what a suitable paradigm is to fuse these two techniques in a simple but effective way. In order to answer these questions, this project focused mainly on developing signal processing and classification technique for hybrid BCI. Hybrid BCI was implemented by extracting the specific information from brain signals, selecting optimum features which contain maximum discrimination information about the speller characters of our interest and by efficiently classifying the hybrid signals. The designed spellers were developed with the aim to improve quality of life of patients with disability by utilizing visually controlled BCI paradigms. The paradigms consist of electrodes to record electroencephalogram signal (EEG) during stimulation, a software to analyze the collected data, and a computing device where the subject’s EEG is the input to estimate the spelled character. Signal processing phase included preliminary tasks as preprocessing, feature extraction, and feature selection. Captured EEG data are usually a superposition of the signals of interest with other unwanted signals from muscles, and from non-biological artifacts. The accuracy of each trial and average accuracy for subjects were computed. Overall, the average accuracy of the P300 and SSVEP spelling paradigm was 84% and 68.5 %. P300 spelling paradigms have better accuracy than both the SSVEP and hybrid paradigm. Hybrid paradigm has the average accuracy of 79 %. However, hybrid system is faster in time and more soothing to look than other paradigms. This work is significant because it has great potential for improving the BCI research in design and application of clinically suitable speller paradigm

Introducing the Edges Paradigm: A P300 Brain-Computer Interface for spelling written words.

P300-based brain–computer interface spellers employ the P300 component, which is derived from scalp measured electroencephalogram during the brain’s electrical response to a flash denoting an attended target character. The most popular P300 speller, the row–column paradigm (RCP), displays characters in a matrix within which rows and columns of characters are flashed eliciting P300 responses when the illuminated row or column contains the attended target character. Despite being a longstanding successful approach, this RCP faces several challenges, including the adjacency and crowding problems. A new P300 speller is introduced—the edges paradigm (EP). Distinct from existing P300 spellers, the EP presents a square adjacent to each column or row in the outer boundary of the matrix. By replacing each flash of a row or column with that square, this EP exhibited attenuated influences of crowding and adjacency—problems known to perturb the RCP. In the copy-spelling mode, 14 neurologically normal participants demonstrated an improved accuracy of 93.3 +\- 2.0% for the EP relative to 81.7 +\- 2.8% for the RCP, alongside a faster communication rate. Subjective ratings also indicated that the EP caused significantly less fatigue, while increasing alertness and comfort.Peer reviewe

Bacteria Hunt: A multimodal, multiparadigm BCI game

Brain-Computer Interfaces (BCIs) allow users to control applications by brain activity. Among their possible applications for non-disabled people, games are promising candidates. BCIs can enrich game play by the mental and affective state information they contain. During the eNTERFACE’09 workshop we developed the Bacteria Hunt game which can be played by keyboard and BCI, using SSVEP and relative alpha power. We conducted experiments in order to investigate what difference positive vs. negative neurofeedback would have on subjects’ relaxation states and how well the different BCI paradigms can be used together. We observed no significant difference in mean alpha band power, thus relaxation, and in user experience between the games applying positive and negative feedback. We also found that alpha power before SSVEP stimulation was significantly higher than alpha power during SSVEP stimulation indicating that there is some interference between the two BCI paradigms

Practical Brain Computer Interfacing

A brain-computer interface (BCI) is a communication system that enables users to voluntary send messages or commands without movement. The classical goal of BCI research is to support communication and control for users with impaired communication due to illness or injury. Typical BCI applications are the operation of computer cursors, spelling programs or external devices, such as wheelchairs, robots and neural prostheses. The user sends modulated information to the BCI by engaging in mental tasks that produce distinct brain patterns. The BCI acquires signals from the user's brain and translates them into suitable communication. This dissertation aims to develop faster and more reliable non-invasive BCI communication based on the study of users learning process and their interaction with the BCI transducer. To date, BCI research has focused on the development of advanced pattern recognition and classification algorithms to improve accuracy and reliability of the classified patterns. However, even with optimal detection methods, successful BCI operation depends on the degree to which the users can voluntary modulate their brain signals. Therefore, learning to operate a BCI requires repeated practice with feedback that engages learning mechanisms in the brain. In this work, several aspects including signal processing techniques, feedback methods, experimental and training protocols, demographics, and applications were explored and investigated. Research was focused on two BCI paradigms, steady-state visual evoked potentials (SSVEP) and event-related (de-)synchronization (ERD/ERS). Signal processing algorithms for the detection of both brain patterns were applied and evaluated. A general application interface for BCI feedback tasks was developed to evaluate the practicability, reliability and acceptance of new feedback methods. The role of feedback and training was fully investigated on studies conducted with healthy subjects. The influence of demographics on BCIs was explored in two field studies with a large number of subjects. Results were supported through advanced statistical analysis. Furthermore, the BCI control was evaluated in a spelling application and a service robotic application. This dissertation demonstrates that BCIs can provide effective communication for most subjects. Presented results showed that improvements in the BCI transducer, training protocols, and feedback methods constituted the basis to achieve faster and more reliable BCI communication. Nevertheless, expert assistance is necessary for both initial configuration and daily operation, which reduces the practicability of BCIs for people who really need them

SSL for Auditory ERP-Based BCI

A brain–computer interface (BCI) is a communication tool that analyzes neural activity and relays the translated commands to carry out actions. In recent years, semi-supervised learning (SSL) has attracted attention for visual event-related potential (ERP)-based BCIs and motor-imagery BCIs as an effective technique that can adapt to the variations in patterns among subjects and trials. The applications of the SSL techniques are expected to improve the performance of auditory ERP-based BCIs as well. However, there is no conclusive evidence supporting the positive effect of SSL techniques on auditory ERP-based BCIs. If the positive effect could be verified, it will be helpful for the BCI community. In this study, we assessed the effects of SSL techniques on two public auditory BCI datasets—AMUSE and PASS2D—using the following machine learning algorithms: step-wise linear discriminant analysis, shrinkage linear discriminant analysis, spatial temporal discriminant analysis, and least-squares support vector machine. These backbone classifiers were firstly trained by labeled data and incrementally updated by unlabeled data in every trial of testing data based on SSL approach. Although a few data of the datasets were negatively affected, most data were apparently improved by SSL in all cases. The overall accuracy was logarithmically increased with every additional unlabeled data. This study supports the positive effect of SSL techniques and encourages future researchers to apply them to auditory ERP-based BCIs

Brain computer interface based neurorehabilitation technique using a commercially available EEG headset

Neurorehabilitation has recently been augmented with the use of virtual reality and rehabilitation robotics. In many systems, some known volitional control must exist in order to synchronize the user intended movement with the therapeutic virtual or robotic movement. Brain Computer Interface (BCI) aims to open up a new rehabilitation option for clinical population having no residual movement due to disease or injury to the central or peripheral nervous system. Brain activity contains a wide variety of electrical signals which can be acquired using many invasive and non-invasive acquisition techniques and holds the potential to be used as an input to BCI. Electroencephalogram (EEG) is a non-invasive method of acquiring brain activity which then, with further processing and classification, can be used to predict various brain states such as an intended motor movement. EEG provides the temporal resolution required to obtain significant result which may not be provided by many other non-invasive techniques. Here, EEG is recorded using a commercially available EEG headset provided by Emotiv Inc. Data is collected and processed using BCI2000 software, and the difference in the Mu-rhythm due to Event Related Synchronization (ERS) and Desynchronization (ERD) is used to distinguish an intended motor movement and resting brain state, without the need for physical movement. The idea is to combine this user intent/free will with an assistive robot to achieve the user initiated, repetitive motor movements required to bring therapeutic changes in the targeted subject group, as per Hebbian type learning

A Brain-Computer Interface based on Colour Dependent Visual Attention

In this thesis we designed a specific visual protocol for a new application in the brain-computer interface field. We evaluated how coloured stimuli affect brain activity in health

A noninvasive brain-actuated wheelchair based on a P300 neurophysiological protocol and automated navigation

This paper describes a new noninvasive brain-actuated wheelchair that relies on a P300 neurophysiological protocol and automated navigation. When in operation, the user faces a screen displaying a real-time virtual reconstruction of the scenario and concentrates on the location of the space to reach. A visual stimulation process elicits the neurological phenomenon, and the electroencephalogram (EEG) signal processing detects the target location. This location is transferred to the autonomous navigation system that drives the wheelchair to the desired location while avoiding collisions with obstacles in the environment detected by the laser scanner. This concept gives the user the flexibility to use the device in unknown and evolving scenarios. The prototype was validated with five healthy participants in three consecutive steps: screening (an analysis of three different groups of visual interface designs), virtual-environment driving, and driving sessions with the wheelchair. On the basis of the results, this paper reports the following evaluation studies: 1) a technical evaluation of the device and all functionalities; 2) a users’ behavior study; and 3) a variability study. The overall result was that all the participants were able to successfully operate the device with relative ease, thus showing a great adaptation as well as a high robustness and low variability of the system

Consciousness level assessment in completely locked-in syndrome patients using soft-clustering

Brain-computer interfaces (BCIs) are very convenient tools to assess locked-in (LIS) and completely locked-in state (CLIS) patients' hidden states of consciousness. For the time being, there is no ground-truth data in respect to these states for above-mentioned patients. This lack of gold standard makes this problem particularly challenging. In addition to consciousness assessment, BCIs also provide them with a communication device that does not require the presence of motor responses, which they are lacking. Communication plays an important role in the patients' quality of life and prognosis. Significant progress have been made to provide them with EEG-based BCIs in particular. Nonetheless, the majority of existing studies directly dive into the communication part without assessing if the patient is even conscious. Additionally, the few studies that do essentially use evoked brain potentials, mostly the P300, that necessitates the patient's voluntary and active participation to be elicited. Patients are easily fatigued, and would consequently be less successful during the main communication task. Furthermore, when the consciousness states are determined using resting state data, only one or two features were used. In this thesis, different sets of EEG features are used to assess the consciousness level of CLIS patients using resting-state data. This is done as a preliminary step that needed to be succeeded in order to engage to the next step, communication with the patient. In other words, the 'conversation' is initiated only if the patient is sufficiently conscious. This variety of EEG features is utilised to increase the probability of correctly estimating the patients' consciousness states. Indeed, each of them captures a particular signal attribute, and combining them would allow the collection of different hidden characteristics that could have not been obtained from a single feature. Furthermore, the proposed method should allow to determine if communication shall be initiated at a specific time with the patient. The EEG features used are frequency-based, complexity related and connectivity metrics. Besides, instead of analysing results from individual channels or specific brain regions, the global activity of the brain is assessed. The estimated consciousness levels are then obtained by applying two different soft-clustering analysis methods, namely Fuzzy c-means (FCM) and Gaussian Mixture Models (GMM), to the individual features and ensembling their results using their average or their product. The proposed approach is first applied to EEG data recorded from patients with unresponsive wakefulness syndrome (UWS) and minimally conscious state (MCS) (patients with disorders of consciousness (DoC)) to evaluate its performance. It is subsequently applied to data from one CLIS patient that is unique in its kind because it contains a time frame during which the experimenters affirmed that he was conscious. Finally, it is used to estimate the levels of consciousness of nine other CLIS patients. The obtained results revealed that the presented approach was able to take into account the variations of the different features and deduce a unique output taking into consideration the individual features contributions. Some of them performed better than others, which is not surprising since each person is different. It was also able to draw very accurate estimations of the level of consciousness under specific conditions. The approach presented in this thesis provides an additional tool for diagnosis to the medical staff. Furthermore, when implemented online, it would enable to determine the optimal time to engage in communication with CLIS patients. Moreover, it could possibly be used to predict patients' cognitive decline and/or death