Electroencephalogram Signal Processing For Hybrid Brain Computer Interface Systems

The goal of this research was to evaluate and compare three types of brain computer interface (BCI) systems, P300, steady state visually evoked potentials (SSVEP) and Hybrid as virtual spelling paradigms. Hybrid BCI is an innovative approach to combine the P300 and SSVEP. However, it is challenging to process the resulting hybrid signals to extract both information simultaneously and effectively. The major step executed toward the advancement to modern BCI system was to move the BCI techniques from traditional LED system to electronic LCD monitor. Such a transition allows not only to develop the graphics of interest but also to generate objects flickering at different frequencies. There were pilot experiments performed for designing and tuning the parameters of the spelling paradigms including peak detection for different range of frequencies of SSVEP BCI, placement of objects on LCD monitor, design of the spelling keyboard, and window time for the SSVEP peak detection processing. All the experiments were devised to evaluate the performance in terms of the spelling accuracy, region error, and adjacency error among all of the paradigms: P300, SSVEP and Hybrid. Due to the different nature of P300 and SSVEP, designing a hybrid P300-SSVEP signal processing scheme demands significant amount of research work in this area. Eventually, two critical questions in hybrid BCl are: (1) which signal processing strategy can best measure the user\u27s intent and (2) what a suitable paradigm is to fuse these two techniques in a simple but effective way. In order to answer these questions, this project focused mainly on developing signal processing and classification technique for hybrid BCI. Hybrid BCI was implemented by extracting the specific information from brain signals, selecting optimum features which contain maximum discrimination information about the speller characters of our interest and by efficiently classifying the hybrid signals. The designed spellers were developed with the aim to improve quality of life of patients with disability by utilizing visually controlled BCI paradigms. The paradigms consist of electrodes to record electroencephalogram signal (EEG) during stimulation, a software to analyze the collected data, and a computing device where the subject’s EEG is the input to estimate the spelled character. Signal processing phase included preliminary tasks as preprocessing, feature extraction, and feature selection. Captured EEG data are usually a superposition of the signals of interest with other unwanted signals from muscles, and from non-biological artifacts. The accuracy of each trial and average accuracy for subjects were computed. Overall, the average accuracy of the P300 and SSVEP spelling paradigm was 84% and 68.5 %. P300 spelling paradigms have better accuracy than both the SSVEP and hybrid paradigm. Hybrid paradigm has the average accuracy of 79 %. However, hybrid system is faster in time and more soothing to look than other paradigms. This work is significant because it has great potential for improving the BCI research in design and application of clinically suitable speller paradigm

Modeling temporal dependency of brain responses to rapidly presented stimuli in ERP based BCIs

Projecte realitzat en el marc d’un programa de mobilitat amb la Northeastern University[ANGLÈS] Brain computer interface technologies aim, among a number of possible goals, to help people with motor and speech disabilities to communicate through a computer using the electrical activity in the brain, allowing them to move robotic arms or typing. We will specifically work with the RSVP-Keyboard, a letter-by-letter brain computer interface (BCI) typing system based on EEG responses to serial presentation of characters on the screen. Target stimulus elicits a P300 ERP, a positive deflection in the scalp voltage around 300 ms after the stimulus. Previous RSVP-Keyboard models approached the problem of detecting the user's intended symbol by fusing a language model with EEG evidence in response to each symbol presentation, with the assumption that EEG for each trial was independent from others. Trials were windowed assuming a limited time effect of the ERPs (500ms). Even though the inter stimulus interval (ISI) is smaller than the window duration (200ms), independence of overlapping ERP responses from consecutive trials given the intended symbol was assumed. Our main objective here is to design a more realistic model that better captures the temporal dependencies inside a sequence. The new model consists of a finite state machine whose states are determined by the presence or nonpresence of a target within the assumed stimulus response time. Thus, the new method looks for the most probable sequence of states given the position of the target, considering the temporal dependencies induced by overlapping ERP responses on the recorded EEG.[CASTELLA] Las tecnologías de “Brain Computer Interface” (BCI) pretenden, entre otras aplicaciones, ayudar a la población con discapacidades motoras o del habla a comunicarse a través del ordenador utilizando la señal eléctrica del cerebro, permitiendo el movimiento de brazos robóticos o escribir. En este proyecto trabajaremos con el RSVP-Keyboard, un sistema de BCI que permite escribir letra a letra. El sistema se basa en la respuesta del señal de electroencefalografía (EEG) a caracteres presentados rápidamente por la pantalla del ordenador. El símbolo buscado por el usuario (“target”) excita el “Event Related Potential (ERP)” llamado P300: una defección positiva de voltaje alrededor de 300ms después del estímulo. Los modelos previos del RSVP-Keyboard presentaban el problema de detectar el símbolo que el usuario pretendía escribir fusionando un modelo de lenguaje con la evidencia sacada de la respuesta de EEG de cada símbolo de la presentación. Las respuestas están enventanadas asumiendo un efecto limitado temporalmente del ERP (500ms) e independencia entre estas respuestas, sin tener en cuenta que el intervalo de aparición entre estímulos (ISI) es más pequeño que el tamaño de la ventana (200ms). Nuestro principal objetivo es diseñar un modelo más realístico que capture mejor las dependencias temporales dentro de una secuencia. El modelo nuevo consiste en una máquina de estados finitos donde los estados están definidos según la presencia o ausencia de un “target” durante el tiempo asumido de respuesta. Así, el nuevo método busca por la secuencia de estados más probable dada la posición del “target”, considerando las dependencias temporales causadas por la superposición de respuestas.[CATALA] Les tecnologies “Brain Computer Interface" (BCI) pretenen, entre altres possibles aplicacions, ajudar a la població amb discapacitats motores o de parla a comunicar-se a través de l’ordinador utilitzant l’activitat elèctrica del cervell, permetent així el moviment de braços robòtics o escriure. En aquest projecte treballarem amb el RSVP-Keyboard, un sistema de BCI que permet escriure lletra a lletra. El sistema es basa en la resposta del senyal d’electroencefalografia (EEG) a caràcters presentats ràpidament a través de la pantalla de l’ordinador. El símbol buscat per l’usuari (“target”) excita el “Event Related Potential (ERP)” anomenat P300: una deflecció positiva de voltatge al voltant dels 300ms després de l’estímul. Els models previs del RSVP-Keyboard presentaven el problema de detectar el símbol que l'usuari pretén escriure fusionant un model de llenguatge amb l’evidència extreta de la resposta en EEG de cada símbol de la presentació. Les respostes estan enfinestrades assumint un effecte limitat temporalment del ERP (500ms) e independència entre aquestes respostes, sense tenir en compte que l’interval d'aparició entre estímuls (ISI) és més petit que el tamany de la finestra (200ms). El nostre principal objectiu és dissenyar un model més realístic que capturi millor les dependències temporals dins d’una seqüència. El nou model consisteix en una màquina d’estats finits on els estats estan definits segons la presència o absència d’un “target” durant el temps assumit de resposta. Així, el nou mètode busca per la seqüència d’estats més probable donada la posició del “target”, considerant les dependències temporals causades per la superposició de respostes

Brain-Computer Interface

Brain-computer interfacing (BCI) with the use of advanced artificial intelligence identification is a rapidly growing new technology that allows a silently commanding brain to manipulate devices ranging from smartphones to advanced articulated robotic arms when physical control is not possible. BCI can be viewed as a collaboration between the brain and a device via the direct passage of electrical signals from neurons to an external system. The book provides a comprehensive summary of conventional and novel methods for processing brain signals. The chapters cover a range of topics including noninvasive and invasive signal acquisition, signal processing methods, deep learning approaches, and implementation of BCI in experimental problems

Non-Intrusive Gait Recognition Employing Ultra Wideband Signal Detection

A self-regulating and non-contact impulse radio ultra wideband (IR-UWB) based 3D human gait analysis prototype has been modeled and developed with the help of supervised machine learning (SML) for this application for the first time. The work intends to provide a rewarding assistive biomedical application which would help doctors and clinicians monitor human gait trait and abnormalities with less human intervention in the fields of physiological examinations, physiotherapy, home assistance, rehabilitation success determination and health diagnostics, etc. The research comprises IR-UWB data gathered from a number of male and female participants in both anechoic chamber and multi-path environments. In total twenty four individuals have been recruited, where twenty individuals were said to have normal gait and four persons complained of knee pain that resulted in compensated spastic walking patterns. A 3D postural model of human movements has been created from the backscattering property of the radar pulses employing understanding of spherical trigonometry and vector fields. This subjective data (height of the body areas from the ground) of an individual have been recorded and implemented to extract the gait trait from associated biomechanical activity and differentiates the lower limb movement patterns from other body areas. Initially, a 2D postural model of human gait is presented from IR-UWB sensing phenomena employing spherical co-ordinate and trigonometry where only two dimensions such as, distance from radar and height of reflection have been determined. There are five pivotal gait parameters; step frequency, cadence, step length, walking speed, total covered distance, and body orientation which have all been measured employing radar principles and short term Fourier transformation (STFT). Subsequently, the proposed gait identification and parameter characterization has been analysed, tested and validated against popularly accepted smartphone applications with resulting variations of less than 5%. Subsequently, the spherical trigonometric model has been elevated to a 3D postural model where the prototype can determine width of motion, distance from radar, and height of reflection. Vector algebra has been incorporated with this 3D model to measure knee angles and hip angles from the extension and flexion of lower limbs to understand the gait behavior throughout the entire range of bipedal locomotion. Simultaneously, the Microsoft Kinect Xbox One has been employed during the experiment to assist in the validation process. The same vector mathematics have been implemented to the skeleton data obtained from Kinect to determine both the hip and knee angles. The outcomes have been compared by statistical graphical approach Bland and Altman (B&A) analysis. Further, the changes of knee angles obtained from the normal gaits have been used to train popular SMLs such as, k-nearest neighbour (kNN) and support vector machines (SVM). The trained model has subsequently been tested with the new data (knee angles extracted from both normal and abnormal gait) to assess the prediction ability of gait abnormality recognition. The outcomes have been validated through standard and wellknown statistical performance metrics with promising results found. The outcomes prove the acceptability of the proposed non-contact IR-UWB gait recognition to detect gait

Developing Machine Learning Algorithms for Behavior Recognition from Deep Brain Signals

Parkinson’s disease (PD) is a neurodegenerative condition and movement disorder that appears with symptoms such as tremor, rigidity of muscles and slowness of movements. Deep brain stimulation (DBS) is an FDA-approved surgical therapy for essential tremor and PD. Despite the fact that DBS substantially alleviates the motor signs of PD, it can cause cognitive side effects and speech malfunction mainly due to the lack of adaptivity and optimality of the stimulation signal to the patients’ current state. A behavior-adapted closed-loop DBS system may reduce the side effects and power consumption by adjusting the stimulation parameters to patients’ need. Behavior recognition based on physiological feedbacks plays a key role in designing the next generation of closed-loop DBS systems. Hence, this dissertation is concentrated on: 1. Investigating the capability of local field potential (LFP) signals recorded from Subthalamic nucleus (STN) in identifying behavioral activities 2. Developing advanced machine learning algorithms to recognize behavioral activities using LFP signals 3. Investigating the effects of medication and stimulation pulse on the behavior recognition task as well as characteristics of the LFP signal. STN-LFP is a great physiological signal candidate since the stimulation device itself can record it, eliminating the need for additional sensors. Continuous wavelet transform is utilized for time-frequency analysis of STN-LFPs. Experimental results demonstrate that different behaviors create different modulation patterns in STN within the beta frequency range. A hierarchical classification structure is proposed to perform the behavior classification through a multi-level framework. The beta frequency components of STN-LFPs recorded from all contacts of DBS leads are combined through an MKL-based SVM classifier for behavior classification. Alternatively, the inter-hemispheric synchronization of the LFP signals measured by an FFT-based synchronization approach is utilized to pair up the LFP signals from left and right STNs. Using these rearranged LFP signals reduces the computational cost significantly while keeping the classification ability almost unchanged. LFP-Net, a customized deep convolutional neural network (CNN) approach for behavior classification, is also proposed. CNNs learn different feature maps based on the beta power patterns associated with different behaviors. The features extracted by CNNs are passed through fully connected layers, and, then to the softmax layer for classification. The effect of medication and stimulation “off/on” conditions on characteristics of LFP signals and the behavior classification performance is studied. The beta power of LFP signals under different stimulation and medication paradigms is investigated. Experimental results confirm that the beta power is suppressed significantly when the patients take medication or therapeutic stimulation. The results also show that the behavior classification performance is not impacted by different medication or stimulation conditions. Identifying human behavioral activities from physiological signals is a stepping-stone toward adaptive closed-loop DBS systems. To design such systems, however, there are other open questions that need to be addressed, which are beyond the scope of this dissertation, such as developing event-related biomarkers, customizing the parameter of DBS system based on the patients’ current state, investigating the power consumption and computational complexity of the behavior recognition algorithms

Role of independent component analysis in intelligent ECG signal processing

The Electrocardiogram (ECG) reflects the activities and the attributes of the human heart and reveals very important hidden information in its structure. The information is extracted by means of ECG signal analysis to gain insights that are very crucial in explaining and identifying various pathological conditions. The feature extraction process can be accomplished directly by an expert through, visual inspection of ECGs printed on paper or displayed on a screen. However, the complexity and the time taken for the ECG signals to be visually inspected and manually analysed means that it‟s a very tedious task thus yielding limited descriptions. In addition, a manual ECG analysis is always prone to errors: human oversights. Moreover ECG signal processing has become a prevalent and effective tool for research and clinical practices. A typical computer based ECG analysis system includes a signal preprocessing, beats detection and feature extraction stages, followed by classification.Automatic identification of arrhythmias from the ECG is one important biomedical application of pattern recognition. This thesis focuses on ECG signal processing using Independent Component Analysis (ICA), which has received increasing attention as a signal conditioning and feature extraction technique for biomedical application. Long term ECG monitoring is often required to reliably identify the arrhythmia. Motion induced artefacts are particularly common in ambulatory and Holter recordings, which are difficult to remove with conventional filters due to their similarity to the shape of ectopic xiiibeats. Feature selection has always been an important step towards more accurate, reliable and speedy pattern recognition. Better feature spaces are also sought after in ECG pattern recognition applications. Two new algorithms are proposed, developed and validated in this thesis, one for removing non-trivial noises in ECGs using the ICA and the other deploys the ICA extracted features to improve recognition of arrhythmias. Firstly, independent component analysis has been studiedand found effective in this PhD project to separate out motion induced artefacts in ECGs, the independent component corresponding to noise is then removed from the ECG according to kurtosis and correlation measurement.The second algorithm has been developed for ECG feature extraction, in which the independent component analysis has been used to obtain a set of features, or basis functions of the ECG signals generated hypothetically by different parts of the heart during the normal and arrhythmic cardiac cycle. ECGs are then classified based on the basis functions along with other time domain features. The selection of the appropriate feature set for classifier has been found important for better performance and quicker response. Artificial neural networks based pattern recognition engines are used to perform final classification to measure the performance of ICA extracted features and effectiveness of the ICA based artefacts reduction algorithm.The motion artefacts are effectively removed from the ECG signal which is shown by beat detection on noisy and cleaned ECG signals after ICA processing. Using the ICA extracted feature sets classification of ECG arrhythmia into eight classes with fewer independent components and very high classification accuracy is achieved