Electroencephalographic Signal Processing and Classification Techniques for Noninvasive Motor Imagery Based Brain Computer Interface

In motor imagery (MI) based brain-computer interface (BCI), success depends on reliable processing of the noisy, non-linear, and non-stationary brain activity signals for extraction of features and effective classification of MI activity as well as translation to the corresponding intended actions. In this study, signal processing and classification techniques are presented for electroencephalogram (EEG) signals for motor imagery based brain-computer interface. EEG signals have been acquired placing the electrodes following the international 10-20 system. The acquired signals have been pre-processed removing artifacts using empirical mode decomposition (EMD) and two extended versions of EMD, ensemble empirical mode decomposition (EEMD), and multivariate empirical mode decomposition (MEMD) leading to better signal to noise ratio (SNR) and reduced mean square error (MSE) compared to independent component analysis (ICA). EEG signals have been decomposed into independent mode function (IMFs) that are further processed to extract features like sample entropy (SampEn) and band power (BP). The extracted features have been used in support vector machines to characterize and identify MI activities. EMD and its variants, EEMD, MEMD have been compared with common spatial pattern (CSP) for different MI activities. SNR values from EMD, EEMD and MEMD (4.3, 7.64, 10.62) are much better than ICA (2.1) but accuracy of MI activity identification is slightly better for ICA than EMD using BP and SampEn. Further work is outlined to include more features with larger database for better classification accuracy

Artifact Removal Methods in EEG Recordings: A Review

To obtain the correct analysis of electroencephalogram (EEG) signals, non-physiological and physiological artifacts should be removed from EEG signals. This study aims to give an overview on the existing methodology for removing physiological artifacts, e.g., ocular, cardiac, and muscle artifacts. The datasets, simulation platforms, and performance measures of artifact removal methods in previous related research are summarized. The advantages and disadvantages of each technique are discussed, including regression method, filtering method, blind source separation (BSS), wavelet transform (WT), empirical mode decomposition (EMD), singular spectrum analysis (SSA), and independent vector analysis (IVA). Also, the applications of hybrid approaches are presented, including discrete wavelet transform - adaptive filtering method (DWT-AFM), DWT-BSS, EMD-BSS, singular spectrum analysis - adaptive noise canceler (SSA-ANC), SSA-BSS, and EMD-IVA. Finally, a comparative analysis for these existing methods is provided based on their performance and merits. The result shows that hybrid methods can remove the artifacts more effectively than individual methods

Heterogeneous data fusion for brain psychology applications

This thesis aims to apply Empirical Mode Decomposition (EMD), Multiscale Entropy (MSE), and collaborative adaptive filters for the monitoring of different brain consciousness states. Both block based and online approaches are investigated, and a possible extension to the monitoring and identification of Electromyograph (EMG) states is provided. Firstly, EMD is employed as a multiscale time-frequency data driven tool to decompose a signal into a number of band-limited oscillatory components; its data driven nature makes EMD an ideal candidate for the analysis of nonlinear and non-stationary data. This methodology is further extended to process multichannel real world data, by making use of recent theoretical advances in complex and multivariate EMD. It is shown that this can be used to robustly measure higher order features in multichannel recordings to robustly indicate ‘QBD’. In the next stage, analysis is performed in an information theory setting on multiple scales in time, using MSE. This enables an insight into the complexity of real world recordings. The results of the MSE analysis and the corresponding statistical analysis show a clear difference in MSE between the patients in different brain consciousness states. Finally, an online method for the assessment of the underlying signal nature is studied. This method is based on a collaborative adaptive filtering approach, and is shown to be able to approximately quantify the degree of signal nonlinearity, sparsity, and non-circularity relative to the constituent subfilters. To further illustrate the usefulness of the proposed data driven multiscale signal processing methodology, the final case study considers a human-robot interface based on a multichannel EMG analysis. A preliminary analysis shows that the same methodology as that applied to the analysis of brain cognitive states gives robust and accurate results. The analysis, simulations, and the scope of applications presented suggest great potential of the proposed multiscale data processing framework for feature extraction in multichannel data analysis. Directions for future work include further development of real-time feature map approaches and their use across brain-computer and brain-machine interface applications

Data-driven multivariate and multiscale methods for brain computer interface

This thesis focuses on the development of data-driven multivariate and multiscale methods for brain computer interface (BCI) systems. The electroencephalogram (EEG), the most convenient means to measure neurophysiological activity due to its noninvasive nature, is mainly considered. The nonlinearity and nonstationarity inherent in EEG and its multichannel recording nature require a new set of data-driven multivariate techniques to estimate more accurately features for enhanced BCI operation. Also, a long term goal is to enable an alternative EEG recording strategy for achieving long-term and portable monitoring. Empirical mode decomposition (EMD) and local mean decomposition (LMD), fully data-driven adaptive tools, are considered to decompose the nonlinear and nonstationary EEG signal into a set of components which are highly localised in time and frequency. It is shown that the complex and multivariate extensions of EMD, which can exploit common oscillatory modes within multivariate (multichannel) data, can be used to accurately estimate and compare the amplitude and phase information among multiple sources, a key for the feature extraction of BCI system. A complex extension of local mean decomposition is also introduced and its operation is illustrated on two channel neuronal spike streams. Common spatial pattern (CSP), a standard feature extraction technique for BCI application, is also extended to complex domain using the augmented complex statistics. Depending on the circularity/noncircularity of a complex signal, one of the complex CSP algorithms can be chosen to produce the best classification performance between two different EEG classes. Using these complex and multivariate algorithms, two cognitive brain studies are investigated for more natural and intuitive design of advanced BCI systems. Firstly, a Yarbus-style auditory selective attention experiment is introduced to measure the user attention to a sound source among a mixture of sound stimuli, which is aimed at improving the usefulness of hearing instruments such as hearing aid. Secondly, emotion experiments elicited by taste and taste recall are examined to determine the pleasure and displeasure of a food for the implementation of affective computing. The separation between two emotional responses is examined using real and complex-valued common spatial pattern methods. Finally, we introduce a novel approach to brain monitoring based on EEG recordings from within the ear canal, embedded on a custom made hearing aid earplug. The new platform promises the possibility of both short- and long-term continuous use for standard brain monitoring and interfacing applications

Using novel stimuli and alternative signal processing techniques to enhance BCI paradigms

A Brain-Computer Interface (BCI) is a device that uses the brain activity of a person as an input to select desired outputs on a computer. BCIs that use surface electroencephalogram (EEG) recordings as their input are the least invasive but also suffer from a very low signal-to-noise ratio (SNR) due to the very low amplitude of the person’s brain activity and the presence of many signal artefacts and background noise. This can be compensated for by subjecting the signals to extensive signal processing, and by using stimuli to trigger a large but consistent change in the signal – these changes are called evoked potentials. The method used to stimulate the evoked potential, and introduce an element of conscious selection in order to allow the user’s intent to modify the evoked potential produced, is called the BCI paradigm. However, even with these additions the performance of BCIs used for assistive communication and control is still significantly below that of other assistive solutions, such as keypads or eye-tracking devices. This thesis examines the paradigm and signal processing components of BCIs and puts forward several methods meant to enhance BCIs’ performance and efficiency. Firstly, two novel signal processing methods based on Empirical Mode Decomposition (EMD) were developed and evaluated. EMD is a technique that divides any oscillating signal into groups of frequency harmonics, called Intrinsic Mode Functions (IMFs). Furthermore, by using Takens’ theorem, a single channel of EEG can be converted into a multi-temporal channel signal by transforming the channel into multiple snapshots of its signal content in time using a series of delay vectors. This signal can then be decomposed into IMFs using a multi-channel variation of EMD, called Multi-variate EMD (MEMD), which uses the spatial information from the signal’s neighbouring channels to inform its decomposition. In the case of a multi-temporal channel signal, this allows the temporal dynamics of the signal to be incorporated into the IMFs. This is called Temporal MEMD (T-MEMD). The second signal processing method based on EMD decomposed both the spatial and temporal channels simultaneously, allowing both spatial and temporal dynamics to be incorporated into the resulting IMFs. This is called Spatio-temporal MEMD (ST-MEMD). Both methods were applied to a large pre-recorded Motor Imagery BCI dataset along with EMD and MEMD for comparison. These results were also compared to those from other studies in the literature that had used the same dataset. T-MEMD performed with an average classification accuracy of 70.2%, performing on a par with EMD that had an average classification accuracy of 68.9%. Both ST-MEMD and MEMD outperformed them with ST-MEMD having an average classification accuracy of 73.6%, and MEMD having an average classification accuracy of 75.3%. The methods containing spatial dynamics, i.e. MEMD and ST-MEMD, outperformed those with only temporal dynamics, i.e. EMD and T-MEMD. The two methods with temporal dynamics each performed on a par with the non-temporal method that had the same level of spatial dynamics. This shows that only the presence of spatial dynamics resulted in a performance increase. This was concluded to be because the differences between the classes of motor-imagery are inherently spatial in nature, not temporal. Next a novel BCI paradigm was developed based on the standard Steady-state Somatosensory Evoked Potential (SSSEP) BCI paradigm. This paradigm uses a tactile stimulus applied to the skin at a certain frequency, generating a resonance signal in the brain’s activity. If two stimuli of different frequency are applied, two resonance signals will be present. However, if the user attends one stimulus over the other, its corresponding SSSEP will increase in amplitude. Unfortunately these changes in amplitude can be very minute. To counter this, a stimulus amplitude and frequency of the vibrotactile stimuli. It was hypothesised that if the stimuli generator was constructed that could alter the were of the same frequency, but one’s amplitude was just below the user’s conscious level of perception and the other was above it, the changes in the SSSEP between classes would be the same as those between an SSSEP being generated and neutral EEG, with differences in α activity between the low-amplitude SSSEP and neutral activity due to the differences in the user’s level of concentration from attending the low-amplitude stimulus. The novel SSSEP BCI paradigm performed on a par with the standard paradigm with an average 61.8% classification accuracy over 16 participants, compared to an average 63.3% classification accuracy respectively, indicating that the hypothesis was false. However, the large presence of electro-magnetic interference (EMI) in the EEG recordings may have compromised the data. Many different noise suppression methods were applied to the stimulus device and the data, and whilst the EMI artefacts were reduced in magnitude they were not eliminated completely. Even with the noise the standard SSSEP stimulus paradigm performed on a par with studies that used the same paradigm, indicating that the results may not have been invalidated by the EMI. Overall the thesis shows that motor-imagery signals are inherently spatial in difference, and that the novel methods of T-MEMD and ST-MEMD may yet out-perform the existing methods of EMD and MEMD if applied to signals that are temporal in nature, such as functional Magnetic Resonance Imaging (fMRI). Whilst the novel SSSEP paradigm did not result in an increase in performance, it highlighted the impact of EMI from stimulus equipment on EEG recordings and potentially confirmed that the amplitude of SSEP stimuli is a minor factor in a BCI paradigm

Analysis of gamma-band activity from human EEG using empirical mode decomposition

The purpose of this paper is to determine whether gamma-band activity detection is improved when a filter, based on empirical mode decomposition (EMD), is added to the pre-processing block of single-channel electroencephalography (EEG) signals. EMD decomposes the original signal into a finite number of intrinsic mode functions (IMFs). EEGs from 25 control subjects were registered in basal and motor activity (hand movements) using only one EEG channel. Over the basic signal, IMF signals are computed. Gamma-band activity is computed using power spectrum density in the 30–60 Hz range. Event-related synchronization (ERS) was defined as the ratio of motor and basal activity. To evaluate the performance of the new EMD based method, ERS was computed from the basic and IMF signals. The ERS obtained using IMFs improves, from 31.00% to 73.86%, on the original ERS for the right hand, and from 22.17% to 47.69% for the left hand. As EEG processing is improved, the clinical applications of gamma-band activity will expand.Universidad de AlcaláInstituto de Salud Carlos II

Unmixing oscillatory brain activity by EEG source localization and empirical mode decomposition

Neuronal activity is composed of synchronous and asynchronous oscillatory activity at different frequencies. The neuronal oscillations occur at time scales well matched to the temporal resolution of electroencephalography (EEG); however, to derive meaning from the electrical brain activity as measured from the scalp, it is useful to decompose the EEG signal in space and time. In this study, we elaborate on the investigations into source-based signal decomposition of EEG. Using source localization, the electrical brain signal is spatially unmixed and the neuronal dynamics from a region of interest are analyzed using empirical mode decomposition (EMD), a technique aimed at detecting periodic signals. We demonstrate, first in simulations, that the EMD is more accurate when applied to the spatially unmixed signal compared to the scalp-level signal. Furthermore, on EEG data recorded simultaneously with transcranial magnetic stimulation (TMS) over the hand area of the primary motor cortex, we observe a link between the peak to peak amplitude of the motor-evoked potential (MEP) and the phase of the decomposed localized electrical activity before TMS onset. The results thus encourage combination of source localization and EMD in the pursuit of further insight into the mechanisms of the brain with respect to the phase and frequency of the electrical oscillations and their cortical origin

A Study of Biomedical Time Series Using Empirical Mode Decomposition : Extracting event-related modes from EEG signals recorded during visual processing of contour stimuli

Noninvasive neuroimaging techniques like functional Magnetic Resonance Imaging (fMRI) and/or Electroencephalography (EEG) allow researchers to investigate and analyze brain activities during visual processing. EEG offers a high temporal resolution at a level of submilliseconds which can be combined favorably with fMRI which has a good spatial resolution on small spatial scales in the millimeter range. These neuroimaging techniques were, and still are instrumental in the diagnoses and treatments of neurological disorders in the clinical applications. In this PhD thesis we concentrate on lectrophysiological signatures within EEG recordings of a combined EEG-fMRI data set which where taken while performing a contour integration task. The estimation of location and distribution of the electrical sources in the brain from surface recordings which are responsible for interesting EEG waves has drawn the attention of many EEG/MEG researchers. However, this process which is called brain source localization is still one of the major problems in EEG. It consists of solving two modeling problems: forward and inverse. In the forward problem, one is interested in predicting the expected potential distribution on the scalp from given electrical sources that represent active neurons in the head. These evaluations are necessary to solve the inverse problem which can be defined as the problem of estimating the brain sources that generated the measured electrical potentials. This thesis presents a data-driven analysis of EEG data recorded during a combined EEG/fMRI study of visual processing during a contour integration task. The analysis is based on an ensemble empirical mode decomposition (EEMD) and discusses characteristic features of event related modes (ERMs) resulting from the decomposition. We identify clear differences in certain ERMs in response to contour vs non-contour Gabor stimuli mainly for response amplitudes peaking around 100 [ms] (called P100) and 200 [ms] (called N200) after stimulus onset, respectively. We observe early P100 and N200 responses at electrodes located in the occipital area of the brain, while late P100 and N200 responses appear at electrodes located in frontal brain areas. Signals at electrodes in central brain areas show bimodal early/late response signatures in certain ERMs. Head topographies clearly localize statistically significant response differences to both stimulus conditions. Our findings provide an independent proof of recent models which suggest that contour integration depends on distributed network activity within the brain. Next and based on the previous analysis, a new approach for source localization of EEG data based on combining ERMs, extracted with EEMD, with inverse models has been presented. As the first step, 64 channel EEG recordings are pooled according to six brain areas and decomposed, by applying an EEMD, into their underlying ERMs. Then, based upon the problem at hand, the most closely related ERM, in terms of frequency and amplitude, is combined with inverse modeling techniques for source localization. More specifically, the standardized low resolution brain electromagnetic tomography (sLORETA) procedure is employed in this work. Accuracy and robustness of the results indicate that this approach deems highly promising in source localization techniques for EEG data. Given the results of analyses above, it can be said that EMD is able to extract intrinsic signal modes, ERMs, which contain decisive information about responses to contour and non-contour stimuli. Hence, we introduce a new toolbox, called EMDLAB, which serves the growing interest of the signal processing community in applying EMD as a decomposition technique. EMDLAB can be used to perform, easily and effectively, four common types of EMD: plain EMD, ensemble EMD (EEMD), weighted sliding EMD (wSEMD) and multivariate EMD (MEMD) on the EEG data. The main goal of EMDLAB toolbox is to extract characteristics of either the EEG signal by intrinsic mode functions (IMFs) or ERMs. Since IMFs reflect characteristics of the original EEG signal, ERMs reflect characteristics of ERPs of the original signal. The new toolbox is provided as a plug-in to the well-known EEGLAB which enables it to exploit the advantageous visualization capabilities of EEGLAB as well as statistical data analysis techniques provided there for extracted IMFs and ERMs of the signal

NEW APPROACHES FOR ASSESSING TIME-VARYING FUNCTIONAL BRAIN CONNECTIVITY USING FMRI DATA

It was long assumed that functional connectivity (FC) among brain regions did not vary substantially during a single resting-state functional magnetic resonance imaging (rs-fMRI) run. However, an increasing number of studies have reported on the existence of time-varying functional connectivity (TVC) in rs-fMRI data taking place in a considerably shorter time window than previously thought (i.e., on the order of seconds and minutes). However, the study of TVFC is a relatively new research area and there remain a number of unaddressed problems hindering its ability to fulfill its promise of increasing our knowledge of human brain function. First, while it has previously been shown that autocorrelation can negatively impact estimates of static functional connectivity, its impact on TVC estimates has not been established. Understanding the influence of autocorrelation on TVFC is of high importance, as we hypothesize the autocorrelation within a time series can inflate the sampling variability of TVC estimated using sliding window techniques, leading to the increase of risk of misinterpreting noise as true TVC and negatively impact subsequent estimation of whole-brain time varying functional connectivity. We thus study the impact of autocorrelation on TVC and how to mitigate it. Second, there is a need for new analytic approaches for estimating TVC. Most studies use a sliding window approach, where the correlation between region is computed locally within a specific time window that is moved across time. A shortcoming of this approach is the need to select an a priori window length for analysis. To circumvent this issue, we focus on the use of instantaneous phase synchronization (IPS), which offers single time-point resolution of time-resolved fMRI connectivity. The use of IPS necessitates bandpass filtering the data to obtain valid results. We seek to show how bandpass filtering affects the estimates of IPS metrics such as phase locking value (PLV) and phase coherence. Further, as current metrics discard the temporal transitions from positive to negative associations common in IPS analysis we introduce a new approach within IPS framework for circumventing this issue. Third, the choice of cut-off frequencies when bandpass filtering in IPS analysis is to some extend arbitrary. We seek to compare standard phase synchronization using the Hilbert transform with empirical mode decomposition (EMD) which eliminates the need for bandpass filtering in a data driven manner. While the use of EMD has a number of benefits compared to the Hilbert transform, it has a couple shortcomings: the susceptibility of the EMD to the SNR of the signal and untangling frequencies close to one another. To circumvent this issue and improve the assessment of IPS, we propose the use of an alternative decomposition approach, multivariate variational mode decomposition (MVMD) for phase synchronization analysis.