Towards Real-World BCI: CCSPNet, A Compact Subject-Independent Motor Imagery Framework

A conventional subject-dependent (SD) brain-computer interface (BCI) requires a complete data-gathering, training, and calibration phase for each user before it can be used. In recent years, a number of subject-independent (SI) BCIs have been developed. However, there are many problems preventing them from being used in real-world BCI applications. A weaker performance compared to the subject-dependent (SD) approach, and a relatively large model requiring high computational power are the most important ones. Therefore, a potential real-world BCI would greatly benefit from a compact low-power subject-independent BCI framework, ready to be used immediately after the user puts it on. To move towards this goal, we propose a novel subject-independent BCI framework named CCSPNet (Convolutional Common Spatial Pattern Network) trained on the motor imagery (MI) paradigm of a large-scale electroencephalography (EEG) signals database consisting of 21600 trials for 54 subjects performing two-class hand-movement MI tasks. The proposed framework applies a wavelet kernel convolutional neural network (WKCNN) and a temporal convolutional neural network (TCNN) in order to represent and extract the diverse spectral features of EEG signals. The outputs of the convolutional layers go through a common spatial pattern (CSP) algorithm for spatial feature extraction. The number of CSP features is reduced by a dense neural network, and the final class label is determined by a linear discriminative analysis (LDA) classifier. The CCSPNet framework evaluation results show that it is possible to have a low-power compact BCI that achieves both SD and SI performance comparable to complex and computationally expensive.Comment: 15 pages, 6 figures, 6 tables, 1 algorith

Signal Processing Combined with Machine Learning for Biomedical Applications

The Master’s thesis is comprised of four projects in the realm of machine learning and signal processing. The abstract of the thesis is divided into four parts and presented as follows, Abstract 1: A Kullback-Leibler Divergence-Based Predictor for Inter-Subject Associative BCI. Inherent inter-subject variability in sensorimotor brain dynamics hinders the transferability of brain-computer interface (BCI) model parameters across subjects. An individual training session is essential for effective BCI control to compensate for variability. We report a Kullback-Leibler Divergence (KLD)-based predictor for inter-subject associative BCI. An online dataset comprising left/right hand, both feet, and tongue motor imagery tasks was used to show correlation between the proposed inter-subject predictor and BCI performance. Linear regression between the KLD predictor and BCI performance showed a strong inverse correlation (r = -0.62). The KLD predictor can act as an indicator for generalized inter-subject associative BCI designs. Abstract 2: Multiclass Sensorimotor BCI Based on Simultaneous EEG and fNIRS. Hybrid BCI (hBCI) utilizes multiple data modalities to acquire brain signals during motor execution (ME) tasks. Studies have shown significant enhancements in the classification of binary class ME-hBCIs; however, four-class ME-hBCI classification is yet to be done using multiclass algorithms. We present a quad-class classification of ME-hBCI tasks from simultaneous EEG-fNIRS recordings. Appropriate features were extracted from EEG-fNIRS signals and combined for hybrid features and classified with support vector machine. Results showed a significant increase in hybrid accuracy over single modalities and show hybrid method’s performance enhancement capability. Abstract 3: Deep Learning for Improved Inter-Subject EEG-fNIRS Hybrid BCI Performance. Multimodality based hybrid BCI has become famous for performance improvement; however, the inherent inter-subject and inter-session variation between participants brain dynamics poses obstacles in achieving high performance. This work presents an inter-subject hBCI to classify right/left-hand MI tasks from simultaneous EEG-fNIRS recordings of 29 healthy subjects. State-of-art features were extracted from EEG-fNIRS signals and combined for hybrid features, and finally, classified using deep Long short-term memory classifier. Results showed an increase in the inter-subject performance for the hybrid system while making the system more robust to brain dynamics change and hints to the feasibility of EEG-fNIRS based inter-subject hBCI. Abstract 4: Microwave Based Glucose Concentration Classification by Machine Learning. Non-invasive blood sugar measurement attracts increased attention in recent years, given the increase in diabetes-related complications and inconvenience in the traditional ways using blood. This work utilized machine learning (ML) algorithms to classify glucose concentration (GC) from the measured broadband microwave scattering signals (S11). An N-type microwave adapter pair was utilized to measure the sweeping frequency scattering-parameter (S-parameter) of the glucose solutions with GC varying from 50-10,000 dg/dL. Dielectric parameters were retrieved from the measured wideband complex S-parameters based on the modified Debye dielectric dispersion model. Results indicate that the best algorithm can achieve a perfect classification accuracy and suggests an alternate way to develop a GC detection method using ML algorithms

Towards Adaptive Classification using Riemannian Geometry approaches in Brain-Computer Interfaces

International audienceThe omnipresence of non-stationarity and noise in Electroencephalogram signals restricts the ubiquitous use of Brain-Computer interface. One of the possible ways to tackle this problem is to adapt the computational model used to detect and classify different mental states. Adapting the model will possibly help us to track the changes and thus reducing the effect of non-stationarities. In this paper, we present different adaptation strategies for state of the art Riemannian geometry based classifiers. The offline evaluation of our proposed methods on two different datasets showed a statistically significant improvement over baseline non-adaptive classifiers. Moreover, we also demonstrate that combining different (hybrid) adaptation strategies generally increased the performance over individual adaptation schemes. Also, the improvement in average classification accuracy for a 3-class mental imagery BCI with hybrid adaption is as high as around 17% above the baseline non-adaptive classifier

A Multifaceted Approach to Covert Attention Brain-Computer Interfaces

Over the last years, brain-computer interfaces (BCIs) have shown their value for assistive technology and neurorehabilitation. Recently, a BCI-approach for the rehabilitation of hemispatial neglect has been proposed on the basis of covert visuospatial attention (CVSA). CVSA is an internal action which can be described as shifting one's attention to the visual periphery without moving the actual point of gaze. Such attention shifts induce a lateralization in parietooccipital blood flow and oscillations in the so-called alpha band (8-14 Hz), which can be detected via electroencephalography (EEG), magnetoencephalography (MEG) or functional magnetic resonance imaging (fMRI). Previous studies have proven the technical feasibility of using CVSA as a control signal for BCIs, but unfortunately, these BCIs could not provide every subject with sufficient control. The aim of this thesis was to investigate the possibility of amplifying the weak lateralization patterns in the alpha band - the main reason behind insufficient CVSA BCI performance. To this end, I have explored three different approaches that could lead to better performing and more inclusive CVSA BCI systems. The first approach illuminated the changes in the behavior and brain patterns by closing the loop between subject and system with continuous real-time feedback at the instructed locus of attention. I could observe that even short (20 minutes) stretches of real-time feedback have an effect on behavioral correlates of attention, even when the changes observed in the EEG remained less conclusive. The second approach attempted to complement the information extracted fromthe EEG signal with another sensing modality that could provide additional information about the state of CVSA. For this reason, I firstly combined functional functional near-infrared spectroscopy (fNIRS) with EEG measurements. The results showed that, while the EEG was able to pick up the expected lateralization in the alpha band, the fNIRS was not able to reliably image changes in blood circulation in the parietooccipital cortex. Secondly, I successfully combined data from the EEG with measures of pupil size changes, induced by a high illumination contrast between the covertly attended target regions, which resulted in an improved BCI decoding performance. The third approach examined the option of using noninvasive electrical brain stimulation to boost the power of the alpha band oscillations and therefore render the lateralization pattern in the alpha band more visible compared to the background activity. However, I could not observe any impact of the stimulation on the ongoing alpha band power, and thus results of the subsequent effect on the lateralization remain inconclusive. Overall, these studies helped to further understand CVSA and lay out a useful basis for further exploration of the connection between behavior and alpha power oscillations in CVSA tasks, as well as for potential directions to improve CVSA-based BCIs

Advancing Pattern Recognition Techniques for Brain-Computer Interfaces: Optimizing Discriminability, Compactness, and Robustness

In dieser Dissertation formulieren wir drei zentrale Zielkriterien zur systematischen Weiterentwicklung der Mustererkennung moderner Brain-Computer Interfaces (BCIs). Darauf aufbauend wird ein Rahmenwerk zur Mustererkennung von BCIs entwickelt, das die drei Zielkriterien durch einen neuen Optimierungsalgorithmus vereint. Darüber hinaus zeigen wir die erfolgreiche Umsetzung unseres Ansatzes für zwei innovative BCI Paradigmen, für die es bisher keine etablierte Mustererkennungsmethodik gibt

Discovering structure in multi-neuron recordings through network modelling

Our brains contain billions of neurons, which are continually producing electrical signals to relay information around the brain. Yet most of our knowledge of how the brain works comes from studying the activity of one neuron at a time. Recently, studies of multiple neurons have shown that they tend to be active together. These coordinated dynamics vary across brain states and impact the way that external sensory information is processed. To investigate the network-level mechanisms that underlie these dynamics, we developed novel computational techniques to fit a deterministic spiking network model directly to multi-neuron recordings from different rodent species, sensory modalities, and behavioral states. We found that inhibition modulates the interactions between intrinsic dynamics and sensory inputs to control the reliability of sensory representations. We next recorded from awake mice using calcium imaging techniques, and acquired activity from 10,000 neurons simultaneously in visual cortex while presenting 2,800 different natural images. In awake mice, these intrinsic population-wide fluctuations were suppressed and responses to visual stimuli were reliable. The stimulus-related information was stored in a high-dimensional neural space: 1,000 dimensions of neural activity accounted for 90\% of the variance. Although awake mice lacked large population-wide fluctuations in activity, we observed several dozen dimensions of spontaneous activity. These dimensions of spontaneous activity were not spatially organized in cortex. Instead they were related to the orofacial behaviors of the mouse: over 50\% of the shared variability of the network could be predicted from the facial movements of the mouse. In simulations of high-dimensional network activity, flexible patterns of activity were reproduced only if the network contained multiple dimensions of inhibitory activity. We tested this hypothesis in our recordings and found that inhibitory neuron activity did track excitatory neuron activity across multiple dimensions

Brain-Computer Interfaces for Non-clinical (Home, Sports, Art, Entertainment, Education, Well-being) Applications

HCI researchers interest in BCI is increasing because the technology industry is expanding into application areas where efficiency is not the main goal of concern. Domestic or public space use of information and communication technology raise awareness of the importance of affect, comfort, family, community, or playfulness, rather than efficiency. Therefore, in addition to non-clinical BCI applications that require efficiency and precision, this Research Topic also addresses the use of BCI for various types of domestic, entertainment, educational, sports, and well-being applications. These applications can relate to an individual user as well as to multiple cooperating or competing users. We also see a renewed interest of artists to make use of such devices to design interactive art installations that know about the brain activity of an individual user or the collective brain activity of a group of users, for example, an audience. Hence, this Research Topic also addresses how BCI technology influences artistic creation and practice, and the use of BCI technology to manipulate and control sound, video, and virtual and augmented reality (VR/AR)

An investigation into current and vibration signatures of three phase induction motors

This research aimed at investigating the relationship between three phase induction motors vibration (MVS) and current signatures (MCS). This is essential due to the cost of vibration measuring equipment and in cases where vibration of interest point is not accessible; such as electrical submersible pumps (ESP) used in oil industry. A mathematical model was developed to understand the effects of two types of induction motors common faults; rotor bar imperfections and phase imbalance on the motor vibration and current signatures. An automated test facility was developed in which 1.1 kW three phase motor could be tested under varying shaft rotation speeds and loads for validating the developed model. Time and frequency domains statistical parameters of the measured signals were calculated for fault detection and assessing its severity. The measured signals were also processed using the short time Fourier transform (STFT), the Wigner-Ville distribution (WVD), the continuous wavelet transform (CWT) and discrete wavelet transform (DWT) and wavelet multi-resolution analysis (MRA). The non-stationary components, representing faults within induction motor measured vibration and current signals, were successfully detected using wavelet decomposition technique. An effective alternative to direct vibration measurement scheme, based on radial basis function networks, was developed to the reconstruction of motor vibration using measurements of one phase of the motor current. It was found that this method captured the features of induction motor faults with reasonable degrees of accuracy. Another method was also developed for the early detection and diagnosis of faults using an enhanced power factor method. Experimental results confirmed that the power factor can be used successfully for induction motor fault diagnosis and is also promising in assessing fault severity. The suggested two methods offer inexpensive, reliable and non-intrusive condition monitoring tools that suits real-time applications. Directions for further work were also outlined

Vibrational Spectroscopy Prospects in Frontline Clinical Diagnosis

The key experimental results from this research are the viable and cost effective methods of diagnosing oral and pancreatic cancer with accuracies over 90%. Furthermore, development of the molecular windowing method to further narrow down the origins of those cancer biomarkers and further improve accuracy.Many papers are being published demonstrating how vibrational spectral biomarkers can be used to diagnose a whole variety of diseases, from cancers to colitis. However, much of the research, proposed as discovering a useful tool for clinical diagnosis, has not yet been widely utilised in clinical practice. This is due mainly to the lack or reproducibility of the findings and current lack of relating the spectral observation to a root biological cause. This thesis aims to highlight the inconsistencies between studies and propose an improved process for spectral biomarker identification, including suggestions for follow up studies to discover the foundation of the spectral change. This thesis reassesses, and adds to, ground covered by previous reviews regarding sample preparation, patient selection and multivariate analysis.Resultantly, this thesis brings light to the need, and suggests solutions, for:• a method to standardise results between detection devices,• knowledge of the additional requirements for using biomarkers for disease monitoring/prognosis,• understanding the biological root cause for the spectral shift.These promising results and suggestions for combined methodology improvements will provide guidance to enable this burgeoning research field to improve patient outcome in the clinical sphere