On Error-related Potentials during Sensorimotor-based Brain-Computer Interface: Explorations with a Pseudo-Online Brain-Controlled Speller

Objective: Brain-computer interface (BCI) spelling is a promising communication solution for people in paralysis. Currently, BCIs suffer from imperfect decoding accuracy which calls for methods to handle spelling mistakes. Detecting error-related potentials (ErrPs) has been early identified as a potential remedy. Nevertheless, few works have studied the elicitation of ErrPs during engagement with other BCI tasks, especially when BCI feedback is provided continuously. Here, we test the possibility of correcting errors during pseudo-online Motor Imagery (MI) BCI spelling through ErrPs, and investigate whether BCI feedback hinders their generation. Results: Ten subjects performed a series of MI spelling tasks with and without observing BCI feedback. The average pseudo-online ErrP detection accuracy was found to be significantly above the chance level in both conditions and did not significantly differ between the two (74% with, and 78% without feedback). Conclusions: Our results support the possibility to detect ErrPs during MI-BCI spelling and suggest the absence of any BCI feedback-related interference

Context–aware Learning for Generative Models

This work studies the class of algorithms for learning with side-information that emerges by extending generative models with embedded context-related variables. Using finite mixture models (FMMs) as the prototypical Bayesian network, we show that maximum-likelihood estimation (MLE) of parameters through expectation-maximization (EM) improves over the regular unsupervised case and can approach the performances of supervised learning, despite the absence of any explicit ground-truth data labeling. By direct application of the missing information principle (MIP), the algorithms' performances are proven to range between the conventional supervised and unsupervised MLE extremities proportionally to the information content of the contextual assistance provided. The acquired benefits regard higher estimation precision, smaller standard errors, faster convergence rates, and improved classification accuracy or regression fitness shown in various scenarios while also highlighting important properties and differences among the outlined situations. Applicability is showcased with three real-world unsupervised classification scenarios employing Gaussian mixture models. Importantly, we exemplify the natural extension of this methodology to any type of generative model by deriving an equivalent context-aware algorithm for variational autoencoders (VAs), thus broadening the spectrum of applicability to unsupervised deep learning with artificial neural networks. The latter is contrasted with a neural-symbolic algorithm exploiting side information

Brain-Machine Interfaces: A Tale of Two Learners

Brain-machine interface (BMI) technology has rapidly matured over the last two decades, mainly thanks to the introduction of artificial intelligence (AI) methods, in particular, machine-learning algorithms. Yet, the need for subjects to learn to modulate their brain activity is a key component of successful BMI control. Blending machine and subject learning, or mutual learning, is widely acknowledged in the BMI field. Nevertheless, we posit that current research trends are heavily biased toward the machine-learning side of BMI training. In this article, we take a critical view of the relevant literature, and our own previous work, to identify the key issues for more effective mutual-learning schemes in translational BMIs that are specifically tailored to promote subject learning. We identify the main caveats in the literature on subject learning in BMI, in particular, the lack of longitudinal studies involving end users and shortcomings in quantifying subject learning, and pinpoint critical improvements for future experimental designs

Machine-learning based monitoring of cognitive workload in rescue missions with drones

In search and rescue missions, drone operations are challenging and cognitively demanding. High levels of cognitive workload can affect rescuers' performance, leading to failure with catastrophic outcomes. To face this problem, we propose a machine learning algorithm for real-time cognitive workload monitoring to understand if a search and rescue operator has to be replaced or if more resources are required. Our multimodal cognitive workload monitoring model combines the information of 25 features extracted from physiological signals, such as respiration, electrocardiogram, photoplethysmogram, and skin temperature, acquired in a noninvasive way. To reduce both subject and day inter-variability of the signals, we explore different feature normalization techniques, and introduce a novel weighted-learning method based on support vector machines suitable for subject-specific optimizations. On an unseen test set acquired from 34 volunteers, our proposed subject-specific model is able to distinguish between low and high cognitive workloads with an average accuracy of 87.3% and 91.2% while controlling a drone simulator using both a traditional controller and a new-generation controller, respectively

Decoding Neural Correlates of Cognitive States to Enhance Driving Experience

Modern cars can support their drivers by assessing and autonomously performing different driving maneuvers based on information gathered by in-car sensors. We propose that brain–machine interfaces (BMIs) can provide complementary information that can ease the interaction with intelligent cars in order to enhance the driving experience. In our approach, the human remains in control, while a BMI is used to monitor the driver's cognitive state and use that information to modulate the assistance provided by the intelligent car. In this paper, we gather our proof-of-concept studies demonstrating the feasibility of decoding electroencephalography correlates of upcoming actions and those reflecting whether the decisions of driving assistant systems are in-line with the drivers' intentions. Experimental results while driving both simulated and real cars consistently showed neural signatures of anticipation, movement preparation, and error processing. Remarkably, despite the increased noise inherent to real scenarios, these signals can be decoded on a single-trial basis, reflecting some of the cognitive processes that take place while driving. However, moderate decoding performance compared to the controlled experimental BMI paradigms indicate there exists room for improvement of the machine learning methods typically used in the state-of-the-art BMIs. We foresee that neural fusion correlates with information extracted from other physiological measures, e.g., eye movements or electromyography as well as contextual information gathered by in-car sensors will allow intelligent cars to provide timely and tailored assistance only if it is required; thus, keeping the user in the loop and allowing him to fully enjoy the driving experience

Learning to control a BMI-driven wheelchair for people with severe tetraplegia

Mind-controlled wheelchairs are an intriguing assistive mobility solution applicable in complete paralysis. Despite technical progress in brain-machine interface (BMI) technology, its translation remains elusive. The primary objective of this study is to probe the hypothesis that BMI skill acquisition by end-users is fundamental to control a non-invasive brain-actuated intelligent wheelchair in real-world settings. Here, we show that three tetraplegic spinal cord injury users could be trained to operate a non-invasive, self-paced thought-controlled wheelchair and execute complex navigation tasks. However, only the two users exhibiting increasing decoding performance and feature discriminancy, as well as significant neuroplasticity changes and improved BMI command latency, achieved high navigation performance. Additionally, we show that dexterous, continuous control of robots is possible through low-degree of freedom, discrete and uncertain control channels like a motor imagery BMI, by blending human and artificial intelligence through shared-control methodologies. We posit that subject learning and shared-control are the key-components paving the way for translational non-invasive BMI

Cortico-muscular coherence is reduced acutely post-stroke and increases bilaterally during motor recovery: a pilot study

Motor recovery following stroke is believed to necessitate alteration in functional connectivity between cortex and muscle. Cortico-muscular coherence has been proposed as a potential biomarker for post-stroke motor deficits, enabling a quantification of recovery, as well as potentially indicating the regions of cortex involved in recovery of function. We recorded simultaneous EEG and EMG during wrist extension from healthy participants and patients following ischaemic stroke, evaluating function at three time points post-stroke. EEG–EMG coherence increased over time, as wrist mobility recovered clinically, and by the final evaluation, coherence was higher in the patient group than in the healthy controls. Moreover, the cortical distribution differed between the groups, with coherence involving larger and more bilaterally scattered areas of cortex in the patients than in the healthy participants. The findings suggest that EEG–EMG coherence has the potential to serve as a biomarker for motor recovery and to provide information about the cortical regions that should be targeted in rehabilitation therapies based on real-time EEG

Functional electrical stimulation driven by a brain–computer interface in acute and subacute stroke patients impacts beta power and long-range temporal correlation

Functional electrical stimulation (FES) is a standard rehabilitation approach applied by therapists to aid motor recovery in a paretic limb post-stroke. Information pertaining to the timing of a movement attempt can be obtained from changes in the power of oscillatory electrophysiological activity in motor cortical regions, derived from scalp electroencephalographic (EEG) recordings. The use of a brain–computer interface (BCI), to enable delivery of FES within a tight temporal window with a movement attempt detected in scalp EEG, is associated with greater motor recovery than conventional FES application in patients in the chronic phase post-stroke. We hypothesized that the heightened neural plasticity early post-stroke could further enhance motor recovery and that motor improvements would be accompanied by changes in the motor cortical sensorimotor rhythm after compared with before treatment. Here we assessed clinical outcome and changes in the sensorimotor rhythm in patients following subcortical stroke affecting the non-dominant hemisphere from a study comparing timing of FES delivery using a BCI, with a Sham group, receiving FES with no such temporal relationship. The BCI group showed greater clinical improvement following the treatment, particularly early post-stroke, and a greater decrease in beta oscillatory power and long-range temporal correlation over contralateral (ipsilesional) motor cortex. The electrophysiological changes are consistent with a reduction in compensatory processes and a transition towards a subcritical state when movement is triggered at the time of movement detection based on motor cortical oscillations