Classification of different reaching movements from the same limb using EEG

Objective. Brain–computer-interfaces (BCIs) have been proposed not only as assistive technologies but also as rehabilitation tools for lost functions. However, due to the stochastic nature, poor spatial resolution and signal to noise ratio from electroencephalography (EEG), multidimensional decoding has been the main obstacle to implement non-invasive BCIs in real-live rehabilitation scenarios. This study explores the classification of several functional reaching movements from the same limb using EEG oscillations in order to create a more versatile BCI for rehabilitation. Approach. Nine healthy participants performed four 3D center-out reaching tasks in four different sessions while wearing a passive robotic exoskeleton at their right upper limb. Kinematics data were acquired from the robotic exoskeleton. Multiclass extensions of Filter Bank Common Spatial Patterns (FBCSP) and a linear discriminant analysis (LDA) classifier were used to classify the EEG activity into four forward reaching movements (from a starting position towards four target positions), a backward movement (from any of the targets to the starting position and rest). Recalibrating the classifier using data from previous or the same session was also investigated and compared. Main results. Average EEG decoding accuracy were significantly above chance with 67%, 62.75%, and 50.3% when decoding three, four and six tasks from the same limb, respectively. Furthermore, classification accuracy could be increased when using data from the beginning of each session as training data to recalibrate the classifier. Significance. Our results demonstrate that classification from several functional movements performed by the same limb is possible with acceptable accuracy using EEG oscillations, especially if data from the same session are used to recalibrate the classifier. Therefore, an ecologically valid decoding could be used to control assistive or rehabilitation mutli-degrees of freedom (DoF) robotic devices using EEG data. These results have important implications towards assistive and rehabilitative neuroprostheses control in paralyzed patients.This study was funded by the Baden-Württemberg Stiftung (GRUENS), the Deutsche Forschungsgemeinschaft (DFG, Koselleck and SP-1533/2-1), Bundes Ministerium für Bildung und Forschung BMBF MOTORBIC (FKZ 13GW0053), the fortune-Program of the University of Tübingen (2422-0-0), and AMORSA (FKZ 16SV7754). A Sarasola-Sanz’s work is supported by the La Caixa-DAAD scholarship, and N IrastorzaLanda’s work by the Basque Government and IKERBASQUE, Basque Foundation for Science

EEG and ECoG features for Brain Computer Interface in Stroke Rehabilitation

The ability of non-invasive Brain-Computer Interface (BCI) to control an exoskeleton was used for motor rehabilitation in stroke patients or as an assistive device for the paralyzed. However, there is still a need to create a more reliable BCI that could be used to control several degrees of Freedom (DoFs) that could improve rehabilitation results. Decoding different movements from the same limb, high accuracy and reliability are some of the main difficulties when using conventional EEG-based BCIs and the challenges we tackled in this thesis. In this PhD thesis, we investigated that the classification of several functional hand reaching movements from the same limb using EEG is possible with acceptable accuracy. Moreover, we investigated how the recalibration could affect the classification results. For this reason, we tested the recalibration in each multi-class decoding for within session, recalibrated between-sessions, and between sessions. It was shown the great influence of recalibrating the generated classifier with data from the current session to improve stability and reliability of the decoding. Moreover, we used a multiclass extension of the Filter Bank Common Spatial Patterns (FBCSP) to improve the decoding accuracy based on features and compared it to our previous study using CSP. Sensorimotor-rhythm-based BCI systems have been used within the same frequency ranges as a way to influence brain plasticity or controlling external devices. However, neural oscillations have shown to synchronize activity according to motor and cognitive functions. For this reason, the existence of cross-frequency interactions produces oscillations with different frequencies in neural networks. In this PhD, we investigated for the first time the existence of cross-frequency coupling during rest and movement using ECoG in chronic stroke patients. We found that there is an exaggerated phase-amplitude coupling between the phase of alpha frequency and the amplitude of gamma frequency, which can be used as feature or target for neurofeedback interventions using BCIs. This coupling has been also reported in another neurological disorder affecting motor function (Parkinson and dystonia) but, to date, it has not been investigated in stroke patients. This finding might change the future design of assistive or therapeuthic BCI systems for motor restoration in stroke patients

Are We the Robots? : Man-Machine Integration

We experience and interact with the world through our body. The founding father of computer science, Alan Turing, correctly realized that one of the most important features of the human being is the interaction between mind and body. Since the original demonstration that electrical activity of the cortical neurons can be employed to directly control a robotic device, the research on the so-called Brain-Machine Interfaces (BMIs) has impressively grown. For example, current BMIs dedicated to both experimental and clinical studies can translate raw neuronal signals into computational commands to reproduce reaching or grasping in artificial actuators. These developments hold promise for the restoration of limb mobility in paralyzed individuals. However, as the authors review in this chapter, before this goal can be achieved, several hurdles have to be overcome, including developments in real-time computational algorithms and in designing fully implantable and biocompatible devices. Future investigations will have to address the best solutions for restoring sensation to the prosthetic limb, which still remains a major challenge to full integration of the limb into the user's self-image

VALIDATION OF A MODEL OF SENSORIMOTOR INTEGRATION WITH CLINICAL BENEFITS

Healthy sensorimotor integration – or how our touch influences our movements – is critical to efficiently interact with our environment. Yet, many aspects of this process are still poorly understood. Importantly, several movement disorders are often considered as originating from purely motor impairments, while a sensory origin could also lead to a similar set of symptoms. To alleviate these issues, we hereby propose a novel biologically-based model of the sensorimotor loop, known as the SMILE model. After describing both the functional, and the corresponding neuroanatomical versions of the SMILE, we tested several aspects of its motor component through functional magnetic resonance imaging (fMRI) and transcranial magnetic stimulation (TMS). Both experimental studies resulted in coherent outcomes with respect to the SMILE predictions, but they also provided novel scientific outcomes about such broad topics as the sub-phases of motor imagery, the neural processing of bodily representations, or the extend of the role of the extrastriate body area. In the final sections of this manuscript, we describe some potential clinical application of the SMILE. The first one presents the identification of plausible neuroanatomical origins for focal hand dystonia, a yet poorly understood sensorimotor disorder. The last chapter then covers possible improvements on brain-machine interfaces, driven by a better understanding of the sensorimotor system. -- La façon dont votre sens du toucher et vos mouvements interagissent est connue sous le nom d’intégration sensorimotrice. Ce procédé est essentiel pour une interaction normale avec tout ce qui nous entoure. Cependant, plusieurs aspects de ce processus sont encore méconnus. Plus important encore, l’origine de certaines déficiences motrices encore trop peu comprises sont parfois considérées comme purement motrice, alors qu’une origine sensorielle pourrait mener à un même ensemble de symptômes. Afin d’améliorer cette situation, nous proposons ici un nouveau modèle d’intégration sensorimotrice, dénommé « SMILE », basé sur les connaissances de neurobiologie actuelles. Dans ce manuscrit, nous commençons par décrire les caractéristiques fonctionnelles et neuroanatomiques du SMILE. Plusieurs expériences sont ensuite effectuées, via l’imagerie par résonance magnétique fonctionnelle (IRMf), et la stimulation magnétique transcranienne (SMT), afin de tester différents aspects de la composante motrice du SMILE. Si les résultats de ces expériences corroborent les prédictions du SMILE, elles ont aussi mis en évidences d’autres résultats scientifiques intéressants et novateurs, dans des domaines aussi divers que les sous-phases de l’imagination motrice, les processus cérébraux liés aux représentations corporelles, ou encore l’extension du rôle de l’extrastriate body area. Dans les dernières parties de ce manuscrit, nous dévoilons quelques applications cliniques potentielles de notre modèle. Nous utilisons le SMILE afin de proposer deux origines cérébrales plausibles de la dystonie focale de la main. Le dernier chapitre présente comment certaines technologies existantes, telles que les interfaces cerveaux-machines, pourraient bénéficier d’une meilleure compréhension du système sensorimoteur

Development of EEG-based technologies for the characterization and treatment of neurological diseases affecting the motor function

This thesis presents a set of studies applying signal processing and data mining techniques in real-time working systems to register, characterize and condition the movement-related cortical activity of healthy subjects and of patients with neurological disorders affecting the motor function. Patients with two of the most widespread neurological affections impairing the motor function are considered here: patients with essential tremor and patients who have suffered a cerebro-vascular accident. The different chapters in the presented thesis show results regarding the normal cortical activity associated with the planning and execution of motor actions with the upper-limb, and the pathological activity related to the patients' motor dysfunction (measurable with muscle electrodes or movement sensors). The initial chapters of the book present i) a revision of the basic concepts regarding the role of the cerebral cortex in the motor control and the way in which the electroencephalographic activity allows its analysis and conditioning, ii) a study on the cortico-muscular interaction at the tremor frequency in patients with essential tremor under the effects of a drug reducing their tremor, and finally iii) a study based on evolutionary algorithms that aims to identify cortical patterns related to the planning of a number of motor tasks performed with a single arm. In the second half of the thesis book, two brain-computer interface systems to be used in rehabilitation scenarios with essential tremor patients and with patients with a stroke are proposed. In the first system, the electroencephalographic activity is used to anticipate voluntary movement actions, and this information is integrated in a multimodal platform estimating and suppressing the pathological tremors. In the second case, a conditioning paradigm for stroke patients based on the identification of the motor intention with temporal precision is presented and tested with a cohort of four patients along a month during which the patients undergo eight intervention sessions. The presented thesis has yielded advances from both the technological and the scientific points of view in all studies proposed. The main contributions from the technological point of view are: ¿ The design of an integrated upper-limb platform working in real-time. The platform was designed to acquire information from different types of noninvasive sensors (EEG, EMG and gyroscopic sensors) characterizing the planning and execution of voluntary movements. The platform was also capable of processing online the acquired data and generating an electrical feedback. ¿ The development of signal processing and classifying techniques adapted to the kind of signal recorded in the two kinds of patients considered in this thesis (patients with essential tremor and patients with a stroke) and to the requirements of online processing and real-time single-trial function desired for BCI applications. Especially in this regard, an original methodology to detect onsets of voluntary movements using slow cortical potentials and cortical rhythms has been presented. ¿ The design and validation in real-time of asynchronous BCI systems using motor planning EEG segments to anticipate or detect when patients begin a voluntary movement with the upper-limb. ¿ The proof of concept of the advantages of an EEG system integrated in a multimodal human-robot interface architecture that constitutes the first multimodal interface using the combined acquisition of EEG, EMG and gyroscopic data, which allows the concurrent characterization of different parts of the body associated with the execution of a movement. The main scientific contributions of this thesis are: ¿ The study of the EEG-based anticipation of voluntary movements presented in Chapter 5 of the thesis was the first demonstration (to the author's knowledge) of the capacity of the EEG signal to provide reliable movement predictions based on single-trial classification of online data of healthy subjects and ET patients. This study also provides, for the first time, the results of a BCI system tested in ET patients and it represents an original approach to BCI applications for this group of patients. ¿ It has been presented the first neurophysiological study using EEG and EMG data to analyze the effects of a drug on cortical activity and tremors of patients with ET. In addition, the obtained results have shown for the first time that a significant correlation exists between the dynamics of specific cortical oscillations and pathological tremor manifestation as a consequence of the drug effects. ¿ It has been proposed for the first time an experiment to inspect whether the EEG signal carries enough information to classify up to seven different tasks performed with a single limb. Both the methodology applied and the validation procedure are also innovative in this sort of studies. ¿ It has been demonstrated for the first time the relevance of combining different cortical sources of information (such as BP and ERD) to estimate the initiation of voluntary movements with the upper-limb. In this line, special relevance may be given to the positive results achieved with stroke patients, improving the results presented by similar previous EEG-based studies by other research groups. It has also been proposed for the first time an upper-limb intervention protocol for stroke patients using BP and ERD patterns to provide proprioceptive feedback tightly associated with the patients' expectations of movement. The effects of the proposed intervention have been studied with a small group of patients

Investigation of differences in cortical activation during wrist flexion and extension performed under real, passive and motor imagined paradigms

The neuromuscular control comparison between flexion and extension of the upper extremities has been conducted in a number of studies. It has been speculated that differences in the corticospinal pathway between flexion and extension may play a role in the cortical difference detected between flexion and extension, resulting in higher cortical activation for extension. However, it is still unclear as to what roles these pathways play, and to what degree other factors (muscle force activation, sensory feedback, frequency of movement, structural and/or functional differences) might influence the cortical activation in the brain. It has been speculated that the difference in cortical muscular pathways is due to flexion movements being used more often in day to day activities, therefore requiring less cortical activation for that movement. Through the investigation of the cortical differences present during different movement types, a deeper understanding into the differences between flexion and extension may be obtained. No previous study has compared the cortical differences between flexion and extension of the upper extremities during different movement types. In this study, an offline investigation is conducted between wrist flexion and extension, during real, passive and motor imaginary movement with the help of a servo controlled hand device. Simultaneous recording of EEG, EMG and wrist dynamics (velocity, angle, strain) were made on fifteen healthy right handed subjects performing 60 randomized repartitions of right wrist flexion and extension, for kinaesthetic motor imaginary, passively moved, and voluntary real active movements. Real movements were conducted at 10% relative subject maximum voluntary contraction (MVC). A servo controlled hand device was used to regulate dynamic force applied for real movements, and provide motion during passive movements. The use of different movement types with the aid of a servo controlled hand device, may give a deeper understanding into the effects of muscle force activation, rate of movement and corticospinal pathway on flexion and extension. In order to investigate the cortical differences between flexion and extension, subjects perceived difficulty, movement dynamics, movement related cortical potential (MRCP), event related desynchronization and synchronization (ERD/ERS), and phase locking value (PLV) were measured. Each measurement examines a different aspect of the cortical activation present in the brain, during the different movement types. Although relative muscle force activation between wrist real flexion and extension was similar, the motor cortex activation during extension was higher than during flexion, by MRCP and mu-band ERD, with subjects also perceiving real wrist extension to be more difficult to perform. Passive movements found higher motor cortex activation for flexion (MRCP, beta-band ERD), however higher somatosensory cortical activation was present during extension, by mu-band ERS and PLV. Motor imagined wrist flexion showed higher cortical activation during wrist flexion, by MRCP and beta-band ERD. Although numerous variables were tested (each in difference frequency bands), with some being significant and others being non-significant, overall it can be suggested that there was higher cortical activation for extension. The higher cortical activation during wrist extension movements may be due to corticospinal and somatosensory motor control pathways to motor neuron and from sensory neuron pools for extensor/flexor muscle and muscle spindle of the upper extremities. This investigation contributes to the current literature relating to cortical differences between flexion and extension of the upper extremities, by including the real, passive and motor imaginary differences between flexion and extension

Brain Computer Interfaces: Challenges to Clinical Viability Addressed in the Laboratory

Paralysis following spinal cord injuries, amyotrophic lateral sclerosis, stroke, and other disorders can intervene with signal transduction from the brain to the motor periphery, and eliminate the ability to perform volitional movements. Brain computer interfaces (BCI) directly measure brain activity associated with the user’s intent and translate the recorded brain activity into control signals for BCI applications, such as moving a computer cursor or a robot arm. While BCI technology has become an active and exciting field of research, much of the field’s development and achievements to date have taken place in the laboratory. The translation of BCI technology to the clinical setting is still not a reality. My thesis research has been dedicated to the objective of facilitating the translation of BCI systems from the primate lab to a clinical setting. That guiding objective has led me to work on several projects including: a technique that vastly improves the longevity of surgical implants in primate studies; a task that pushes the limits of sensorimotor performance – improving our knowledge of the function of primary motor cortex during realistic reaches and allowing us to quantify feedback effectiveness; characterizing the long-term tissue response to chronically implanted electrodes, and investigating how to optimally select parameters for neural information extraction. Each of these contributions will help bring BCI systems one step closer to clinical reality

AN INVESTIGATION OF ELECTROMYOGRAPHIC (EMG) CONTROL OF DEXTROUS HAND PROSTHESES FOR TRANSRADIAL AMPUTEES

In reference to IEEE copyrighted material which is used with permission in this thesis, the IEEE does not endorse any of Plymouth University's products or services.There are many amputees around the world who have lost a limb through conflict, disease or an accident. Upper-limb prostheses controlled using surface Electromyography (sEMG) offer a solution to help the amputees; however, their functionality is limited by the small number of movements they can perform and their slow reaction times. Pattern recognition (PR)-based EMG control has been proposed to improve the functional performance of prostheses. It is a very promising approach, offering intuitive control, fast reaction times and the ability to control a large number of degrees of freedom (DOF). However, prostheses controlled with PR systems are not available for everyday use by amputees, because there are many major challenges and practical problems that need to be addressed before clinical implementation is possible. These include lack of individual finger control, an impractically large number of EMG electrodes, and the lack of deployment protocols for EMG electrodes site selection and movement optimisation. Moreover, the inability of PR systems to handle multiple forces is a further practical problem that needs to be addressed. The main aim of this project is to investigate the research challenges mentioned above via non-invasive EMG signal acquisition, and to propose practical solutions to help amputees. In a series of experiments, the PR systems presented here were tested with EMG signals acquired from seven transradial amputees, which is unique to this project. Previous studies have been conducted using non-amputees. In this work, the challenges described are addressed and a new protocol is proposed that delivers a fast clinical deployment of multi-functional upper limb prostheses controlled by PR systems. Controlling finger movement is a step towards the restoration of lost human capabilities, and is psychologically important, as well as physically. A central thread running through this work is the assertion that no two amputees are the same, each suffering different injuries and retaining differing nerve and muscle structures. This work is very much about individualised healthcare, and aims to provide the best possible solution for each affected individual on a case-by-case basis. Therefore, the approach has been to optimise the solution (in terms of function and reliability) for each individual, as opposed to developing a generic solution, where performance is optimised against a test population. This work is unique, in that it contributes to improving the quality of life for each individual amputee by optimising function and reliability. The main four contributions of the thesis are as follows: 1- Individual finger control was achieved with high accuracy for a large number of finger movements, using six optimally placed sEMG channels. This was validated on EMG signals for ten non-amputee and six amputee subjects. Thumb movements were classified successfully with high accuracy for the first time. The outcome of this investigation will help to add more movements to the prosthesis, and reduce hardware and computational complexity. 2- A new subject-specific protocol for sEMG site selection and reliable movement subset optimisation, based on the amputee’s needs, has been proposed and validated on seven amputees. This protocol will help clinicians to perform an efficient and fast deployment of prostheses, by finding the optimal number and locations of EMG channels. It will also find a reliable subset of movements that can be achieved with high performance. 3- The relationship between the force of contraction and the statistics of EMG signals has been investigated, utilising an experimental design where visual feedback from a Myoelectric Control Interface (MCI) helped the participants to produce the correct level of force. Kurtosis values were found to decrease monotonically when the contraction level increased, thus indicating that kurtosis can be used to distinguish different forces of contractions. 4- The real practical problem of the degradation of classification performance as a result of the variation of force levels during daily use of the prosthesis has been investigated, and solved by proposing a training approach and the use of a robust feature extraction method, based on the spectrum. The recommendations of this investigation improve the practical robustness of prostheses controlled with PR systems and progress a step further towards clinical implementation and improving the quality of life of amputees. The project showed that PR systems achieved a reliable performance for a large number of amputees, taking into account real life issues such as individual finger control for high dexterity, the effect of force level variation, and optimisation of the movements and EMG channels for each individual amputee. The findings of this thesis showed that the PR systems need to be appropriately tuned before usage, such as training with multiple forces to help to reduce the effect of force variation, aiming to improve practical robustness, and also finding the optimal EMG channel for each amputee, to improve the PR system’s performance. The outcome of this research enables the implementation of PR systems in real prostheses that can be used by amputees.Ministry of Higher Education and Scientific Research and Baghdad University- Baghdad/Ira

Virtual Reality Games for Motor Rehabilitation

This paper presents a fuzzy logic based method to track user satisfaction without the need for devices to monitor users physiological conditions. User satisfaction is the key to any product’s acceptance; computer applications and video games provide a unique opportunity to provide a tailored environment for each user to better suit their needs. We have implemented a non-adaptive fuzzy logic model of emotion, based on the emotional component of the Fuzzy Logic Adaptive Model of Emotion (FLAME) proposed by El-Nasr, to estimate player emotion in UnrealTournament 2004. In this paper we describe the implementation of this system and present the results of one of several play tests. Our research contradicts the current literature that suggests physiological measurements are needed. We show that it is possible to use a software only method to estimate user emotion