Control of an ambulatory exoskeleton with a brain-machine interface for spinal cord injury gait rehabilitation

The closed-loop control of rehabilitative technologies by neural commands has shown a great potential to improve motor recovery in patients suffering from paralysis. Brain-machine interfaces (BMI) can be used as a natural control method for such technologies. BMI provides a continuous association between the brain activity and peripheral stimulation, with the potential to induce plastic changes in the nervous system. Paraplegic patients, and especially the ones with incomplete injuries, constitute a potential target population to be rehabilitated with brain-controlled robotic systems, as they may improve their gait function after the reinforcement of their spared intact neural pathways. This paper proposes a closed-loop BMI system to control an ambulatory exoskeleton-without any weight or balance support-for gait rehabilitation of incomplete spinal cord injury (SCI) patients. The integrated system was validated with three healthy subjects, and its viability in a clinical scenario was tested with four SCI patients. Using a cue-guided paradigm, the electroencephalographic signals of the subjects were used to decode their gait intention and to trigger the movements of the exoskeleton. We designed a protocol with a special emphasis on safety, as patients with poor balance were required to stand and walk. We continuously monitored their fatigue and exertion level, and conducted usability and user-satisfaction tests after the experiments. The results show that, for the three healthy subjects, 84.44 ± 14.56% of the trials were correctly decoded. Three out of four patients performed at least one successful BMI session, with an average performance of 77.6 1 ± 14.72%. The shared control strategy implemented (i.e., the exoskeleton could only move during specific periods of time) was effective in preventing unexpected movements during periods in which patients were asked to relax. On average, 55.22 ± 16.69% and 40.45 ± 16.98% of the trials (for healthy subjects and patients, respectively) would have suffered from unexpected activations (i.e., false positives) without the proposed control strategy. All the patients showed low exertion and fatigue levels during the performance of the experiments. This paper constitutes a proof-of-concept study to validate the feasibility of a BMI to control an ambulatory exoskeleton by patients with incomplete paraplegia (i.e., patients with good prognosis for gait rehabilitation)

Development of an EEG-based recurrent neural network for online gait decoding

Recent neuroscientific literature has shown that the use of brain-controlled robotic exoskeletons in walking rehabilitation induces neuroplasticity modi- fications, possibly leading to a higher likelihood of recovery and maintenance of lost motor functions due to a neural lesion, with respect to traditional re- habilitation. However, the gait decoding from brain signals remains an open challenge. The aim of this work is to implement and validate a deep learning model for online gait decoding that exploits Electroencephalography (EEG) infor- mation to predict the intention of initiating a step, which could be used to trigger the assistance of a lower-limb exoskeleton. In particular, the model exploits a Gated Recurrent Units (GRU) deep neural network to handle the time-dependent features which were identified by analysing the neural cor- relates preceding the step onset (i.e., Movement-Related Cortical Potentials (MRCP)). The network was evaluated on a pre-recorded dataset of 11 healthy subjects walking on a treadmill. The network’s architecture (e.g., number of GRU units) was optimized through grid search. In addition, to deal with the data scarcity problem of neurophysiological applications, I proposed a data augmentation procedure to increase the dataset available to train the model of each subject. With the proposed approach, the model achieved an average accuracy in detecting the step onset of 89.7 ± 7.7% with just the 15% of the dataset for each subject (∼70 steps), and up to 97.8 ± 1.3% with the whole dataset (∼440 steps). This thesis support the use of a memory-based deep learning model to de- code walking activity from non-invasive brain recordings. In future works, this model will be exploited in real time as a more effective input for devices restoring locomotion in impaired people, such as robotic exoskeletons.Recent neuroscientific literature has shown that the use of brain-controlled robotic exoskeletons in walking rehabilitation induces neuroplasticity modi- fications, possibly leading to a higher likelihood of recovery and maintenance of lost motor functions due to a neural lesion, with respect to traditional re- habilitation. However, the gait decoding from brain signals remains an open challenge. The aim of this work is to implement and validate a deep learning model for online gait decoding that exploits Electroencephalography (EEG) infor- mation to predict the intention of initiating a step, which could be used to trigger the assistance of a lower-limb exoskeleton. In particular, the model exploits a Gated Recurrent Units (GRU) deep neural network to handle the time-dependent features which were identified by analysing the neural cor- relates preceding the step onset (i.e., Movement-Related Cortical Potentials (MRCP)). The network was evaluated on a pre-recorded dataset of 11 healthy subjects walking on a treadmill. The network’s architecture (e.g., number of GRU units) was optimized through grid search. In addition, to deal with the data scarcity problem of neurophysiological applications, I proposed a data augmentation procedure to increase the dataset available to train the model of each subject. With the proposed approach, the model achieved an average accuracy in detecting the step onset of 89.7 ± 7.7% with just the 15% of the dataset for each subject (∼70 steps), and up to 97.8 ± 1.3% with the whole dataset (∼440 steps). This thesis support the use of a memory-based deep learning model to de- code walking activity from non-invasive brain recordings. In future works, this model will be exploited in real time as a more effective input for devices restoring locomotion in impaired people, such as robotic exoskeletons

Enhancement of Robot-Assisted Rehabilitation Outcomes of Post-Stroke Patients Using Movement-Related Cortical Potential

Post-stroke rehabilitation is essential for stroke survivors to help them regain independence and to improve their quality of life. Among various rehabilitation strategies, robot-assisted rehabilitation is an efficient method that is utilized more and more in clinical practice for motor recovery of post-stroke patients. However, excessive assistance from robotic devices during rehabilitation sessions can make patients perform motor training passively with minimal outcome. Towards the development of an efficient rehabilitation strategy, it is necessary to ensure the active participation of subjects during training sessions. This thesis uses the Electroencephalography (EEG) signal to extract the Movement-Related Cortical Potential (MRCP) pattern to be used as an indicator of the active engagement of stroke patients during rehabilitation training sessions. The MRCP pattern is also utilized in designing an adaptive rehabilitation training strategy that maximizes patients’ engagement. This project focuses on the hand motor recovery of post-stroke patients using the AMADEO rehabilitation device (Tyromotion GmbH, Austria). AMADEO is specifically developed for patients with fingers and hand motor deficits. The variations in brain activity are analyzed by extracting the MRCP pattern from the acquired EEG data during training sessions. Whereas, physical improvement in hand motor abilities is determined by two methods. One is clinical tests namely Fugl-Meyer Assessment (FMA) and Motor Assessment Scale (MAS) which include FMA-wrist, FMA-hand, MAS-hand movements, and MAS-advanced hand movements’ tests. The other method is the measurement of hand-kinematic parameters using the AMADEO assessment tool which contains hand strength measurements during flexion (force-flexion), and extension (force-extension), and Hand Range of Movement (HROM)

Volitional Control of Lower-limb Prosthesis with Vision-assisted Environmental Awareness

Early and reliable prediction of user’s intention to change locomotion mode or speed is critical for a smooth and natural lower limb prosthesis. Meanwhile, incorporation of explicit environmental feedback can facilitate context aware intelligent prosthesis which allows seamless operation in a variety of gait demands. This dissertation introduces environmental awareness through computer vision and enables early and accurate prediction of intention to start, stop or change speeds while walking. Electromyography (EMG), Electroencephalography (EEG), Inertial Measurement Unit (IMU), and Ground Reaction Force (GRF) sensors were used to predict intention to start, stop or increase walking speed. Furthermore, it was investigated whether external emotional music stimuli could enhance the predictive capability of intention prediction methodologies. Application of advanced machine learning and signal processing techniques on pre-movement EEG resulted in an intention prediction system with low latency, high sensitivity and low false positive detection. Affective analysis of EEG suggested that happy music stimuli significantly (

Fusion of virtual reality and brain-machine interfaces for the assessment and rehabilitation of patients with spinal cord injury

La presente tesis está centrada en la utilización de nuevas tecnologías (Interfaces Cerebro-Máquina y Realidad Virtual). En la primera parte de la tesis se describe la definición y la aplicación de un conjunto de métricas para evaluar el estado funcional de los pacientes con lesión medular en el contexto de un sistema de realidad virtual para la rehabilitación de los miembros superiores. El objetivo de este primer estudio es demostrar que la realidad virtual puede utilizarse, en combinación con sensores inerciales para rehabilitar y evaluar simultáneamente. 15 pacientes con lesión medular llevaron a cabo 3 sesiones con el sistema de realidad virtual Toyra y se aplicó el conjunto definido de métricas a las grabaciones obtenidas con los sensores inerciales. Se encontraron correlaciones entre algunas de las métricas definidas y algunas de las escalas clínicas utilizadas con frecuencia en el contexto de la rehabilitación. En la segunda parte de la tesis se ha combinado una retroalimentación virtual con un estimulador eléctrico funcional (en adelante FES, por sus siglas en inglés Functional Electrical Stimulator), ambos controlados por un Interfaz Cerebro-Máquina (BMI por sus siglas en inglés Brain-Machine Interface), para desarrollar un nuevo tipo de enfoque terapéutico para los pacientes. El sistema ha sido utilizado por 4 pacientes con lesión medular que intentaron mover sus manos. Esta intención desencadenó simultáneamente el FES y la retroalimentación virtual, cerrando la mano de los pacientes y mostrándoles una fuente adicional de retroalimentación para complementar la terapia. Este trabajo es, de acuerdo al estado del arte revisado, el primero que integra BMI, FES y realidad virtual como terapia para pacientes con lesión medular. Se han obtenido resultados clínicos prometedores por 4 pacientes con lesión medular después de realizar 5 sesiones de terapia con el sistema, mostrando buenos niveles de precisión en las diferentes sesiones (79,13% en promedio). En la tercera parte de la tesis se ha definido una nueva métrica para estudiar los cambios de conectividad cerebral en los pacientes con lesión medular, que incluye información de las interacciones neuronales entre diferentes áreas. El objetivo de este estudio ha sido extraer información clínicamente relevante de la actividad del EEG cuando se realizan terapias basadas en BMI

Electroencephalographic recording of the movement-related cortical potential in ecologically-valid movements:A scoping review

The movement-related cortical potential (MRCP) is a brain signal that can be recorded using surface electroencephalography (EEG) and represents the cortical processes involved in movement preparation. The MRCP has been widely researched in simple, single-joint movements, however, these movements often lack ecological validity. Ecological validity refers to the generalizability of the findings to real-world situations, such as neurological rehabilitation. This scoping review aimed to synthesize the research evidence investigating the MRCP in ecologically valid movement tasks. A search of six electronic databases identified 102 studies that investigated the MRCP during multi-joint movements; 59 of these studies investigated ecologically valid movement tasks and were included in the review. The included studies investigated 15 different movement tasks that were applicable to everyday situations, but these were largely carried out in healthy populations. The synthesized findings suggest that the recording and analysis of MRCP signals is possible in ecologically valid movements, however the characteristics of the signal appear to vary across different movement tasks (i.e., those with greater complexity, increased cognitive load, or a secondary motor task) and different populations (i.e., expert performers, people with Parkinson’s Disease, and older adults). The scarcity of research in clinical populations highlights the need for further research in people with neurological and age-related conditions to progress our understanding of the MRCPs characteristics and to determine its potential as a measure of neurological recovery and intervention efficacy. MRCP-based neuromodulatory interventions applied during ecologically valid movements were only represented in one study in this review as these have been largely delivered during simple joint movements. No studies were identified that used ecologically valid movements to control BCI-driven external devices; this may reflect the technical challenges associated with accurately classifying functional movements from MRCPs. Future research investigating MRCP-based interventions should use movement tasks that are functionally relevant to everyday situations. This will facilitate the application of this knowledge into the rehabilitation setting

Systematic Review of Intelligent Algorithms in Gait Analysis and Prediction for Lower Limb Robotic Systems

The rate of development of robotic technologies has been meteoric, as a result of compounded advancements in hardware and software. Amongst these robotic technologies are active exoskeletons and orthoses, used in the assistive and rehabilitative fields. Artificial intelligence techniques are increasingly being utilised in gait analysis and prediction. This review paper systematically explores the current use of intelligent algorithms in gait analysis for robotic control, specifically the control of active lower limb exoskeletons and orthoses. Two databases, IEEE and Scopus, were screened for papers published between 1989 to May 2020. 41 papers met the eligibility criteria and were included in this review. 66.7% of the identified studies used classification models for the classification of gait phases and locomotion modes. Meanwhile, 33.3% implemented regression models for the estimation/prediction of kinematic parameters such as joint angles and trajectories, and kinetic parameters such as moments and torques. Deep learning algorithms have been deployed in ∼15% of the machine learning implementations. Other methodological parameters were reviewed, such as the sensor selection and the sample sizes used for training the models

Homology Characteristics of EEG and EMG for Lower Limb Voluntary Movement Intention

In the field of lower limb exoskeletons, besides its electromechanical system design and control, attention has been paid to realizing the linkage of exoskeleton robots to humans via electroencephalography (EEG) and electromyography (EMG). However, even the state of the art performance of lower limb voluntary movement intention decoding still faces many obstacles. In the following work, focusing on the perspective of the inner mechanism, a homology characteristic of EEG and EMG for lower limb voluntary movement intention was conducted. A mathematical model of EEG and EMG was built based on its mechanism, which consists of a neural mass model (NMM), neuromuscular junction model, EMG generation model, decoding model, and musculoskeletal biomechanical model. The mechanism analysis and simulation results demonstrated that EEG and EMG signals were both excited by the same movement intention with a response time difference. To assess the efficiency of the proposed model, a synchronous acquisition system for EEG and EMG was constructed to analyze the homology and response time difference from EEG and EMG signals in the limb movement intention. An effective method of wavelet coherence was used to analyze the internal correlation between EEG and EMG signals in the same limb movement intention. To further prove the effectiveness of the hypothesis in this paper, six subjects were involved in the experiments. The experimental results demonstrated that there was a strong EEG-EMG coherence at 1 Hz around movement onset, and the phase of EEG was leading the EMG. Both the simulation and experimental results revealed that EEG and EMG are homologous, and the response time of the EEG signals are earlier than EMG signals during the limb movement intention. This work can provide a theoretical basis for the feasibility of EEG-based pre-perception and fusion perception of EEG and EMG in human movement detection

Analysis of Human Gait Using Hybrid EEG-fNIRS-Based BCI System: A Review

Human gait is a complex activity that requires high coordination between the central nervous system, the limb, and the musculoskeletal system. More research is needed to understand the latter coordination\u27s complexity in designing better and more effective rehabilitation strategies for gait disorders. Electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS) are among the most used technologies for monitoring brain activities due to portability, non-invasiveness, and relatively low cost compared to others. Fusing EEG and fNIRS is a well-known and established methodology proven to enhance brain–computer interface (BCI) performance in terms of classification accuracy, number of control commands, and response time. Although there has been significant research exploring hybrid BCI (hBCI) involving both EEG and fNIRS for different types of tasks and human activities, human gait remains still underinvestigated. In this article, we aim to shed light on the recent development in the analysis of human gait using a hybrid EEG-fNIRS-based BCI system. The current review has followed guidelines of preferred reporting items for systematic reviews and meta-Analyses (PRISMA) during the data collection and selection phase. In this review, we put a particular focus on the commonly used signal processing and machine learning algorithms, as well as survey the potential applications of gait analysis. We distill some of the critical findings of this survey as follows. First, hardware specifications and experimental paradigms should be carefully considered because of their direct impact on the quality of gait assessment. Second, since both modalities, EEG and fNIRS, are sensitive to motion artifacts, instrumental, and physiological noises, there is a quest for more robust and sophisticated signal processing algorithms. Third, hybrid temporal and spatial features, obtained by virtue of fusing EEG and fNIRS and associated with cortical activation, can help better identify the correlation between brain activation and gait. In conclusion, hBCI (EEG + fNIRS) system is not yet much explored for the lower limb due to its complexity compared to the higher limb. Existing BCI systems for gait monitoring tend to only focus on one modality. We foresee a vast potential in adopting hBCI in gait analysis. Imminent technical breakthroughs are expected using hybrid EEG-fNIRS-based BCI for gait to control assistive devices and Monitor neuro-plasticity in neuro-rehabilitation. However, although those hybrid systems perform well in a controlled experimental environment when it comes to adopting them as a certified medical device in real-life clinical applications, there is still a long way to go

Deep learning for gait prediction: an application to exoskeletons for children with neurological disorders

Cerebral Palsy, a non-progressive neurological disorder, is a lifelong condition. While it has no cure, clinical intervention aims to minimise the impact of the disability on individuals' lives. Wearable robotic devices, like exoskeletons, have been rapidly advancing and proving to be effective in rehabilitating individuals with gait pathologies. The utilization of artificial intelligence (AI) algorithms in controlling exoskeletons, particularly at the supervisory level, has emerged as a valuable approach. These algorithms rely on input from onboard sensors to predict gait phase, user intention, or joint kinematics. Using AI to improve the control of robotic devices not only enhances human-robot interaction but also has the potential to improve user comfort and functional outcomes of rehabilitation, and reduce accidents and injuries. In this research study, a comprehensive systematic literature review is conducted, exploring the various applications of AI in lower-limb robotic control. This review focuses on methodological parameters such as sensor usage, training demographics, sample size, and types of models while identifying gaps in the existing literature. Building on the findings of the review, subsequent research leveraged the power of deep learning to predict gait trajectories for the application of rehabilitative exoskeleton control. This study addresses a gap in the existing literature by focusing on predicting pathological gait trajectories, which exhibit higher inter- and intra-subject variability compared to the gait of healthy individuals. The research focused on the gait of children with neurological disorders, particularly Cerebral Palsy, as they stand to benefit greatly from rehabilitative exoskeletons. State-of-the-art deep learning algorithms, including transformers, fully connected neural networks, convolutional neural networks, and long short-term memory networks, were implemented for gait trajectory prediction. This research presents findings on the performance of these models for short-term and long-term recursive predictions, the impact of varying input and output window sizes on prediction errors, the effect of adding variable levels of Gaussian noise, and the robustness of the models in predicting gait at speeds within and outside the speed range of the training set. Moreover, the research outlines a methodology for optimising the stability of long-term forecasts and provides a comparative analysis of gait trajectory forecasting for typically developing children and children with Cerebral Palsy. A novel approach to generating adaptive trajectories for children with Cerebral Palsy, which can serve as reference trajectories for position-controlled exoskeletons, is also presented