Volitional Control of Lower-limb Prosthesis with Vision-assisted Environmental Awareness

Early and reliable prediction of user’s intention to change locomotion mode or speed is critical for a smooth and natural lower limb prosthesis. Meanwhile, incorporation of explicit environmental feedback can facilitate context aware intelligent prosthesis which allows seamless operation in a variety of gait demands. This dissertation introduces environmental awareness through computer vision and enables early and accurate prediction of intention to start, stop or change speeds while walking. Electromyography (EMG), Electroencephalography (EEG), Inertial Measurement Unit (IMU), and Ground Reaction Force (GRF) sensors were used to predict intention to start, stop or increase walking speed. Furthermore, it was investigated whether external emotional music stimuli could enhance the predictive capability of intention prediction methodologies. Application of advanced machine learning and signal processing techniques on pre-movement EEG resulted in an intention prediction system with low latency, high sensitivity and low false positive detection. Affective analysis of EEG suggested that happy music stimuli significantly (

Electroencephalographic recording of the movement-related cortical potential in ecologically-valid movements:A scoping review

The movement-related cortical potential (MRCP) is a brain signal that can be recorded using surface electroencephalography (EEG) and represents the cortical processes involved in movement preparation. The MRCP has been widely researched in simple, single-joint movements, however, these movements often lack ecological validity. Ecological validity refers to the generalizability of the findings to real-world situations, such as neurological rehabilitation. This scoping review aimed to synthesize the research evidence investigating the MRCP in ecologically valid movement tasks. A search of six electronic databases identified 102 studies that investigated the MRCP during multi-joint movements; 59 of these studies investigated ecologically valid movement tasks and were included in the review. The included studies investigated 15 different movement tasks that were applicable to everyday situations, but these were largely carried out in healthy populations. The synthesized findings suggest that the recording and analysis of MRCP signals is possible in ecologically valid movements, however the characteristics of the signal appear to vary across different movement tasks (i.e., those with greater complexity, increased cognitive load, or a secondary motor task) and different populations (i.e., expert performers, people with Parkinson’s Disease, and older adults). The scarcity of research in clinical populations highlights the need for further research in people with neurological and age-related conditions to progress our understanding of the MRCPs characteristics and to determine its potential as a measure of neurological recovery and intervention efficacy. MRCP-based neuromodulatory interventions applied during ecologically valid movements were only represented in one study in this review as these have been largely delivered during simple joint movements. No studies were identified that used ecologically valid movements to control BCI-driven external devices; this may reflect the technical challenges associated with accurately classifying functional movements from MRCPs. Future research investigating MRCP-based interventions should use movement tasks that are functionally relevant to everyday situations. This will facilitate the application of this knowledge into the rehabilitation setting

Ongoing temporal dynamics of broadband EEG during movement intention for BCI

Brain Computer Interface (BCI) empowers individuals with severe movement impairing conditions to interact with the computers directly by their thoughts, without the involvement of any motor pathways. Motor-based BCIs can offer intuitive control by merely intending to move. Hence, to develop effective motor-based non-invasive BCIs, it is essential to understand the mechanisms of neural processes involved in motor command generation in electroencephalography (EEG). The EEG consists of complex narrowband oscillatory and broadband arrhythmic processes. However, there is more focus on the oscillations in different frequency bands for studying motor command generation in the literature. The narrowband processes such as event-related (de)synchronisation (ERD/S) and movement-related cortical potential (MRCP) are commonly used for movement detection. Analysis of these narrowband EEG components disregards the information existing in the rest of the frequencies and their dynamics. Hence, this thesis investigates various facets of previously unexplored temporal dynamics of neuronal processes in the broadband arrhythmic EEG to fill the gap in the knowledge of motor command generation on a single trial basis in the BCI framework. The temporal dynamics of the broadband EEG were characterised by the decay of its autocorrelation. The autocorrelation decayed according to the power-law resulting in the longrange temporal correlations (LRTC). The instantaneous ongoing changes in the broadband LRTC were uniquely quantified by the Hurst exponent on very short EEG sliding windows. There was an increase in the temporal dependencies in the EEG leading to slower decay of autocorrelation during the movement and significant increase in the LRTC (p<0.05). Different types of temporal dependencies in the broadband EEG were comprehensively examined further by modelling the long and short-range correlations together using autoregressive fractionally integrated moving average model (ARFIMA). The short-range correlations also changed significantly (p<0.05) during the movement. These ongoing changes in the dynamics of the broadband EEG were able to predict the movement 1 s before its onset with accuracy higher than ERD and MRCP. The LRTCs were robust across participants and did not require determination of participant specific parameters such as most responsive spectral or spatial components

Development of an EEG-based recurrent neural network for online gait decoding

Recent neuroscientific literature has shown that the use of brain-controlled robotic exoskeletons in walking rehabilitation induces neuroplasticity modi- fications, possibly leading to a higher likelihood of recovery and maintenance of lost motor functions due to a neural lesion, with respect to traditional re- habilitation. However, the gait decoding from brain signals remains an open challenge. The aim of this work is to implement and validate a deep learning model for online gait decoding that exploits Electroencephalography (EEG) infor- mation to predict the intention of initiating a step, which could be used to trigger the assistance of a lower-limb exoskeleton. In particular, the model exploits a Gated Recurrent Units (GRU) deep neural network to handle the time-dependent features which were identified by analysing the neural cor- relates preceding the step onset (i.e., Movement-Related Cortical Potentials (MRCP)). The network was evaluated on a pre-recorded dataset of 11 healthy subjects walking on a treadmill. The network’s architecture (e.g., number of GRU units) was optimized through grid search. In addition, to deal with the data scarcity problem of neurophysiological applications, I proposed a data augmentation procedure to increase the dataset available to train the model of each subject. With the proposed approach, the model achieved an average accuracy in detecting the step onset of 89.7 ± 7.7% with just the 15% of the dataset for each subject (∼70 steps), and up to 97.8 ± 1.3% with the whole dataset (∼440 steps). This thesis support the use of a memory-based deep learning model to de- code walking activity from non-invasive brain recordings. In future works, this model will be exploited in real time as a more effective input for devices restoring locomotion in impaired people, such as robotic exoskeletons.Recent neuroscientific literature has shown that the use of brain-controlled robotic exoskeletons in walking rehabilitation induces neuroplasticity modi- fications, possibly leading to a higher likelihood of recovery and maintenance of lost motor functions due to a neural lesion, with respect to traditional re- habilitation. However, the gait decoding from brain signals remains an open challenge. The aim of this work is to implement and validate a deep learning model for online gait decoding that exploits Electroencephalography (EEG) infor- mation to predict the intention of initiating a step, which could be used to trigger the assistance of a lower-limb exoskeleton. In particular, the model exploits a Gated Recurrent Units (GRU) deep neural network to handle the time-dependent features which were identified by analysing the neural cor- relates preceding the step onset (i.e., Movement-Related Cortical Potentials (MRCP)). The network was evaluated on a pre-recorded dataset of 11 healthy subjects walking on a treadmill. The network’s architecture (e.g., number of GRU units) was optimized through grid search. In addition, to deal with the data scarcity problem of neurophysiological applications, I proposed a data augmentation procedure to increase the dataset available to train the model of each subject. With the proposed approach, the model achieved an average accuracy in detecting the step onset of 89.7 ± 7.7% with just the 15% of the dataset for each subject (∼70 steps), and up to 97.8 ± 1.3% with the whole dataset (∼440 steps). This thesis support the use of a memory-based deep learning model to de- code walking activity from non-invasive brain recordings. In future works, this model will be exploited in real time as a more effective input for devices restoring locomotion in impaired people, such as robotic exoskeletons

Enhancement of Robot-Assisted Rehabilitation Outcomes of Post-Stroke Patients Using Movement-Related Cortical Potential

Post-stroke rehabilitation is essential for stroke survivors to help them regain independence and to improve their quality of life. Among various rehabilitation strategies, robot-assisted rehabilitation is an efficient method that is utilized more and more in clinical practice for motor recovery of post-stroke patients. However, excessive assistance from robotic devices during rehabilitation sessions can make patients perform motor training passively with minimal outcome. Towards the development of an efficient rehabilitation strategy, it is necessary to ensure the active participation of subjects during training sessions. This thesis uses the Electroencephalography (EEG) signal to extract the Movement-Related Cortical Potential (MRCP) pattern to be used as an indicator of the active engagement of stroke patients during rehabilitation training sessions. The MRCP pattern is also utilized in designing an adaptive rehabilitation training strategy that maximizes patients’ engagement. This project focuses on the hand motor recovery of post-stroke patients using the AMADEO rehabilitation device (Tyromotion GmbH, Austria). AMADEO is specifically developed for patients with fingers and hand motor deficits. The variations in brain activity are analyzed by extracting the MRCP pattern from the acquired EEG data during training sessions. Whereas, physical improvement in hand motor abilities is determined by two methods. One is clinical tests namely Fugl-Meyer Assessment (FMA) and Motor Assessment Scale (MAS) which include FMA-wrist, FMA-hand, MAS-hand movements, and MAS-advanced hand movements’ tests. The other method is the measurement of hand-kinematic parameters using the AMADEO assessment tool which contains hand strength measurements during flexion (force-flexion), and extension (force-extension), and Hand Range of Movement (HROM)

Brain-Computer Interface Based on Unsupervised Methods to Recognize Motor Intention for Command of a Robotic Exoskeleton

Stroke and road traffic injuries may severely affect movements of lower-limbs in humans, and consequently the locomotion, which plays an important role in daily activities, and the quality of life. Robotic exoskeleton is an emerging alternative, which may be used on patients with motor deficits in the lower extremities to provide motor rehabilitation and gait assistance. However, the effectiveness of robotic exoskeletons may be reduced by the autonomous ability of the robot to complete the movement without the patient involvement. Then, electroencephalography signals (EEG) have been addressed to design brain-computer interfaces (BCIs), in order to provide a communication pathway for patients perform a direct control on the exoskeleton using the motor intention, and thus increase their participation during the rehabilitation. Specially, activations related to motor planning may help to improve the close loop between user and exoskeleton, enhancing the cortical neuroplasticity. Motor planning begins before movement onset, thus, the training stage of BCIs may be affected by the intuitive labeling process, as it is not possible to use reference signals, such as goniometer or footswitch, to select those time periods really related to motor planning. Therefore, the gait planning recognition is a challenge, due to the high uncertainty of selected patterns, However, few BCIs based on unsupervised methods to recognize gait planning/stopping have been explored. This Doctoral Thesis presents unsupervised methods to improve the performance of BCIs during gait planning/ stopping recognition. At this context, an adaptive spatial filter for on-line processing based on the Concordance Correlation Coefficient (CCC) was addressed to preserve the useful information on EEG signals, while rejecting neighbor electrodes around the electrode of interest. Here, two methods for electrode selection were proposed. First, both standard deviation and CCC between target electrodes and their correspondent neighbor electrodes are analyzed on sliding windows to select those neighbors that are highly correlated. Second, Zscore analysis is performed to reject those neighbor electrodes whose amplitude values presented significant difference in relation to other neighbors. Furthermore, another method that uses the representation entropy and the maximal information compression index (MICI) was proposed for feature selection, which may be robust to select patterns, as only it depends on cluster distribution. In addition, a statistical analysis was introduced here to adjust, in the training stage of BCIs, regularized classifiers, such as support vector machine (SVM) and regularized discriminant analysis (RDA). Six subjects were adopted to evaluate the performance of different BCIs based on the proposed viii methods, during gait planning/stopping recognition. The unsupervised approach for feature selection showed similar performance to other methods based on linear discriminant analysis (LDA), when it was applied in a BCI based on the traditional Weighted Average to recognize gait planning. Additionally, the proposed adaptive filter improved the performance of BCIs based on traditional spatial filters, such as Local Average Reference (LAR) and WAR, as well as others BCIs based on powerful methods, such as Common Spatial Pattern (CSP), Filter Bank Common Spatial Pattern (FBCSP) and Riemannian kernel (RK). RK presented the best performance in comparison to CSP and FBCSP, which agrees with the hypothesis that unsupervised methods may be more appropriate to analyze clusters of high uncertainty, as those formed by motor planning. BCIs using adaptive filter based on Zscore analysis, with an unsupervised approach for feature selection and RDA showed promising results to recognize both gait planning and gait stopping, achieving for three subjects, good values of true positive rate (>70%) and false positive (<16%). Thus, the proposed methods may be used to obtain an optimized BCI that preserves the useful information, enhancing the gait planning/stopping recognition. In addition, the method for feature selection has low computational cost, which may be suitable for applications that demand short time of training, such as clinical application time

Signal Processing Using Non-invasive Physiological Sensors

Non-invasive biomedical sensors for monitoring physiological parameters from the human body for potential future therapies and healthcare solutions. Today, a critical factor in providing a cost-effective healthcare system is improving patients' quality of life and mobility, which can be achieved by developing non-invasive sensor systems, which can then be deployed in point of care, used at home or integrated into wearable devices for long-term data collection. Another factor that plays an integral part in a cost-effective healthcare system is the signal processing of the data recorded with non-invasive biomedical sensors. In this book, we aimed to attract researchers who are interested in the application of signal processing methods to different biomedical signals, such as an electroencephalogram (EEG), electromyogram (EMG), functional near-infrared spectroscopy (fNIRS), electrocardiogram (ECG), galvanic skin response, pulse oximetry, photoplethysmogram (PPG), etc. We encouraged new signal processing methods or the use of existing signal processing methods for its novel application in physiological signals to help healthcare providers make better decisions

Електроенцефалографски сигнали за управљање рачунарским интерфејсом у неурорехабилитацији

Мозак-рачунар интерфејс (МоРИ) системи могу искористити карактеристичне промене мождане активности корисника као контролне сигнале уређаја (рачунара). Различити ментални задаци или спољашњи стимулуси (визуелни, аудитивни или соматосензорни) индукују промене које су кодиране у спонтаној неуралној активности. Генерисане промене се могу идентификовати мерењем можданих сигнала који представљају директну или индиректну меру електричне активности мозга...Brain Computer Interface (BCI) systems can use characteristic brain neural alterations as control signals of the device/computer. Various mental tasks or external stimulation (visual, auditory or somatosensory) induce changes which are embedded in the spontaneous neural activity. Generated changes can be extracted and identified from the brain-signal recordings that represent the (direct or indirect) measure of electrical neural activity..

Analysis of Human Gait Using Hybrid EEG-fNIRS-Based BCI System: A Review

Human gait is a complex activity that requires high coordination between the central nervous system, the limb, and the musculoskeletal system. More research is needed to understand the latter coordination\u27s complexity in designing better and more effective rehabilitation strategies for gait disorders. Electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS) are among the most used technologies for monitoring brain activities due to portability, non-invasiveness, and relatively low cost compared to others. Fusing EEG and fNIRS is a well-known and established methodology proven to enhance brain–computer interface (BCI) performance in terms of classification accuracy, number of control commands, and response time. Although there has been significant research exploring hybrid BCI (hBCI) involving both EEG and fNIRS for different types of tasks and human activities, human gait remains still underinvestigated. In this article, we aim to shed light on the recent development in the analysis of human gait using a hybrid EEG-fNIRS-based BCI system. The current review has followed guidelines of preferred reporting items for systematic reviews and meta-Analyses (PRISMA) during the data collection and selection phase. In this review, we put a particular focus on the commonly used signal processing and machine learning algorithms, as well as survey the potential applications of gait analysis. We distill some of the critical findings of this survey as follows. First, hardware specifications and experimental paradigms should be carefully considered because of their direct impact on the quality of gait assessment. Second, since both modalities, EEG and fNIRS, are sensitive to motion artifacts, instrumental, and physiological noises, there is a quest for more robust and sophisticated signal processing algorithms. Third, hybrid temporal and spatial features, obtained by virtue of fusing EEG and fNIRS and associated with cortical activation, can help better identify the correlation between brain activation and gait. In conclusion, hBCI (EEG + fNIRS) system is not yet much explored for the lower limb due to its complexity compared to the higher limb. Existing BCI systems for gait monitoring tend to only focus on one modality. We foresee a vast potential in adopting hBCI in gait analysis. Imminent technical breakthroughs are expected using hybrid EEG-fNIRS-based BCI for gait to control assistive devices and Monitor neuro-plasticity in neuro-rehabilitation. However, although those hybrid systems perform well in a controlled experimental environment when it comes to adopting them as a certified medical device in real-life clinical applications, there is still a long way to go

Topological Changes in the Functional Brain Networks Induced by Isometric Force Exertions Using a Graph Theoretical Approach: An EEG-based Neuroergonomics Study

Neuroergonomics, the application of neuroscience to human factors and ergonomics, is an emerging science focusing on the human brain concerning performance at work and in everyday settings. The advent of portable neurophysiological methods, including electroencephalography (EEG), has enabled measurements of real-time brain activity during physical tasks without restricting body movements. However, the EEG signatures of different physical exertion activity levels that involve the musculoskeletal system in everyday settings remain poorly understood. Furthermore, the assessment of functional connectivity among different brain regions during different force exertion levels remains unclear. One approach to investigating the brain connectome is to model the underlying mechanism of the brain as a complex network. This study applied employed a graph-theoretical approach to characterize the topological properties of the functional brain network induced by predefined force exertion levels, namely extremely light (EL), light (L), somewhat hard (SWH), hard (H), and extremely hard (EH) in two frequency bands, i.e., alpha and beta. Twelve female participants performed an isometric force exertion task and rated their perception of physical comfort at different physical exertion levels. A CGX-Mobile-64 EEG was used for recording spontaneous brain electrical activity. After preprocessing the EEG data, a source localization method was applied to study the functional brain connectivity at the source level. Subsequently, the alpha and beta networks were constructed by calculating the coherence between all pairs of 84 brain regions of interests that were selected using Brodmann Areas. Graph -theoretical measures were then employed to quantify the topological properties of the functional brain networks at different levels of force exertions at each frequency band. During an \u27extremely hard\u27 exertion level, a small-world network was observed for the alpha coherence network, whereas an ordered network was observed for the beta coherence network. The results suggest that high-level force exertions are associated with brain networks characterized by a more significant clustering coefficient, more global and local efficiency, and shorter characteristic path length under alpha coherence. The above suggests that brain regions are communicating and cooperating to a more considerable degree when the muscle force exertions increase to meet physically challenging tasks. The exploration of the present study extends the current understanding of the neurophysiological basis of physical efforts with different force levels of human physical exertion to reduce work-related musculoskeletal disorders