Deep Learning Based Upper-limb Motion Estimation Using Surface Electromyography

To advance human-machine interfaces (HMI) that can help disabled people reconstruct lost functions of upper-limbs, machine learning (ML) techniques, particularly classification-based pattern recognition (PR), have been extensively implemented to decode human movement intentions from surface electromyography (sEMG) signals. However, performances of ML can be substantially affected, or even limited, by feature engineering that requires expertise in both domain knowledge and experimental experience. To overcome this limitation, researchers are now focusing on deep learning (DL) techniques to derive informative, representative, and transferable features from raw data automatically. Despite some progress reported in recent literature, it is still very challenging to achieve reliable and robust interpretation of user intentions in practical scenarios. This is mainly because of the high complexity of upper-limb motions and the non-stable characteristics of sEMG signals. Besides, the PR scheme only identifies discrete states of motion. To complete coordinated tasks such as grasping, users have to rely on a sequential on/off control of each individual function, which is inherently different from the simultaneous and proportional control (SPC) strategy adopted by the natural motions of upper-limbs. The aim of this thesis is to develop and advance several DL techniques for the estimation of upper-limb motions from sEMG, and the work is centred on three themes: 1) to improve the reliability of gesture recognition by rejecting uncertain classification outcomes; 2) to build regression frameworks for joint kinematics estimation that enables SPC; and 3) to reduce the degradation of estimation performances when DL model is applied to a new individual. In order to achieve these objectives, the following efforts were made: 1) a confidence model was designed to predict the possibility of correctness with regard to each classification of convolutional neural networks (CNN), such that the uncertain recognition can be identified and rejected; 2) a hybrid framework using CNN for deep feature extraction and long short-term memory neural network (LSTM) was constructed to conduct sequence regression, which could simultaneously exploit the temporal and spatial information in sEMG data; 3) the hybrid framework was further extended by integrating Kalman filter with LSTM units in the recursive learning process, obtaining a deep Kalman filter network (DKFN) to perform kinematics estimation more effectively; and 4) a novel regression scheme was proposed for supervised domain adaptation (SDA), based on which the model generalisation among subjects can be substantially enhanced

Computational Intelligence in Electromyography Analysis

Electromyography (EMG) is a technique for evaluating and recording the electrical activity produced by skeletal muscles. EMG may be used clinically for the diagnosis of neuromuscular problems and for assessing biomechanical and motor control deficits and other functional disorders. Furthermore, it can be used as a control signal for interfacing with orthotic and/or prosthetic devices or other rehabilitation assists. This book presents an updated overview of signal processing applications and recent developments in EMG from a number of diverse aspects and various applications in clinical and experimental research. It will provide readers with a detailed introduction to EMG signal processing techniques and applications, while presenting several new results and explanation of existing algorithms. This book is organized into 18 chapters, covering the current theoretical and practical approaches of EMG research

Automatic Monitoring of Physical Activity Related Affective States for Chronic Pain Rehabilitation

Chronic pain is a prevalent disorder that affects engagement in valued activities. This is a consequence of cognitive and affective barriers, particularly low self-efficacy and emotional distress (i.e. fear/anxiety and depressed mood), to physical functioning. Although clinicians intervene to reduce these barriers, their support is limited to clinical settings and its effects do not easily transfer to everyday functioning which is key to self-management for the person with pain. Analysis carried out in parallel with this thesis points to untapped opportunities for technology to support pain self-management or improved function in everyday activity settings. With this long-term goal for technology in mind, this thesis investigates the possibility of building systems that can automatically detect relevant psychological states from movement behaviour, making three main contributions. First, extension of the annotation of an existing dataset of participants with and without chronic pain performing physical exercises is used to develop a new model of chronic disabling pain where anxiety acts as mediator between pain and self-efficacy, emotional distress, and movement behaviour. Unlike previous models, which are largely theoretical and draw from broad measures of these variables, the proposed model uses event-specific data that better characterise the influence of pain and related states on engagement in physical activities. The model further shows that the relationship between these states and guarding during movement (the behaviour specified in the pain behaviour literature) is complex and behaviour descriptions of a lower level of granularity are needed for automatic classification of the states. The model also suggests that some of the states may be expressed via other movement behaviour types. Second, addressing this using the aforementioned dataset with the additional labels, and through an in-depth analysis of movement, this thesis provides an extended taxonomy of bodily cues for the automatic classification of pain, self-efficacy and emotional distress. In particular, the thesis provides understanding of novel cues of these states and deeper understanding of known cues of pain and emotional distress. Using machine learning algorithms, average F1 scores (mean across movement types) of 0.90, 0.87, and 0.86 were obtained for automatic detection of three levels of pain and self-efficacy and of two levels of emotional distress respectively, based on the bodily cues described and thus supporting the discriminative value of the proposed taxonomy. Third, based on this, the thesis acquired a new dataset of both functional and exercise movements of people with chronic pain based on low-cost wearable sensors designed for this thesis and informed by the previous studies. The modelling results of average F1 score of 0.78 for two-level detection of both pain and self-efficacy point to the possibility of automatic monitoring of these states in everyday functioning. With these contributions, the thesis provides understanding and tools necessary to advance the area of pain-related affective computing and groundbreaking insight that is critical to the understanding of chronic pain. Finally, the contributions lay the groundwork for physical rehabilitation technology to facilitate everyday functioning of people with chronic pain

Machine learning-based dexterous control of hand prostheses

Upper-limb myoelectric prostheses are controlled by muscle activity information recorded on the skin surface using electromyography (EMG). Intuitive prosthetic control can be achieved by deploying statistical and machine learning (ML) tools to decipher the user’s movement intent from EMG signals. This thesis proposes various means of advancing the capabilities of non-invasive, ML-based control of myoelectric hand prostheses. Two main directions are explored, namely classification-based hand grip selection and proportional finger position control using regression methods. Several practical aspects are considered with the aim of maximising the clinical impact of the proposed methodologies, which are evaluated with offline analyses as well as real-time experiments involving both able-bodied and transradial amputee participants. It has been generally accepted that the EMG signal may not always be a reliable source of control information for prostheses, mainly due to its stochastic and non-stationary properties. One particular issue associated with the use of surface EMG signals for upper-extremity myoelectric control is the limb position effect, which is related to the lack of decoding generalisation under novel arm postures. To address this challenge, it is proposed to make concurrent use of EMG sensors and inertial measurement units (IMUs). It is demonstrated this can lead to a significant improvement in both classification accuracy (CA) and real-time prosthetic control performance. Additionally, the relationship between surface EMG and inertial measurements is investigated and it is found that these modalities are partially related due to reflecting different manifestations of the same underlying phenomenon, that is, the muscular activity. In the field of upper-limb myoelectric control, the linear discriminant analysis (LDA) classifier has arguably been the most popular choice for movement intent decoding. This is mainly attributable to its ease of implementation, low computational requirements, and acceptable decoding performance. Nevertheless, this particular method makes a strong fundamental assumption, that is, data observations from different classes share a common covariance structure. Although this assumption may often be violated in practice, it has been found that the performance of the method is comparable to that of more sophisticated algorithms. In this thesis, it is proposed to remove this assumption by making use of general class-conditional Gaussian models and appropriate regularisation to avoid overfitting issues. By performing an exhaustive analysis on benchmark datasets, it is demonstrated that the proposed approach based on regularised discriminant analysis (RDA) can offer an impressive increase in decoding accuracy. By combining the use of RDA classification with a novel confidence-based rejection policy that intends to minimise the rate of unintended hand motions, it is shown that it is feasible to attain robust myoelectric grip control of a prosthetic hand by making use of a single pair of surface EMG-IMU sensors. Most present-day commercial prosthetic hands offer the mechanical abilities to support individual digit control; however, classification-based methods can only produce pre-defined grip patterns, a feature which results in prosthesis under-actuation. Although classification-based grip control can provide a great advantage over conventional strategies, it is far from being intuitive and natural to the user. A potential way of approaching the level of dexterity enjoyed by the human hand is via continuous and individual control of multiple joints. To this end, an exhaustive analysis is performed on the feasibility of reconstructing multidimensional hand joint angles from surface EMG signals. A supervised method based on the eigenvalue formulation of multiple linear regression (MLR) is then proposed to simultaneously reduce the dimensionality of input and output variables and its performance is compared to that of typically used unsupervised methods, which may produce suboptimal results in this context. An experimental paradigm is finally designed to evaluate the efficacy of the proposed finger position control scheme during real-time prosthesis use. This thesis provides insight into the capacity of deploying a range of computational methods for non-invasive myoelectric control. It contributes towards developing intuitive interfaces for dexterous control of multi-articulated prosthetic hands by transradial amputees

Estimation des forces musculaires du membre supérieur humain par optimisation dynamique en utilisant une méthode directe de tir multiple

La modélisation musculo-squelettique permet d’estimer les forces internes du corps humain, à savoir, les forces musculaires et articulaires. Ces estimations sont nécessaires pour comprendre l’anatomie fonctionnelle, les mécanismes de blessures ou encore de concevoir des aides techniques à la motricité. Le défi est d’utiliser l’ensemble des données biomécaniques existantes pour prédire des forces internes qui tiennent compte des stratégies neuro-musculo-squelettiques propres à chacun. L’objectif de cette thèse était d’estimer les forces musculaires du membre supérieur humain par optimisation dynamique, en proposant une méthode innovante de suivi simultané des données électromyographiques (EMG) et cinématiques. À cet égard, nos quatre objectifs spécifiques étaient de : (1) résoudre ce problème d’optimisation dynamique en utilisant une méthode directe de tir multiple ; (2) déterminer sa pertinence et sa performance par rapport aux autres algorithmes existants ; (3) valider son applicabilité à des données expérimentales ; et (4) caractériser des techniques d’identification (numériques et expérimentales) des propriétés musculaires, notamment à l’aide d’un ergomètre isocinétique. Nos différentes études ont permis d’établir que, en un temps de calcul raisonnable (~ 1 heure), notre nouvelle méthode de suivi simultané en optimisation dynamique est à-même de reproduire la cinématique attendue avec une précision de l’ordre de 5°. En outre, l'erreur quadratique moyenne sur les forces musculaires a été réduite d’au moins cinq fois avec notre nouvelle méthode, comparativement aux optimisations statique, hybride et dynamique reposant sur des fonctions-objectif de moindres-activations/excitations (erreur sur les forces musculaires de 18,45 ± 12,60 N avec notre nouvelle méthode contre 85,10 ± 116,40 N avec une optimisation hybride faisant le suivi des moments articulaires). Notre algorithme a également montré son efficacité lors de l’identification des propriétés musculaires d’un modèle musculo-squelettique générique : ce faisant, des excitations musculaires avec deux fois moins d’erreurs vis-à-vis de l’EMG expérimental ont été obtenues, comparativement à l’optimisation statique. Finalement, en termes de calibration du modèle musculo-squelettique, nous avons pu établir que la mesure expérimentale du moment articulaire à l’épaule au moyen de l’ergomètre isocinétique est inadéquate, en particulier lors de mouvements de rotation interne/externe de l’épaule. En effet, les composantes en flexion et abduction du moment à l’épaule mesurées par l’ergomètre isocinétique sont significativement sous-estimées (jusqu'à 94,9% par rapport au moment résultant calculé à partir des efforts tridimensionnels à la main et au coude, mesurés par des capteurs de force six axes). Par conséquent, cette thèse a mis en évidence l’importance du suivi simultané de l’EMG et de la cinématique en optimisation dynamique, afin de rendre fiables les estimations de forces musculaires du membre supérieur – notamment, dans les cas de forte co-contraction musculaire. Elle également a permis d’établir des recommandations qui serviront lors de la calibration du modèle à partir de l’ergomètre isocinétique. Notre méthode innovante pourra être appliquée à des populations pathologiques, afin de comprendre la pathomécanique et mieux intervenir auprès des professionnels de la santé et de leurs patients.Musculoskeletal modeling is used to estimate the internal forces of the human body, namely, muscle and joint forces. These estimates are necessary to understand functional anatomy and pathogenesis or to design technical devices supporting the movement. The challenge is to use all existing biomechanical data to predict internal forces that account for the neuro-musculoskeletal strategies of each individual. The purpose of this thesis was to estimate the human upper-limb muscles forces using forward dynamic optimisation. To do so, we proposed an innovative method tracking both electromyographic (EMG) and kinematic data directly into the optimisation objective-function. In this regard, our four specific objectives were: (1) solving the forward-dynamic optimisation problem using a direct multiple shooting method; (2) determining its relevance and performance compared to other existing algorithms in the literature; (3) validating its applicability to experimental data; and (4) characterizing techniques to identify the model muscle properties using the isokinetic dynamometer. In our different studies, we have demonstrated that, in a reasonable computation time (~ 1 hour), our new dynamic-optimisation method is able to predict the joint kinematics with an accuracy of about 5°. In addition, the muscle forces root-mean-square error was reduced by at least five times with our new method compared to static, hybrid, and dynamic optimisations based on least-activations/excitations objective-functions (muscle forces error of 18.45 ± 12.60 N with our new method vs. 85.10 ± 116.40 N with a traditional hybrid optimisation tracking the joint torques). Our new algorithm also proved to be efficient in identifying the muscle properties of a generic musculoskeletal model: in doing so, the error between the optimised muscle excitations and the experimental EMG was two time lower than the one obtained with static optimisation. Finally, regarding the calibration of the musculoskeletal model, we established that the experimental joint torque measurement at the shoulder using the isokinetic dynamometer was not suitable, especially during internal/external rotation movements of the shoulder. In fact, the flexion and abduction components of the shoulder torque measured by the isokinetic dynamometer are significantly underestimated (up to 94.9% compared to the resulting torque calculated from the three-dimensional forces at the hand and at the elbow, measured by six-axis force sensors). Therefore, this thesis has emphasized the importance of tracking both EMG and kinematics in dynamic optimisation, in order to make reliable estimations of the upper-limb muscle forces – specifically when high co-contraction occurs. Besides, recommendations were issued about calibrating the musculoskeletal model from the experimental torques measured with the isokinetic dynamometer. It will be possible to apply our innovative forward-dynamic optimisation method to pathological populations to increase understanding of the pathomechanics of human movement and better assist health professionals and their patients

sEMG-MMG State-Space Model for the Continuous Estimation of Multijoint Angle

Continuous joint angle estimation plays an important role in motion intention recognition and rehabilitation training. In this study, a surface electromyography- (sEMG-) mechanomyography (MMG) state-space model is proposed to estimate continuous multijoint movements from sEMG and MMG signals accurately. The model combines forward dynamics with a Hill-based muscle model that estimates joint torque only in a nonfeedback form, making the extended model capable of predicting the multijoint motion directly. The sEMG and MMG features, including the Wilson amplitude and permutation entropy, are then extracted to construct a measurement equation to reduce system error and external disturbances. Using the proposed model, a closed-loop prediction-correction approach, unscented particle filtering, is used to estimate the joint angle from sEMG and MMG signals. Comprehensive experiments are conducted on the human elbow and shoulder joint, and remarkable improvements are demonstrated compared with conventional methods