Continuous Kalman Estimation Method for Finger Kinematics Tracking from Surface Electromyography

Deciphering hand motion intention from surface electromyography (sEMG) encounters challenges posed by the requisites of multiple degrees of freedom (DOF) and adaptability. Unlike discrete action classification grounded in pattern recognition, the pursuit of continuous kinematics estimation is appreciated for its inherent naturalness and intuitiveness. However, prevailing estimation techniques contend with accuracy limitations and substantial computational demands. Kalman estimation technology, celebrated for its ease of implementation and real-time adaptability, finds extensive application across diverse domains. This study introduces a continuous Kalman estimation method, leveraging a system model with sEMG and joint angles as inputs and outputs. Facilitated by model parameter training methods, the approach deduces multiple DOF finger kinematics simultaneously. The method's efficacy is validated using a publicly accessible database, yielding a correlation coefficient (CC) of 0.73. With over 45,000 windows for training Kalman model parameters, the average computation time remains under 0.01 seconds. This pilot study amplifies its potential for further exploration and application within the realm of continuous finger motion estimation technology

Analysis of ANN and Fuzzy Logic Dynamic Modelling to Control the Wrist Exoskeleton

Human intention has long been a primary emphasis in the field of electromyography (EMG) research. This being considered, the movement of the exoskeleton hand can be accurately predicted based on the user's preferences. The EMG is a nonlinear signal formed by muscle contractions as the human hand moves and easily captured noise signal from its surroundings. Due to this fact, this study aims to estimate wrist desired velocity based on EMG signals using ANN and FL mapping methods. The output was derived using EMG signals and wrist position were directly proportional to control wrist desired velocity. Ten male subjects, ranging in age from 21 to 40, supplied EMG signal data set used for estimating the output in single and double muscles experiments. To validate the performance, a physical model of an exoskeleton hand was created using Sim-mechanics program tool. The ANN used Levenberg training method with 1 hidden layer and 10 neurons, while FL used a triangular membership function to represent muscles contraction signals amplitude at different MVC levels for each wrist position. As a result, PID was substituted to compensate fluctuation of mapping outputs, resulting in a smoother signal reading while improving the estimation of wrist desired velocity performance. As a conclusion, ANN compensates for complex nonlinear input to estimate output, but it works best with large data sets. FL allowed designers to design rules based on their knowledge, but the system will struggle due to the large number of inputs. Based on the results achieved, FL was able to show a distinct separation of wrist desired velocity hand movement when compared to ANN for similar testing datasets due to the decision making based on rules setting setup by the designer

Gaussian process autoregression for simultaneous proportional multi-modal prosthetic control with natural hand kinematics

Matching the dexterity, versatility, and robustness of the human hand is still an unachieved goal in bionics, robotics, and neural engineering. A major limitation for hand prosthetics lies in the challenges of reliably decoding user intention from muscle signals when controlling complex robotic hands. Most of the commercially available prosthetic hands use muscle-related signals to decode a finite number of predefined motions and some offer proportional control of open/close movements of the whole hand. Here, in contrast, we aim to offer users flexible control of individual joints of their artificial hand. We propose a novel framework for decoding neural information that enables a user to independently control 11 joints of the hand in a continuous manner-much like we control our natural hands. Toward this end, we instructed six able-bodied subjects to perform everyday object manipulation tasks combining both dynamic, free movements (e.g., grasping) and isometric force tasks (e.g., squeezing). We recorded the electromyographic and mechanomyographic activities of five extrinsic muscles of the hand in the forearm, while simultaneously monitoring 11 joints of hand and fingers using a sensorized data glove that tracked the joints of the hand. Instead of learning just a direct mapping from current muscle activity to intended hand movement, we formulated a novel autoregressive approach that combines the context of previous hand movements with instantaneous muscle activity to predict future hand movements. Specifically, we evaluated a linear vector autoregressive moving average model with exogenous inputs and a novel Gaussian process (gP) autoregressive framework to learn the continuous mapping from hand joint dynamics and muscle activity to decode intended hand movement. Our gP approach achieves high levels of performance (RMSE of 8°/s and ρ = 0.79). Crucially, we use a small set of sensors that allows us to control a larger set of independently actuated degrees of freedom of a hand. This novel undersensored control is enabled through the combination of nonlinear autoregressive continuous mapping between muscle activity and joint angles. The system evaluates the muscle signals in the context of previous natural hand movements. This enables us to resolve ambiguities in situations, where muscle signals alone cannot determine the correct action as we evaluate the muscle signals in their context of natural hand movements. gP autoregression is a particularly powerful approach which makes not only a prediction based on the context but also represents the associated uncertainty of its predictions, thus enabling the novel notion of risk-based control in neuroprosthetics. Our results suggest that gP autoregressive approaches with exogenous inputs lend themselves for natural, intuitive, and continuous control in neurotechnology, with the particular focus on prosthetic restoration of natural limb function, where high dexterity is required for complex movements

Neural network committees for finger joint angle estimation from surface EMG signals

<p>Abstract</p> <p>Background</p> <p>In virtual reality (VR) systems, the user's finger and hand positions are sensed and used to control the virtual environments. Direct biocontrol of VR environments using surface electromyography (SEMG) signals may be more synergistic and unconstraining to the user. The purpose of the present investigation was to develop a technique to predict the finger joint angle from the surface EMG measurements of the extensor muscle using neural network models.</p> <p>Methodology</p> <p>SEMG together with the actual joint angle measurements were obtained while the subject was performing flexion-extension rotation of the index finger at three speeds. Several neural networks were trained to predict the joint angle from the parameters extracted from the SEMG signals. The best networks were selected to form six committees. The neural network committees were evaluated using data from new subjects.</p> <p>Results</p> <p>There was hysteresis in the measured SMEG signals during the flexion-extension cycle. However, neural network committees were able to predict the joint angle with reasonable accuracy. RMS errors ranged from 0.085 ± 0.036 for fast speed finger-extension to 0.147 ± 0.026 for slow speed finger extension, and from 0.098 ± 0.023 for the fast speed finger flexion to 0.163 ± 0.054 for slow speed finger flexion.</p> <p>Conclusion</p> <p>Although hysteresis was observed in the measured SEMG signals, the committees of neural networks were able to predict the finger joint angle from SEMG signals.</p

A non-invasive human-machine interfacing framework for investigating dexterous control of hand muscles

The recent fast development of virtual reality and robotic assistive devices enables to augment the capabilities of able-body individuals as well as to overcome the motor missing functions of neurologically impaired or amputee individuals. To control these devices, movement intentions can be captured from biological structures involved in the process of motor planning and execution, such as the central nervous system (CNS), the peripheral nervous system (in particular the spinal motor neurons) and the musculoskeletal system. Thus, human-machine interfaces (HMI) enable to transfer neural information from the neuro-muscular system to machines. To prevent any risks due to surgical operations or tissue damage in implementing these HMIs, a non-invasive approach is proposed in this thesis. In the last five decades, surface electromyography (sEMG) has been extensively explored as a non-invasive source of neural information. EMG signals are constituted by the mixed electrical activity of several recruited motor units, the fundamental components of muscle contraction. High-density sEMG (HD-sEMG) with the use of blind source separation methods enabled to identify the discharge patterns of many of these active motor units. From these decomposed discharge patterns, the net common synaptic input (CSI) to the corresponding spinal motor neurons was quantified with cross-correlation in the time and frequency domain or with principal component analysis (PCA) on one or few muscles. It has been hypothesised that this CSI would result from the contribution of spinal descending commands sent by supra-spinal structures and afferences integrated by spinal interneurons. Another motor strategy implying the integration of descending commands at the spinal level is the one regarding the coordination of many muscles to control a large number of articular joints. This neurophysiological mechanism was investigated by measuring a single EMG amplitude per muscle, thus without the use of HD-sEMG and decomposition. In this case, the aim was to understand how the central nervous system (CNS) could control a large set of muscles actuating a vast set of combinations of degrees of freedom in a modular way. Thus, time-invariant patterns of muscle coordination, i.e. muscle synergies , were found in animals and humans from EMG amplitude of many muscles, modulated by time-varying commands to be combined to fulfil complex movements. In this thesis, for the first time, we present a non-invasive framework for human-machine interfaces based on both spinal motor neuron recruitment strategy and muscle synergistic control for unifying the understanding of these two motor control strategies and producing control signals correlated to biomechanical quantities. This implies recording both from many muscles and using HD-sEMG for each muscle. We investigated 14 muscles of the hand, 6 extrinsic and 8 intrinsic. The first two studies, (in Chapters 2 and 3, respectively) present the framework for CSI quantification by PCA and the extraction of the synergistic organisation of spinal motor neurons innervating the 14 investigated muscles. For the latter analysis, in Chapter 3, we proposed the existence of what we named as motor neuron synergies extracted with non-negative matrix factorisation (NMF) from the identified motor neurons. In these first two studies, we considered 7 subjects and 7 grip types involving differently all the four fingers in opposition with the thumb. In the first study, we found that the variance explained by the CSI among all motor neuron spike trains was (53.0 ± 10.9) % and its cross-correlation with force was 0.67 ± 0.10, remarkably high with respect to previous findings. In the second study, 4 motor neuron synergies were identified and associated with the actuation of one finger in opposition with the thumb, finding even higher correlation values with force (over 0.8) for each synergy associated with a finger during the actuation of the relative finger. In Chapter 4, we then extended the set of analysed movements in a vast repertoire of gestures and repeated the analysis of Chapter 3 by finding a different synergistic organisation during the execution of tens of tasks. We divided the contribution among extrinsic and intrinsic muscles and we found that intrinsic better enable single-finger spatial discrimination, while no difference was found in regression of joint angles by dividing the two groups of muscles. Finally, in Chapter 5 we proposed the techniques of the previous chapters for cases of impairment due both to amputation and stroke. We analysed one case of pre and post rehabilitation sessions of a trans-humeral amputee, the case of a post-stroke trans-radial amputee and three cases of acute stroke, i.e. less than one month from the stroke event. We present future perspectives (Chapter 6) aimed to design and implement a platform for both rehabilitation monitoring and myoelectric control. Thus, this thesis provides a bridge between two extensively studied motor control mechanisms, i.e. motor neuron recruitment and muscle synergies, and proposes this framework as suitable for rehabilitation monitoring and control of assistive devices.Open Acces

Modelling and EMG based Control of Upper Limb Exoskeletons for Hand Impairments

Functional losses associated with hand impairments have led to the growing development of hand exoskeletons. The main challenges are to develop the exoskeletons that work according to the user’s motion intention, which can be done by utilizing the electromyogram signals generated by forearm muscles contributed from the movement and/or grasping abilities of the hand. In this research, modelling and EMG based control of hand exoskeletons with the aim to assist stroke survivors in regaining their hand strength and functionality, and improve their quality of life is presented

Continuous Myoelectric Prediction of Future Ankle Angle and Moment Across Ambulation Conditions and Their Transitions

A hallmark of human locomotion is that it continuously adapts to changes in the environment and predictively adjusts to changes in the terrain, both of which are major challenges to lower limb amputees due to the limitations in prostheses and control algorithms. Here, the ability of a single-network nonlinear autoregressive model to continuously predict future ankle kinematics and kinetics simultaneously across ambulation conditions using lower limb surface electromyography (EMG) signals was examined. Ankle plantarflexor and dorsiflexor EMG from ten healthy young adults were mapped to normal ranges of ankle angle and ankle moment during level overground walking, stair ascent, and stair descent, including transitions between terrains (i.e., transitions to/from staircase). Prediction performance was characterized as a function of the time between current EMG/angle/moment inputs and future angle/moment model predictions (prediction interval), the number of past EMG/angle/moment input values over time (sampling window), and the number of units in the network hidden layer that minimized error between experimentally measured values (targets) and model predictions of ankle angle and moment. Ankle angle and moment predictions were robust across ambulation conditions with root mean squared errors less than 1° and 0.04 Nm/kg, respectively, and cross-correlations (R2) greater than 0.99 for prediction intervals of 58 ms. Model predictions at critical points of trip-related fall risk fell within the variability of the ankle angle and moment targets (Benjamini-Hochberg adjusted p \u3e 0.065). EMG contribution to ankle angle and moment predictions occurred consistently across ambulation conditions and model outputs. EMG signals had the greatest impact on noncyclic regions of gait such as double limb support, transitions between terrains, and around plantarflexion and moment peaks. The use of natural muscle activation patterns to continuously predict variations in normal gait and the model’s predictive capabilities to counteract electromechanical inherent delays suggest that this approach could provide robust and intuitive user-driven real-time control of a wide variety of lower limb robotic devices, including active powered ankle-foot prostheses

A MECHANISTIC APPROACH TO POSTURAL DEVELOPMENT IN CHILDREN

Upright standing is intrinsically unstable and requires active control. The central nervous system's feedback process is the active control that integrates multi-sensory information to generate appropriate motor commands to control the plant (the body with its musculotendon actuators). Maintaining standing balance is not trivial for a developing child because the feedback and the plant are both developing and the sensory inputs used for feedback are continually changing. Knowledge gaps exist in characterizing the critical ability of adaptive multi-sensory reweighting for standing balance control in children. Furthermore, the separate contributions of the plant and feedback and their relationship are poorly understood in children, especially when considering that the body is multi-jointed and feedback is multi-sensory. The purposes of this dissertation are to use a mechanistic approach to study multi-sensory abilities of typically developing (TD) children and children with Developmental Coordination Disorder (DCD). The specific aims are: 1) to characterize postural control under different multi-sensory conditions in TD children and children with DCD; 2) to characterize the development of adaptive multi-sensory reweighting in TD children and children with DCD; and, 3) to identify the plant and feedback for postural control in TD children and how they change in response to visual reweighting. In the first experiment (Aim 1), TD children, adults, and 7-year-old children with DCD are tested under four sensory conditions (no touch/no vision, with touch/no vision, no touch/with vision, and with touch/with vision). We found that touch robustly attenuated standing sway in all age groups. Children with DCD used touch less effectively than their TD peers and they also benefited from using vision to reduce sway. In the second experiment (Aim 2), TD children (4- to 10-year-old) and children with DCD (6- to 11-year-old) were presented with simultaneous small-amplitude touch bar and visual scene movement at 0.28 and 0.2 Hz, respectively, within five conditions that independently varied the amplitude of the stimuli. We found that TD children can reweight to both touch and vision from 4 years on and the amount of reweighting increased with age. However, multi-sensory fusion (i.e., inter-modal reweighting) was only observed in the older children. Children with DCD reweight to both touch and vision at a later age (10.8 years) than their TD peers. Even older children with DCD do not show advanced multisensory fusion. Two signature deficits of multisensory reweighting are a weak vision reweighting and a general phase lag to both sensory modalities. The final aim involves closed-loop system identification of the plant and feedback using electromyography (EMG) and kinematic responses to a high- or low-amplitude visual perturbation and two mechanical perturbations in children ages six and ten years and adults. We found that the plant is different between children and adults. Children demonstrate a smaller phase difference between trunk and leg than adults at higher frequencies. Feedback in children is qualitatively similar to adults. Quantitatively, children show less phase advance at the peak of the feedback curve which may be due to a longer time delay. Under the high and low visual amplitude conditions, children show less gain change (interpreted as reweighting) than adults in the kinematic and EMG responses. The observed kinematic and EMG reweighting are mainly due to the different use of visual information by the central nervous system as measured by the open-loop mapping from visual scene angle to EMG activity. The plant and the feedback do not contribute to reweighting