Regressing Grasping Using Force Myography: An Exploratory Study

Background: Partial hand amputation forms more than 90% of all upper limb amputations. This amputation has a notable efect on the amputee’s life. To improve the quality of life for partial hand amputees diferent prosthesis options, including externallypowered prosthesis, have been investigated. The focus of this work is to explore force myography (FMG) as a technique for regressing grasping movement accompanied by wrist position variations. This study can lay the groundwork for a future investigation of FMG as a technique for controlling externally-powered prostheses continuously. Methods: Ten able-bodied participants performed three hand movements while their wrist was fxed in one of six predefned positions. The angle between Thumb and Index fnger (θTI), and Thumb and Middle fnger (θTM) were calculated as measures of grasping movements. Two approaches were examined for estimating each angle: (i) one regression model, trained on data from all wrist positions and hand movements; (ii) a classifer that identifed the wrist position followed by a separate regression model for each wrist position. The possibility of training the system using a limited number of wrist positions and testing it on all positions was also investigated. Results: The frst approach had a correlation of determination (R2) of 0.871 for θTI and R2 θTM = 0.941. Using the second approach R2 θTI = 0.874 and R2 θTM = 0.942 were obtained. The frst approach is over two times faster than the second approach while having similar performance; thus the frst approach was selected to investigate the efect of the wrist position variations. Training with 6 or 5 wrist positions yielded results which were not statistically signifcant. A statistically signifcant decrease in performance resulted when less than fve wrist positions were used for training. Conclusions: The results indicate the potential of FMG to regress grasping movement, accompanied by wrist position variations, with a regression model for each angle. Also, it is necessary to include more than one wrist position in the training phase

Improving the Performance of Dynamic Electromyogram-to-Force Models for the Hand-Wrist and Multiple Fingers

Relating surface electromyogram (EMG) activity to force/torque models is used in many areas including: prosthesis control systems, to regulate direction and speed of movement in reaching and matching tasks; clinical biomechanics, to assess muscle deficiency and effort levels; and ergonomics analysis, to assess risk of work-related injury such as back pain, fatigue and skill tests. This thesis work concentrated on improving the performance of dynamic EMG-to-force models for the hand-wrist and multiple fingers. My contributions include: 1) rapid calibration of dynamic hand-wrist EMG-force models using a minimum number of electrodes, 2) efficiently training two degree of freedom (DoF) hand-wrist EMG-force models, and 3) estimating individual and combined fingertip forces from forearm EMG during constant-pose, force-varying tasks. My calibration approach for hand-wrist EMG-force models optimized three main factors for 1-DoF and 2-DoF tasks: training duration (14, 22, 30, 38, 44, 52, 60, 68, 76 s), number of electrodes (2 through 16), and model forms (subject-specific, DoF-specific, universal). The results show that training duration can be reduced from historical 76 s to 40–60 s without statistically affecting the average error for both 1-DoF and 2-DoF tasks. Reducing the number of electrodes depended on the number of DoFs. One-DoF models can be reduced to 2 electrodes with average test error range of 8.3–9.2% maximum voluntary contraction (MVC), depending on the DoF (e.g., flexion-extension, radial-ulnar deviation, pronation-supination, open-close). Additionally, 2-DoF models can be reduced to 6 electrodes with average error of 7.17–9.21 %MVC. Subject-specific models had the lowest error for 1-DoF tasks while DoF-specific and universal were the lowest for 2-DoF tasks. In the EMG-finger project, we studied independent contraction of one, two, three or four fingers (thumb excluded), as well as contraction of four fingers in unison. Using regression, we found that a pseudo-inverse tolerance (ratio of largest to smallest singular value) of 0.01 was optimal. Lower values produced erratic models and higher values produced models with higher errors. EMG-force errors using one finger ranged from 2.5–3.8 %MVC, using the optimal pseudoinverse tolerance. With additional fingers (two, three or four), the average error ranged from 5–8 %MVC. When four fingers contracted in unison, the average error was 4.3 %MVC. Additionally, I participated in two team projects—EMG-force dynamic models about the elbow and relating forearm muscle EMG to finger force during slowly force varying contractions. This work is also described herein

Force-Aware Interface via Electromyography for Natural VR/AR Interaction

While tremendous advances in visual and auditory realism have been made for virtual and augmented reality (VR/AR), introducing a plausible sense of physicality into the virtual world remains challenging. Closing the gap between real-world physicality and immersive virtual experience requires a closed interaction loop: applying user-exerted physical forces to the virtual environment and generating haptic sensations back to the users. However, existing VR/AR solutions either completely ignore the force inputs from the users or rely on obtrusive sensing devices that compromise user experience. By identifying users' muscle activation patterns while engaging in VR/AR, we design a learning-based neural interface for natural and intuitive force inputs. Specifically, we show that lightweight electromyography sensors, resting non-invasively on users' forearm skin, inform and establish a robust understanding of their complex hand activities. Fuelled by a neural-network-based model, our interface can decode finger-wise forces in real-time with 3.3% mean error, and generalize to new users with little calibration. Through an interactive psychophysical study, we show that human perception of virtual objects' physical properties, such as stiffness, can be significantly enhanced by our interface. We further demonstrate that our interface enables ubiquitous control via finger tapping. Ultimately, we envision our findings to push forward research towards more realistic physicality in future VR/AR.Comment: ACM Transactions on Graphics (SIGGRAPH Asia 2022

High performance wearable ultrasound as a human-machine interface for wrist and hand kinematic tracking

Objective: Non-invasive human machine interfaces (HMIs) have high potential in medical, entertainment, and industrial applications. Traditionally, surface electromyography (sEMG) has been used to track muscular activity and infer motor intention. Ultrasound (US) has received increasing attention as an alternative to sEMG-based HMIs. Here, we developed a portable US armband system with 24 channels and a multiple receiver approach, and compared it with existing sEMG- and US-based HMIs on movement intention decoding. Methods: US and motion capture data was recorded while participants performed wrist and hand movements of four degrees of freedom (DoFs) and their combinations. A linear regression model was used to offline predict hand kinematics from the US (or sEMG, for comparison) features. The method was further validated in real-time for a 3-DoF target reaching task. Results: In the offline analysis, the wearable US system achieved an average R2 of 0.94 in the prediction of four DoFs of the wrist and hand while sEMG reached a performance of R2=0.06 . In online control, the participants achieved an average 93% completion rate of the targets. Conclusion: When tailored for HMIs, the proposed US A-mode system and processing pipeline can successfully regress hand kinematics both in offline and online settings with performances comparable or superior to previously published interfaces. Significance: Wearable US technology may provide a new generation of HMIs that use muscular deformation to estimate limb movements. The wearable US system allowed for robust proportional and simultaneous control over multiple DoFs in both offline and online settings

Towards electrodeless EMG linear envelope signal recording for myo-activated prostheses control

After amputation, the residual muscles of the limb may function in a normal way, enabling the electromyogram (EMG) signals recorded from them to be used to drive a replacement limb. These replacement limbs are called myoelectric prosthesis. The prostheses that use EMG have always been the first choice for both clinicians and engineers. Unfortunately, due to the many drawbacks of EMG (e.g. skin preparation, electromagnetic interferences, high sample rate, etc.); researchers have aspired to find suitable alternatives. One proposes the dry-contact, low-cost sensor based on a force-sensitive resistor (FSR) as a valid alternative which instead of detecting electrical events, detects mechanical events of muscle. FSR sensor is placed on the skin through a hard, circular base to sense the muscle contraction and to acquire the signal. Similarly, to reduce the output drift (resistance) caused by FSR edges (creep) and to maintain the FSR sensitivity over a wide input force range, signal conditioning (Voltage output proportional to force) is implemented. This FSR signal acquired using FSR sensor can be used directly to replace the EMG linear envelope (an important control signal in prosthetics applications). To find the best FSR position(s) to replace a single EMG lead, the simultaneous recording of EMG and FSR output is performed. Three FSRs are placed directly over the EMG electrodes, in the middle of the targeted muscle and then the individual (FSR1, FSR2 and FSR3) and combination of FSR (e.g. FSR1+FSR2, FSR2-FSR3) is evaluated. The experiment is performed on a small sample of five volunteer subjects. The result shows a high correlation (up to 0.94) between FSR output and EMG linear envelope. Consequently, the usage of the best FSR sensor position shows the ability of electrode less FSR-LE to proportionally control the prosthesis (3-D claw). Furthermore, FSR can be used to develop a universal programmable muscle signal sensor that can be suitable to control the myo-activated prosthesis

On the Utility of Representation Learning Algorithms for Myoelectric Interfacing

Electrical activity produced by muscles during voluntary movement is a reflection of the firing patterns of relevant motor neurons and, by extension, the latent motor intent driving the movement. Once transduced via electromyography (EMG) and converted into digital form, this activity can be processed to provide an estimate of the original motor intent and is as such a feasible basis for non-invasive efferent neural interfacing. EMG-based motor intent decoding has so far received the most attention in the field of upper-limb prosthetics, where alternative means of interfacing are scarce and the utility of better control apparent. Whereas myoelectric prostheses have been available since the 1960s, available EMG control interfaces still lag behind the mechanical capabilities of the artificial limbs they are intended to steer—a gap at least partially due to limitations in current methods for translating EMG into appropriate motion commands. As the relationship between EMG signals and concurrent effector kinematics is highly non-linear and apparently stochastic, finding ways to accurately extract and combine relevant information from across electrode sites is still an active area of inquiry.This dissertation comprises an introduction and eight papers that explore issues afflicting the status quo of myoelectric decoding and possible solutions, all related through their use of learning algorithms and deep Artificial Neural Network (ANN) models. Paper I presents a Convolutional Neural Network (CNN) for multi-label movement decoding of high-density surface EMG (HD-sEMG) signals. Inspired by the successful use of CNNs in Paper I and the work of others, Paper II presents a method for automatic design of CNN architectures for use in myocontrol. Paper III introduces an ANN architecture with an appertaining training framework from which simultaneous and proportional control emerges. Paper Iv introduce a dataset of HD-sEMG signals for use with learning algorithms. Paper v applies a Recurrent Neural Network (RNN) model to decode finger forces from intramuscular EMG. Paper vI introduces a Transformer model for myoelectric interfacing that do not need additional training data to function with previously unseen users. Paper vII compares the performance of a Long Short-Term Memory (LSTM) network to that of classical pattern recognition algorithms. Lastly, paper vIII describes a framework for synthesizing EMG from multi-articulate gestures intended to reduce training burden

Control of a hand prosthesis using mixed electromyography and pressure sensing

During the last years, new technologies approaches have helped to develop realistic robotic hands for prosthetic use. Even so, the strategies to control them (input signals, prediction algorithms) are still limiting a complete match between the robotic hand and the real hand movements and behaviors. On this thesis, two different input signals (FMG and sEMG) were evaluated. From this analysis characteristic properties from each kind of signal were obtained, related with wrist and hand movements. In this way two different learning methods were implemented for the first time on robotic hand research. The goal of these two methods was to combine both kind of input signals, supported by the feature analysis previously done, in order to improve the movements prediction performance. The methods’ performance were compared with the separate input signals methods, so the improvement could be measured. Both mixing methods presented better results than the single input signal ones. These results along with other considerations defined, could lead to a robotic hand performance improvement from different perspective