Force Myography-Based Human Robot Interactions via Deep Domain Adaptation and Generalization

Estimating applied force using force myography (FMG) technique can be effective in human-robot interactions (HRI) using data-driven models. A model predicts well when adequate training and evaluation are observed in same session, which is sometimes time consuming and impractical. In real scenarios, a pretrained transfer learning model predicting forces quickly once fine-tuned to target distribution would be a favorable choice and hence needs to be examined. Therefore, in this study a unified supervised FMG-based deep transfer learner (SFMG-DTL) model using CNN architecture was pretrained with multiple sessions FMG source data (Ds, Ts) and evaluated in estimating forces in separate target domains (Dt, Tt) via supervised domain adaptation (SDA) and supervised domain generalization (SDG). For SDA, case (i) intra-subject evaluation (Ds ≠ Dt-SDA, Ts ≈ Tt-SDA) was examined, while for SDG, case (ii) cross-subject evaluation (Ds ≠ Dt-SDG, Ts ≠ Tt-SDG) was examined. Fine tuning with few “target training data” calibrated the model effectively towards target adaptation. The proposed SFMG-DTL model performed better with higher estimation accuracies and lower errors (R2 ≥ 88%, NRMSE ≤ 0.6) in both cases. These results reveal that interactive force estimations via transfer learning will improve daily HRI experiences where “target training data” is limited, or faster adaptation is require

Estimation of User-Applied Isometric Force/Torque Using Upper Extremity Force Myography

Hand force estimation is critical for applications that involve physical human-machine interactions for force monitoring and machine control. Force Myography (FMG) is a potential technique to be used for estimating hand force/torque. The FMG signals reflect the volumetric changes in the arm muscles due to muscle contraction or expansion. This paper investigates the feasibility of employing force-sensing resistors (FSRs) worn on the arm to measure the FMG signals for isometric force/torque estimation. Nine participants were recruited in this study and were asked to exert isometric force along three perpendicular axes, torque about the same three axes, and force and torque simultaneously. During the tests, the isometric force and torque were measured using a 6-degree-of-freedom (DoF) (i.e., force in three axes and torque around the same axes) load cell for ground truth labels whereas the FMG signals were recorded using a total number of 60 FSRs, which were embedded into four bands worn on the different locations of the arm. A two-stage regression strategy was employed to enhance the performance of the FMG bands, where three regression algorithms including general regression neural network (GRNN), support vector regression (SVR), and random forest regression (RF) models were employed, respectively, in the first stage and GRNN was used in the second stage. Two cases were considered to explore the performance of the FMG bands in estimating: (1) 3-DoF force and 3-DoF torque at once and (2) 6-DoF force and torque. In addition, the impact of sensor placement and the spatial coverage of FMG measurements were studied. This preliminary investigation demonstrates promising potential of FMG to estimate multi-DoF isometric force/torque. Specifically, R2 accuracies of 0.83 for the 3-DoF force, 0.84 for 3-DoF torque, and 0.77 for the combination of force and torque (6-DoF) regressions were obtained using the four bands on the arm in cross-trial evaluation