Combined spatial and frequency encoding for electrotactile feedback of myoelectric signals

Electrotactile stimulation has been commonly used in human–machine interfaces to provide feedback to the user, thereby closing the control loop and improving performance. The encoding approach, which defines the mapping of the feedback information into stimulation profiles, is a critical component of an electrotactile interface. Ideally, the encoding will provide a high-fidelity representation of the feedback variable while being easy to perceive and interpret by the subject. In the present study, we performed a closed-loop experiment wherein discrete and continuous coding schemes are combined to exploit the benefits of both techniques. Subjects performed a muscle activation-matching task relying solely on electrotactile feedback representing the generated myoelectric signal (EMG). In particular, we investigated the performance of two different coding schemes (spatial and spatial combined with frequency) at two feedback resolutions (low: 3 and high: 5 intervals). In both schemes, the stimulation electrodes were placed circumferentially around the upper arm. The magnitude of the normalized EMG was divided into intervals, and each electrode was associated with one interval. When the generated EMG entered one of the intervals, the associated electrode started stimulating. In the combined encoding, the additional frequency modulation of the active electrode also indicated the momentary magnitude of the signal within the interval. The results showed that combined coding decreased the undershooting rate, variability and absolute deviation when the resolution was low but not when the resolution was high, where it actually worsened the performance. This demonstrates that combined coding can improve the effectiveness of EMG feedback, but that this effect is limited by the intrinsic variability of myoelectric control. Our findings, therefore, provide important insights as well as elucidate limitations of the information encoding methods when using electrotactile stimulation to convey a feedback signal characterized by high variability (EMG biofeedback)

Exploring the relationship between EMG feature space characteristics and control performance in machine learning myoelectric control

In myoelectric machine learning (ML) based control, it has been demonstrated that control performance usually increases with training, but it remains largely unknown which underlying factors govern these improvements. It has been suggested that the increase in performance originates from changes in characteristics of the Electromyography (EMG) patterns, such as separability or repeatability. However, the relation between these EMG metrics and control performance has hardly been studied. We assessed the relation between three common EMG feature space metrics (separability, variability and repeatability) in 20 able bodied participants who learned ML myoelectric control in a virtual task over 15 training blocks on 5 days. We assessed the change in offline and real-time performance, as well as the change of each EMG metric over the training. Subsequently, we assessed the relation between individual EMG metrics and offline and real-time performance via correlation analysis. Last, we tried to predict real-time performance from all EMG metrics via L2-regularized linear regression. Results showed that real-time performance improved with training, but there was no change in offline performance or in any of the EMG metrics. Furthermore, we only found a very low correlation between separability and real-time performance and no correlation between any other EMG metric and real-time performance. Finally, real-time performance could not be successfully predicted from all EMG metrics employing L2-regularized linear regression. We concluded that the three EMG metrics and real-time performance appear to be unrelated

A compact system for simultaneous stimulation and recording for closed-loop myoelectric control

Background.Despite important advancements in control and mechatronics of myoelectric prostheses, the communication between the user and his/her bionic limb is still unidirectional, as these systems do not provide somatosensory feedback. Electrotactile stimulation is an attractive technology to close the control loop since it allows flexible modulation of multiple parameters and compact interface design via multi-pad electrodes. However, the stimulation interferes with the recording of myoelectric signals and this can be detrimental to control.The work in this study was supported by the project ROBIN (8022-00243A and 8022-00226B) funded by the Independent Research Fund Denmark

Simultaneous assessment and training of an upper-limb amputee using incremental machine-learning-based myocontrol:a single-case experimental design

Background Machine-learning-based myocontrol of prosthetic devices suffers from a high rate of abandonment due to dissatisfaction with the training procedure and with the reliability of day-to-day control. Incremental myocontrol is a promising approach as it allows on-demand updating of the system, thus enforcing continuous interaction with the user. Nevertheless, a long-term study assessing the efficacy of incremental myocontrol is still missing, partially due to the lack of an adequate tool to do so. In this work we close this gap and report about a person with upper-limb absence who learned to control a dexterous hand prosthesis using incremental myocontrol through a novel functional assessment protocol called SATMC (Simultaneous Assessment and Training of Myoelectric Control). Methods The participant was fitted with a custom-made prosthetic setup with a controller based on Ridge Regression with Random Fourier Features (RR-RFF), a non-linear, incremental machine learning method, used to build and progressively update the myocontrol system. During a 13-month user study, the participant performed increasingly complex daily-living tasks, requiring fine bimanual coordination and manipulation with a multi-fingered hand prosthesis, in a realistic laboratory setup. The SATMC was used both to compose the tasks and continually assess the participant's progress. Patient satisfaction was measured using Visual Analog Scales. Results Over the course of the study, the participant progressively improved his performance both objectively, e.g., the time required to complete each task became shorter, and subjectively, meaning that his satisfaction improved. The SATMC actively supported the improvement of the participant by progressively increasing the difficulty of the tasks in a structured way. In combination with the incremental RR-RFF allowing for small adjustments when required, the participant was capable of reliably using four actions of the prosthetic hand to perform all required tasks at the end of the study. Conclusions Incremental myocontrol enabled an upper-limb amputee to reliably control a dexterous hand prosthesis while providing a subjectively satisfactory experience. The SATMC can be an effective tool to this aim

Grasp force estimation from the transient EMG using high-density surface recordings.

Objective: Understanding the neurophysiological signals underlying voluntary motor control and decoding them for prosthesis control are among the major challenges in applied neuroscience and bioengineering. Usually, information from the electrical activity of residual forearm muscles (i.e. the electromyogram, EMG) is used to control different functions of a prosthesis. Noteworthy, forearm EMG patterns at the onset of a contraction (transient phase) have shown to contain predictive information about upcoming grasps. However, decoding this information for the estimation of grasp force was so far overlooked. Approach: High Density-EMG signals (192 channels) were recorded from twelve participants performing a pick-and-lift task. The final grasp force was estimated offline using linear regressors, with four subsets of channels and ten features obtained using three channels-features selection methods. Two different evaluation metrics (absolute error and R2), complemented with statistical analysis, were used to select the optimal configuration of the parameters. Different windows of data starting at the grasp force (GF) onset were compared to determine the time at which the grasp force can be ascertained from the EMG signals. Main results: The prediction accuracy improved by increasing the window length from the moment of the onset and kept improving until the steady state at which a plateau of performances was reached. With our methodology, estimations of the grasp force through 16 EMG channels reached an absolute error of 2.52% the maximum voluntary force using only transient information and 1.99% with the first 500ms of data following the onset. Significance: The final GF estimation from transient EMG was comparable to the one obtained using steady state data, confirming our hypothesis that the transient phase contains information about the final grasp force. This result paves the way to fast online myoelectric controllers capable of decoding grasp strength from the very early portion of the EMG signal

Discrete vibro-tactile feedback prevents object slippage in hand prostheses more intuitively than other modalities

In the case of a hand amputation, the affected can use myoelectric prostheses to substitute the missing limb and regain motor functionality. Unfortunately, these prostheses do not restore sensory feedback, thus users are forced to rely on vision to avoid object slippage. This is cognitively taxing, as it requires continuous attention to the task. Thus, providing functionally effective sensory feedback is pivotal to reduce the occurrence of slip events and reduce the users’ cognitive burden. However, only a few studies investigated which kind of feedback is the most effective for this purpose, mostly using unrealistic experimental scenarios. Here we attempt a more realistic simulation of involuntary hand opening and subsequent recovery of a stable grasp of the slipping object using a robotic hand operated by the subjects through a standard myoelectric control interface. We compared three stimulation modalities (vision, continuous grip force feedback and discrete slip feedback) and found that the discrete feedback allowed subjects to have higher success rates (close to 100%) in terms of objects recovered from slippage, basically requiring no learning. These results suggest that this simple yet effective feedback can be used to reduce grasp failures in prosthetic users, increasing their confidence in the device