Temporal-Difference Learning to Assist Human Decision Making during the Control of an Artificial Limb

In this work we explore the use of reinforcement learning (RL) to help with human decision making, combining state-of-the-art RL algorithms with an application to prosthetics. Managing human-machine interaction is a problem of considerable scope, and the simplification of human-robot interfaces is especially important in the domains of biomedical technology and rehabilitation medicine. For example, amputees who control artificial limbs are often required to quickly switch between a number of control actions or modes of operation in order to operate their devices. We suggest that by learning to anticipate (predict) a user's behaviour, artificial limbs could take on an active role in a human's control decisions so as to reduce the burden on their users. Recently, we showed that RL in the form of general value functions (GVFs) could be used to accurately detect a user's control intent prior to their explicit control choices. In the present work, we explore the use of temporal-difference learning and GVFs to predict when users will switch their control influence between the different motor functions of a robot arm. Experiments were performed using a multi-function robot arm that was controlled by muscle signals from a user's body (similar to conventional artificial limb control). Our approach was able to acquire and maintain forecasts about a user's switching decisions in real time. It also provides an intuitive and reward-free way for users to correct or reinforce the decisions made by the machine learning system. We expect that when a system is certain enough about its predictions, it can begin to take over switching decisions from the user to streamline control and potentially decrease the time and effort needed to complete tasks. This preliminary study therefore suggests a way to naturally integrate human- and machine-based decision making systems.Comment: 5 pages, 4 figures, This version to appear at The 1st Multidisciplinary Conference on Reinforcement Learning and Decision Making, Princeton, NJ, USA, Oct. 25-27, 201

Spatial distribution of HD-EMG improves identification of task and force in patients with incomplete spinal cord injury

Background: Recent studies show that spatial distribution of High Density surface EMG maps (HD-EMG) improves the identification of tasks and their corresponding contraction levels. However, in patients with incomplete spinal cord injury (iSCI), some nerves that control muscles are damaged, leaving some muscle parts without an innervation. Therefore, HD-EMG maps in patients with iSCI are affected by the injury and they can be different for every patient. The objective of this study is to investigate the spatial distribution of intensity in HD-EMG recordings to distinguish co-activation patterns for different tasks and effort levels in patients with iSCI. These patterns are evaluated to be used for extraction of motion intention.; Method: HD-EMG was recorded in patients during four isometric tasks of the forearm at three different effort levels. A linear discriminant classifier based on intensity and spatial features of HD-EMG maps of five upper-limb muscles was used to identify the attempted tasks. Task and force identification were evaluated for each patient individually, and the reliability of the identification was tested with respect to muscle fatigue and time interval between training and identification. Results: Three feature sets were analyzed in the identification: 1) intensity of the HD-EMG map, 2) intensity and center of gravity of HD-EMG maps and 3) intensity of a single differential EMG channel (gold standard).; Results show that the combination of intensity and spatial features in classification identifies tasks and effort levels properly (Acc = 98.8 %; S = 92.5 %; P = 93.2 %; SP = 99.4 %) and outperforms significantly the other two feature sets (p < 0.05).; Conclusion: In spite of the limited motor functionality, a specific co-activation pattern for each patient exists for both intensity, and spatial distribution of myoelectric activity. The spatial distribution is less sensitive than intensity to myoelectric changes that occur due to fatigue, and other time-dependent influences.Peer ReviewedPostprint (published version

Proceedings of the first workshop on Peripheral Machine Interfaces: going beyond traditional surface electromyography

abstract: One of the hottest topics in rehabilitation robotics is that of proper control of prosthetic devices. Despite decades of research, the state of the art is dramatically behind the expectations. To shed light on this issue, in June, 2013 the first international workshop on Present and future of non-invasive peripheral nervous system (PNS)–Machine Interfaces (MI; PMI) was convened, hosted by the International Conference on Rehabilitation Robotics. The keyword PMI has been selected to denote human–machine interfaces targeted at the limb-deficient, mainly upper-limb amputees, dealing with signals gathered from the PNS in a non-invasive way, that is, from the surface of the residuum. The workshop was intended to provide an overview of the state of the art and future perspectives of such interfaces; this paper represents is a collection of opinions expressed by each and every researcher/group involved in it

Real-Time Control of a Multi-Degree-of-Freedom Mirror Myoelectric Interface During Functional Task Training

Motor learning mediated by motor training has in the past been explored for rehabilitation. Myoelectric interfaces together with exoskeletons allow patients to receive real-time feedback about their muscle activity. However, the number of degrees of freedom that can be simultaneously controlled is limited, which hinders the training of functional tasks and the effectiveness of the rehabilitation therapy. The objective of this study was to develop a myoelectric interface that would allow multi-degree-of-freedom control of an exoskeleton involving arm, wrist and hand joints, with an eye toward rehabilitation. We tested the effectiveness of a myoelectric decoder trained with data from one upper limb and mirrored to control a multi-degree-of-freedom exoskeleton with the opposite upper limb (i.e., mirror myoelectric interface) in 10 healthy participants. We demonstrated successful simultaneous control of multiple upper-limb joints by all participants. We showed evidence that subjects learned the mirror myoelectric model within the span of a five-session experiment, as reflected by a significant decrease in the time to execute trials and in the number of failed trials. These results are the necessary precursor to evaluating if a decoder trained with EMG from the healthy limb could foster learning of natural EMG patterns and lead to motor rehabilitation in stroke patients.This study was funded by the Eurostars Project E! 113928 Subliminal Home Rehab (SHR), BMBF (Bundesministerium für Bildung und Forschung) (FKZ: SHR 01QE2023; and REHOME 16SV8606), Fortüne-Program of the University of Tübingen (2452-0-0/1), Ministry of Science of the Basque Country (Elkartek: MODULA KK-2019/00018) and H2020- FETPROACT-EIC-2018-2020 (MAIA 951910)

Myoelectric Control Architectures to Drive Upper Limb Exoskeletons

Myoelectric interfaces are sensing devices based on electromyography (EMG) able to read the electrical activity of motoneurons and muscles. These interfaces can be used to infer movement volition and to control assistive devices. Currently, these interfaces are widely used to control robotic prostheses for amputees, but their use could be beneficial even for people suffering from motor disabilities where the peripheral nervous system is intact and the impairment is only due to the muscles, e.g. muscular dystrophy, myopathies, or ageing. In combination with recent robotic orthoses and exoskeletons, myoelectric interfaces could dramatically improve these patients’ quality of life. Unfortunately, despite a wide plethora of methodologies has been proposed so far, a natural, intuitive, and reliable interface able to follow impaired subjects’ volition is still missing. The first contribution of this work is to provide a review of existing approaches. In this work we found that existing EMG-based control interfaces can be viewed as specific cases of a generic myoelectric control architecture composed by three distinct functional modules: a decoder to extract the movement intention from EMG signals, a controller to accomplish the desired motion through an actual command given to the actuators, and an adapter to connect them. The latter is responsible for translating the signal from decoder’s output to controller’s input domain and for modulating the level of provided assistance. We used this concept to analyse the case of study of linear regression decoders and an elbow exoskeleton. This thesis has the scientific objective to determine how these modules affect performance of EMG-driven exoskeletons and wearer’s fatigue. To experimentally test and compare myoelectric interfaces this work proposes: (1) a procedure to automatically tune the decoder module in order to equally compare or to normalize the decoder output among different sessions and subjects; (2) a procedure to automatically tune gravity compensation even for subjects suffering from severe disabilities, allowing them to perform the experimental tests; (3) a methodology to guide the impaired patients through the experimental session; (4) an evaluation procedure and metrics allowing statistically significant and unbiased comparison of different myoelectric interfaces. A further contribution of this work is the design of an experimental test bed composed by an elbow exoskeleton and by a software framework able to collect EMG signals and make them available to the exoskeleton’s actuators with minimal latency. Using this test bed, we were able to test different myoelectric interfaces based on our architecture, with different modules choices and tunings. We used linear regression decoders calibrated to predict the muscular torque, low-level controllers having torque or velocity as reference, and adapters consisting of a properly dimensioned gain or simple dynamic systems, such as an integrator or a mass-damping system. The results we obtained allow to conclude that EMG-based control is a viable technology to assist muscular weakness patients. Moreover, all the components of the myoelectric control architecture – decoder, adapter, controller, and their tuning – significantly affect the task-based performance measures we collect. Further investigations should be devoted to a methodology to automatically tune all the components, not the decoders only, and to the quantitative study of the effect the adapter has on the regulation of the assistance level and of the tradeoff between speed and accuracy