Simultaneously encoding movement and sEMG-based stiffness for robotic skill learning

Transferring human stiffness regulation strategies to robots enables them to effectively and efficiently acquire adaptive impedance control policies to deal with uncertainties during the accomplishment of physical contact tasks in an unstructured environment. In this work, we develop such a physical human-robot interaction (pHRI) system which allows robots to learn variable impedance skills from human demonstrations. Specifically, the biological signals, i.e., surface electromyography (sEMG) are utilized for the extraction of human arm stiffness features during the task demonstration. The estimated human arm stiffness is then mapped into a robot impedance controller. The dynamics of both movement and stiffness are simultaneously modeled by using a model combining the hidden semi-Markov model (HSMM) and the Gaussian mixture regression (GMR). More importantly, the correlation between the movement information and the stiffness information is encoded in a systematic manner. This approach enables capturing uncertainties over time and space and allows the robot to satisfy both position and stiffness requirements in a task with modulation of the impedance controller. The experimental study validated the proposed approach

Guidage non-intrusif d'un bras robotique à l'aide d'un bracelet myoélectrique à électrode sèche

Depuis plusieurs années la robotique est vue comme une solution clef pour améliorer la qualité de vie des personnes ayant subi une amputation. Pour créer de nouvelles prothèses intelligentes qui peuvent être facilement intégrées à la vie quotidienne et acceptée par ces personnes, celles-ci doivent être non-intrusives, fiables et peu coûteuses. L’électromyographie de surface fournit une interface intuitive et non intrusive basée sur l’activité musculaire de l’utilisateur permettant d’interagir avec des robots. Cependant, malgré des recherches approfondies dans le domaine de la classification des signaux sEMG, les classificateurs actuels manquent toujours de fiabilité, car ils ne sont pas robustes face au bruit à court terme (par exemple, petit déplacement des électrodes, fatigue musculaire) ou à long terme (par exemple, changement de la masse musculaire et des tissus adipeux) et requiert donc de recalibrer le classifieur de façon périodique. L’objectif de mon projet de recherche est de proposer une interface myoélectrique humain-robot basé sur des algorithmes d’apprentissage par transfert et d’adaptation de domaine afin d’augmenter la fiabilité du système à long-terme, tout en minimisant l’intrusivité (au niveau du temps de préparation) de ce genre de système. L’aspect non intrusif est obtenu en utilisant un bracelet à électrode sèche possédant dix canaux. Ce bracelet (3DC Armband) est de notre (Docteur Gabriel Gagnon-Turcotte, mes co-directeurs et moi-même) conception et a été réalisé durant mon doctorat. À l’heure d’écrire ces lignes, le 3DC Armband est le bracelet sans fil pour l’enregistrement de signaux sEMG le plus performant disponible. Contrairement aux dispositifs utilisant des électrodes à base de gel qui nécessitent un rasage de l’avant-bras, un nettoyage de la zone de placement et l’application d’un gel conducteur avant l’utilisation, le brassard du 3DC peut simplement être placé sur l’avant-bras sans aucune préparation. Cependant, cette facilité d’utilisation entraîne une diminution de la qualité de l’information du signal. Cette diminution provient du fait que les électrodes sèches obtiennent un signal plus bruité que celle à base de gel. En outre, des méthodes invasives peuvent réduire les déplacements d’électrodes lors de l’utilisation, contrairement au brassard. Pour remédier à cette dégradation de l’information, le projet de recherche s’appuiera sur l’apprentissage profond, et plus précisément sur les réseaux convolutionels. Le projet de recherche a été divisé en trois phases. La première porte sur la conception d’un classifieur permettant la reconnaissance de gestes de la main en temps réel. La deuxième porte sur l’implémentation d’un algorithme d’apprentissage par transfert afin de pouvoir profiter des données provenant d’autres personnes, permettant ainsi d’améliorer la classification des mouvements de la main pour un nouvel individu tout en diminuant le temps de préparation nécessaire pour utiliser le système. La troisième phase consiste en l’élaboration et l’implémentation des algorithmes d’adaptation de domaine et d’apprentissage faiblement supervisé afin de créer un classifieur qui soit robuste au changement à long terme.For several years, robotics has been seen as a key solution to improve the quality of life of people living with upper-limb disabilities. To create new, smart prostheses that can easily be integrated into everyday life, they must be non-intrusive, reliable and inexpensive. Surface electromyography provides an intuitive interface based on a user’s muscle activity to interact with robots. However, despite extensive research in the field of sEMG signal classification, current classifiers still lack reliability due to their lack of robustness to short-term (e.g. small electrode displacement, muscle fatigue) or long-term (e.g. change in muscle mass and adipose tissue) noise. In practice, this mean that to be useful, classifier needs to be periodically re-calibrated, a time consuming process. The goal of my research project is to proposes a human-robot myoelectric interface based on transfer learning and domain adaptation algorithms to increase the reliability of the system in the long term, while at the same time reducing the intrusiveness (in terms of hardware and preparation time) of this kind of systems. The non-intrusive aspect is achieved from a dry-electrode armband featuring ten channels. This armband, named the 3DC Armband is from our (Dr. Gabriel Gagnon-Turcotte, my co-directors and myself) conception and was realized during my doctorate. At the time of writing, the 3DC Armband offers the best performance for currently available dry-electrodes, surface electromyographic armbands. Unlike gel-based electrodes which require intrusive skin preparation (i.e. shaving, cleaning the skin and applying conductive gel), the 3DC Armband can simply be placed on the forearm without any preparation. However, this ease of use results in a decrease in the quality of information. This decrease is due to the fact that the signal recorded by dry electrodes is inherently noisier than gel-based ones. In addition, other systems use invasive methods (intramuscular electromyography) to capture a cleaner signal and reduce the source of noises (e.g. electrode shift). To remedy this degradation of information resulting from the non-intrusiveness of the armband, this research project will rely on deep learning, and more specifically on convolutional networks. The research project was divided into three phases. The first is the design of a classifier allowing the recognition of hand gestures in real-time. The second is the implementation of a transfer learning algorithm to take advantage of the data recorded across multiple users, thereby improving the system’s accuracy, while decreasing the time required to use the system. The third phase is the development and implementation of a domain adaptation and self-supervised learning to enhance the classifier’s robustness to long-term changes

A framework of human impedance recognition

A framework for recognizing the human intention of human forearm is developed. For a cooperative task, friendly and safe interaction is a key issue when humans directly interaction with the robots. Therefore, estimating the dynamics and intention of the human hand are very meaningful in the human machine interaction. A human subject puts his hand on the force sensor when a haptic device sets force in the proposed framework, the measured force, the surface electromyographic signal and the motion of the hand are employed to estimate the parameters of human forearm's impedance. The performance and feasibility of developed framework are verified

Intramuscular EMG-driven Musculoskeletal Modelling: Towards Implanted Muscle Interfacing in Spinal Cord Injury Patients

Objective: Surface EMG-driven modelling has been proposed as a means to control assistive devices by estimating joint torques. Implanted EMG sensors have several advantages over wearable sensors but provide a more localized information on muscle activity, which may impact torque estimates. Here, we tested and compared the use of surface and intramuscular EMG measurements for the estimation of required assistive joint torques using EMG driven modelling. Methods: Four healthy subjects and three incomplete spinal cord injury (SCI) patients performed walking trials at varying speeds. Motion capture marker trajectories, surface and intramuscular EMG, and ground reaction forces were measured concurrently. Subject-specific musculoskeletal models were developed for all subjects, and inverse dynamics analysis was performed for all individual trials. EMG-driven modelling based joint torque estimates were obtained from surface and intramuscular EMG. Results: The correlation between the experimental and predicted joint torques was similar when using intramuscular or surface EMG as input to the EMG-driven modelling estimator in both healthy individuals and patients. Conclusion: We have provided the first comparison of non-invasive and implanted EMG sensors as input signals for torque estimates in healthy individuals and SCI patients. Significance: Implanted EMG sensors have the potential to be used as a reliable input for assistive exoskeleton joint torque actuation

Assessment of a Wearable Force- and Electromyography Device and Comparison of the Related Signals for Myocontrol

In the frame of assistive robotics, multi-finger prosthetic hand/wrists have recently appeared,offering an increasing level of dexterity; however, in practice their control is limited to a few handgrips and still unreliable, with the effect that pattern recognition has not yet appeared in the clinicalenvironment. According to the scientific community, one of the keys to improve the situation ismulti-modal sensing, i.e., using diverse sensor modalities to interpret the subject’s intent andimprove the reliability and safety of the control system in daily life activities. In this work, wefirst describe and test a novel wireless, wearable force- and electromyography device; throughan experiment conducted on ten intact subjects, we then compare the obtained signals bothqualitatively and quantitatively, highlighting their advantages and disadvantages. Our resultsindicate that force-myography yields signals which are more stable across time during whenevera pattern is held, than those obtained by electromyography. We speculate that fusion of the twomodalities might be advantageous to improve the reliability of myocontrol in the near future

Surface Electromyography for Direct Vocal Control

This paper introduces a new method for direct control using the voice via measurement of vocal muscular activation with surface electromyography (sEMG). Digital musical interfaces based on the voice have typically used indirect control, in which features extracted from audio signals control the parameters of sound generation, for example in audio to MIDI controllers. By contrast, focusing on the musculature of the singing voice allows direct muscular control, or alternatively, combined direct and indirect control in an augmented vocal instrument. In this way we aim to both preserve the intimate relationship a vocalist has with their instrument and key timbral and stylistic characteristics of the voice while expanding its sonic capabilities. This paper discusses other digital instruments which effectively utilise a combination of indirect and direct control as well as a history of controllers involving the voice. Subsequently, a new method of direct control from physiological aspects of singing through sEMG and its capabilities are discussed. Future developments of the system are further outlined along with usage in performance studies, interactive live vocal performance, and educational and practice tools

Intramuscular EMG-driven musculoskeletal modelling: towards implanted muscle interfacing in spinal cord injury patients

OBJECTIVE: Surface EMG-driven modelling has been proposed as a means to control assistive devices by estimating joint torques. Implanted EMG sensors have several advantages over wearable sensors but provide a more localized information on muscle activity, which may impact torque estimates. Here, we tested and compared the use of surface and intramuscular EMG measurements for the estimation of required assistive joint torques using EMG driven modelling. METHODS: Four healthy subjects and three incomplete spinal cord injury (SCI) patients performed walking trials at varying speeds. Motion capture marker trajectories, surface and intramuscular EMG, and ground reaction forces were measured concurrently. Subject-specific musculoskeletal models were developed for all subjects, and inverse dynamics analysis was performed for all individual trials. EMG-driven modelling based joint torque estimates were obtained from surface and intramuscular EMG. RESULTS: The correlation between the experimental and predicted joint torques was similar when using intramuscular or surface EMG as input to the EMG-driven modelling estimator in both healthy individuals and patients. CONCLUSION: We have provided the first comparison of non-invasive and implanted EMG sensors as input signals for torque estimates in healthy individuals and SCI patients. SIGNIFICANCE: Implanted EMG sensors have the potential to be used as a reliable input for assistive exoskeleton joint torque actuation

Decoding subtle forearm flexions using fractal features of surface electromyogram from single and multiple sensors

Identifying finger and wrist flexion based actions using a single channel surface electromyogram (sEMG) can lead to a number of applications such as sEMG based controllers for near elbow amputees, human computer interface (HCI) devices for elderly and for defence personnel. These are currently infeasible because classification of sEMG is unreliable when the level of muscle contraction is low and there are multiple active muscles. The presence of noise and cross-talk from closely located and simultaneously active muscles is exaggerated when muscles are weakly active such as during sustained wrist and finger flexion. This paper reports the use of fractal properties of sEMG to reliably identify individual wrist and finger flexion, overcoming the earlier shortcomings