Pattern recognition based on HD-sEMG spatial features extraction for an efficient proportional control of a robotic arm.

To enable an efficient alternative control of an assistive robotic arm using electromyographic (EMG) signals, the control method must simultaneously provide both the direction and the velocity. However, the contraction variations of the forearm muscles, used to proportionally control the device’s velocity using a regression method, can disturb the accuracy of the classification used to estimate its direction at the same time. In this paper, the original set of spatial features takes advantage of the 2D structure of an 8 × 8 high-density surface EMG (HD-sEMG) sensor to perform a high accuracy classification while improving the robustness to the contraction variations. Based on the HD-sEMG sensor, different muscular activity images are extracted by applying different spatial filters. In order to characterize their distribution specific to each movement, instead of the EMG signals’ amplitudes, these muscular images are divided in sub-images upon which the proposed spatial features, such as the centers of the gravity coordinates and the percentages of influence, are computed. These features permits to achieve average accuracies of 97% and 96.7% to detect respectively 16 forearm movements performed by a healthy subject with prior experience with the control approach and 10 movements by ten inexperienced healthy subjects. Compared with the time-domain features, the proposed method exhibits significant higher accuracies in presence of muscular contraction variations, requires less training data and is more robust against the time of use. Furthermore, two fine real-time tasks illustrate the potential of the proposed approach to efficiently control a robotic arm

Real-time EMG based pattern recognition control for hand prostheses : a review on existing methods, challenges and future implementation

Upper limb amputation is a condition that significantly restricts the amputees from performing their daily activities. The myoelectric prosthesis, using signals from residual stump muscles, is aimed at restoring the function of such lost limbs seamlessly. Unfortunately, the acquisition and use of such myosignals are cumbersome and complicated. Furthermore, once acquired, it usually requires heavy computational power to turn it into a user control signal. Its transition to a practical prosthesis solution is still being challenged by various factors particularly those related to the fact that each amputee has different mobility, muscle contraction forces, limb positional variations and electrode placements. Thus, a solution that can adapt or otherwise tailor itself to each individual is required for maximum utility across amputees. Modified machine learning schemes for pattern recognition have the potential to significantly reduce the factors (movement of users and contraction of the muscle) affecting the traditional electromyography (EMG)-pattern recognition methods. Although recent developments of intelligent pattern recognition techniques could discriminate multiple degrees of freedom with high-level accuracy, their efficiency level was less accessible and revealed in real-world (amputee) applications. This review paper examined the suitability of upper limb prosthesis (ULP) inventions in the healthcare sector from their technical control perspective. More focus was given to the review of real-world applications and the use of pattern recognition control on amputees. We first reviewed the overall structure of pattern recognition schemes for myo-control prosthetic systems and then discussed their real-time use on amputee upper limbs. Finally, we concluded the paper with a discussion of the existing challenges and future research recommendations

Spatial distribution of HD-EMG improves identification of task and force in patients with incomplete spinal cord injury

Background: Recent studies show that spatial distribution of High Density surface EMG maps (HD-EMG) improves the identification of tasks and their corresponding contraction levels. However, in patients with incomplete spinal cord injury (iSCI), some nerves that control muscles are damaged, leaving some muscle parts without an innervation. Therefore, HD-EMG maps in patients with iSCI are affected by the injury and they can be different for every patient. The objective of this study is to investigate the spatial distribution of intensity in HD-EMG recordings to distinguish co-activation patterns for different tasks and effort levels in patients with iSCI. These patterns are evaluated to be used for extraction of motion intention.; Method: HD-EMG was recorded in patients during four isometric tasks of the forearm at three different effort levels. A linear discriminant classifier based on intensity and spatial features of HD-EMG maps of five upper-limb muscles was used to identify the attempted tasks. Task and force identification were evaluated for each patient individually, and the reliability of the identification was tested with respect to muscle fatigue and time interval between training and identification. Results: Three feature sets were analyzed in the identification: 1) intensity of the HD-EMG map, 2) intensity and center of gravity of HD-EMG maps and 3) intensity of a single differential EMG channel (gold standard).; Results show that the combination of intensity and spatial features in classification identifies tasks and effort levels properly (Acc = 98.8 %; S = 92.5 %; P = 93.2 %; SP = 99.4 %) and outperforms significantly the other two feature sets (p < 0.05).; Conclusion: In spite of the limited motor functionality, a specific co-activation pattern for each patient exists for both intensity, and spatial distribution of myoelectric activity. The spatial distribution is less sensitive than intensity to myoelectric changes that occur due to fatigue, and other time-dependent influences.Peer ReviewedPostprint (published version

Towards electrodeless EMG linear envelope signal recording for myo-activated prostheses control

After amputation, the residual muscles of the limb may function in a normal way, enabling the electromyogram (EMG) signals recorded from them to be used to drive a replacement limb. These replacement limbs are called myoelectric prosthesis. The prostheses that use EMG have always been the first choice for both clinicians and engineers. Unfortunately, due to the many drawbacks of EMG (e.g. skin preparation, electromagnetic interferences, high sample rate, etc.); researchers have aspired to find suitable alternatives. One proposes the dry-contact, low-cost sensor based on a force-sensitive resistor (FSR) as a valid alternative which instead of detecting electrical events, detects mechanical events of muscle. FSR sensor is placed on the skin through a hard, circular base to sense the muscle contraction and to acquire the signal. Similarly, to reduce the output drift (resistance) caused by FSR edges (creep) and to maintain the FSR sensitivity over a wide input force range, signal conditioning (Voltage output proportional to force) is implemented. This FSR signal acquired using FSR sensor can be used directly to replace the EMG linear envelope (an important control signal in prosthetics applications). To find the best FSR position(s) to replace a single EMG lead, the simultaneous recording of EMG and FSR output is performed. Three FSRs are placed directly over the EMG electrodes, in the middle of the targeted muscle and then the individual (FSR1, FSR2 and FSR3) and combination of FSR (e.g. FSR1+FSR2, FSR2-FSR3) is evaluated. The experiment is performed on a small sample of five volunteer subjects. The result shows a high correlation (up to 0.94) between FSR output and EMG linear envelope. Consequently, the usage of the best FSR sensor position shows the ability of electrode less FSR-LE to proportionally control the prosthesis (3-D claw). Furthermore, FSR can be used to develop a universal programmable muscle signal sensor that can be suitable to control the myo-activated prosthesis

Analysis of forearm muscles activity by means of new protocols of multichannel EMG signal recording and processing

Los movimientos voluntarios del cuerpo son controlados por el sistema nervioso central y periférico a través de la contracción de los músculos esqueléticos. La contracción se inicia al liberarse un neurotransmisor sobre la unión neuromuscular, iniciando la propagación de un biopotencial sobre la membrana de las fibras musculares que se desplaza hacia los tendones: el Potencial de Acción de la Unidad Motora (MUAP). La señal electromiográfica de superficie registra la activación continua de dichos potenciales sobre la superficie de la piel y constituye una valiosa herramienta para la investigación, diagnóstico y seguimiento clínico de trastornos musculares, así como para la identificación de la intención movimiento tanto en términos de dirección como de potencia. En el estudio de las enfermedades del sistema neuromuscular es necesario analizar el nivel de actividad, la capacidad de producción de fuerza, la activación muscular conjunta y la predisposición a la fatiga muscular, todos ellos asociados con factores fisiológicos que determinan la resultante contracción mioeléctrica. Además, el uso de matrices de electrodos facilita la investigación de las propiedades periféricas de las unidades motoras activas, las características anatómicas del músculo y los cambios espaciales en su activación, ocasionados por el tipo de tarea motora o la potencia de la misma. El objetivo principal de esta tesis es el diseño e implementación de protocolos experimentales y algoritmos de procesado para extraer información fiable de señales sEMG multicanal en 1 y 2 dimensiones del espacio. Dicha información ha sido interpretada y relacionada con dos patologías específicas de la extremidad superior: Epicondilitis Lateral y Lesión de Esfuerzo Repetitivo. También fue utilizada para identificar la dirección de movimiento y la fuerza asociada a la contracción muscular, cuyos patrones podrían ser de utilidad en aplicaciones donde la señal electromiográfica se utilice para controlar interfaces hombre-máquina como es el caso de terapia física basada en robots, entornos virtuales de rehabilitación o realimentación de la actividad muscular. En resumen, las aportaciones más relevantes de esta tesis son: * La definición de protocolos experimentales orientados al registro de señales sEMG en una región óptima del músculo. * Definición de índices asociados a la co-activación de diferentes músculos * Identificación de señales artefactuadas en registros multicanal * Selección de los canales mas relevantes para el análisis Extracción de un conjunto de características que permita una alta exactitud en la identificación de tareas motoras Los protocolos experimentales y los índices propuestos permitieron establecer que diversos desequilibrios entre músculos extrínsecos del antebrazo podrían desempeñar un papel clave en la fisiopatología de la epicondilitis lateral. Los resultados fueron consistentes en diferentes ejercicios y pueden definir un marco de evaluación para el seguimiento y evaluación de pacientes en programas de rehabilitación motora. Por otra parte, se encontró que las características asociadas con la distribución espacial de los MUAPs mejoran la exactitud en la identificación de la intención de movimiento. Lo que es más, las características extraídas de registros sEMG de alta densidad son más robustas que las extraídas de señales bipolares simples, no sólo por la redundancia de contacto implicada en HD-EMG, sino también porque permite monitorizar las regiones del músculo donde la amplitud de la señal es máxima y que varían con el tipo de ejercicio, permitiendo así una mejor estimación de la activación muscular mediante el análisis de los canales mas relevantes.Voluntary movements are achieved by the contraction of skeletal muscles controlled by the Central and Peripheral Nervous system. The contraction is initiated by the release of a neurotransmitter that promotes a reaction in the walls of the muscular fiber, producing a biopotential known as Motor Unit Action Potential (MUAP) that travels from the neuromuscular junction to the tendons. The surface electromyographic signal records the continuous activation of such potentials over the surface of the skin and constitutes a valuable tool for the diagnosis, monitoring and clinical research of muscular disorders as well as to infer motion intention not only regarding the direction of the movement but also its power. In the study of diseases of the neuromuscular system it is necessary to analyze the level of activity, the capacity of production of strength, the load-sharing between muscles and the probably predisposition to muscular fatigue, all of them associated with physiological factors determining the resultant muscular contraction. Moreover, the use of electrode arrays facilitate the investigation of the peripheral properties of the active Motor Units, the anatomical characteristics of the muscle and the spatial changes induced in their activation of as product of type of movement or power of the contraction.The main objective of this thesis was the design and implementation of experimental protocols, and algorithms to extract information from multichannel sEMG signals in 1 and 2 dimensions of the space. Such information was interpreted and related to pathological events associated to two upper-limb conditions: Lateral Epicondylitis and Repetitive Strain Injury. It was also used to identify the direction of movement and contraction strength which could be useful in applications concerning the use of biofeedback from EMG like in robotic- aided therapies and computer-based rehabilitation training.In summary, the most relevant contributions are:§The definition of experimental protocols intended to find optimal regions for the recording of sEMG signals. §The definition of indices associated to the co- activation of different muscles. §The detection of low-quality signals in multichannel sEMG recordings.§ The selection of the most relevant EMG channels for the analysis§The extraction of a set of features that led to high classification accuracy in the identification of tasks.The experimental protocols and the proposed indices allowed establishing that imbalances between extrinsic muscles of the forearm could play a key role in the pathophysiology of lateral epicondylalgia. Results were consistent in different types of motor task and may define an assessment framework for the monitoring and evaluation of patients during rehabilitation programs.On the other hand, it was found that features associated with the spatial distribution of the MUAPs improve the accuracy of the identification of motion intention. What is more, features extracted from high density EMG recordings are more robust not only because it implies contact redundancy but also because it allows the tracking of (task changing) skin surface areas where EMG amplitude is maximal and a better estimation of muscle activity by the proper selection of the most significant channels

Surface EMG-Based Inter-Session/Inter-Subject Gesture Recognition by Leveraging Lightweight All-ConvNet and Transfer Learning

Gesture recognition using low-resolution instantaneous HD-sEMG images opens up new avenues for the development of more fluid and natural muscle-computer interfaces. However, the data variability between inter-session and inter-subject scenarios presents a great challenge. The existing approaches employed very large and complex deep ConvNet or 2SRNN-based domain adaptation methods to approximate the distribution shift caused by these inter-session and inter-subject data variability. Hence, these methods also require learning over millions of training parameters and a large pre-trained and target domain dataset in both the pre-training and adaptation stages. As a result, it makes high-end resource-bounded and computationally very expensive for deployment in real-time applications. To overcome this problem, we propose a lightweight All-ConvNet+TL model that leverages lightweight All-ConvNet and transfer learning (TL) for the enhancement of inter-session and inter-subject gesture recognition performance. The All-ConvNet+TL model consists solely of convolutional layers, a simple yet efficient framework for learning invariant and discriminative representations to address the distribution shifts caused by inter-session and inter-subject data variability. Experiments on four datasets demonstrate that our proposed methods outperform the most complex existing approaches by a large margin and achieve state-of-the-art results on inter-session and inter-subject scenarios and perform on par or competitively on intra-session gesture recognition. These performance gaps increase even more when a tiny amount (e.g., a single trial) of data is available on the target domain for adaptation. These outstanding experimental results provide evidence that the current state-of-the-art models may be overparameterized for sEMG-based inter-session and inter-subject gesture recognition tasks

On the Utility of Representation Learning Algorithms for Myoelectric Interfacing

Electrical activity produced by muscles during voluntary movement is a reflection of the firing patterns of relevant motor neurons and, by extension, the latent motor intent driving the movement. Once transduced via electromyography (EMG) and converted into digital form, this activity can be processed to provide an estimate of the original motor intent and is as such a feasible basis for non-invasive efferent neural interfacing. EMG-based motor intent decoding has so far received the most attention in the field of upper-limb prosthetics, where alternative means of interfacing are scarce and the utility of better control apparent. Whereas myoelectric prostheses have been available since the 1960s, available EMG control interfaces still lag behind the mechanical capabilities of the artificial limbs they are intended to steer—a gap at least partially due to limitations in current methods for translating EMG into appropriate motion commands. As the relationship between EMG signals and concurrent effector kinematics is highly non-linear and apparently stochastic, finding ways to accurately extract and combine relevant information from across electrode sites is still an active area of inquiry.This dissertation comprises an introduction and eight papers that explore issues afflicting the status quo of myoelectric decoding and possible solutions, all related through their use of learning algorithms and deep Artificial Neural Network (ANN) models. Paper I presents a Convolutional Neural Network (CNN) for multi-label movement decoding of high-density surface EMG (HD-sEMG) signals. Inspired by the successful use of CNNs in Paper I and the work of others, Paper II presents a method for automatic design of CNN architectures for use in myocontrol. Paper III introduces an ANN architecture with an appertaining training framework from which simultaneous and proportional control emerges. Paper Iv introduce a dataset of HD-sEMG signals for use with learning algorithms. Paper v applies a Recurrent Neural Network (RNN) model to decode finger forces from intramuscular EMG. Paper vI introduces a Transformer model for myoelectric interfacing that do not need additional training data to function with previously unseen users. Paper vII compares the performance of a Long Short-Term Memory (LSTM) network to that of classical pattern recognition algorithms. Lastly, paper vIII describes a framework for synthesizing EMG from multi-articulate gestures intended to reduce training burden

Increasing the robustness of active upper limb prostheses

This thesis is based on my work done at the Institute for Neurorehabilitation Systems at the University Medical Center Goettingen. My work has been partially founded by German Ministry for Education and Research (BMBF) via the Bernstein Focus Neurotechnology (BFNT) Göttingen under grant number 1GQ0810 The local ethics committee approved all studies involving human subjects, and all subjects signed informed consents prior to their participation in the studies. The entire thesis has been originally written by me. Part of the materials used in this thesis have also been published in journals or conferences, where I am the first or corresponding author. All rights for re-use of previously published material were obtained. Reused figures and tables of IEEE publications are marked with © [Year] IEEE. Hereby I declare that I have written this thesis independently and with no other aids and sources than quoted