Subtle hand gesture identification for HCI using temporal decorrelation source separation BSS of surface EMG

Hand gesture identification has various human computer interaction (HCI) applications. This paper presents a method for subtle hand gesture identification from sEMG of the forearm by decomposing the signal into components originating from different muscles. The processing requires the decomposition of the surface EMG by temporal decorrelation source separation (TDSEP) based blind source separation technique. Pattern classification of the separated signal is performed in the second step with a back propagation neural network. The focus of this work is to establish a simple, yet robust system that can be used to identify subtle complex hand actions and gestures for control of prosthesis and other HCI based devices. The proposed model based approach is able to overcome the ambiguity problems (order and magnitude problem) of BSS methods by selecting an a priori mixing matrix based on known hand muscle anatomy. The paper reports experimental results, where the system was able to reliably recognize different subtle hand gesture with an overall accuracy of 97%. The advantage of such a system is that it is easy to train by a lay user, and can easily be implemented in real time after the initial training. The paper also highlights the importance of mixing matrix analysis in BSS technique

Stationary wavelet processing and data imputing in myoelectric pattern recognition on a low-cost embedded system

Pattern recognition-based decoding of surface electromyography allows for intuitive and flexible control of prostheses but comes at the cost of sensitivity to in-band noise and sensor faults. System robustness can be improved with wavelet-based signal processing and data imputing, but no attempt has been made to implement such algorithms on real-time, portable systems. The aim of this work was to investigate the feasibility of low-latency, wavelet-based processing and data imputing on an embedded device capable of controlling upper-arm prostheses. Nine able-bodied subjects performed Motion Tests while inducing transient disturbances. Additional investigation was performed on pre-recorded Motion Tests from 15 able-bodied subjects with simulated disturbances. Results from real-time tests were inconclusive, likely due to the low number of disturbance episodes, but simulated tests showed significant improvements in most metrics for both algorithms. However, both algorithms also showed reduced responsiveness during disturbance episodes. These results suggest wavelet-based processing and data imputing can be implemented in portable, real-time systems to potentially improve robustness to signal distortion in prosthetic devices with the caveat of reduced responsiveness for the typically short duration of signal disturbances. The trade-off between large-scale signal corruption robustness and system responsiveness warrants further studies in daily life activities

Characteristics of muscle activation patterns at the ankle in stroke patients during walking.

Stroke causes impairment of the sensory and motor systems; this can lead to difficulties in walking and participation in society. For effective rehabilitation it is important to measure the essential characteristics of impairment and associate these with the nature of disability. Efficient gait requires a complex interplay of muscles. Surface electromyography(sEMG) can be used to measure muscle activity and to observe disruption to this interplay after stroke. Yet, classification of this disruption in stroke patients has not been achieved. It is hypothesised that features identified from the sEMG signal can be used to classify underlying impairments. A clinically viable gait analysis system has been developed, integrating an in-house wireless sEMG system synchronised with bilateral video and inertial orientation sensors. Signal processing techniques have been extended and implemented, appropriate for use with sEMG. These techniques have focussed on frequency domain features using wavelet analysis and muscle activation patterns using principal component analysis. The system has been used to measure gait from stroke patients and un-impaired subjects. Characteristic patterns of activity from the ankle musculature were defined using principal component analysis of the linear envelope. Patients with common patterns of tibialis anterior activity did not necessarily share common patterns of gastrocnemius or soleus activity. Patients with similar linear envelope patterns did not always present with the same kinematic profiles. The relationship between observable impairments, kinematics and sEMG is seen to be complex and there is therefore a need for a multidimensional view of gait data in relation to stroke impairment. The analysis of instantaneous mean frequency and time-frequency has revealed additional periods of activity not obvious in the linear or raw signal representation. Furthermore, characteristic calf activity was identified that may relate to abnormal reflex activity. This has provided additional information with which to group characteristic muscle activity. An evaluation of the co-activation of gastrocnemius and tibialis anterior muscles using a sub-band filtering technique revealed three groups; those with distinct co-activation, those with little co-activation and those with continuous activity in the antagonistic pair across the stride. Signal features have been identified in sEMG recordings from stroke patients whilst walking extending current signal processing techniques. Common features of the sEMG and movement have been grouped creating a decision matrix. These results have contributed to the field of clinical measurement and diagnosis because interpretation of this decision matrix is related to underlying impairment. This has provided a framework from which subsequent studies can classify characteristic patterns of impairment within the stroke population; and thus assist in the provision of rehabilitative interventions

Guidage non-intrusif d'un bras robotique à l'aide d'un bracelet myoélectrique à électrode sèche

Depuis plusieurs années la robotique est vue comme une solution clef pour améliorer la qualité de vie des personnes ayant subi une amputation. Pour créer de nouvelles prothèses intelligentes qui peuvent être facilement intégrées à la vie quotidienne et acceptée par ces personnes, celles-ci doivent être non-intrusives, fiables et peu coûteuses. L’électromyographie de surface fournit une interface intuitive et non intrusive basée sur l’activité musculaire de l’utilisateur permettant d’interagir avec des robots. Cependant, malgré des recherches approfondies dans le domaine de la classification des signaux sEMG, les classificateurs actuels manquent toujours de fiabilité, car ils ne sont pas robustes face au bruit à court terme (par exemple, petit déplacement des électrodes, fatigue musculaire) ou à long terme (par exemple, changement de la masse musculaire et des tissus adipeux) et requiert donc de recalibrer le classifieur de façon périodique. L’objectif de mon projet de recherche est de proposer une interface myoélectrique humain-robot basé sur des algorithmes d’apprentissage par transfert et d’adaptation de domaine afin d’augmenter la fiabilité du système à long-terme, tout en minimisant l’intrusivité (au niveau du temps de préparation) de ce genre de système. L’aspect non intrusif est obtenu en utilisant un bracelet à électrode sèche possédant dix canaux. Ce bracelet (3DC Armband) est de notre (Docteur Gabriel Gagnon-Turcotte, mes co-directeurs et moi-même) conception et a été réalisé durant mon doctorat. À l’heure d’écrire ces lignes, le 3DC Armband est le bracelet sans fil pour l’enregistrement de signaux sEMG le plus performant disponible. Contrairement aux dispositifs utilisant des électrodes à base de gel qui nécessitent un rasage de l’avant-bras, un nettoyage de la zone de placement et l’application d’un gel conducteur avant l’utilisation, le brassard du 3DC peut simplement être placé sur l’avant-bras sans aucune préparation. Cependant, cette facilité d’utilisation entraîne une diminution de la qualité de l’information du signal. Cette diminution provient du fait que les électrodes sèches obtiennent un signal plus bruité que celle à base de gel. En outre, des méthodes invasives peuvent réduire les déplacements d’électrodes lors de l’utilisation, contrairement au brassard. Pour remédier à cette dégradation de l’information, le projet de recherche s’appuiera sur l’apprentissage profond, et plus précisément sur les réseaux convolutionels. Le projet de recherche a été divisé en trois phases. La première porte sur la conception d’un classifieur permettant la reconnaissance de gestes de la main en temps réel. La deuxième porte sur l’implémentation d’un algorithme d’apprentissage par transfert afin de pouvoir profiter des données provenant d’autres personnes, permettant ainsi d’améliorer la classification des mouvements de la main pour un nouvel individu tout en diminuant le temps de préparation nécessaire pour utiliser le système. La troisième phase consiste en l’élaboration et l’implémentation des algorithmes d’adaptation de domaine et d’apprentissage faiblement supervisé afin de créer un classifieur qui soit robuste au changement à long terme.For several years, robotics has been seen as a key solution to improve the quality of life of people living with upper-limb disabilities. To create new, smart prostheses that can easily be integrated into everyday life, they must be non-intrusive, reliable and inexpensive. Surface electromyography provides an intuitive interface based on a user’s muscle activity to interact with robots. However, despite extensive research in the field of sEMG signal classification, current classifiers still lack reliability due to their lack of robustness to short-term (e.g. small electrode displacement, muscle fatigue) or long-term (e.g. change in muscle mass and adipose tissue) noise. In practice, this mean that to be useful, classifier needs to be periodically re-calibrated, a time consuming process. The goal of my research project is to proposes a human-robot myoelectric interface based on transfer learning and domain adaptation algorithms to increase the reliability of the system in the long term, while at the same time reducing the intrusiveness (in terms of hardware and preparation time) of this kind of systems. The non-intrusive aspect is achieved from a dry-electrode armband featuring ten channels. This armband, named the 3DC Armband is from our (Dr. Gabriel Gagnon-Turcotte, my co-directors and myself) conception and was realized during my doctorate. At the time of writing, the 3DC Armband offers the best performance for currently available dry-electrodes, surface electromyographic armbands. Unlike gel-based electrodes which require intrusive skin preparation (i.e. shaving, cleaning the skin and applying conductive gel), the 3DC Armband can simply be placed on the forearm without any preparation. However, this ease of use results in a decrease in the quality of information. This decrease is due to the fact that the signal recorded by dry electrodes is inherently noisier than gel-based ones. In addition, other systems use invasive methods (intramuscular electromyography) to capture a cleaner signal and reduce the source of noises (e.g. electrode shift). To remedy this degradation of information resulting from the non-intrusiveness of the armband, this research project will rely on deep learning, and more specifically on convolutional networks. The research project was divided into three phases. The first is the design of a classifier allowing the recognition of hand gestures in real-time. The second is the implementation of a transfer learning algorithm to take advantage of the data recorded across multiple users, thereby improving the system’s accuracy, while decreasing the time required to use the system. The third phase is the development and implementation of a domain adaptation and self-supervised learning to enhance the classifier’s robustness to long-term changes

Biosignal‐based human–machine interfaces for assistance and rehabilitation : a survey

As a definition, Human–Machine Interface (HMI) enables a person to interact with a device. Starting from elementary equipment, the recent development of novel techniques and unobtrusive devices for biosignals monitoring paved the way for a new class of HMIs, which take such biosignals as inputs to control various applications. The current survey aims to review the large literature of the last two decades regarding biosignal‐based HMIs for assistance and rehabilitation to outline state‐of‐the‐art and identify emerging technologies and potential future research trends. PubMed and other databases were surveyed by using specific keywords. The found studies were further screened in three levels (title, abstract, full‐text), and eventually, 144 journal papers and 37 conference papers were included. Four macrocategories were considered to classify the different biosignals used for HMI control: biopotential, muscle mechanical motion, body motion, and their combinations (hybrid systems). The HMIs were also classified according to their target application by considering six categories: prosthetic control, robotic control, virtual reality control, gesture recognition, communication, and smart environment control. An ever‐growing number of publications has been observed over the last years. Most of the studies (about 67%) pertain to the assistive field, while 20% relate to rehabilitation and 13% to assistance and rehabilitation. A moderate increase can be observed in studies focusing on robotic control, prosthetic control, and gesture recognition in the last decade. In contrast, studies on the other targets experienced only a small increase. Biopotentials are no longer the leading control signals, and the use of muscle mechanical motion signals has experienced a considerable rise, especially in prosthetic control. Hybrid technologies are promising, as they could lead to higher performances. However, they also increase HMIs’ complex-ity, so their usefulness should be carefully evaluated for the specific application

Analysis of forearm muscles activity by means of new protocols of multichannel EMG signal recording and processing

Los movimientos voluntarios del cuerpo son controlados por el sistema nervioso central y periférico a través de la contracción de los músculos esqueléticos. La contracción se inicia al liberarse un neurotransmisor sobre la unión neuromuscular, iniciando la propagación de un biopotencial sobre la membrana de las fibras musculares que se desplaza hacia los tendones: el Potencial de Acción de la Unidad Motora (MUAP). La señal electromiográfica de superficie registra la activación continua de dichos potenciales sobre la superficie de la piel y constituye una valiosa herramienta para la investigación, diagnóstico y seguimiento clínico de trastornos musculares, así como para la identificación de la intención movimiento tanto en términos de dirección como de potencia. En el estudio de las enfermedades del sistema neuromuscular es necesario analizar el nivel de actividad, la capacidad de producción de fuerza, la activación muscular conjunta y la predisposición a la fatiga muscular, todos ellos asociados con factores fisiológicos que determinan la resultante contracción mioeléctrica. Además, el uso de matrices de electrodos facilita la investigación de las propiedades periféricas de las unidades motoras activas, las características anatómicas del músculo y los cambios espaciales en su activación, ocasionados por el tipo de tarea motora o la potencia de la misma. El objetivo principal de esta tesis es el diseño e implementación de protocolos experimentales y algoritmos de procesado para extraer información fiable de señales sEMG multicanal en 1 y 2 dimensiones del espacio. Dicha información ha sido interpretada y relacionada con dos patologías específicas de la extremidad superior: Epicondilitis Lateral y Lesión de Esfuerzo Repetitivo. También fue utilizada para identificar la dirección de movimiento y la fuerza asociada a la contracción muscular, cuyos patrones podrían ser de utilidad en aplicaciones donde la señal electromiográfica se utilice para controlar interfaces hombre-máquina como es el caso de terapia física basada en robots, entornos virtuales de rehabilitación o realimentación de la actividad muscular. En resumen, las aportaciones más relevantes de esta tesis son: * La definición de protocolos experimentales orientados al registro de señales sEMG en una región óptima del músculo. * Definición de índices asociados a la co-activación de diferentes músculos * Identificación de señales artefactuadas en registros multicanal * Selección de los canales mas relevantes para el análisis Extracción de un conjunto de características que permita una alta exactitud en la identificación de tareas motoras Los protocolos experimentales y los índices propuestos permitieron establecer que diversos desequilibrios entre músculos extrínsecos del antebrazo podrían desempeñar un papel clave en la fisiopatología de la epicondilitis lateral. Los resultados fueron consistentes en diferentes ejercicios y pueden definir un marco de evaluación para el seguimiento y evaluación de pacientes en programas de rehabilitación motora. Por otra parte, se encontró que las características asociadas con la distribución espacial de los MUAPs mejoran la exactitud en la identificación de la intención de movimiento. Lo que es más, las características extraídas de registros sEMG de alta densidad son más robustas que las extraídas de señales bipolares simples, no sólo por la redundancia de contacto implicada en HD-EMG, sino también porque permite monitorizar las regiones del músculo donde la amplitud de la señal es máxima y que varían con el tipo de ejercicio, permitiendo así una mejor estimación de la activación muscular mediante el análisis de los canales mas relevantes.Voluntary movements are achieved by the contraction of skeletal muscles controlled by the Central and Peripheral Nervous system. The contraction is initiated by the release of a neurotransmitter that promotes a reaction in the walls of the muscular fiber, producing a biopotential known as Motor Unit Action Potential (MUAP) that travels from the neuromuscular junction to the tendons. The surface electromyographic signal records the continuous activation of such potentials over the surface of the skin and constitutes a valuable tool for the diagnosis, monitoring and clinical research of muscular disorders as well as to infer motion intention not only regarding the direction of the movement but also its power. In the study of diseases of the neuromuscular system it is necessary to analyze the level of activity, the capacity of production of strength, the load-sharing between muscles and the probably predisposition to muscular fatigue, all of them associated with physiological factors determining the resultant muscular contraction. Moreover, the use of electrode arrays facilitate the investigation of the peripheral properties of the active Motor Units, the anatomical characteristics of the muscle and the spatial changes induced in their activation of as product of type of movement or power of the contraction.The main objective of this thesis was the design and implementation of experimental protocols, and algorithms to extract information from multichannel sEMG signals in 1 and 2 dimensions of the space. Such information was interpreted and related to pathological events associated to two upper-limb conditions: Lateral Epicondylitis and Repetitive Strain Injury. It was also used to identify the direction of movement and contraction strength which could be useful in applications concerning the use of biofeedback from EMG like in robotic- aided therapies and computer-based rehabilitation training.In summary, the most relevant contributions are:§The definition of experimental protocols intended to find optimal regions for the recording of sEMG signals. §The definition of indices associated to the co- activation of different muscles. §The detection of low-quality signals in multichannel sEMG recordings.§ The selection of the most relevant EMG channels for the analysis§The extraction of a set of features that led to high classification accuracy in the identification of tasks.The experimental protocols and the proposed indices allowed establishing that imbalances between extrinsic muscles of the forearm could play a key role in the pathophysiology of lateral epicondylalgia. Results were consistent in different types of motor task and may define an assessment framework for the monitoring and evaluation of patients during rehabilitation programs.On the other hand, it was found that features associated with the spatial distribution of the MUAPs improve the accuracy of the identification of motion intention. What is more, features extracted from high density EMG recordings are more robust not only because it implies contact redundancy but also because it allows the tracking of (task changing) skin surface areas where EMG amplitude is maximal and a better estimation of muscle activity by the proper selection of the most significant channels

On the Utility of Representation Learning Algorithms for Myoelectric Interfacing

Electrical activity produced by muscles during voluntary movement is a reflection of the firing patterns of relevant motor neurons and, by extension, the latent motor intent driving the movement. Once transduced via electromyography (EMG) and converted into digital form, this activity can be processed to provide an estimate of the original motor intent and is as such a feasible basis for non-invasive efferent neural interfacing. EMG-based motor intent decoding has so far received the most attention in the field of upper-limb prosthetics, where alternative means of interfacing are scarce and the utility of better control apparent. Whereas myoelectric prostheses have been available since the 1960s, available EMG control interfaces still lag behind the mechanical capabilities of the artificial limbs they are intended to steer—a gap at least partially due to limitations in current methods for translating EMG into appropriate motion commands. As the relationship between EMG signals and concurrent effector kinematics is highly non-linear and apparently stochastic, finding ways to accurately extract and combine relevant information from across electrode sites is still an active area of inquiry.This dissertation comprises an introduction and eight papers that explore issues afflicting the status quo of myoelectric decoding and possible solutions, all related through their use of learning algorithms and deep Artificial Neural Network (ANN) models. Paper I presents a Convolutional Neural Network (CNN) for multi-label movement decoding of high-density surface EMG (HD-sEMG) signals. Inspired by the successful use of CNNs in Paper I and the work of others, Paper II presents a method for automatic design of CNN architectures for use in myocontrol. Paper III introduces an ANN architecture with an appertaining training framework from which simultaneous and proportional control emerges. Paper Iv introduce a dataset of HD-sEMG signals for use with learning algorithms. Paper v applies a Recurrent Neural Network (RNN) model to decode finger forces from intramuscular EMG. Paper vI introduces a Transformer model for myoelectric interfacing that do not need additional training data to function with previously unseen users. Paper vII compares the performance of a Long Short-Term Memory (LSTM) network to that of classical pattern recognition algorithms. Lastly, paper vIII describes a framework for synthesizing EMG from multi-articulate gestures intended to reduce training burden