Decoding motor neuron behavior for advanced control of upper limb prostheses

One of the main challenges in upper limb prosthesis control to date is to provide devices intuitive to use and capable to reproduce the natural movements of the arm and hand. One approach to solve this challenge is to use the same control signals for prosthesis control that our nervous system uses to control its muscles. This thesis aims to investigate the possibility of natural, intuitive prosthesis control using neural information obtained with available surface EMG decomposition methods. In order to explore all aspects of such a novel approach, a series of five studies were performed with the final goal of implementing a proof of concept and comparing its performance with state of the art myoelectric control. The performed investigations revealed important insights in motor unit physiology after targeted muscle reinnervation, EMG decomposition in dynamic voluntary contractions of the forearm, and the properties and challenges of neural information based prosthesis control. The main outcome of the thesis is that neural information based prosthesis control is capable to outperform myoelectric approaches in pattern recognition, linear regression and nonlinear regression, as determined by offline performance comparisons. The final proof of concept for this novel approach was a robust regression method based on neuromusculoskeletal modeling. The kinematics estimation of the proposed approach outperformed EMG-based nonlinear regression in both able-bodied subjects and patients with limb deficiency, indicating that using neural information is a promising avenue for advanced myoelectric control.2017-11-3

Virtual sensor of surface electromyography in a new extensive fault-tolerant classification system

A few prosthetic control systems in the scientific literature obtain pattern recognition algorithms adapted to changes that occur in the myoelectric signal over time and, frequently, such systems are not natural and intuitive. These are some of the several challenges for myoelectric prostheses for everyday use. The concept of the virtual sensor, which has as its fundamental objective to estimate unavailable measures based on other available measures, is being used in other fields of research. The virtual sensor technique applied to surface electromyography can help to minimize these problems, typically related to the degradation of the myoelectric signal that usually leads to a decrease in the classification accuracy of the movements characterized by computational intelligent systems. This paper presents a virtual sensor in a new extensive fault-tolerant classification system to maintain the classification accuracy after the occurrence of the following contaminants: ECG interference, electrode displacement, movement artifacts, power line interference, and saturation. The Time-Varying Autoregressive Moving Average (TVARMA) and Time-Varying Kalman filter (TVK) models are compared to define the most robust model for the virtual sensor. Results of movement classification were presented comparing the usual classification techniques with the method of the degraded signal replacement and classifier retraining The experimental results were evaluated for these five noise types in 16 surface electromyography (sEMG) channel degradation case studies. The proposed system without using classifier retraining techniques recovered of mean classification accuracy was of 4% to 38% for electrode displacement, movement artifacts, and saturation noise. The best mean classification considering all signal contaminants and channel combinations evaluated was the classification using the retraining method, replacing the degraded channel by the virtual sensor TVARMA model. This method recovered the classification accuracy after the degradations, reaching an average of 5.7% below the classification of the clean signal, that is the signal without the contaminants or the original signal. Moreover, the proposed intelligent technique minimizes the impact of the motion classification caused by signal contamination related to degrading events over time. There are improvements in the virtual sensor model and in the algorithm optimization that need further development to provide an increase the clinical application of myoelectric prostheses but already presents robust results to enable research with virtual sensors on biological signs with stochastic behavior

On the Utility of Representation Learning Algorithms for Myoelectric Interfacing

Electrical activity produced by muscles during voluntary movement is a reflection of the firing patterns of relevant motor neurons and, by extension, the latent motor intent driving the movement. Once transduced via electromyography (EMG) and converted into digital form, this activity can be processed to provide an estimate of the original motor intent and is as such a feasible basis for non-invasive efferent neural interfacing. EMG-based motor intent decoding has so far received the most attention in the field of upper-limb prosthetics, where alternative means of interfacing are scarce and the utility of better control apparent. Whereas myoelectric prostheses have been available since the 1960s, available EMG control interfaces still lag behind the mechanical capabilities of the artificial limbs they are intended to steer—a gap at least partially due to limitations in current methods for translating EMG into appropriate motion commands. As the relationship between EMG signals and concurrent effector kinematics is highly non-linear and apparently stochastic, finding ways to accurately extract and combine relevant information from across electrode sites is still an active area of inquiry.This dissertation comprises an introduction and eight papers that explore issues afflicting the status quo of myoelectric decoding and possible solutions, all related through their use of learning algorithms and deep Artificial Neural Network (ANN) models. Paper I presents a Convolutional Neural Network (CNN) for multi-label movement decoding of high-density surface EMG (HD-sEMG) signals. Inspired by the successful use of CNNs in Paper I and the work of others, Paper II presents a method for automatic design of CNN architectures for use in myocontrol. Paper III introduces an ANN architecture with an appertaining training framework from which simultaneous and proportional control emerges. Paper Iv introduce a dataset of HD-sEMG signals for use with learning algorithms. Paper v applies a Recurrent Neural Network (RNN) model to decode finger forces from intramuscular EMG. Paper vI introduces a Transformer model for myoelectric interfacing that do not need additional training data to function with previously unseen users. Paper vII compares the performance of a Long Short-Term Memory (LSTM) network to that of classical pattern recognition algorithms. Lastly, paper vIII describes a framework for synthesizing EMG from multi-articulate gestures intended to reduce training burden

Bio-signal based control in assistive robots: a survey

Recently, bio-signal based control has been gradually deployed in biomedical devices and assistive robots for improving the quality of life of disabled and elderly people, among which electromyography (EMG) and electroencephalography (EEG) bio-signals are being used widely. This paper reviews the deployment of these bio-signals in the state of art of control systems. The main aim of this paper is to describe the techniques used for (i) collecting EMG and EEG signals and diving these signals into segments (data acquisition and data segmentation stage), (ii) dividing the important data and removing redundant data from the EMG and EEG segments (feature extraction stage), and (iii) identifying categories from the relevant data obtained in the previous stage (classification stage). Furthermore, this paper presents a summary of applications controlled through these two bio-signals and some research challenges in the creation of these control systems. Finally, a brief conclusion is summarized

The effect of accelerometer location on the classification of single-site forearm mechanomyograms

<p>Abstract</p> <p>Background</p> <p>Recently, pattern recognition methods have been deployed in the classification of multiple activation states from mechanomyogram (MMG) signals for the purpose of controlling switching interfaces. Given the propagative properties of MMG signals, it has been suggested that MMG classification should be robust to changes in sensor placement. Nonetheless, this purported robustness remains speculative to date. This study sought to quantify the change in classification accuracy, if any, when a classifier trained with MMG signals from the muscle belly, is subsequently tested with MMG signals from a nearby location.</p> <p>Methods</p> <p>An arrangement of 5 accelerometers was attached to the flexor carpi radialis muscle of 12 able-bodied participants; a reference accelerometer was located over the muscle belly, two peripheral accelerometers were positioned along the muscle's transverse axis and two more were aligned to the muscle's longitudinal axis. Participants performed three classes of muscle activity: wrist flexion, wrist extension and semi-pronation. A collection of time, frequency and time-frequency features were considered and reduced by genetic feature selection. The classifier, trained using features from the reference accelerometer, was tested with signals from the longitudinally and transversally displaced accelerometers.</p> <p>Results</p> <p>Classification degradation due to accelerometer displacement was significant for all participants, and showed no consistent trend with the direction of displacement. Further, the displaced accelerometer signals showed task-dependent de-correlations with respect to the reference accelerometer.</p> <p>Conclusions</p> <p>These results indicate that MMG signal features vary with spatial location and that accelerometer displacements of only 1-2 cm cause sufficient feature drift to significantly diminish classification accuracy. This finding emphasizes the importance of consistent sensor placement between MMG classifier training and deployment for accurate control of switching interfaces.</p

Deep learning and feature engineering techniques applied to the myoelectric signal for accurate prediction of movements

Técnicas de reconhecimento de padrões no Sinal Mioelétrico (EMG) são empregadas no desenvolvimento de próteses robóticas, e para isso, adotam diversas abordagens de Inteligência Artificial (IA). Esta Tese se propõe a resolver o problema de reconhecimento de padrões EMG através da adoção de técnicas de aprendizado profundo de forma otimizada. Para isso, desenvolveu uma abordagem que realiza a extração da característica a priori, para alimentar os classificadores que supostamente não necessitam dessa etapa. O estudo integrou a plataforma BioPatRec (estudo e desenvolvimento avançado de próteses) a dois algoritmos de classificação (Convolutional Neural Network e Long Short-Term Memory) de forma híbrida, onde a entrada fornecida à rede já possui características que descrevem o movimento (nível de ativação muscular, magnitude, amplitude, potência e outros). Assim, o sinal é rastreado como uma série temporal ao invés de uma imagem, o que nos permite eliminar um conjunto de pontos irrelevantes para o classificador, tornando a informação expressivas. Na sequência, a metodologia desenvolveu um software que implementa o conceito introduzido utilizando uma Unidade de Processamento Gráfico (GPU) de modo paralelo, esse incremento permitiu que o modelo de classificação aliasse alta precisão com um tempo de treinamento inferior a 1 segundo. O modelo paralelizado foi chamado de BioPatRec-Py e empregou algumas técnicas de Engenharia de Features que conseguiram tornar a entrada da rede mais homogênea, reduzindo a variabilidade, o ruído e uniformizando a distribuição. A pesquisa obteve resultados satisfatórios e superou os demais algoritmos de classificação na maioria dos experimentos avaliados. O trabalho também realizou uma análise estatística dos resultados e fez o ajuste fino dos hiper-parâmetros de cada uma das redes. Em última instancia, o BioPatRec-Py forneceu um modelo genérico. A rede foi treinada globalmente entre os indivíduos, permitindo a criação de uma abordagem global, com uma precisão média de 97,83%.Pattern recognition techniques in the Myoelectric Signal (EMG) are employed in the development of robotic prostheses, and for that, they adopt several approaches of Artificial Intelligence (AI). This Thesis proposes to solve the problem of recognition of EMG standards through the adoption of profound learning techniques in an optimized way. The research developed an approach that extracts the characteristic a priori to feed the classifiers that supposedly do not need this step. The study integrated the BioPatRec platform (advanced prosthesis study and development) to two classification algorithms (Convolutional Neural Network and Long Short-Term Memory) in a hybrid way, where the input provided to the network already has characteristics that describe the movement (level of muscle activation, magnitude, amplitude, power, and others). Thus, the signal is tracked as a time series instead of an image, which allows us to eliminate a set of points irrelevant to the classifier, making the information expressive. In the sequence, the methodology developed software that implements the concept introduced using a Graphical Processing Unit (GPU) in parallel this increment allowed the classification model to combine high precision with a training time of less than 1 second. The parallel model was called BioPatRec-Py and employed some Engineering techniques of Features that managed to make the network entry more homogeneous, reducing variability, noise, and standardizing distribution. The research obtained satisfactory results and surpassed the other classification algorithms in most of the evaluated experiments. The work performed a statistical analysis of the outcomes and fine-tuned the hyperparameters of each of the networks. Ultimately, BioPatRec-Py provided a generic model. The network was trained globally between individuals, allowing the creation of a standardized approach, with an average accuracy of 97.83%

Causes of Performance Degradation in Non-invasive Electromyographic Pattern Recognition in Upper Limb Prostheses

Surface Electromyography (EMG)-based pattern recognition methods have been investigated over the past years as a means of controlling upper limb prostheses. Despite the very good reported performance of myoelectric controlled prosthetic hands in lab conditions, real-time performance in everyday life conditions is not as robust and reliable, explaining the limited clinical use of pattern recognition control. The main reason behind the instability of myoelectric pattern recognition control is that EMG signals are non-stationary in real-life environments and present a lot of variability over time and across subjects, hence affecting the system's performance. This can be the result of one or many combined changes, such as muscle fatigue, electrode displacement, difference in arm posture, user adaptation on the device over time and inter-subject singularity. In this paper an extensive literature review is performed to present the causes of the drift of EMG signals, ways of detecting them and possible techniques to counteract for their effects in the application of upper limb prostheses. The suggested techniques are organized in a table that can be used to recognize possible problems in the clinical application of EMG-based pattern recognition methods for upper limb prosthesis applications and state-of-the-art methods to deal with such problems