PaWFE ::fast signal feature extraction using parallel time windows

Motivation: Hand amputations can dramatically affect the quality of life of a person. Researchers are developing surface electromyography and machine learning solutions to control dexterous and robotic prosthetic hands, however long computational times can slow down this process. Objective: This paper aims at creating a fast signal feature extraction algorithm that can extract widely used features and allow researchers to easily add new ones. Methods: PaWFE (Parallel Window Feature Extractor) extracts the signal features from several time windows in parallel. The MATLAB code is publicly available and supports several time domain and frequency features. The code was tested and benchmarked using 1,2,4,8,16,32, and 48 threads on a server with four Xeon E7- 4820 and 128 GB RAM using the first 5 datasets of the Ninapro database, that are recorded with different acquisition setups. Results: The parallel time window analysis approach allows to reduce the computational time up to 20 times when using 32 cores, showing a very good scalability. Signal features can be extracted in few seconds from an entire data acquisition and in <100ms from a single time window, easily reducing of up to over 15 times the feature extraction procedure in comparison to traditional approaches. The code allows users to easily add new signal feature extraction scripts, that can be added to the code and on the Ninapro website upon request. Significance: The code allows researchers in machine learning and biosignals data analysis to easily and quickly test modern machine learning approaches on big datasets and it can be used as a resource for real time data analysis too

putEMG -- a surface electromyography hand gesture recognition dataset

In this paper, we present a putEMG dataset intended for evaluation of hand gesture recognition methods based on sEMG signal. The dataset was acquired for 44 able-bodied subjects and include 8 gestures (3 full hand gestures, 4 pinches, and idle). It consists of uninterrupted recordings of 24 sEMG channels from the subject's forearm, RGB video stream and depth camera images used for hand motion tracking. Moreover, exemplary processing scripts are also published. putEMG dataset is available under Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) license at: https://www.biolab.put.poznan.pl/putemg-dataset/. The dataset was validated regarding sEMG amplitudes and gesture recognition performance. The classification was performed using state-of-the-art classifiers and feature sets. Accuracy of 90% was achieved for SVM classifier utilising RMS feature and for LDA classifier using Hudgin's and Du's feature sets. Analysis of performance for particular gestures showed that LDA/Du combination has significantly higher accuracy for full hand gestures, while SVM/RMS performs better for pinch gestures. Presented dataset can be used as a benchmark for various classification methods, evaluation of electrode localisation concepts, or development of classification methods invariant to user-specific features or electrode displacement

From Unimodal to Multimodal: improving the sEMG-Based Pattern Recognition via deep generative models

Multimodal hand gesture recognition (HGR) systems can achieve higher recognition accuracy. However, acquiring multimodal gesture recognition data typically requires users to wear additional sensors, thereby increasing hardware costs. This paper proposes a novel generative approach to improve Surface Electromyography (sEMG)-based HGR accuracy via virtual Inertial Measurement Unit (IMU) signals. Specifically, we trained a deep generative model based on the intrinsic correlation between forearm sEMG signals and forearm IMU signals to generate virtual forearm IMU signals from the input forearm sEMG signals at first. Subsequently, the sEMG signals and virtual IMU signals were fed into a multimodal Convolutional Neural Network (CNN) model for gesture recognition. To evaluate the performance of the proposed approach, we conducted experiments on 6 databases, including 5 publicly available databases and our collected database comprising 28 subjects performing 38 gestures, containing both sEMG and IMU data. The results show that our proposed approach outperforms the sEMG-based unimodal HGR method (with increases of 2.15%-13.10%). It demonstrates that incorporating virtual IMU signals, generated by deep generative models, can significantly enhance the accuracy of sEMG-based HGR. The proposed approach represents a successful attempt to transition from unimodal HGR to multimodal HGR without additional sensor hardware

Understanding Forearm Muscle Coordination in Children

A combination of surface electromyography (EMG) and pattern recognition algorithms have led to improvements in the functionality of upper limb prosthetics. This method of control relies on user\u27s ability to repeatedly generate consistent muscle contractions. Research in EMG based control of prosthesis has mainly utilized adult subjects who have fully developed neuromuscular control. Little is known about children\u27s ability to generate consistent EMG signals necessary to control artificial limbs with multiple degrees of freedom. To address this gap, two experiments were designed to validate and benchmark an experimental protocol that quantifies the ability to coordinate forearm muscle contractions in able-bodied children across adolescent ages. Able-bodied, healthy adults (n = 8) and children (n = 9) participated in the first experiment that aimed to measure the subject\u27s ability to produce distinguishable EMG signals. Each subject performed 8 repetitions of 16 different hand/wrist movements. We quantify the number of movement types that can be classified by Support Vector Machine with \u3e 90% accuracy. Additional adults (n=8) and children (n=12) were recruited for the second experiment which measured the subjects\u27 ability to control the position of a virtual cursor on a 1-DoF slide using proportional EMG control under three different gain levels. We demonstrated that children had a smaller number of highly independent movements than adults did, due to higher variability. Furthermore, we found that children had higher failure rates and slower average target acquisitions due to increased time-to-target and follow-up correction time. We also found significant correlation between forearm circumference/age and performance. The results of this study provide novel insights into the technical and empirical basis to better understand neuromuscular development in pediatric upper-limb amputees

Repeatability of grasp recognition for robotic hand prosthesis control based on sEMG data

none6Control methods based on sEMG obtained promising results for hand prosthetics. Control system robustness is still often inadequate and does not allow the amputees to perform a large number of movements useful for everyday life. Only few studies analyzed the repeatability of sEMG classification of hand grasps. The main goals of this paper are to explore repeatability in sEMG data and to release a repeatability database with the recorded experiments. The data are recorded from 10 intact subjects repeating 7 grasps 12 times, twice a day for 5 days. The data are publicly available on the Ninapro web page. The analysis for the repeatability is based on the comparison of movement classification accuracy in several data acquisitions and for different subjects. The analysis is performed using mean absolute value and waveform length features and a Random Forest classifier. The accuracy obtained by training and testing on acquisitions at different times is on average 27.03% lower than training and testing on the same acquisition. The results obtained by training and testing on different acquisitions suggest that previous acquisitions can be used to train the classification algorithms. The inter-subject variability is remarkable, suggesting that specific characteristics of the subjects can affect repeatibility and sEMG classification accuracy. In conclusion, the results of this paper can contribute to develop more robust control systems for hand prostheses, while the presented data allows researchers to test repeatability in further analyses.nonePalermo F.; Cognolato M.; Gijsberts A.; Muller H.; Caputo B.; Atzori M.Palermo, F.; Cognolato, M.; Gijsberts, A.; Muller, H.; Caputo, B.; Atzori, M

Desenvolvimento de metodologia baseada em aprendizado por reforço e Sistema de Inferência Fuzzy para identificação e minimização de contaminantes em sinais de sEMG com aplicação em identificação de movimentos do segmento mão-braço

A incessante busca por novas tecnologias que proporcionem aumento da qualidade de vida do ser humano tem norteado a pesquisa acadêmica ao longo da história. Isso é observado na evolução dos meios de transporte, dos dispositivos de comunicação e até mesmo de serviços como o bancário. No entanto, para pessoas com deficiência motora, em especial aquelas que sofreram amputação ou não possuem parte do membro superior, a conquista de melhores condições de vida está potencialmente relacionada com liberdade e independência. Visando suprir esta necessidade, muitos pesquisadores têm trabalhado no desenvolvimento de algoritmos preditores de movimento do segmento mão-braço a partir de sinais de eletromiografia para o controle de próteses na expectativa de aumentar o número de graus de liberdade do dispositivo. Contudo, para que se obtenha sistemas eficientes e que tenham elevados índices de assertividade, é imprescindível que o nível de interferência e ruído, os quais inevitavelmente estão presentes nos registros de eletromiografia devido à instrumentação, ambiente, aspectos fisiológicos, dentre outros, seja o menor possível. Neste contexto, alguns trabalhos foram desenvolvidos visando a minimização do efeito de interferências no classificador, contudo todos aqueles abrangidos pela pesquisa realizada demandam um estágio de treinamento off-line, não são adaptáveis às variações do sinal de EMG e/ou dependem do sinal dos outros canais de medição para a minimização do efeito degradador. Diante disso, a presente proposta de tese apresenta uma metodologia baseada em aprendizagem por reforço (Reinforcement Learning) e Sistema de Inferência Fuzzy para detecção, identificação do tipo e atenuação do efeito de contaminantes em registros de eletromiografia, com aplicação em sistemas de reconhecimento de gestos do membro superior. O mesmo está fundamentado em um modelo de agente e ambiente, sendo constituído dos seguintes elementos: ambiente (atividade elétrica muscular), estado (conjunto de 6 características extraídas do sinal de EMG), ações (aplicação de filtros/procedimentos específicos para a redução do impacto de cada interferência) e agente (controlador que fará a identificação do tipo da contaminação e executará a ação adequada). Para cada ação exercida pelo agente será atribuída uma recompensa a qual, por sua vez, é determinada em virtude do impacto da primeira nas características do sinal (estado) por meio de um Sistema de Inferência Fuzzy. O treinamento, realizado através do método Ator-Crítico, consiste na obtenção de uma política de ações que maximize a recompensa percebida a longo prazo. Por meio de um experimento realizado de forma off-line conseguiu-se taxas de acerto de 92,96% na identificação de 4 tipos de contaminantes (interferência por eletrocardiografia (ECG), artefato de movimento, interferência eletromagnética oriunda da rede de energia elétrica e ruído branco gaussiano) e 69,5% quando se considerou também sinal íntegro. Além disso, por meio de um estudo de caso simulando-se o treinamento online do agente evidenciou-se que o modelo de Transfer Learning adotado foi eficaz na dispensa da necessidade do uso de dados adquiridos previamente do usuário além de acelerar o processo de aprendizado. Estas propriedades são fundamentais para a implementação de qualquer sistema de forma online. Logo, verificou-se indícios de que o SIF-ACRL tem, de fato, potencial para ser implementado de forma online.The incessant search for new technologies that provide increased quality of life for human beings has guided academic research throughout history. This is observed in the evolution of transports, communication devices and even services such as banking. However, for people with motor disabilities, especially those who have had an amputation or do not have part of the upper limb, achieving better living conditions is potentially related to freedom and independence. To meet this need, many researchers have been working on the development of hand-arm segment movement predictors algorithms from electromyography signals for the control of prostheses in the hope of increasing the device's degrees of freedom. However, to obtain efficient systems that have high levels of assertiveness, it is essential that the interference and noise level, which are inevitably present in the electromyography records due to the instrumentation, environment, physiological aspects, among others, is the lowest possible. In this context, some works were developed aiming at minimizing the effect of interference in the classifier, however, all those covered by the performed research demand an offline training stage, are not adaptable to the EMG signal variations, and/or depend on the signal of others measurement channels to minimize the degrading effect. In view of this, the present thesis proposal presents a methodology based on Reinforcement Learning and Fuzzy Inference System for detection, identification of the type and mitigation of the effect of contaminants in electromyography records, with application in gesture recognition systems of the upper limb. It is based on an agent and environment model, consisting of the following elements: environment (muscle electrical activity), state (set of 6 characteristics extracted from the EMG signal), actions (application of specific filters/procedures to reduce impact of each interference) and agent (controller who will identify the type of contamination and take the appropriate action). For each action performed by the agent, a reward will be attributed which, in turn, is determined by the impact of the actions on the signal features (state) by means of a Fuzzy Inference System. The training, carried out through the Actor-Critic method, consists of obtaining an action policy that maximizes the long term perceived reward. Through an experiment carried out offline, success rates of 92.96% were achieved in the identification of 4 types of contaminants (interference by electrocardiography (ECG), motion artifact, electromagnetic interference from the electricity network and Gaussian white noise) and 69.5% when a clean signal class was added. In addition, a case study simulating the agent's online training showed that the Transfer Learning model adopted was effective in dispensing with the need to use data previously acquired from the user, in addition to accelerating the learning process. These properties are fundamental for the implementation of any system online. Therefore, there were indications that the SIF-ACRL has the potential to be implemented online

Métodos de classificação confiável e resiliente de movimentos de membros superiores baseado em extreme learning machines e sinais de eletromiografia de superfície

Apesar de avanços recentes, a classificação confiável de sinais de eletromiografia de superfície (sEMG) permanece uma tarefa árdua sob a perspectiva de Aprendizagem de Máquina. Sinais de sEMG possuem uma sobreposição de classes inerente à sua natureza, o que impede a separação perfeita das amostras e produz ruídos de classificação. Alternativas ao problema geralmente baseiam-se na filtragem do sEMG ou métodos de pós-processamento como o Major-Voting, soluções estas que necessariamente geram atrasos na classificação do sinal e frequentemente não geram melhoras substanciais. A abordagem deste trabalho baseia-se no desenvolvimento de métodos confiáveis e resilientes sob a perspectiva de classificação que gerem saídas mais estáveis e consistentes para o classificador baseado em Extreme Learning Machines (ELM) utilizado. Para tanto, métodos envolvendo o pré-processamento e pós-processamento, a suavização do arg max do classificador, thresholds adaptativos e um classificador binário auxiliar foram utilizados. Os sinais classificados derivam de 12 canais de sEMG envolvendo três bases de dados diferentes onde 99 ensaios compostos pela execução de 17 movimentos distintos do segmento mão-braço foram realizados. Nos melhores resultados, os métodos utilizados atingiram taxas de acerto médio global de 66,99 ± 23,6% para a base de voluntários amputados, 87,10 ± 5,89% para a base de voluntários não-amputados e taxas superiores a 99% para todas as variações de diferentes ensaios que compõe a base de dados adquirida em laboratório. Já para a taxa de acerto média ponderada por classes, nos melhores resultados foram de 53,36 ± 18,2% para a base de voluntários amputados, 77,94 ± 6,22% para a base de voluntários não-amputados e taxas superiores a 91% para os ensaios da base de dados adquirida em laboratório. Ambas as métricas de taxa de acerto consideradas superam ou equivalem-se a alternativas descritas na literatura, utilizando abordagens que não demandam grandes mudanças estruturais no classificador.Despite recent advances, reliable classification of surface electromyography (sEMG) signals remains an arduous task from the perspective of Machine Learning. sEMG signals have inherent class overlaps that prevent optimal labeling due to classification noises. Alternatives to classification ripples usually rely on stochastic sEMG filtering or post-processing methods, like Major-Voting, both solutions that insert constraints and additional delays in signal classification and often do not generate substantial improvements. The approach of this paper focuses on the development of reliable and resilient methods used in combination with an Extreme Learning Machines (ELM) classifier to generate more stable and consistent outputs. Methods of pre-processing and post-processing, a smoothed arg max version of the ELM, adaptive thresholds, and an auxiliary binary classifier were used to process signals derived from 12 EMG channels from three different databases. In total, 99 trials were performed, each one containing 17 different upper-limb movements. The proposed methods reached an average overall accuracy rate of 66.99 ± 23.6% for the amputee individuals’ database, 87.10 ± 5.89% for the non-amputee individuals’ database, and rates over 99% for all variations of our own lab-generated database. The average weighted accuracy rates were 53.36 ± 18.2% for the amputee individuals’ database, 77.94 ± 6.22% for the base of the non-amputee individuals’ database, and higher than 91% for the best-case scenario of our own lab-generated database. In both metrics considered, the results outperform, or match alternatives described in the literature using approaches that do not require significant changes in the classifier's architecture