Estimation and Early Prediction of Grip Force Based on sEMG Signals and Deep Recurrent Neural Networks

Hands are used for communicating with the surrounding environment and have a complex structure that enables them to perform various tasks with their multiple degrees of freedom. Hand amputation can prevent a person from performing their daily activities. In that event, finding a suitable, fast, and reliable alternative for the missing limb can affect the lives of people who suffer from such conditions. As the most important use of the hands is to grasp objects, the purpose of this study is to accurately predict gripping force from surface electromyography (sEMG) signals during a pinch-type grip. In that regard, gripping force and sEMG signals are derived from 10 healthy subjects. Results show that for this task, recurrent networks outperform nonrecurrent ones, such as a fully connected multilayer perceptron (MLP) network. Gated recurrent unit (GRU) and long short-term memory (LSTM) networks can predict the gripping force with R-squared values of 0.994 and 0.992, respectively, and a prediction rate of over 1300 predictions per second. The predominant advantage of using such frameworks is that the gripping force can be predicted straight from preprocessed sEMG signals without any form of feature extraction, not to mention the ability to predict future force values using larger prediction horizons adequately. The methods presented in this study can be used in the myoelectric control of prosthetic hands or robotic grippers.Comment: 9 pages, accepted for publication in journal of the Brazilian Society of Mechanical Sciences and Engineerin

EEG-EMG Analysis Method in Hybrid Brain Computer Interface for Hand Rehabilitation Training

Brain-computer interfaces (BCIs) have demonstrated immense potential in aiding stroke patients during their physical rehabilitation journey. By reshaping the neural circuits connecting the patient’s brain and limbs, these interfaces contribute to the restoration of motor functions, ultimately leading to a significant improvement in the patient’s overall quality of life. However, the current BCI primarily relies on Electroencephalogram (EEG) motor imagery (MI), which has relatively coarse recognition granularity and struggles to accurately recognize specific hand movements. To address this limitation, this paper proposes a hybrid BCI framework based on Electroencephalogram and Electromyography (EEG-EMG). The framework utilizes a combination of techniques: decoding EEG by using Graph Convolutional LSTM Networks (GCN-LSTM) to recognize the subject’s motion intention, and decoding EMG by using a convolutional neural network (CNN) to accurately identify hand movements. In EEG decoding, the correlation between channels is calculated using Standardized Permutation Mutual Information (SPMI), and the decoding process is further explained by analyzing the correlation matrix. In EMG decoding, experiments are conducted on two task paradigms, both achieving promising results. The proposed framework is validated using the publicly available WAL-EEG-GAL (Wearable interfaces for hand function recovery Electroencephalography Grasp-And-Lift) dataset, where the average classification accuracies of EEG and EMG are 0.892 and 0.954, respectively. This research aims to establish an efficient and user-friendly EEG-EMG hybrid BCI, thereby facilitating the hand rehabilitation training of stroke patients

Wearable fusion system for assessment of motor function in lesion-symptom mapping studies

Lesion-symptom mapping studies are a critical component of addressing the relationship between brain and behaviour. Recent developments have yielded significant improvements in the imaging and detection of lesion profiles, but the quantification of motor outcomes is still largely performed by subjective and low-resolution standard clinical rating scales. This mismatch means than lesion-symptom mapping studies are limited in scope by scores which lack the necessary accuracy to fully quantify the subcomponents of motor function. The first study conducted aimed to develop a new automated system of motor function which addressed the limitations inherent in the clinical rating scales. A wearable fusion system was designed that included the attachment of inertial sensors to record the kinematics of upper extremity. This was combined with the novel application of mechanomyographic sensors in this field, to enable the quantification of hand/wrist function. Novel outputs were developed for this system which aimed to combine the validity of the clinical rating scales with the high accuracy of measurements possible with a wearable sensor system. This was achieved by the development of a sophisticated classification model which was trained on series of kinematic and myographic measures to classify the clinical rating scale. These classified scores were combined with a series of fine-grained clinical features derived from higher-order sensor metrics. The developed automated system graded the upper-extremity tasks of the Fugl-Meyer Assessment with a mean accuracy of 75\% for gross motor tasks and 66\% for the wrist/hand tasks. This accuracy increased to 85\% and 74\% when distinguishing between healthy and impaired function for each of these tasks. Several clinical features were computed to describe the subcomponents of upper extremity motor function. This fine-grained clinical feature set offers a novel means to complement the low resolution but well-validated standardised clinical rating scales. A second study was performed to utilise the fine-grained clinical feature set calculated in the previous study in a large-scale region-of-interest lesion-symptom mapping study. Statistically significant regions of motor dysfunction were found in the corticospinal tract and the internal capsule, which are consistent with other motor-based lesion-symptom mapping studies. In addition, the cortico-ponto-cerebellar tract was found to be statistically significant when testing with a clinical feature of hand/wrist motor function. This is a novel finding, potentially due to prior studies being limited to quantifying this subcomponent of motor function using standard clinical rating scales. These results indicate the validity and potential of the clinical feature set to provide a more detailed picture of motor dysfunction in lesion-symptom mapping studies.Open Acces

Prosthetic hand control: phase-based grasping pattern recognition using sEMG and computer vision

Pattern recognition using surface Electromyography (sEMG) applied on prosthesis control has attracted much attention. The strong relationship between visual perception and hand manipulation makes vision play an essential role in prosthetic hand control. Utilizing both sEMG and visual information to improve prosthetic hand control became a promising research direction. In most existing hand grasping classification research using sEMG, the signals collected during the firmly grasped period were used for classification because stable signals facilitated classification performance. However, using signals collected from the firm grasp period may cause a delay in controlling the prosthetic hand. Targeting this issue, we explored a new way for grasp classification using signals collected before the firm grasp. We examined accuracy changes during the reaching and grasping process and identified an sEMG sweet period, starts at 1100 ms and ends at 1400 ms in the early grasping phase, that can leverage the grasp classi- fication accuracy for the earlier grasp detection. Although Surface Electromyography (sEMG) achieved a feasible solution in a laboratory environment, the classification accuracy is not high enough for real-time application. Researchers proposed integrating sEMG signals with another feature not affected by amputation. The muscular coordination between vision and hand manipulation makes us consider including the visual information in prosthetic hand control. In this study, we identify another sweet period, starts at 0 ms and ends at 320 ms during the early reaching phase, in which the vision data could better classify the grasp patterns. Moreover, the visual classification results from the sweet period could be naturally integrated with sEMG data collected during the grasp phase. After the integration, the accuracy of grasp classification increased from 85.5% (only sEMG) to 90.06% (integrated). Knowledge gained from this study encourages us to further explore the methods for incorporating computer vision into myoelectric data to enhance the movement control of prosthetic hands

Intelligent Biosignal Processing in Wearable and Implantable Sensors

This reprint provides a collection of papers illustrating the state-of-the-art of smart processing of data coming from wearable, implantable or portable sensors. Each paper presents the design, databases used, methodological background, obtained results, and their interpretation for biomedical applications. Revealing examples are brain–machine interfaces for medical rehabilitation, the evaluation of sympathetic nerve activity, a novel automated diagnostic tool based on ECG data to diagnose COVID-19, machine learning-based hypertension risk assessment by means of photoplethysmography and electrocardiography signals, Parkinsonian gait assessment using machine learning tools, thorough analysis of compressive sensing of ECG signals, development of a nanotechnology application for decoding vagus-nerve activity, detection of liver dysfunction using a wearable electronic nose system, prosthetic hand control using surface electromyography, epileptic seizure detection using a CNN, and premature ventricular contraction detection using deep metric learning. Thus, this reprint presents significant clinical applications as well as valuable new research issues, providing current illustrations of this new field of research by addressing the promises, challenges, and hurdles associated with the synergy of biosignal processing and AI through 16 different pertinent studies. Covering a wide range of research and application areas, this book is an excellent resource for researchers, physicians, academics, and PhD or master students working on (bio)signal and image processing, AI, biomaterials, biomechanics, and biotechnology with applications in medicine

Intelligent Sensors for Human Motion Analysis

The book, "Intelligent Sensors for Human Motion Analysis," contains 17 articles published in the Special Issue of the Sensors journal. These articles deal with many aspects related to the analysis of human movement. New techniques and methods for pose estimation, gait recognition, and fall detection have been proposed and verified. Some of them will trigger further research, and some may become the backbone of commercial systems

Desenvolvimento de metodologia baseada em aprendizado por reforço e Sistema de Inferência Fuzzy para identificação e minimização de contaminantes em sinais de sEMG com aplicação em identificação de movimentos do segmento mão-braço

A incessante busca por novas tecnologias que proporcionem aumento da qualidade de vida do ser humano tem norteado a pesquisa acadêmica ao longo da história. Isso é observado na evolução dos meios de transporte, dos dispositivos de comunicação e até mesmo de serviços como o bancário. No entanto, para pessoas com deficiência motora, em especial aquelas que sofreram amputação ou não possuem parte do membro superior, a conquista de melhores condições de vida está potencialmente relacionada com liberdade e independência. Visando suprir esta necessidade, muitos pesquisadores têm trabalhado no desenvolvimento de algoritmos preditores de movimento do segmento mão-braço a partir de sinais de eletromiografia para o controle de próteses na expectativa de aumentar o número de graus de liberdade do dispositivo. Contudo, para que se obtenha sistemas eficientes e que tenham elevados índices de assertividade, é imprescindível que o nível de interferência e ruído, os quais inevitavelmente estão presentes nos registros de eletromiografia devido à instrumentação, ambiente, aspectos fisiológicos, dentre outros, seja o menor possível. Neste contexto, alguns trabalhos foram desenvolvidos visando a minimização do efeito de interferências no classificador, contudo todos aqueles abrangidos pela pesquisa realizada demandam um estágio de treinamento off-line, não são adaptáveis às variações do sinal de EMG e/ou dependem do sinal dos outros canais de medição para a minimização do efeito degradador. Diante disso, a presente proposta de tese apresenta uma metodologia baseada em aprendizagem por reforço (Reinforcement Learning) e Sistema de Inferência Fuzzy para detecção, identificação do tipo e atenuação do efeito de contaminantes em registros de eletromiografia, com aplicação em sistemas de reconhecimento de gestos do membro superior. O mesmo está fundamentado em um modelo de agente e ambiente, sendo constituído dos seguintes elementos: ambiente (atividade elétrica muscular), estado (conjunto de 6 características extraídas do sinal de EMG), ações (aplicação de filtros/procedimentos específicos para a redução do impacto de cada interferência) e agente (controlador que fará a identificação do tipo da contaminação e executará a ação adequada). Para cada ação exercida pelo agente será atribuída uma recompensa a qual, por sua vez, é determinada em virtude do impacto da primeira nas características do sinal (estado) por meio de um Sistema de Inferência Fuzzy. O treinamento, realizado através do método Ator-Crítico, consiste na obtenção de uma política de ações que maximize a recompensa percebida a longo prazo. Por meio de um experimento realizado de forma off-line conseguiu-se taxas de acerto de 92,96% na identificação de 4 tipos de contaminantes (interferência por eletrocardiografia (ECG), artefato de movimento, interferência eletromagnética oriunda da rede de energia elétrica e ruído branco gaussiano) e 69,5% quando se considerou também sinal íntegro. Além disso, por meio de um estudo de caso simulando-se o treinamento online do agente evidenciou-se que o modelo de Transfer Learning adotado foi eficaz na dispensa da necessidade do uso de dados adquiridos previamente do usuário além de acelerar o processo de aprendizado. Estas propriedades são fundamentais para a implementação de qualquer sistema de forma online. Logo, verificou-se indícios de que o SIF-ACRL tem, de fato, potencial para ser implementado de forma online.The incessant search for new technologies that provide increased quality of life for human beings has guided academic research throughout history. This is observed in the evolution of transports, communication devices and even services such as banking. However, for people with motor disabilities, especially those who have had an amputation or do not have part of the upper limb, achieving better living conditions is potentially related to freedom and independence. To meet this need, many researchers have been working on the development of hand-arm segment movement predictors algorithms from electromyography signals for the control of prostheses in the hope of increasing the device's degrees of freedom. However, to obtain efficient systems that have high levels of assertiveness, it is essential that the interference and noise level, which are inevitably present in the electromyography records due to the instrumentation, environment, physiological aspects, among others, is the lowest possible. In this context, some works were developed aiming at minimizing the effect of interference in the classifier, however, all those covered by the performed research demand an offline training stage, are not adaptable to the EMG signal variations, and/or depend on the signal of others measurement channels to minimize the degrading effect. In view of this, the present thesis proposal presents a methodology based on Reinforcement Learning and Fuzzy Inference System for detection, identification of the type and mitigation of the effect of contaminants in electromyography records, with application in gesture recognition systems of the upper limb. It is based on an agent and environment model, consisting of the following elements: environment (muscle electrical activity), state (set of 6 characteristics extracted from the EMG signal), actions (application of specific filters/procedures to reduce impact of each interference) and agent (controller who will identify the type of contamination and take the appropriate action). For each action performed by the agent, a reward will be attributed which, in turn, is determined by the impact of the actions on the signal features (state) by means of a Fuzzy Inference System. The training, carried out through the Actor-Critic method, consists of obtaining an action policy that maximizes the long term perceived reward. Through an experiment carried out offline, success rates of 92.96% were achieved in the identification of 4 types of contaminants (interference by electrocardiography (ECG), motion artifact, electromagnetic interference from the electricity network and Gaussian white noise) and 69.5% when a clean signal class was added. In addition, a case study simulating the agent's online training showed that the Transfer Learning model adopted was effective in dispensing with the need to use data previously acquired from the user, in addition to accelerating the learning process. These properties are fundamental for the implementation of any system online. Therefore, there were indications that the SIF-ACRL has the potential to be implemented online

東北大学電気通信研究所研究活動報告 第29号(2022年度)

紀要類（bulletin）departmental bulletin pape

Preparation, Physico-Chemical Properties and Biomedical Applications of Nanoparticles

Nowadays, the impact of nanotechnology on applications in medicine and biomedical sciences has broader societal and economic effects, enhancing awareness of the business, regulatory, and administrative aspects of medical applications. The selected papers included in the present Special Issue gives readers a critical, balanced and realistic evaluation of existing nanomedicine developments and future prospects, allowing practitioners to plan and make decisions.The topics of this book covers the use of nanoparticles and nanotechnology in medical applications including biomaterials for tissue regeneration, diagnosis and monitoring, surgery, prosthetics, drug delivery systems, nanocarriers, and wound dressing. I would like to express my gratitude to all contributors to this issue, who have given so much of their time and effort to help create this collection of high quality papers