Deep learning approach to control of prosthetic hands with electromyography signals

Natural muscles provide mobility in response to nerve impulses. Electromyography (EMG) measures the electrical activity of muscles in response to a nerve's stimulation. In the past few decades, EMG signals have been used extensively in the identification of user intention to potentially control assistive devices such as smart wheelchairs, exoskeletons, and prosthetic devices. In the design of conventional assistive devices, developers optimize multiple subsystems independently. Feature extraction and feature description are essential subsystems of this approach. Therefore, researchers proposed various hand-crafted features to interpret EMG signals. However, the performance of conventional assistive devices is still unsatisfactory. In this paper, we propose a deep learning approach to control prosthetic hands with raw EMG signals. We use a novel deep convolutional neural network to eschew the feature-engineering step. Removing the feature extraction and feature description is an important step toward the paradigm of end-to-end optimization. Fine-tuning and personalization are additional advantages of our approach. The proposed approach is implemented in Python with TensorFlow deep learning library, and it runs in real-time in general-purpose graphics processing units of NVIDIA Jetson TX2 developer kit. Our results demonstrate the ability of our system to predict fingers position from raw EMG signals. We anticipate our EMG-based control system to be a starting point to design more sophisticated prosthetic hands. For example, a pressure measurement unit can be added to transfer the perception of the environment to the user. Furthermore, our system can be modified for other prosthetic devices.Comment: Conference. Houston, Texas, USA. September, 201

Medical system based on thermal optical system and neural network

Military personnel in the training or operational phases always need constant medical examination, but the presence of efficient medical care is difficult to implement in real-time for such cases. A wireless system for thermal tracking of soldiers was proposed, as well as tracking their vital signs in real time. Thermal cameras are used with an optical system designed to increase the accuracy of the thermal images captured as the change in the electro-cardiogram, heart rate, and temperature measurements are measured using a specially designed circuit. The results from both the thermal system and the biometric system are combined and sent to a computer for analysis using a model prepared with neural network technology. The proposed system was tested, and a database was created for 127 males and 110 females during training and rest times. The neural network model achieved a response time of 85 seconds until the release of the final analysis, and the accuracy of the proposed tracking system is 96%. The main contribution of this paper is the design of an integrated portable system for rapid, in-field, real-time military medical diagnostics

Electromiographic Signal Processing Using Embedded Artificial Intelligence: An Adaptive Filtering Approach

In recent times, Artificial Intelligence (AI) has become ubiquitous in technological fields, mainly due to its ability to perform computations in distributed systems or the cloud. Nevertheless, for some applications -as the case of EMG signal processing- it may be highly advisable or even mandatory an on-the-edge processing, i.e., an embedded processing methodology. On the other hand, sEMG signals have been traditionally processed using LTI techniques for simplicity in computing. However, making this strong assumption leads to information loss and spurious results. Considering the current advances in silicon technology and increasing computer power, it is possible to process these biosignals with AI-based techniques correctly. This paper presents an embedded-processing-based adaptive filtering system (here termed edge AI) being an outstanding alternative in contrast to a sensor-computer- actuator system and a classical digital signal processor (DSP) device. Specifically, a PYNQ-Z1 embedded system is used. For experimental purposes, three methodologies on similar processing scenarios are compared. The results show that the edge AI methodology is superior to benchmark approaches by reducing the processing time compared to classical DSPs and general standards while maintaining the signal integrity and processing it, considering that the EMG system is not LTI. Likewise, due to the nature of the proposed architecture, handling information exhibits no leakages. Findings suggest that edge computing is suitable for EMG signal processing when an on-device analysis is required

Stationary wavelet processing and data imputing in myoelectric pattern recognition on a low-cost embedded system

Pattern recognition-based decoding of surface electromyography allows for intuitive and flexible control of prostheses but comes at the cost of sensitivity to in-band noise and sensor faults. System robustness can be improved with wavelet-based signal processing and data imputing, but no attempt has been made to implement such algorithms on real-time, portable systems. The aim of this work was to investigate the feasibility of low-latency, wavelet-based processing and data imputing on an embedded device capable of controlling upper-arm prostheses. Nine able-bodied subjects performed Motion Tests while inducing transient disturbances. Additional investigation was performed on pre-recorded Motion Tests from 15 able-bodied subjects with simulated disturbances. Results from real-time tests were inconclusive, likely due to the low number of disturbance episodes, but simulated tests showed significant improvements in most metrics for both algorithms. However, both algorithms also showed reduced responsiveness during disturbance episodes. These results suggest wavelet-based processing and data imputing can be implemented in portable, real-time systems to potentially improve robustness to signal distortion in prosthetic devices with the caveat of reduced responsiveness for the typically short duration of signal disturbances. The trade-off between large-scale signal corruption robustness and system responsiveness warrants further studies in daily life activities

Pattern recognition-based real-time myoelectric control for anthropomorphic robotic systems : a thesis presented in partial fulfilment of the requirements for the degree of Doctor of Philosophy in Mechatronics at Massey University, Manawatū, New Zealand

All copyrighted Figures have been removed but may be accessed via their source cited in their respective captions.Advanced human-computer interaction (HCI) or human-machine interaction (HMI) aims to help humans interact with computers smartly. Biosignal-based technology is one of the most promising approaches in developing intelligent HCI systems. As a means of convenient and non-invasive biosignal-based intelligent control, myoelectric control identifies human movement intentions from electromyogram (EMG) signals recorded on muscles to realise intelligent control of robotic systems. Although the history of myoelectric control research has been more than half a century, commercial myoelectric-controlled devices are still mostly based on those early threshold-based methods. The emerging pattern recognition-based myoelectric control has remained an active research topic in laboratories because of insufficient reliability and robustness. This research focuses on pattern recognition-based myoelectric control. Up to now, most of effort in pattern recognition-based myoelectric control research has been invested in improving EMG pattern classification accuracy. However, high classification accuracy cannot directly lead to high controllability and usability for EMG-driven systems. This suggests that a complete system that is composed of relevant modules, including EMG acquisition, pattern recognition-based gesture discrimination, output equipment and its controller, is desirable and helpful as a developing and validating platform that is able to closely emulate real-world situations to promote research in myoelectric control. This research aims at investigating feasible and effective EMG signal processing and pattern recognition methods to extract useful information contained in EMG signals to establish an intelligent, compact and economical biosignal-based robotic control system. The research work includes in-depth study on existing pattern recognition-based methodologies, investigation on effective EMG signal capturing and data processing, EMG-based control system development, and anthropomorphic robotic hand design. The contributions of this research are mainly in following three aspects: Developed precision electronic surface EMG (sEMG) acquisition methods that are able to collect high quality sEMG signals. The first method was designed in a single-ended signalling manner by using monolithic instrumentation amplifiers to determine and evaluate the analog sEMG signal processing chain architecture and circuit parameters. This method was then evolved into a fully differential analog sEMG detection and collection method that uses common commercial electronic components to implement all analog sEMG amplification and filtering stages in a fully differential way. The proposed fully differential sEMG detection and collection method is capable of offering a higher signal-to-noise ratio in noisy environments than the single-ended method by making full use of inherent common-mode noise rejection capability of balanced signalling. To the best of my knowledge, the literature study has not found similar methods that implement the entire analog sEMG amplification and filtering chain in a fully differential way by using common commercial electronic components. Investigated and developed a reliable EMG pattern recognition-based real-time gesture discrimination approach. Necessary functional modules for real-time gesture discrimination were identified and implemented using appropriate algorithms. Special attention was paid to the investigation and comparison of representative features and classifiers for improving accuracy and robustness. A novel EMG feature set was proposed to improve the performance of EMG pattern recognition. Designed an anthropomorphic robotic hand construction methodology for myoelectric control validation on a physical platform similar to in real-world situations. The natural anatomical structure of the human hand was imitated to kinematically model the robotic hand. The proposed robotic hand is a highly underactuated mechanism, featuring 14 degrees of freedom and three degrees of actuation. This research carried out an in-depth investigation into EMG data acquisition and EMG signal pattern recognition. A series of experiments were conducted in EMG signal processing and system development. The final myoelectric-controlled robotic hand system and the system testing confirmed the effectiveness of the proposed methods for surface EMG acquisition and human hand gesture discrimination. To verify and demonstrate the proposed myoelectric control system, real-time tests were conducted onto the anthropomorphic prototype robotic hand. Currently, the system is able to identify five patterns in real time, including hand open, hand close, wrist flexion, wrist extension and the rest state. With more motion patterns added in, this system has the potential to identify more hand movements. The research has generated a few journal and international conference publications

Advanced Interfaces for HMI in Hand Gesture Recognition

The present thesis investigates techniques and technologies for high quality Human Machine Interfaces (HMI) in biomedical applications. Starting from a literature review and considering market SoA in this field, the thesis explores advanced sensor interfaces, wearable computing and machine learning techniques for embedded resource-constrained systems. The research starts from the design and implementation of a real-time control system for a multifinger hand prosthesis based on pattern recognition algorithms. This system is capable to control an artificial hand using a natural gesture interface, considering the challenges related to the trade-off between responsiveness, accuracy and light computation. Furthermore, the thesis addresses the challenges related to the design of a scalable and versatile system for gesture recognition with the integration of a novel sensor interface for wearable medical and consumer application

Computational Intelligence in Electromyography Analysis

Electromyography (EMG) is a technique for evaluating and recording the electrical activity produced by skeletal muscles. EMG may be used clinically for the diagnosis of neuromuscular problems and for assessing biomechanical and motor control deficits and other functional disorders. Furthermore, it can be used as a control signal for interfacing with orthotic and/or prosthetic devices or other rehabilitation assists. This book presents an updated overview of signal processing applications and recent developments in EMG from a number of diverse aspects and various applications in clinical and experimental research. It will provide readers with a detailed introduction to EMG signal processing techniques and applications, while presenting several new results and explanation of existing algorithms. This book is organized into 18 chapters, covering the current theoretical and practical approaches of EMG research

AN INVESTIGATION OF ELECTROMYOGRAPHIC (EMG) CONTROL OF DEXTROUS HAND PROSTHESES FOR TRANSRADIAL AMPUTEES

In reference to IEEE copyrighted material which is used with permission in this thesis, the IEEE does not endorse any of Plymouth University's products or services.There are many amputees around the world who have lost a limb through conflict, disease or an accident. Upper-limb prostheses controlled using surface Electromyography (sEMG) offer a solution to help the amputees; however, their functionality is limited by the small number of movements they can perform and their slow reaction times. Pattern recognition (PR)-based EMG control has been proposed to improve the functional performance of prostheses. It is a very promising approach, offering intuitive control, fast reaction times and the ability to control a large number of degrees of freedom (DOF). However, prostheses controlled with PR systems are not available for everyday use by amputees, because there are many major challenges and practical problems that need to be addressed before clinical implementation is possible. These include lack of individual finger control, an impractically large number of EMG electrodes, and the lack of deployment protocols for EMG electrodes site selection and movement optimisation. Moreover, the inability of PR systems to handle multiple forces is a further practical problem that needs to be addressed. The main aim of this project is to investigate the research challenges mentioned above via non-invasive EMG signal acquisition, and to propose practical solutions to help amputees. In a series of experiments, the PR systems presented here were tested with EMG signals acquired from seven transradial amputees, which is unique to this project. Previous studies have been conducted using non-amputees. In this work, the challenges described are addressed and a new protocol is proposed that delivers a fast clinical deployment of multi-functional upper limb prostheses controlled by PR systems. Controlling finger movement is a step towards the restoration of lost human capabilities, and is psychologically important, as well as physically. A central thread running through this work is the assertion that no two amputees are the same, each suffering different injuries and retaining differing nerve and muscle structures. This work is very much about individualised healthcare, and aims to provide the best possible solution for each affected individual on a case-by-case basis. Therefore, the approach has been to optimise the solution (in terms of function and reliability) for each individual, as opposed to developing a generic solution, where performance is optimised against a test population. This work is unique, in that it contributes to improving the quality of life for each individual amputee by optimising function and reliability. The main four contributions of the thesis are as follows: 1- Individual finger control was achieved with high accuracy for a large number of finger movements, using six optimally placed sEMG channels. This was validated on EMG signals for ten non-amputee and six amputee subjects. Thumb movements were classified successfully with high accuracy for the first time. The outcome of this investigation will help to add more movements to the prosthesis, and reduce hardware and computational complexity. 2- A new subject-specific protocol for sEMG site selection and reliable movement subset optimisation, based on the amputee’s needs, has been proposed and validated on seven amputees. This protocol will help clinicians to perform an efficient and fast deployment of prostheses, by finding the optimal number and locations of EMG channels. It will also find a reliable subset of movements that can be achieved with high performance. 3- The relationship between the force of contraction and the statistics of EMG signals has been investigated, utilising an experimental design where visual feedback from a Myoelectric Control Interface (MCI) helped the participants to produce the correct level of force. Kurtosis values were found to decrease monotonically when the contraction level increased, thus indicating that kurtosis can be used to distinguish different forces of contractions. 4- The real practical problem of the degradation of classification performance as a result of the variation of force levels during daily use of the prosthesis has been investigated, and solved by proposing a training approach and the use of a robust feature extraction method, based on the spectrum. The recommendations of this investigation improve the practical robustness of prostheses controlled with PR systems and progress a step further towards clinical implementation and improving the quality of life of amputees. The project showed that PR systems achieved a reliable performance for a large number of amputees, taking into account real life issues such as individual finger control for high dexterity, the effect of force level variation, and optimisation of the movements and EMG channels for each individual amputee. The findings of this thesis showed that the PR systems need to be appropriately tuned before usage, such as training with multiple forces to help to reduce the effect of force variation, aiming to improve practical robustness, and also finding the optimal EMG channel for each amputee, to improve the PR system’s performance. The outcome of this research enables the implementation of PR systems in real prostheses that can be used by amputees.Ministry of Higher Education and Scientific Research and Baghdad University- Baghdad/Ira

Wired, wireless and wearable bioinstrumentation for high-precision recording of bioelectrical signals in bidirectional neural interfaces

It is widely accepted by the scientific community that bioelectrical signals, which can be used for the identification of neurophysiological biomarkers indicative of a diseased or pathological state, could direct patient treatment towards more effective therapeutic strategies. However, the design and realisation of an instrument that can precisely record weak bioelectrical signals in the presence of strong interference stemming from a noisy clinical environment is one of the most difficult challenges associated with the strategy of monitoring bioelectrical signals for diagnostic purposes. Moreover, since patients often have to cope with the problem of limited mobility being connected to bulky and mains-powered instruments, there is a growing demand for small-sized, high-performance and ambulatory biopotential acquisition systems in the Intensive Care Unit (ICU) and in High-dependency wards. Furthermore, electrical stimulation of specific target brain regions has been shown to alleviate symptoms of neurological disorders, such as Parkinson’s disease, essential tremor, dystonia, epilepsy etc. In recent years, the traditional practice of continuously stimulating the brain using static stimulation parameters has shifted to the use of disease biomarkers to determine the intensity and timing of stimulation. The main motivation behind closed-loop stimulation is minimization of treatment side effects by providing only the necessary stimulation required within a certain period of time, as determined from a guiding biomarker. Hence, it is clear that high-quality recording of local field potentials (LFPs) or electrocorticographic (ECoG) signals during deep brain stimulation (DBS) is necessary to investigate the instantaneous brain response to stimulation, minimize time delays for closed-loop neurostimulation and maximise the available neural data. To our knowledge, there are no commercial, small, battery-powered, wearable and wireless recording-only instruments that claim the capability of recording ECoG signals, which are of particular importance in closed-loop DBS and epilepsy DBS. In addition, existing recording systems lack the ability to provide artefact-free high-frequency (> 100 Hz) LFP recordings during DBS in real time primarily because of the contamination of the neural signals of interest by the stimulation artefacts. To address the problem of limited mobility often encountered by patients in the clinic and to provide a wide variety of high-precision sensor data to a closed-loop neurostimulation platform, a low-noise (8 nV/√Hz), eight-channel, battery-powered, wearable and wireless multi-instrument (55 × 80 mm2) was designed and developed. The performance of the realised instrument was assessed by conducting both ex vivo and in vivo experiments. The combination of desirable features and capabilities of this instrument, namely its small size (~one business card), its enhanced recording capabilities, its increased processing capabilities, its manufacturability (since it was designed using discrete off-the-shelf components), the wide bandwidth it offers (0.5 – 500 Hz) and the plurality of bioelectrical signals it can precisely record, render it a versatile tool to be utilized in a wide range of applications and environments. Moreover, in order to offer the capability of sensing and stimulating via the same electrode, novel real-time artefact suppression methods that could be used in bidirectional (recording and stimulation) system architectures are proposed and validated. More specifically, a novel, low-noise and versatile analog front-end (AFE), which uses a high-order (8th) analog Chebyshev notch filter to suppress the artefacts originating from the stimulation frequency, is presented. After defining the system requirements for concurrent LFP recording and DBS artefact suppression, the performance of the realised AFE is assessed by conducting both in vitro and in vivo experiments using unipolar and bipolar DBS (monophasic pulses, amplitude ranging from 3 to 6 V peak-to-peak, frequency 140 Hz and pulse width 100 µs). Under both in vitro and in vivo experimental conditions, the proposed AFE provided real-time, low-noise and artefact-free LFP recordings (in the frequency range 0.5 – 250 Hz) during stimulation. Finally, a family of tunable hardware filter designs and a novel method for real-time artefact suppression that enables wide-bandwidth biosignal recordings during stimulation are also presented. This work paves the way for the development of miniaturized research tools for closed-loop neuromodulation that use a wide variety of bioelectrical signals as control signals.Open Acces

Conception, development and evaluation of polymer-based screen-printed textile electrodes for biopotential monitoring

Wearable technologies represent the new frontier of vital signs monitoring in different applications, from fitness to health. With the progressive miniaturization of the electronic components, enabling the implementation of portable and hand-held acquisition and recording devices, the research focus has shifted toward the development of effective and unobtrusive textile electrodes. This work deals with the study, development and characterization of organic-polymer-based electrodes for biopotentials. After an overview of the main materials and fabrication technologies presented so far in the scientific literature, the possibility to use these electrodes as an alternative to the Ag/AgCl disposable gelled electrodes usually adopted in clinical practice was tested. For this purpose, several textile electrode realization techniques were studied and optimized, in order to create electrodes with adequate features to detect two fundamental physiological signals: the electrocardiogram (ECG) and the electromyogram (EMG). The electrodes were obtained by depositing on the fabric the organic bio-compatible polymer poly(3,4-ethylenedioxythiophene) doped with poly(4-styrenesulfonate) (PEDOT:PSS) with three deposition procedures: dipcoating, ink-jet printing and screen printing. The physical\u2013chemical properties of the polymer solution were varied for each procedure to obtain an optimal and reproducible result. For what concerns the ECG signal, the research activity focused on screen-printed textile electrodes and their performance was first assessed by benchtop measurements and then by human trials. The first tests demonstrated that, by adding solid or liquid electrolytes the electrodes, the largest part of the characteristics required by the ANSI/AAMI EC12:2000 standard for gelled ECG electrodes can be achieved. Tests performed in different conditions showed that the skin contact impedance and the ECG morphological features are highly similar to those obtainable with disposable gelled Ag/AgCl electrodes (\u3c1 > 0.99). A trial with ten subjects revealed also the capability of the proposed electrodes to accurately capture with clinical instruments an ECG morphology with performance comparable to off-the-shelf disposable electrodes. Furthermore, the proposed textile electrodes preserve their electrical properties and functionality even after several mild washing cycles, while they suffered physical stretching. Similar tests were performed on screen-printed textile electrodes fabricated in two different sizes to test them as EMG sensors, with and without electrolytes. After a series of controlled acquisitions performed by electro-stimulating the muscles in order to analyze the waveform morphologu of the M-wave, the statistical analysis showed a high similarity in terms of rms of the noise and electrode-skin impedance between conventional and textile electrodes with the addition of solid hydrogel and saline solution. Furthermore, the M-wave recorded on the tibialis anterior muscle during the stimulation of the peroneal nerve was comparatively analyzed between conventional and textile electrodes. The comparison provided an R2 value higher than 97% in all measurement conditions. These results opened their use in smart garments for real application scenarios and for this purpose were developed a couple of smart shirts able to detect the EGC and the EMG signal. The results indicated that this approach could be adopted in the future for the development of smart garments able to comfortably detect physiological signals