Far-field electric potentials provide access to the output from the spinal cord from wrist-mounted sensors

OBJECTIVE: Neural interfaces need to become more unobtrusive and socially acceptable to appeal to general consumers outside rehabilitation settings. APPROACH: We developed a non-invasive neural interface that provides access to spinal motor neuron activities from the wrist, which is the preferred location for a wearable. The interface decodes far-field potentials present at the tendon endings of the forearm muscles using blind source separation. First, we evaluated the reliability of the interface to detect motor neuron firings based on far-field potentials, and thereafter we used the decoded motor neuron activity for the prediction of finger contractions in offline and real-time conditions. MAIN RESULTS: The results showed that motor neuron activity decoded from the far-field potentials at the wrist accurately predicted individual and combined finger commands and therefore allowed for highly accurate real-time task classification. SIGNIFICANCE: These findings demonstrate the feasibility of a non-invasive, neural interface at the wrist for precise real-time control based on the output of the spinal cord

Neuromorphic decoding of spinal motor neuron behaviour during natural hand movements for a new generation of wearable neural interfaces

We propose a neuromorphic framework to process the activity of human spinal motor neurons for movement intention recognition. This framework is integrated into a non-invasive interface that decodes the activity of motor neurons innervating intrinsic and extrinsic hand muscles. One of the main limitations of current neural interfaces is that machine learning models cannot exploit the efficiency of the spike encoding operated by the nervous system. Spiking-based pattern recognition would detect the spatio-temporal sparse activity of a neuronal pool and lead to adaptive and compact implementations, eventually running locally in embedded systems. Emergent Spiking Neural Networks (SNN) have not yet been used for processing the activity of in-vivo human neurons. Here we developed a convolutional SNN to process a total of 467 spinal motor neurons whose activity was identified in 5 participants while executing 10 hand movements. The classification accuracy approached 0.95 ±0.14 for both isometric and non-isometric contractions. These results show for the first time the potential of highly accurate motion intent detection by combining non-invasive neural interfaces and SNN

Adaptive real-time identification of motor unit discharges from non-stationary high-density surface electromyographic signals

Objective. Estimation of the discharge pattern of motor units by electromyography (EMG) decomposition has been applied for neurophysiologic investigations, clinical diagnosis, and human-machine interfacing. However, most of the methods for EMG decomposition are currently applied offline. Here, we propose an approach for high-density surface EMG decomposition in real-time. Methods. A real-time decomposition scheme including two sessions, offline training and online decomposition, is proposed based on the convolutional kernel compensation algorithm. The estimation parameters, separation vectors and the thresholds for spike extraction, are first computed during offline training, and then they are directly applied to estimate motor unit spike trains (MUSTs) during the online decomposition. The estimation parameters are updated with the identification of new discharges to adapt to non-stationary conditions. The decomposition accuracy was validated on simulated EMG signals by convolving synthetic MUSTs with motor unit action potentials (MUAPs). Moreover, the accuracy of the online decomposition was assessed from experimental signals recorded from forearm muscles using a signal-based performance metrics (pulse-to-noise ratio, PNR). Main results. The proposed algorithm yielded a high decomposition accuracy and robustness to non-stationary conditions. The accuracy of MUSTs identified from simulated EMG signals was > 80% for most conditions. From experimental EMG signals, on average, 12±2 MUSTs were identified from each electrode grid with PNR of 25.0±1.8 dB, corresponding to an estimated decomposition accuracy > 75%. Conclusion and Significance. These results indicate the feasibility of real-time identification of motor unit activities non-invasively during variable force contractions, extending the potential applications of high-density EMG as a neural interface

Optimal Motor Unit Subset Selection for Accurate Motor Intention Decoding: Towards Dexterous Real-Time Interfacing

Objective: Motor unit (MU) discharge timings encode human motor intentions to the finest degree. Whilst tapping into such information can bring significant gains to a range of applications, current approaches to MU decoding from surface signals do not scale well with the demands of dexterous human-machine interfacing (HMI). To optimize the forward estimation accuracy and time-efficiency of such systems, we propose the inclusion of task-wise initialization and MU subset selection. Methods: Offline analyses were conducted on data recorded from 11 non-disabled subjects. Task-wise decomposition was applied to identify MUs from high-density surface electromyography (HD-sEMG) pertaining to 18 wrist/forearm motor tasks. The activities of a selected subset of MUs were extracted from test data and used for forward estimation of intended motor tasks and joint kinematics. To that end, various combinations of subset selection and estimation algorithms (both regression and classification-based) were tested for a range of subset sizes. Results: The mutual information-based minimum Redundancy Maximum Relevance (mRMR-MI) criterion retained MUs with the highest predicative power. When the portion of tracked MUs was reduced down to 25%, the regression performance decreased only by 3% (R2=0.79) while classification accuracy dropped by 2.7% (accuracy = 74%) when kernel-based estimators were considered. Conclusion and Significance: Careful selection of tracked MUs can optimize the efficiency of MU-driven interfacing. In particular, prioritization of MUs exhibiting strong nonlinear relationships with target motions is best leveraged by kernel-based estimators. Hence, this frees resources for more robust and adaptive MU decoding techniques to be implemented in future

Deep Learning Methods for Hand Gesture Recognition via High-Density Surface Electromyogram (HD-sEMG) Signals

Hand Gesture Recognition (HGR) using surface Electromyogram (sEMG) signals can be considered as one of the most important technologies in making efficient Human Machine Interface (HMI) systems. In particular, sEMG-based hand gesture has been a topic of growing interest for development of assistive systems to improve the quality of life in individuals suffering from amputated limbs. Generally speaking, myoelectric prosthetic devices work by classifying existing patterns of the collected sEMG signals and synthesizing intended gestures. While conventional myoelectric control systems, e.g., on/off control or direct-proportional, have potential advantages, challenges such as limited Degree of Freedom (DoF) due to crosstalk have resulted in the emergence of data-driven solutions. More specifically, to improve efficiency, intuitiveness, and the control performance of hand prosthetic systems, several Artificial Intelligence (AI) algorithms ranging from conventional Machine Learning (ML) models to highly complicated Deep Neural Network (DNN) architectures have been designed for sEMG-based hand gesture recognition in myoelectric prosthetic devices. In this thesis, we, first, perform a literature review on hand gesture recognition methods and elaborate on the recently proposed Deep Learning/Machine Learning (DL/ML) models in the literature. Then, our utilized High-Density sEMG (HD-sEMG) dataset is introduced and the rationales behind our main focus on this particular type of sEMG dataset are explained. We, then, develop a Vision Transformer (ViT)-based model for gesture recognition with HD-sEMG signals and evaluate its performance under different conditions such as variable window sizes, number of electrode channels, and model's complexity. We compare its performance with that of two conventional ML and one DL algorithm that are typically adopted in this domain. Furthermore, we introduce another capability of our proposed framework for instantaneous training, which is its ability to classify hand gestures based on a single frame of HD-sEMG dataset. Following that, we introduce the idea of integrating the macroscopic and microscopic neural drive information obtained from HD-sEMG data into a hybrid ViT-based framework for gesture recognition, which outperforms a standalone ViT architecture in terms of classification accuracy. Here, microscopic neural drive information (also called Motor Unit Spike Trains) refers to the neural commands sent by the brain and spinal cord to individual muscle fibers and are extracted from HD-sEMG signals using Blind Source Separation (BSP) algorithms. Finally, we design an alternative and novel hand gesture recognition model based on the less-explored topic of Spiking Neural Networks (SNN), which performs spatio-temporal gesture recognition in an event-based fashion. As opposed to the classical DNN architectures, SNNs are of the capacity to imitate human brain's cognitive function by using biologically inspired models of neurons and synapses. Therefore, they are more biologically explainable and computationally efficient

Advancing Medical Technology for Motor Impairment Rehabilitation: Tools, Protocols, and Devices

Excellent motor control skills are necessary to live a high-quality life. Activities such as walking, getting dressed, and feeding yourself may seem mundane, but injuries to the neuromuscular system can render these tasks difficult or even impossible to accomplish without assistance. Statistics indicate that well over 100 million people are affected by diseases or injuries, such as stroke, Parkinson’s Disease, Multiple Sclerosis, Cerebral Palsy, peripheral nerve injury, spinal cord injury, and amputation, that negatively impact their motor abilities. This wide array of injuries presents a challenge to the medical field as optimal treatment paradigms are often difficult to implement due to a lack of availability of appropriate assessment tools, the inability for people to access the appropriate medical centers for treatment, or altogether gaps in technology for treating the underlying impairments causing the disability. Addressing each of these challenges will improve the treatment of movement impairments, provide more customized and continuous treatment to a larger number of patients, and advance rehabilitative and assistive device technology. In my research, the key approach was to develop tools to assess and treat upper extremity movement impairment. In Chapter 2.1, I challenged a common biomechanical[GV1] modeling technique of the forearm. Comparing joint torque values through inverse dynamics simulation between two modeling platforms, I discovered that representing the forearm as a single cylindrical body was unable to capture the inertial parameters of a physiological forearm which is made up of two segments, the radius and ulna. I split the forearm segment into a proximal and distal segment, with the rationale being that the inertial parameters of the proximal segment could be tuned to those of the ulna and the inertial parameters of the distal segment could be tuned to those of the radius. Results showed a marked increase in joint torque calculation accuracy for those degrees of freedom that are affected by the inertial parameters of the radius and ulna. In Chapter 2.2, an inverse kinematic upper extremity model was developed for joint angle calculations from experimental motion capture data, with the rationale being that this would create an easy-to-use tool for clinicians and researchers to process their data. The results show accurate angle calculations when compared to algebraic solutions. Together, these chapters provide easy-to-use models and tools for processing movement assessment data. In Chapter 3.1, I developed a protocol to collect high-quality movement data in a virtual reality task that is used to assess hand function as part of a Box and Block Test. The goal of this chapter is to suggest a method to not only collect quality data in a research setting but can also be adapted for telehealth and at home movement assessment and rehabilitation. Results indicate that the data collected in this protocol are good and the virtual nature of this approach can make it a useful tool for continuous, data driven care in clinic or at home. In Chapter 3.2 I developed a high-density electromyography device for collecting motor unit action potentials of the arm. Traditional surface electromyography is limited by its ability to obtain signals from deep muscles and can also be time consuming to selectively place over appropriate muscles. With this high-density approach, muscle coverage is increased, placement time is decreased, and deep muscle activity can potentially be collected due to the high-density nature of the device[GV2] . Furthermore, the high-density electromyography device is built as a precursor to a high-density electromyography-electrical stimulation device for functional electrical stimulation. The customizable nature of the prototype in Chapter 3.2 allows for the implementation both recording and stimulating electrodes. Furthermore, signal results show that the electromyography data obtained from the device are of high quality and are correlated with gold standard surface electromyography sensors. One key factor in a device that can record and then stimulate based on the information from the recorded signals is an accurate movement intent decoder. High-quality movement decoders have been designed by closed-loop device controllers in the past, but they still struggle when the user interacts with objects of varying weight due to underlying alterations in muscle signals. In Chapter 4, I investigate this phenomenon by administering an experiment where participants perform a Box and Block Task with objects of 3 different weights, 0 kg, 0.02 kg, and 0.1 kg. Electromyography signals of the participants right arm were collected and co-contraction levels between antagonistic muscles were analyzed to uncover alterations in muscle forces and joint dynamics. Results indicated contraction differences between the conditions and also between movement stages (contraction levels before grabbing the block vs after touching the block) for each condition. This work builds a foundation for incorporating object weight estimates into closed-loop electromyography device movement decoders. Overall, we believe the chapters in this thesis provide a basis for increasing availability to movement assessment tools, increasing access to effective movement assessment and rehabilitation, and advance the medical device and technology field

Biosignal‐based human–machine interfaces for assistance and rehabilitation : a survey

As a definition, Human–Machine Interface (HMI) enables a person to interact with a device. Starting from elementary equipment, the recent development of novel techniques and unobtrusive devices for biosignals monitoring paved the way for a new class of HMIs, which take such biosignals as inputs to control various applications. The current survey aims to review the large literature of the last two decades regarding biosignal‐based HMIs for assistance and rehabilitation to outline state‐of‐the‐art and identify emerging technologies and potential future research trends. PubMed and other databases were surveyed by using specific keywords. The found studies were further screened in three levels (title, abstract, full‐text), and eventually, 144 journal papers and 37 conference papers were included. Four macrocategories were considered to classify the different biosignals used for HMI control: biopotential, muscle mechanical motion, body motion, and their combinations (hybrid systems). The HMIs were also classified according to their target application by considering six categories: prosthetic control, robotic control, virtual reality control, gesture recognition, communication, and smart environment control. An ever‐growing number of publications has been observed over the last years. Most of the studies (about 67%) pertain to the assistive field, while 20% relate to rehabilitation and 13% to assistance and rehabilitation. A moderate increase can be observed in studies focusing on robotic control, prosthetic control, and gesture recognition in the last decade. In contrast, studies on the other targets experienced only a small increase. Biopotentials are no longer the leading control signals, and the use of muscle mechanical motion signals has experienced a considerable rise, especially in prosthetic control. Hybrid technologies are promising, as they could lead to higher performances. However, they also increase HMIs’ complex-ity, so their usefulness should be carefully evaluated for the specific application

On the Utility of Representation Learning Algorithms for Myoelectric Interfacing

Electrical activity produced by muscles during voluntary movement is a reflection of the firing patterns of relevant motor neurons and, by extension, the latent motor intent driving the movement. Once transduced via electromyography (EMG) and converted into digital form, this activity can be processed to provide an estimate of the original motor intent and is as such a feasible basis for non-invasive efferent neural interfacing. EMG-based motor intent decoding has so far received the most attention in the field of upper-limb prosthetics, where alternative means of interfacing are scarce and the utility of better control apparent. Whereas myoelectric prostheses have been available since the 1960s, available EMG control interfaces still lag behind the mechanical capabilities of the artificial limbs they are intended to steer—a gap at least partially due to limitations in current methods for translating EMG into appropriate motion commands. As the relationship between EMG signals and concurrent effector kinematics is highly non-linear and apparently stochastic, finding ways to accurately extract and combine relevant information from across electrode sites is still an active area of inquiry.This dissertation comprises an introduction and eight papers that explore issues afflicting the status quo of myoelectric decoding and possible solutions, all related through their use of learning algorithms and deep Artificial Neural Network (ANN) models. Paper I presents a Convolutional Neural Network (CNN) for multi-label movement decoding of high-density surface EMG (HD-sEMG) signals. Inspired by the successful use of CNNs in Paper I and the work of others, Paper II presents a method for automatic design of CNN architectures for use in myocontrol. Paper III introduces an ANN architecture with an appertaining training framework from which simultaneous and proportional control emerges. Paper Iv introduce a dataset of HD-sEMG signals for use with learning algorithms. Paper v applies a Recurrent Neural Network (RNN) model to decode finger forces from intramuscular EMG. Paper vI introduces a Transformer model for myoelectric interfacing that do not need additional training data to function with previously unseen users. Paper vII compares the performance of a Long Short-Term Memory (LSTM) network to that of classical pattern recognition algorithms. Lastly, paper vIII describes a framework for synthesizing EMG from multi-articulate gestures intended to reduce training burden