Transfer learning in hand movement intention detection based on surface electromyography signals

Over the past several years, electromyography (EMG) signals have been used as a natural interface to interact with computers and machines. Recently, deep learning algorithms such as Convolutional Neural Networks (CNNs) have gained interest for decoding the hand movement intention from EMG signals. However, deep networks require a large dataset to train appropriately. Creating such a database for a single subject could be very time-consuming. In this study, we addressed this issue from two perspectives: (i) we proposed a subject-transfer framework to use the knowledge learned from other subjects to compensate for a target subject’s limited data; (ii) we proposed a task-transfer framework in which the knowledge learned from a set of basic hand movements is used to classify more complex movements, which include a combination of mentioned basic movements. We introduced two CNN-based architectures for hand movement intention detection and a subject-transfer learning approach. Classifiers are tested on the Nearlab dataset, a sEMG hand/wrist movement dataset including 8 movements and 11 subjects, along with their combination, and on open-source hand sEMG dataset “NinaPro DataBase 2 (DB2).” For the Nearlab database, the subject-transfer learning approach improved the average classification accuracy of the proposed deep classifier from 92.60 to 93.30% when classifier was utilizing 10 other subjects’ data via our proposed framework. For Ninapro DB2 exercise B (17 hand movement classes), this improvement was from 81.43 to 82.87%. Moreover, three stages of analysis in task-transfer approach proved that it is possible to classify combination hand movements using the knowledge learned from a set of basic hand movements with zero, few samples and few seconds of data from the target movement classes. First stage takes advantage of shared muscle synergies to classify combined movements, while second and third stages take advantage of novel algorithms using few-shot learning and fine-tuning to use samples from target domain to further train the classifier trained on the source database. The use of information learned from basic hand movements improved classification accuracy of combined hand movements by 10%

Wearable Robotics for Impaired Upper-Limb Assistance and Rehabilitation: State of the Art and Future Perspectives

Despite more than thirty-five years of research on wearable technologies to assist the upper-limb and a multitude of promising preliminary results, the goal of restoring pre-impairment quality of life of people with physical disabilities has not been fully reached yet. Whether it is for rehabilitation or for assistance, nowadays robotics is still only used in a few high-tech clinics and hospitals, limiting the access to a small amount of people. This work provides a description of the three major 'revolutions' occurred in the field (end-effector robots, rigid exoskeletons, and soft exosuits), reviewing forty-eight systems for the upper-limb (excluding hand-only devices) used in eighty-nine studies enrolling a clinical population before June 2022. The review critically discusses the state of the art, analyzes the different technologies, and compares the clinical outcomes, with the goal of determine new potential directions to follow

Development and Validation of a Spike Detection and Classification Algorithm Aimed at Implementation on Hardware Devices

Neurons cultured in vitro on MicroElectrode Array (MEA) devices connect to each other, forming a network. To study electrophysiological activity and long term plasticity effects, long period recording and spike sorter methods are needed. Therefore, on-line and real time analysis, optimization of memory use and data transmission rate improvement become necessary. We developed an algorithm for amplitude-threshold spikes detection, whose performances were verified with (a) statistical analysis on both simulated and real signal and (b) Big O Notation. Moreover, we developed a PCA-hierarchical classifier, evaluated on simulated and real signal. Finally we proposed a spike detection hardware design on FPGA, whose feasibility was verified in terms of CLBs number, memory occupation and temporal requirements; once realized, it will be able to execute on-line detection and real time waveform analysis, reducing data storage problems

Brain-Inspired Spiking Neural Network Controller for a Neurorobotic Whisker System

It is common for animals to use self-generated movements to actively sense the surrounding environment. For instance, rodents rhythmically move their whiskers to explore the space close to their body. The mouse whisker system has become a standard model for studying active sensing and sensorimotor integration through feedback loops. In this work, we developed a bioinspired spiking neural network model of the sensorimotor peripheral whisker system, modeling trigeminal ganglion, trigeminal nuclei, facial nuclei, and central pattern generator neuronal populations. This network was embedded in a virtual mouse robot, exploiting the Human Brain Project's Neurorobotics Platform, a simulation platform offering a virtual environment to develop and test robots driven by brain-inspired controllers. Eventually, the peripheral whisker system was adequately connected to an adaptive cerebellar network controller. The whole system was able to drive active whisking with learning capability, matching neural correlates of behavior experimentally recorded in mice

Reshaping the full body illusion through visuoelectro-tactile sensations

The physical boundaries of our body do not define what we perceive as self. This malleable representation arises from the neural integration of sensory information coming from the environment. Manipulating the visual and haptic cues produces changes in body perception, inducing the Full Body Illusion (FBI), a vastly used approach to exploring humans’ perception. After pioneering FBI demonstrations, issues arose regarding its setup, using experimenter-based touch and pre-recorded videos. Moreover, its outcome measures are based mainly on subjective reports, leading to biased results, or on heterogeneous objective ones giving poor consensus on their validity. To address these limitations, we developed and tested a multisensory platform allowing highly controlled experimental conditions, thanks to the leveraged use of innovative technologies: Virtual Reality (VR) and Transcutaneous Electrical Nerve Stimulation (TENS). This enabled a high spatial and temporal precision of the visual and haptic cues, efficiently eliciting FBI. While it matched the classic approach in subjective measures, our setup resulted also in significant results for all objective measurements. Importantly, FBI was elicited when all 4 limbs were multimodally stimulated but also in a single limb condition. Our results behoove the adoption of a comprehensive set of measures, introducing a new neuroscientific platform to investigate body representations

The effects of robotic assistance on upper limb spatial muscle synergies in healthy people during planar upper-limb training

Background Robotic rehabilitation is a commonly adopted technique used to restore motor functionality of neurological patients. However, despite promising results were achieved, the effects of human-robot interaction on human motor control and the recovery mechanisms induced with robot assistance can be further investigated even on healthy subjects before translating to clinical practice. In this study, we adopt a standard paradigm for upper-limb rehabilitation (a planar device with assistive control) with linear and challenging curvilinear trajectories to investigate the effect of the assistance in human-robot interaction in healthy people. Methods Ten healthy subjects were instructed to perform a large set of radial and curvilinear movements in two interaction modes: 1) free movement (subjects hold the robot handle with no assistance) and 2) assisted movement (with a force tunnel assistance paradigm). Kinematics and EMGs from representative upper-limb muscles were recorded to extract phasic muscle synergies. The free and assisted interaction modes were compared assessing the level of assistance, error, and muscle synergy comparison between the two interaction modes. Results It was found that in free movement error magnitude is higher than with assistance, proving that task complexity required assistance also on healthy controls. Moreover, curvilinear tasks require more assistance than standard radial paths and error is higher. Interestingly, while assistance improved task performance, we found only a slight modification of phasic synergies when comparing assisted and free movement. Conclusions We found that on healthy people, the effect of assistance was significant on task performance, but limited on muscle synergies. The findings of this study can find applications for assessing human-robot interaction and to design training to maximize motor recovery

Bayesian Integration in a Spiking Neural System for Sensorimotor Control

The brain continuously estimates the state of body and environment, with specific regions that are thought to act as Bayesian estimator, optimally integrating noisy and delayed sensory feedback with sensory predictions generated by the cerebellum. In control theory, Bayesian estimators are usually implemented using high-level representations. In this work, we designed a new spike-based computational model of a Bayesian estimator. The state estimator receives spiking activity from two neural populations encoding the sensory feedback and the cerebellar prediction, and it continuously computes the spike variability within each population as a reliability index of the signal these populations encode. The state estimator output encodes the current state estimate. We simulated a reaching task at different stages of cerebellar learning. The activity of the sensory feedback neurons encoded a noisy version of the trajectory after actual movement, with an almost constant intrapopulation spiking variability. Conversely, the activity of the cerebellar output neurons depended on the phase of the learning process. Before learning, they fired at their baseline not encoding any relevant information, and the variability was set to be higher than that of the sensory feedback (more reliable, albeit delayed). When learning was complete, their activity encoded the trajectory before the actual execution, providing an accurate sensory prediction; in this case, the variability was set to be lower than that of the sensory feedback. The state estimator model optimally integrated the neural activities of the afferent populations, so that the output state estimate was primarily driven by sensory feedback in prelearning and by the cerebellar prediction in postlearning. It was able to deal even with more complex scenarios, for example, by shifting the dominant source during the movement execution if information availability suddenly changed. The proposed tool will be a critical block within integrated spiking, brain-inspired control systems for simulations of sensorimotor tasks

Brain plasticity mechanisms underlying motor control reorganization: Pilot longitudinal study on post-stroke subjects

Functional Electrical Stimulation (FES) has demonstrated to improve walking ability and to induce the carryover effect, long-lasting persisting improvement. Functional magnetic resonance imaging has been used to investigate effective connectivity differences and longitudinal changes in a group of chronic stroke patients that attended a FES-based rehabilitation program for foot-drop correction, distinguishing between carryover effect responders and non-responders, and in comparison with a healthy control group. Bayesian hierarchical procedures were employed, involving nonlinear models at within-subject level—dynamic causal models—and linear models at between-subjects level. Selected regions of interest were primary sensorimotor cortices (M1, S1), supplementary motor area (SMA), and angular gyrus. Our results suggest the following: (i) The ability to correctly plan the movement and integrate proprioception information might be the features to update the motor control loop, towards the carryover effect, as indicated by the reduced sensitivity to pro-prioception input to S1 of FES non-responders; (ii) FES-related neural plasticity supports the active inference account for motor control, as indicated by the modulation of SMA and M1 connections to S1 area; (iii) SMA has a dual role of higher order motor processing unit responsible for complex movements, and a superintendence role in suppressing standard motor plans as external conditions changes