Fast human motion prediction for human-robot collaboration with wearable interfaces

In this paper, we aim at improving human motion prediction during human-robot collaboration in industrial facilities by exploiting contributions from both physical and physiological signals. Improved human-machine collaboration could prove useful in several areas, while it is crucial for interacting robots to understand human movement as soon as possible to avoid accidents and injuries. In this perspective, we propose a novel human-robot interface capable to anticipate the user intention while performing reaching movements on a working bench in order to plan the action of a collaborative robot. The proposed interface can find many applications in the Industry 4.0 framework, where autonomous and collaborative robots will be an essential part of innovative facilities. A motion intention prediction and a motion direction prediction levels have been developed to improve detection speed and accuracy. A Gaussian Mixture Model (GMM) has been trained with IMU and EMG data following an evidence accumulation approach to predict reaching direction. Novel dynamic stopping criteria have been proposed to flexibly adjust the trade-off between early anticipation and accuracy according to the application. The output of the two predictors has been used as external inputs to a Finite State Machine (FSM) to control the behaviour of a physical robot according to user's action or inaction. Results show that our system outperforms previous methods, achieving a real-time classification accuracy of $94.3\pm2.9\%$ after $160.0msec\pm80.0msec$ from movement onset

Biomedical Sensing and Imaging

This book mainly deals with recent advances in biomedical sensing and imaging. More recently, wearable/smart biosensors and devices, which facilitate diagnostics in a non-clinical setting, have become a hot topic. Combined with machine learning and artificial intelligence, they could revolutionize the biomedical diagnostic field. The aim of this book is to provide a research forum in biomedical sensing and imaging and extend the scientific frontier of this very important and significant biomedical endeavor

Signal and image processing methods for imaging mass spectrometry data

Imaging mass spectrometry (IMS) has evolved as an analytical tool for many biomedical applications. This thesis focuses on algorithms for the analysis of IMS data produced by matrix assisted laser desorption/ionization (MALDI) time-of-flight (TOF) mass spectrometer. IMS provides mass spectra acquired at a grid of spatial points that can be represented as hyperspectral data or a so-called datacube. Analysis of this large and complex data requires efficient computational methods for matrix factorization and for spatial segmentation. In this thesis, state of the art processing methods are reviewed, compared and improved versions are proposed. Mathematical models for peak shapes are reviewed and evaluated. A simulation model for MALDI-TOF is studied, expanded and developed into a simulator for 2D or 3D MALDI-TOF-IMS data. The simulation approach paves way to statistical evaluation of algorithms for analysis of IMS data by providing a gold standard dataset. [...

Simulation and implementation of novel deep learning hardware architectures for resource constrained devices

Corey Lammie designed mixed signal memristive-complementary metal–oxide–semiconductor (CMOS) and field programmable gate arrays (FPGA) hardware architectures, which were used to reduce the power and resource requirements of Deep Learning (DL) systems; both during inference and training. Disruptive design methodologies, such as those explored in this thesis, can be used to facilitate the design of next-generation DL systems

EMG space similarity feedback promotes learning of expert-like muscle activation patterns in a complex motor skill

Augmented feedback provided by a coach or augmented reality system can facilitate the acquisition of a motor skill. Verbal instructions and visual aids can be effective in providing feedback about the kinematics of the desired movements. However, many skills require mastering not only kinematic, but also complex kinetic patterns, for which feedback is harder to convey. Here, we propose the electromyography (EMG) space similarity feedback, which may indirectly convey kinematic and kinetic feedback by comparing the muscle activations of the learner and an expert in the task. The EMG space similarity feedback is a score that reflects how well a set of muscle synergies extracted from the expert can reconstruct the learner’s EMG when performing the task. We tested the EMG space similarity feedback in a virtual bimanual polishing task that uses a robotic system to simulate the dynamics of a real polishing operation. We measured the expert’s and learner’s EMG from eight muscles in each arm during the real and virtual polishing tasks, respectively. The goal of the virtual task was to smoothen the surface of a virtual object. Therefore, we defined performance in the task as the smoothness of the object at the end of a trial. We separated learners into real feedback and null feedback groups to assess the effects of the EMG space similarity feedback. The real and null feedback groups received veridic and no EMG space similarity feedback, respectively. Subjects participated in five training sessions on different days, and we evaluated their performance on each day. Subjects in both groups were able to increase smoothness throughout the training sessions, with no significant differences between groups. However, subjects in the real feedback group were able to improve in the EMG space similarity score to a significantly greater extent than the null feedback group. Additionally, subjects in the real feedback group produced muscle activations that became increasingly consistent with an important muscle synergy found in the expert. Our results indicate that the EMG space similarity feedback promotes acquiring expert-like muscle activation patterns, suggesting that it may assist in the acquisition of complex motor skills

Neuromechanical Tuning for Arm Motor Control

Movement is a fundamental behavior that allows us to interact with the external world. Its importance to human health is most evident when it becomes impaired due to disease or injury. Physical and occupational rehabilitation remains the most common treatment for these types of disorders. Although therapeutic interventions may improve motor function, residual deficits are common for many pathologies, such as stroke. The development of novel therapeutics is dependent upon a better understanding of the underlying mechanisms that govern movement. Movement of the human body adheres to the principles of classic Newtonian mechanics. However, due to the inherent complexity of the body and the highly variable repertoire of environmental contexts in which it operates, the musculoskeletal system presents a challenging control problem and the onus is on the central nervous system to reliably solve this problem. The neural motor system is comprised of numerous efferent and afferent pathways with a hierarchical organization which create a complex arrangement of feedforward and feedback circuits. However, the strategy that the neural motor system employs to reliably control these complex mechanics is still unknown. This dissertation will investigate the neural control of mechanics employing a “bottom-up” approach. It is organized into three research chapters with an additional introductory chapter and a chapter addressing final conclusions. Chapter 1 provides a brief description of the anatomical and physiological principles of the human motor system and the challenges and strategies that may be employed to control it. Chapter 2 describes a computational study where we developed a musculoskeletal model of the upper limb to investigate the complex mechanical interactions due to muscle geometry. Muscle lengths and moment arms contribute to force and torque generation, but the inherent redundancy of these actuators create a high-dimensional control problem. By characterizing these relationships, we found mechanical coupling of muscle lengths which the nervous system could exploit. Chapter 3 describes a study of muscle spindle contribution to muscle coactivation using a computational model of primary afferent activity. We investigated whether these afferents could contribute to motoneuron recruitment during voluntary reaching tasks in humans and found that afferent activity was orthogonal to that of muscle activity. Chapter 4 describes a study of the role of the descending corticospinal tract in the compensation of limb dynamics during arm reaching movements. We found evidence that corticospinal excitability is modulated in proportion to muscle activity and that the coefficients of proportionality vary in the course of these movements. Finally, further questions and future directions for this work are discussed in the Chapter 5

Single Lead EMG signal to Control an Upper Limb Exoskeleton Using Embedded Machine Learning on Raspberry Pi

Post-stroke can cause partial or complete paralysis of the human limb. Delayed rehabilitation steps in post-stroke patients can cause muscle atrophy and limb stiffness. Post-stroke patients require an upper limb exoskeleton device for the rehabilitation process. Several previous studies used more than one electrode lead to control the exoskeleton. The use of many electrode leads can lead to an increase in complexity in terms of hardware and software. Therefore, this research aims to develop single lead EMG pattern recognition to control an upper limb exoskeleton. The main contribution of this research is that the robotic upper limb exoskeleton device can be controlled using a single lead EMG. EMG signals were tapped at the biceps point with a sampling frequency of 2000 Hz. A Raspberry Pi 3B+ was used to embed the data acquisition, feature extraction, classification and motor control by using multithread algorithm. The exoskeleton arm frame is made using 3D printing technology using a high torque servo motor drive. The control process is carried out by extracting EMG signals using EMG features (mean absolute value, root mean square, variance) further extraction results will be trained on machine learning (decision tree (DT), linear regression (LR), polynomial regression (PR), and random forest (RF)). The results show that machine learning decision tree and random forest produce the highest accuracy compared to other classifiers. The accuracy of DT and RF are of 96.36±0.54% and 95.67±0.76%, respectively. Combining the EMG features, shows that there is no significant difference in accuracy (p-value 0.05). A single lead EMG electrode can control the upper limb exoskeleton robot device well

Myoelectric Control Architectures to Drive Upper Limb Exoskeletons

Myoelectric interfaces are sensing devices based on electromyography (EMG) able to read the electrical activity of motoneurons and muscles. These interfaces can be used to infer movement volition and to control assistive devices. Currently, these interfaces are widely used to control robotic prostheses for amputees, but their use could be beneficial even for people suffering from motor disabilities where the peripheral nervous system is intact and the impairment is only due to the muscles, e.g. muscular dystrophy, myopathies, or ageing. In combination with recent robotic orthoses and exoskeletons, myoelectric interfaces could dramatically improve these patients’ quality of life. Unfortunately, despite a wide plethora of methodologies has been proposed so far, a natural, intuitive, and reliable interface able to follow impaired subjects’ volition is still missing. The first contribution of this work is to provide a review of existing approaches. In this work we found that existing EMG-based control interfaces can be viewed as specific cases of a generic myoelectric control architecture composed by three distinct functional modules: a decoder to extract the movement intention from EMG signals, a controller to accomplish the desired motion through an actual command given to the actuators, and an adapter to connect them. The latter is responsible for translating the signal from decoder’s output to controller’s input domain and for modulating the level of provided assistance. We used this concept to analyse the case of study of linear regression decoders and an elbow exoskeleton. This thesis has the scientific objective to determine how these modules affect performance of EMG-driven exoskeletons and wearer’s fatigue. To experimentally test and compare myoelectric interfaces this work proposes: (1) a procedure to automatically tune the decoder module in order to equally compare or to normalize the decoder output among different sessions and subjects; (2) a procedure to automatically tune gravity compensation even for subjects suffering from severe disabilities, allowing them to perform the experimental tests; (3) a methodology to guide the impaired patients through the experimental session; (4) an evaluation procedure and metrics allowing statistically significant and unbiased comparison of different myoelectric interfaces. A further contribution of this work is the design of an experimental test bed composed by an elbow exoskeleton and by a software framework able to collect EMG signals and make them available to the exoskeleton’s actuators with minimal latency. Using this test bed, we were able to test different myoelectric interfaces based on our architecture, with different modules choices and tunings. We used linear regression decoders calibrated to predict the muscular torque, low-level controllers having torque or velocity as reference, and adapters consisting of a properly dimensioned gain or simple dynamic systems, such as an integrator or a mass-damping system. The results we obtained allow to conclude that EMG-based control is a viable technology to assist muscular weakness patients. Moreover, all the components of the myoelectric control architecture – decoder, adapter, controller, and their tuning – significantly affect the task-based performance measures we collect. Further investigations should be devoted to a methodology to automatically tune all the components, not the decoders only, and to the quantitative study of the effect the adapter has on the regulation of the assistance level and of the tradeoff between speed and accuracy