Extraction of Nonlinear Synergies for Proportional and Simultaneous Estimation of Finger Kinematics

Objective: Proportional and simultaneous estimation of finger kinematics from surface EMG based on the assumption that there exists a correlation between muscle activations and finger kinematics in low dimensional space. Methods: We employ Manifold Relevance Determination (MRD), a multi-view learning model with a nonparametric Bayesian approach, to extract the nonlinear muscle and kinematics synergies and the relationship between them by studying muscle activations (input-space) together with the finger kinematics (output-space). Results: This study finds that there exist muscle synergies which are associated with kinematic synergies. The acquired nonlinear synergies and the association between them has further been utilized for the estimation of finger kinematics from muscle activation inputs, and the proposed approach has outperformed other commonly used linear and nonlinear regression approaches with an average correlation coefficient of 0.91±0.03. Conclusion: There exists an association between muscle and kinematic synergies which can be used for the proportional and simultaneous estimation of finger kinematics from the muscle activation inputs. Significance: The findings of this study not only presents a viable approach for accurate and intuitive myoelectric control but also provides a new perspective on the muscle synergies in the motor control community

Synergy-Based Human Grasp Representations and Semi-Autonomous Control of Prosthetic Hands

Das sichere und stabile Greifen mit humanoiden Roboterhänden stellt eine große Herausforderung dar. Diese Dissertation befasst sich daher mit der Ableitung von Greifstrategien für Roboterhände aus der Beobachtung menschlichen Greifens. Dabei liegt der Fokus auf der Betrachtung des gesamten Greifvorgangs. Dieser umfasst zum einen die Hand- und Fingertrajektorien während des Greifprozesses und zum anderen die Kontaktpunkte sowie den Kraftverlauf zwischen Hand und Objekt vom ersten Kontakt bis zum statisch stabilen Griff. Es werden nichtlineare posturale Synergien und Kraftsynergien menschlicher Griffe vorgestellt, die die Generierung menschenähnlicher Griffposen und Griffkräfte erlauben. Weiterhin werden Synergieprimitive als adaptierbare Repräsentation menschlicher Greifbewegungen entwickelt. Die beschriebenen, vom Menschen gelernten Greifstrategien werden für die Steuerung robotischer Prothesenhände angewendet. Im Rahmen einer semi-autonomen Steuerung werden menschenähnliche Greifbewegungen situationsgerecht vorgeschlagen und vom Nutzenden der Prothese überwacht

Neuromechanical Tuning for Arm Motor Control

Movement is a fundamental behavior that allows us to interact with the external world. Its importance to human health is most evident when it becomes impaired due to disease or injury. Physical and occupational rehabilitation remains the most common treatment for these types of disorders. Although therapeutic interventions may improve motor function, residual deficits are common for many pathologies, such as stroke. The development of novel therapeutics is dependent upon a better understanding of the underlying mechanisms that govern movement. Movement of the human body adheres to the principles of classic Newtonian mechanics. However, due to the inherent complexity of the body and the highly variable repertoire of environmental contexts in which it operates, the musculoskeletal system presents a challenging control problem and the onus is on the central nervous system to reliably solve this problem. The neural motor system is comprised of numerous efferent and afferent pathways with a hierarchical organization which create a complex arrangement of feedforward and feedback circuits. However, the strategy that the neural motor system employs to reliably control these complex mechanics is still unknown. This dissertation will investigate the neural control of mechanics employing a “bottom-up” approach. It is organized into three research chapters with an additional introductory chapter and a chapter addressing final conclusions. Chapter 1 provides a brief description of the anatomical and physiological principles of the human motor system and the challenges and strategies that may be employed to control it. Chapter 2 describes a computational study where we developed a musculoskeletal model of the upper limb to investigate the complex mechanical interactions due to muscle geometry. Muscle lengths and moment arms contribute to force and torque generation, but the inherent redundancy of these actuators create a high-dimensional control problem. By characterizing these relationships, we found mechanical coupling of muscle lengths which the nervous system could exploit. Chapter 3 describes a study of muscle spindle contribution to muscle coactivation using a computational model of primary afferent activity. We investigated whether these afferents could contribute to motoneuron recruitment during voluntary reaching tasks in humans and found that afferent activity was orthogonal to that of muscle activity. Chapter 4 describes a study of the role of the descending corticospinal tract in the compensation of limb dynamics during arm reaching movements. We found evidence that corticospinal excitability is modulated in proportion to muscle activity and that the coefficients of proportionality vary in the course of these movements. Finally, further questions and future directions for this work are discussed in the Chapter 5

比例筋電位制御に向けた筋シナジーの抽出、解釈、および応用の研究

Transfer of human intentions into myoelectric hand prostheses is generally achieved by learning a mapping, directly from sEMG signals to the Kinematics using linear or nonlinear regression approaches. Due to the highly random and nonlinear nature of sEMG signals such approaches are not able to exploit the functions of the modern pros- thesis, completely. Inspired from the muscle synergy hypothesis in the motor control community, some studies in the past have shown that better estimation accuracies can be achieved by learning a mapping to kinematics space from the synergistic features extracted from sEMG. However, mainly linear algorithms such as Principle Compo- nent Analysis (PCA), and Non-negative matrix factorization (NNMF) were employed to extract synergistic features, separately, from EMG and kinematics data and have not considered the nonlinearity and the strong correlation that exist between finger kine- matics and muscles. To exploit the relationship between EMG and Finger Kinematics for myoelectric control, we propose the use of the Manifold Relevance Determination (MRD) model (multi-view learning) to find the correspondence between muscular and kinematics by learning a shared low-dimensional representation. In the first part of the study, we present the approach of multi-view learning, interpretation of extracted non- linear muscle synergies from the joint study of sEMG and finger kinematics and their use in estimating the finger kinematics for the upper-limb prosthesis. Applicability of the proposed approach is then demonstrated by comparing the kinematics estimation accuracies against linear synergies and direct mapping. In the second part of the study, we propose a new approach to extract nonlinear muscle synergies from sEMG using multiview learning which addresses the two main drawbacks (1. Inconsistent synergistic patterns upon addition of sEMG signals from more muscles, 2. Weak metric for accessing the quality and quantity of muscle synergies) of established algorithms and discuss the potential of the proposed approach for reducing the number of electrodes with negligible degradation in predicted kinematics.九州工業大学博士学位論文 学位記番号：生工博甲第372号 学位授与年月日：令和2年3月25日1 Introduction|2 Related Work|3 Extraction of nonlinear synergies for proportional and simultaneous estimation of finger kinematics|4 An Approach to Extract Nonlinear Muscle Synergies from sEMG through Multi-Model Learning|5 Conclusion and Future Work九州工業大学令和元年

Description of motor control using inverse models

Humans can perform complicated movements like writing or running without giving them much thought. The scientific understanding of principles guiding the generation of these movements is incomplete. How the nervous system ensures stability or compensates for injury and constraints – are among the unanswered questions today. Furthermore, only through movement can a human impose their will and interact with the world around them. Damage to a part of the motor control system can lower a person’s quality of life. Understanding how the central nervous system (CNS) forms control signals and executes them helps with the construction of devices and rehabilitation techniques. This allows the user, at least in part, to bypass the damaged area or replace its function, thereby improving their quality of life. CNS forms motor commands, for example a locomotor velocity or another movement task. These commands are thought to be processed through an internal model of the body to produce patterns of motor unit activity. An example of one such network in the spinal cord is a central pattern generator (CPG) that controls the rhythmic activation of synergistic muscle groups for overground locomotion. The descending drive from the brainstem and sensory feedback pathways initiate and modify the activity of the CPG. The interactions between its inputs and internal dynamics are still under debate in experimental and modelling studies. Even more complex neuromechanical mechanisms are responsible for some non-periodic voluntary movements. Most of the complexity stems from internalization of the body musculoskeletal (MS) system, which is comprised of hundreds of joints and muscles wrapping around each other in a sophisticated manner. Understanding their control signals requires a deep understanding of their dynamics and principles, both of which remain open problems. This dissertation is organized into three research chapters with a bottom-up investigation of motor control, plus an introduction and a discussion chapter. Each of the three research chapters are organized as stand-alone articles either published or in preparation for submission to peer-reviewed journals. Chapter two introduces a description of the MS kinematic variables of a human hand. In an effort to simulate human hand motor control, an algorithm was defined that approximated the moment arms and lengths of 33 musculotendon actuators spanning 18 degrees of freedom. The resulting model could be evaluated within 10 microseconds and required less than 100 KB of memory. The structure of the approximating functions embedded anatomical and functional features of the modelled muscles, providing a meaningful description of the system. The third chapter used the developments in musculotendon modelling to obtain muscle activity profiles controlling hand movements and postures. The agonist-antagonist coactivation mechanism was responsible for producing joint stability for most degrees of freedom, similar to experimental observations. Computed muscle excitations were used in an offline control of a myoelectric prosthesis for a single subject. To investigate the higher-order generation of control signals, the fourth chapter describes an analytical model of CPG. Its parameter space was investigated to produce forward locomotion when controlled with a desired speed. The model parameters were varied to produce asymmetric locomotion, and several control strategies were identified. Throughout the dissertation the balance between analytical, simulation, and phenomenological modelling for the description of simple and complex behavior is a recurrent theme of discussion

On the Relationship Between Muscle Synergies and Redundant Degrees of Freedom in Musculoskeletal Systems

It has been suggested that the human nervous system controls motions in the task (or operational) space. However, little attention has been given to the separation of the control of the task-related and task-irrelevant degrees of freedom.Aim: We investigate how muscle synergies may be used to separately control the task-related and redundant degrees of freedom in a computational model.Approach: We generalize an existing motor control model, and assume that the task and redundant spaces have orthogonal basis vectors. This assumption originates from observations that the human nervous system tightly controls the task-related variables, and leaves the rest uncontrolled. In other words, controlling the variables in one space does not affect the other space; thus, the actuations must be orthogonal in the two spaces. We implemented this assumption in the model by selecting muscle synergies that produce force vectors with orthogonal directions in the task and redundant spaces.Findings: Our experimental results show that the orthogonality assumption performs well in reconstructing the muscle activities from the measured kinematics/dynamics in the task and redundant spaces. Specifically, we found that approximately 70% of the variation in the measured muscle activity can be captured with the orthogonality assumption, while allowing efficient separation of the control in the two spaces.Implications: The developed motor control model is a viable tool in real-time simulations of musculoskeletal systems, as well as model-based control of bio-mechatronic systems, where a computationally efficient representation of the human motion controller is needed

Machine learning-based dexterous control of hand prostheses

Upper-limb myoelectric prostheses are controlled by muscle activity information recorded on the skin surface using electromyography (EMG). Intuitive prosthetic control can be achieved by deploying statistical and machine learning (ML) tools to decipher the user’s movement intent from EMG signals. This thesis proposes various means of advancing the capabilities of non-invasive, ML-based control of myoelectric hand prostheses. Two main directions are explored, namely classification-based hand grip selection and proportional finger position control using regression methods. Several practical aspects are considered with the aim of maximising the clinical impact of the proposed methodologies, which are evaluated with offline analyses as well as real-time experiments involving both able-bodied and transradial amputee participants. It has been generally accepted that the EMG signal may not always be a reliable source of control information for prostheses, mainly due to its stochastic and non-stationary properties. One particular issue associated with the use of surface EMG signals for upper-extremity myoelectric control is the limb position effect, which is related to the lack of decoding generalisation under novel arm postures. To address this challenge, it is proposed to make concurrent use of EMG sensors and inertial measurement units (IMUs). It is demonstrated this can lead to a significant improvement in both classification accuracy (CA) and real-time prosthetic control performance. Additionally, the relationship between surface EMG and inertial measurements is investigated and it is found that these modalities are partially related due to reflecting different manifestations of the same underlying phenomenon, that is, the muscular activity. In the field of upper-limb myoelectric control, the linear discriminant analysis (LDA) classifier has arguably been the most popular choice for movement intent decoding. This is mainly attributable to its ease of implementation, low computational requirements, and acceptable decoding performance. Nevertheless, this particular method makes a strong fundamental assumption, that is, data observations from different classes share a common covariance structure. Although this assumption may often be violated in practice, it has been found that the performance of the method is comparable to that of more sophisticated algorithms. In this thesis, it is proposed to remove this assumption by making use of general class-conditional Gaussian models and appropriate regularisation to avoid overfitting issues. By performing an exhaustive analysis on benchmark datasets, it is demonstrated that the proposed approach based on regularised discriminant analysis (RDA) can offer an impressive increase in decoding accuracy. By combining the use of RDA classification with a novel confidence-based rejection policy that intends to minimise the rate of unintended hand motions, it is shown that it is feasible to attain robust myoelectric grip control of a prosthetic hand by making use of a single pair of surface EMG-IMU sensors. Most present-day commercial prosthetic hands offer the mechanical abilities to support individual digit control; however, classification-based methods can only produce pre-defined grip patterns, a feature which results in prosthesis under-actuation. Although classification-based grip control can provide a great advantage over conventional strategies, it is far from being intuitive and natural to the user. A potential way of approaching the level of dexterity enjoyed by the human hand is via continuous and individual control of multiple joints. To this end, an exhaustive analysis is performed on the feasibility of reconstructing multidimensional hand joint angles from surface EMG signals. A supervised method based on the eigenvalue formulation of multiple linear regression (MLR) is then proposed to simultaneously reduce the dimensionality of input and output variables and its performance is compared to that of typically used unsupervised methods, which may produce suboptimal results in this context. An experimental paradigm is finally designed to evaluate the efficacy of the proposed finger position control scheme during real-time prosthesis use. This thesis provides insight into the capacity of deploying a range of computational methods for non-invasive myoelectric control. It contributes towards developing intuitive interfaces for dexterous control of multi-articulated prosthetic hands by transradial amputees

Implantable Neural Interfaces for Direct Control of Hand Prostheses

State-of-the art robotic hands can mimic many functions of the human hand. These devices are capable of actuating individual finger and multi-joint movements while providing adequate gripping force for daily activities. However, for patients with spinal cord injuries or amputations, there are few options to control these functions seamlessly or intuitively. A common barrier to restoring hand function to both populations is a lack of high-fidelity control signals. Non-invasive electrophysiological techniques record global summations of activity and lack the spatial or temporal resolution to extract or “decode” precise movement commands. The ability to decode finger movements from the motor system would allow patients to directly control hand functions and provide intuitive and scalable prosthetic solutions. This thesis investigates the capabilities of implantable devices to provide finger-specific commands for prosthetic hands. We adapt existing reasoning algorithms to two different sensing technologies. The first is intracortical electrode arrays implanted into primary motor cortex of two non-human primates. Both subjects controlled a virtual hand with a regression algorithm that decoded brain activity into finger kinematics. Performance was evaluated with single degree of freedom target matching tasks. Bit rate is a throughput metric that accounts for task difficulty and movement precision. A state-of-the-art re-calibration approach improved throughputs by an average of 33.1%. Notably, decoding performance was not dependent on subjects moving their intact hands. In future research, this approach can improve grasp precision for patients with spinal cord injuries. The second sensing technology is intramuscular electrodes implanted into residual muscles and Regenerative Peripheral Nerve Interfaces of two patients with transradial amputations. Both participants used a high-speed pattern recognition system to switch between 10 individual finger and wrist postures in a virtual environment with an average completion rate of 96.3% and a movement delay of 0.26 seconds. When the set was reduced to five grasp postures, average metrics improved to 100% completion and a 0.14 second delay. These results are a significant improvement over previous studies which report average completion rates ranging from 53.9% to 86.9% and delays of 0.45 to 0.86 seconds. Furthermore, grasp performance remained reliable across arm positions and both participants used this controller to complete a functional assessment with robotic prostheses. For a more dexterous solution, we combined the high-speed pattern recognition system with a regression algorithm that enabled simultaneous position control of both the index finger and middle-ring-small finger group. Both patients used this system to complete a virtual two degree of freedom target matching task with throughputs of 1.79 and 1.15 bits per second each. The controllers in this study used only four and five differentiated inputs, which can likely be processed with portable or implantable hardware. These results demonstrate that implantable sensors can provide patients with fluid and precise control of hand prostheses. However, clinically translatable implantable electronics need to be developed to realize the potential of these sensing and reasoning approaches. Further advancement of this technology will likely increase the utility and demand of robotic prostheses.PHDRoboticsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/169798/1/akvaskov_1.pd

A Methodology Towards Comprehensive Evaluation of Shape Memory Alloy Actuators for Prosthetic Finger Design

Presently, DC motors are the actuator of choice within intelligent upper limb prostheses. However, the weight and dimensions associated with suitable DC motors are not always compatible with the geometric restrictions of a prosthetic hand; reducing available degrees of freedom and ultimately rendering the prosthesis uncomfortable for the end-user. As a result, the search is on-going to find a more appropriate actuation solution that is lightweight, noiseless, strong and cheap. Shape memory alloy (SMA) actuators offer the potential to meet these requirements. To date, no viable upper limb prosthesis using SMA actuators has been developed. The primary reasons lie in low force generation as a result of unsuitable actuator designs, and significant difficulties in control owing to the highly nonlinear response of SMAs when subjected to joule heating. This work presents a novel and comprehensive methodology to facilitate evaluation of SMA bundle actuators for prosthetic finger design. SMA bundle actuators feature multiple SMA wires in parallel. This allows for increased force generation without compromising on dynamic performance. The SMA bundle actuator is tasked with reproducing the typical forces and contractions associated with the human finger in a prosthetic finger design, whilst maintaining a high degree of energy efficiency. A novel approach to SMA control is employed, whereby an adaptive controller is developed and tuned using the underlying thermo-mechanical principles of operation of SMA wires. A mathematical simulation of the kinematics and dynamics of motion provides a platform for designing, optimizing and evaluating suitable SMA bundle actuators offline. This significantly reduces the time and cost involved in implementing an appropriate actuation solution. Experimental results show iii that the performance of SMA bundle actuators is favourable for prosthesis applications. Phalangeal tip forces are shown to improve significantly through bundling of SMA wire actuators, while dynamic performance is maintained owing to the design and implementation of the selected control strategy. The work is intended to serve as a roadmap for fellow researchers seeking to design, implement and control SMA bundle actuators in a prosthesis design. Furthermore, the methodology can also be adopted to serve as a guide in the evaluation of other non-conventional actuation technologies in alternative applications

Redundancy reduction in motor control

Research in machine learning and neuroscience has made remarkable progress by investigating statistical redundancy in representations of natural environments, but to date much of this work has focused on sensory information like images and sounds. This dissertation explores the notions of redundancy and efficiency in the motor domain, where several different forms of independence exist. The dissertation begins by discussing redundancy at a conceptual level and presents relevant background material. Next, three main branches of original research are described. The first branch consists of a novel control framework for integrating low-bandwidth sensory updates with model uncertainty and action selection for navigating complex, multi-task environments. The second branch of research applies existing machine learning techniques to movement information and explores the mismatch between these methods for extracting independent components and the forms of redundancy that exist in the motor domain. The third branch of work analyzes full-body, goal-directed reaching movements gathered in a novel laboratory experiment, using explicitly measured information about the goal of each movement to uncover patterns in the movement dynamics. Each branch of research explores redundancy reduction in movement from a different perspective, building up a sort of catalog of the types of information present in movements. Redundancy is discussed throughout as an an important aspect of movement in the natural world. The dissertation concludes by summarizing the contributions of these three branches of work, and discussing promising areas for future work spurred by these investigations. More detailed models of voluntary movements hold promise not only for better treatments, improved prosthetics, smoother animations, and more fluid robots, but also as an avenue for scientific insight into the very foundations of cognition.Computer Science