Kineto-dynamic modeling of human upper limb for robotic manipulators and assistive applications

The sensory-motor architecture of human upper limb and hand is characterized by a complex inter-relation of multiple elements, such as ligaments, muscles, and joints. Nonetheless, humans are able to generate coordinated and meaningful motor actions to interact-and eventually explore-the external environment. Such a complexity reduction is usually studied within the framework of synergistic control, whose focus has been mostly limited on human grasping and manipulation. Little attention has been devoted to the spatio-temporal characterization of human upper limb kinematic strategies and how the purposeful exploitation of the environmental constraints shapes human execution of manipulative actions. In this chapter, we report results on the evidence of a synergistic control of human upper limb and during manipulation with the environment. We propose functional analysis to characterize main spatio-temporal coordinated patterns of arm joints. Furthermore, we study how the environment influences human grasping synergies. The effect of cutaneous impairment is also evaluated. Applications to the design and control of robotic and assistive devices are finally discussed

Data-driven Mechanical Design and Control Method of Dexterous Upper-Limb Prosthesis

With an increasing number of people, 320,000 per year, suffering from impaired upper limb function due to various medical conditions like stroke and blunt trauma, the demand for highly functional upper limb prostheses is increasing; however, the rates of rejection of prostheses are high due to factors such as lack of functionality, high cost, weight, and lack of sensory feedback. Modern robotics has led to the development of more affordable and dexterous upper limb prostheses with mostly anthropomorphic designs. However, due to the highly sophisticated ergonomics of anthropomorphic hands, most are economically prohibitive and suffer from control complexity due to increased cognitive load on the user. Thus, this thesis work aims to design a prosthesis that relies on the emulation of the kinematics and contact forces involved in grasping tasks with healthy human hands rather than on biomimicry for reduction of mechanical complexity and utilization of technologically advanced engineering components. This is accomplished by 1) experimentally characterizing human grasp kinematics and kinetics as a basis for data-driven prosthesis design. Using the grasp data, steps are taken to 2) develop a data-driven design and control method of an upper limb prosthesis that shares the kinematics and kinetics required for healthy human grasps without taking the anthropomorphic design. This thesis demonstrates an approach to decrease the gap between the functionality of the human hand and robotic upper limb prostheses by introducing a method to optimize the design and control method of an upper limb prosthesis. This is accomplished by first, collecting grasp data from human subjects with a motion and force capture glove. The collected data are used to minimize control complexity by reducing the dimensionality of the device while fulfilling the kinematic and kinetic requirements of daily grasping tasks. Using these techniques, a task-oriented upper limb prosthesis is prototyped and tested in simulation and physical environment.Ph.D

Validation of a Biomechanical Injury and Disease Assessment Platform Applying an Inertial-Based Biosensor and Axis Vector Computation

Inertial kinetics and kinematics have substantial influences on human biomechanical function. A new algorithm for Inertial Measurement Unit (IMU)-based motion tracking is presented in this work. The primary aims of this paper are to combine recent developments in improved biosensor technology with mainstream motion-tracking hardware to measure the overall performance of human movement based on joint axis-angle representations of limb rotation. This work describes an alternative approach to representing three-dimensional rotations using a normalized vector around which an identified joint angle defines the overall rotation, rather than a traditional Euler angle approach. Furthermore, IMUs allow for the direct measurement of joint angular velocities, offering the opportunity to increase the accuracy of instantaneous axis of rotation estimations. Although the axis-angle representation requires vector quotient algebra (quaternions) to define rotation, this approach may be preferred for many graphics, vision, and virtual reality software applications. The analytical method was validated with laboratory data gathered from an infant dummy leg’s flexion and extension knee movements and applied to a living subject’s upper limb movement. The results showed that the novel approach could reasonably handle a simple case and provide a detailed analysis of axis-angle migration. The described algorithm could play a notable role in the biomechanical analysis of human joints and offers a harbinger of IMU-based biosensors that may detect pathological patterns of joint disease and injury

Using humanoid robots to study human behavior

Our understanding of human behavior advances as our humanoid robotics work progresses-and vice versa. This team's work focuses on trajectory formation and planning, learning from demonstration, oculomotor control and interactive behaviors. They are programming robotic behavior based on how we humans “program” behavior in-or train-each other

Autonomy Infused Teleoperation with Application to BCI Manipulation

Robot teleoperation systems face a common set of challenges including latency, low-dimensional user commands, and asymmetric control inputs. User control with Brain-Computer Interfaces (BCIs) exacerbates these problems through especially noisy and erratic low-dimensional motion commands due to the difficulty in decoding neural activity. We introduce a general framework to address these challenges through a combination of computer vision, user intent inference, and arbitration between the human input and autonomous control schemes. Adjustable levels of assistance allow the system to balance the operator's capabilities and feelings of comfort and control while compensating for a task's difficulty. We present experimental results demonstrating significant performance improvement using the shared-control assistance framework on adapted rehabilitation benchmarks with two subjects implanted with intracortical brain-computer interfaces controlling a seven degree-of-freedom robotic manipulator as a prosthetic. Our results further indicate that shared assistance mitigates perceived user difficulty and even enables successful performance on previously infeasible tasks. We showcase the extensibility of our architecture with applications to quality-of-life tasks such as opening a door, pouring liquids from containers, and manipulation with novel objects in densely cluttered environments

Intrinsic and Extrinsic Biomechanical Factors in a Co-adaptive ECoG-based Brain Computer Interface

Paralysis, due to spinal cord injury, amyotrophic lateral sclerosis (ALS), or stroke, is the result of severed communication between the brain and the motor periphery. Brain computer interfaces (BCIs) are neuroprosthetic devices that create novel communication pathways by measuring and transforming neural activity into operational commands. State of the art BCI systems measure brain activity using penetrating electrode arrays able to record from hundreds of individual cortical neurons simultaneously. Unfortunately, these systems are highly susceptible to signal degradation which limits their efficacy to 1-2 years. However, electrocorticography (ECoG) signals recorded from the surface of the brain deliver a more competitive balance between surgical risk, long-term stability, signal bandwidth, and signal-to-noise ratio when compared to both the aforementioned intracortical systems and the more common non-invasive electroencephalography (EEG) technologies. Historically, neural signals for controlling a computer cursor or robotic arm have been mapped to extrinsic, kinematic (i.e. position or velocity) variables. Although this strategy is adequate for use in simple environments, it may not be ideal for control of real-world prosthetic devices that are subject to external and unexpected forces. When reaching for an object, the trajectory of the hand through space can be defined in either extrinsic (e.g. Cartesian) or intrinsic (e.g. joint angles, muscle forces) frames of reference. During this movement, the brain has to perform a series of sensorimotor transformations that involve solving a complex, 2nd order differential equation (i.e. musculoskeletal biomechanics) in order to determine the appropriate muscle activations. Functional neuromuscular stimulation (FNS) is a desirable BCI application because it attempts to restore motor function to paralyzed limbs through electrical excitation of muscles. Rather than applying the conventional extrinsic kinematic control signals to such a system, it may be more appropriate to map neural activity to muscle activation directly and allow the brain to develop its own transfer function. This dissertation examines the application of intrinsic decoding schemes to control an upper limb using ECoG in non-human primates. ECoG electrode arrays were chronically implanted in rhesus monkeys over sensorimotor cortex. A novel multi-joint reaching task was developed to train the subjects to control a virtual arm simulating muscle and inertial forces. Utilizing a co-adaptive algorithm (where both the brain adapts via biofeedback and the decoding algorithm adapts to improve performance), new decoding models were initially built over the course of the first 3-5 minutes of each daily experimental session and then continually adapted throughout the day. Three subjects performed the task using neural control signals mapped to 1) joint angular velocity, 2) joint torque, and 3) muscle forces of the virtual arm. Performance exceeded 97%, 93%, and 89% accuracy for the three control paradigms respectively. Neural control features in the upper gamma frequency bands (70-115 and 130-175 Hz) were found to be directionally tuned in an ordered fashion, with preferred directions varying topographically in the mediolateral direction without distinction between motor and sensory areas. Long-term stability was demonstrated by all three monkeys, which maintained performance at 42, 55, and 57 months post-implantation. These results provide insights into the capabilities of sensorimotor cortex for control of non-linear multi-joint reaching dynamics and present a first step toward design of intrinsic, force-based BCI systems suitable for long-term FNS applications

A Continuous Grasp Representation for the Imitation Learning of Grasps on Humanoid Robots

Models and methods are presented which enable a humanoid robot to learn reusable, adaptive grasping skills. Mechanisms and principles in human grasp behavior are studied. The findings are used to develop a grasp representation capable of retaining specific motion characteristics and of adapting to different objects and tasks. Based on the representation a framework is proposed which enables the robot to observe human grasping, learn grasp representations, and infer executable grasping actions

Stochastic optimal control with learned dynamics models

The motor control of anthropomorphic robotic systems is a challenging computational task mainly because of the high levels of redundancies such systems exhibit. Optimality principles provide a general strategy to resolve such redundancies in a task driven fashion. In particular closed loop optimisation, i.e., optimal feedback control (OFC), has served as a successful motor control model as it unifies important concepts such as costs, noise, sensory feedback and internal models into a coherent mathematical framework. Realising OFC on realistic anthropomorphic systems however is non-trivial: Firstly, such systems have typically large dimensionality and nonlinear dynamics, in which case the optimisation problem becomes computationally intractable. Approximative methods, like the iterative linear quadratic gaussian (ILQG), have been proposed to avoid this, however the transfer of solutions from idealised simulations to real hardware systems has proved to be challenging. Secondly, OFC relies on an accurate description of the system dynamics, which for many realistic control systems may be unknown, difficult to estimate, or subject to frequent systematic changes. Thirdly, many (especially biologically inspired) systems suffer from significant state or control dependent sources of noise, which are difficult to model in a generally valid fashion. This thesis addresses these issues with the aim to realise efficient OFC for anthropomorphic manipulators. First we investigate the implementation of OFC laws on anthropomorphic hardware. Using ILQG we optimally control a high-dimensional anthropomorphic manipulator without having to specify an explicit inverse kinematics, inverse dynamics or feedback control law. We achieve this by introducing a novel cost function that accounts for the physical constraints of the robot and a dynamics formulation that resolves discontinuities in the dynamics. The experimental hardware results reveal the benefits of OFC over traditional (open loop) optimal controllers in terms of energy efficiency and compliance, properties that are crucial for the control of modern anthropomorphic manipulators. We then propose a new framework of OFC with learned dynamics (OFC-LD) that, unlike classic approaches, does not rely on analytic dynamics functions but rather updates the internal dynamics model continuously from sensorimotor plant feedback. We demonstrate how this approach can compensate for unknown dynamics and for complex dynamic perturbations in an online fashion. A specific advantage of a learned dynamics model is that it contains the stochastic information (i.e., noise) from the plant data, which corresponds to the uncertainty in the system. Consequently one can exploit this information within OFC-LD in order to produce control laws that minimise the uncertainty in the system. In the domain of antagonistically actuated systems this approach leads to improved motor performance, which is achieved by co-contracting antagonistic actuators in order to reduce the negative effects of the noise. Most importantly the shape and source of the noise is unknown a priory and is solely learned from plant data. The model is successfully tested on an antagonistic series elastic actuator (SEA) that we have built for this purpose. The proposed OFC-LD model is not only applicable to robotic systems but also proves to be very useful in the modelling of biological motor control phenomena and we show how our model can be used to predict a wide range of human impedance control patterns during both, stationary and adaptation tasks