Understanding motor control in humans to improve rehabilitation robots

Recent reviews highlighted the limited results of robotic rehabilitation and the low quality of evidences in this field. Despite the worldwide presence of several robotic infrastructures, there is still a lack of knowledge about the capabilities of robotic training effect on the neural control of movement. To fill this gap, a step back to motor neuroscience is needed: the understanding how the brain works in the generation of movements, how it adapts to changes and how it acquires new motor skills is fundamental. This is the rationale behind my PhD project and the contents of this thesis: all the studies included in fact examined changes in motor control due to different destabilizing conditions, ranging from external perturbations, to self-generated disturbances, to pathological conditions. Data on healthy and impaired adults have been collected and quantitative and objective information about kinematics, dynamics, performance and learning were obtained for the investigation of motor control and skill learning. Results on subjects with cervical dystonia show how important assessment is: possibly adequate treatments are missing because the physiological and pathological mechanisms underlying sensorimotor control are not routinely addressed in clinical practice. These results showed how sensory function is crucial for motor control. The relevance of proprioception in motor control and learning is evident also in a second study. This study, performed on healthy subjects, showed that stiffness control is associated with worse robustness to external perturbations and worse learning, which can be attributed to the lower sensitiveness while moving or co-activating. On the other hand, we found that the combination of higher reliance on proprioception with \u201cdisturbance training\u201d is able to lead to a better learning and better robustness. This is in line with recent findings showing that variability may facilitate learning and thus can be exploited for sensorimotor recovery. Based on these results, in a third study, we asked participants to use the more robust and efficient strategy in order to investigate the control policies used to reject disturbances. We found that control is non-linear and we associated this non-linearity with intermittent control. As the name says, intermittent control is characterized by open loop intervals, in which movements are not actively controlled. We exploited the intermittent control paradigm for other two modeling studies. In these studies we have shown how robust is this model, evaluating it in two complex situations, the coordination of two joints for postural balance and the coordination of two different balancing tasks. It is an intriguing issue, to be addressed in future studies, to consider how learning affects intermittency and how this can be exploited to enhance learning or recovery. The approach, that can exploit the results of this thesis, is the computational neurorehabilitation, which mathematically models the mechanisms underlying the rehabilitation process, with the aim of optimizing the individual treatment of patients. Integrating models of sensorimotor control during robotic neurorehabilitation, might lead to robots that are fully adaptable to the level of impairment of the patient and able to change their behavior accordingly to the patient\u2019s intention. This is one of the goals for the development of rehabilitation robotics and in particular of Wristbot, our robot for wrist rehabilitation: combining proper assessment and training protocols, based on motor control paradigms, will maximize robotic rehabilitation effects

Computational and Robotic Models of Human Postural Control

Currently, no bipedal robot exhibits fully human-like characteristics in terms of its postural control and movement. Current biped robots move more slowly than humans and are much less stable. Humans utilize a variety of sensory systems to maintain balance, primary among them being the visual, vestibular and proprioceptive systems. A key finding of human postural control experiments has been that the integration of sensory information appears to be dynamically regulated to adapt to changing environmental conditions and the available sensory information, a process referred to as "sensory re-weighting." In contrast, in robotics, the emphasis has been on controlling the location of the center of pressure based on proprioception, with little use of vestibular signals (inertial sensing) and no use of vision. Joint-level PD control with only proprioceptive feedback forms the core of robot standing balance control. More advanced schemes have been proposed but not yet implemented. The multiple sensory sources used by humans to maintain balance allow for more complex sensorimotor strategies not seen in biped robots, and arguably contribute to robust human balance function across a variety of environments and perturbations. Our goal is to replicate this robust human balance behavior in robots.In this work, we review results exploring sensory re-weighting in humans, through a series of experimental protocols, and describe implementations of sensory re-weighting in simulation and on a robot

Humanoid robot control of complex postural tasks based on learning from demostration

Mención Internacional en el título de doctorThis thesis addresses the problem of planning and controlling complex tasks in a humanoid robot from a postural point of view. It is motivated by the growth of robotics in our current society, where simple robots are being integrated. Its objective is to make an advancement in the development of complex behaviors in humanoid robots, in order to allow them to share our environment in the future. The work presents different contributions in the areas of humanoid robot postural control, behavior planning, non-linear control, learning from demonstration and reinforcement learning. First, as an introduction of the thesis, a group of methods and mathematical formulations are presented, describing concepts such as humanoid robot modelling, generation of locomotion trajectories and generation of whole-body trajectories. Next, the process of human learning is studied in order to develop a novel method of postural task transference between a human and a robot. It uses the demonstrated action goal as a metrics of comparison, which is codified using the reward associated to the task execution. As an evolution of the previous study, this process is generalized to a set of sequential behaviors, which are executed by the robot based on human demonstrations. Afterwards, the execution of postural movements using a robust control approach is proposed. This method allows to control the desired trajectory even with mismatches in the robot model. Finally, an architecture that encompasses all methods of postural planning and control is presented. It is complemented by an environment recognition module that identifies the free space in order to perform path planning and generate safe movements for the robot. The experimental justification of this thesis was developed using the humanoid robot HOAP-3. Tasks such as walking, standing up from a chair, dancing or opening a door have been implemented using the techniques proposed in this work.Esta tesis aborda el problema de la planificación y control de tareas complejas de un robot humanoide desde el punto de vista postural. Viene motivada por el auge de la robótica en la sociedad actual, donde ya se están incorporando robots sencillos y su objetivo es avanzar en el desarrollo de comportamientos complejos en robots humanoides, para que en el futuro sean capaces de compartir nuestro entorno. El trabajo presenta diferentes contribuciones en las áreas de control postural de robots humanoides, planificación de comportamientos, control no lineal, aprendizaje por demostración y aprendizaje por refuerzo. En primer lugar se desarrollan un conjunto de métodos y formulaciones matemáticas sobre los que se sustenta la tesis, describiendo conceptos de modelado de robots humanoides, generación de trayectorias de locomoción y generación de trayectorias del cuerpo completo. A continuación se estudia el proceso de aprendizaje humano, para desarrollar un novedoso método de transferencia de una tarea postural de un humano a un robot, usando como métrica de comparación el objetivo de la acción demostrada, que es codificada a través del refuerzo asociado a la ejecución de dicha tarea. Como evolución del trabajo anterior, se generaliza este proceso para la realización de un conjunto de comportamientos secuenciales, que son de nuevo realizados por el robot basándose en las demostraciones de un ser humano. Seguidamente se estudia la ejecución de movimientos posturales utilizando un método de control robusto ante imprecisiones en el modelado del robot. Para analizar, se presenta una arquitectura que aglutina los métodos de planificación y el control postural desarrollados en los capítulos anteriores. Esto se complementa con un módulo de reconocimiento del entorno y extracción del espacio libre para poder planificar y generar movimientos seguros en dicho entorno. La justificación experimental de la tesis se ha desarrollado con el robot humanoide HOAP-3. En este robot se han implementado tareas como caminar, levantarse de una silla, bailar o abrir una puerta. Todo ello haciendo uso de las técnicas propuestas en este trabajo.Programa Oficial de Doctorado en Ingeniería Eléctrica, Electrónica y AutomáticaPresidente: Manuel Ángel Armada Rodríguez.- Secretario: Luis Santiago Garrido Bullón.- Vocal: Sylvain Calino

Postural control: learning to balance and responses to mechanical and sensory perturbations

The purpose of the current research was to examine how a novel balance task is learnt by individuals with a mature neurological system, and to investigate the responses of experienced hand balancers to mechanical and sensory perturbations. Balance in each posture was assessed by various techniques, including: traditional measures of centre of pressure, nonlinear time series analysis of centre of pressure, estimates of feedback time delay from cross correlations and delayed regression models, and calculation of small, medium, and large movement corrections. Data from this study suggests that the best balance metric for distinguishing between each of the balance conditions was the traditional balance measure of sway velocity. However, sway velocity cannot provide any further information on the underlying process of balance. Nonlinear measures of balance offer insight into the underlying deterministic processes that control balance, offering measures of system determinism, complexity, and predictability. Assessments of feedback time delay and movement corrections provide both an insight into the control of posture and help distinguish one condition from another. Both feedback time delay and movement corrections and magnitudes may be used simultaneously to delve further into the control of posture. Delayed regression models seem to be an appropriate and useful tool for estimating feedback time delays during balance. Findings support the use of the third term in the adapted regression model as a means of estimating the effect of passive stiffness on feedback time delay. Generally, with increased duration in handstand subjects displayed reduced sway as measured by traditional measures of balance. A more marked change in nonlinear measures of balance can be seen, with quicker reductions in variance for some nonlinear measures of balance than in the traditional measures. It may be that more pronounced changes in nonlinear measures represent changes in the subjects underlying process of postural control, whereas less pronounced changes in traditional measures relate more to their general ability or performance in the balance task

Biomechanics of Leaning and Downward Reaching Tasks in Young and Older Women.

Stooping crouching or kneeling (SCK) difficulty is prevalent among older adults yet few studies have explored the mechanisms underlying downward reaching and pick-up difficulty. During targeted movements tradeoffs are expected between the speed and accuracy of center of pressure (COP) movements as balance is maintained. Thus, this research focused on how age-related changes in COP control strategies affect the performance of tasks with a large range of truncal motion and momenta. It was hypothesized that while performing leaning and downward reaching movements, older women, compared to younger women, would exhibit slower but more frequent COP submovements in order to accomplish the task and regain the upright posture. First, we investigated the limiting factors in downward reach and pick-up movements. Using an age-adjusted proportional odds model, increased SCK difficulty was found to be independently associated with balance confidence, leg joint limitations, and knee extension strength. Secondly, we explored the age-related changes in COP control in healthy women. Despite being 27% slower, older women rely on nearly twice as many submovements to maintain a similar level of endpoint accuracy in volitional COP movements, particularly when moving posteriorly. Furthermore, older women used slower primary submovements that more often undershot their target, in comparison to young women, particularly as movement amplitude increased. Lastly, healthy older women were found to lose their balance more often than young women in downward reaching tasks, but rely on similar COP control strategies when successful. Modeling results suggest that a simple forward dynamic model that accounts for changes in musculoskeletal factors may distinguish between healthy young and healthy older women with and without SCK difficulty. We conclude that biomechanical factors can distinguish between older women with and without SCK difficulty. Given the significance of the rate of torque development in arresting downward reaching movements, changes in COP control may be effective tools in evaluating early signs of physical impairment. Undershooting primary submovements and increased secondary submovements are indicative of an increasingly conservative strategy used by older adults near the limits of the base of support that may explain their slower speeds during whole body movements to maintain balance.Ph.D.Biomedical EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/91565/1/manueleh_1.pd

An Optimal Control Model for Analyzing Human Postural Balance

Abstract-The question posed in this study is whether optimal control and state estimation can explain selection of control strategies used by humans, in response to small perturbations to stable upright balance. To answer this question, a human sensorimotor control model, compatible with previous work by others, was assembled. This model incorporates linearized equations and full-state feedback with provision for state estimation. A form of gain-scheduling is employed to account for nonlinearities caused by control and biomechanical constraints. By decoupling the mechanics and transforming the controls into the space of experimentally observed strategies, the model is made amenable to the study of a number of possible control objectives. The objectives studied include cost functions on the state deviations, so as to control the center of mass, provide a stable platform for the head, or maintain upright stance, along with a cost function on control effort. Also studied was the effect of time delay on the stability of controls produced using various control strategies. An objective function weighting excursion of the center of mass and deviations from the upright stable position, while taking advantage of fast modes of the system, as dictated by inertial parameters and musculoskeletal geometry, produces a control that reasonably matches experimental data. Given estimates of sensor performance, the model is also suited for prediction of uncertainty in the response

Identifying Plant and Feedback in Human Posture Control

Human upright bipedal stance is a classic example of a control system consisting of a plant (i.e., the physical body and its actuators) and feedback (i.e., neural control) operating continuously in a closed loop. Determining the mechanistic basis of behavior in a closed loop control system is problematic because experimental manipulations or deficits due to trauma/injury influence all parts of the loop. Moreover, experimental techniques to open the loop (e.g., isolate the plant) are not viable because bipedal upright stance is not possible without feedback. The goal of the proposed study is to use a technique called closed loop system identification (CLSI) to investigate properties of the plant and feedback separately. Human upright stance has typically been approximated as a single-joint inverted pendulum, simplifying not only the control of a multi-linked body but also how sensory information is processed relative to body dynamics. However, a recent study showed that a single-joint approximation is inadequate. Trunk and leg segments are in-phase at frequencies below 1 Hz of body sway and simultaneously anti-phase at frequencies above 1 Hz during quiet stance. My dissertation studies have investigated the coordination between the leg and trunk segments and how sensory information is processed relative to that coordination. For example, additional sensory information provided through visual or light touch information led to a change of the in-phase pattern but not the anti-phase pattern, indicating that the anti-phase pattern may not be neurally controlled, but more a function of biomechanical properties of a two-segment body. In a subsequent study, I probed whether an internal model of the body processes visual information relative to a single or double-linked body. The results suggested a simple control strategy that processes sensory information relative to a single-joint internal model providing further evidence that the anti-phase pattern is biomechanically driven. These studies suggest potential mechanisms but cannot rule out alternative hypotheses because the source of behavioral changes can be attributed to properties of the plant and/or feedback. Here I adopt the CLSI approach using perturbations to probe separate processes within the postural control loop. Mechanical perturbations introduce sway as an input to the feedback, which in turn generates muscle activity as an output. Visual perturbations elicit muscle activity (a motor command) as an input to the plant, which then triggers body sway as an output. Mappings of muscle activity to body sway and body sway to muscle activity are used to identify properties of the plant and feedback, respectively. The results suggest that feedback compensates for the low-pass properties of the plant, except at higher frequencies. An optimal control model minimizing the amount of muscle activation suggests that the mechanism underlying this lack of compensation may be due to an uncompensated time delay. These techniques have the potential for more precise identification of the source of deficits in the postural control loop, leading to improved rehabilitation techniques and treatment of balance deficits, which currently contributes to 40% of nursing home admissions and costs the US health care system over $20B per year

Sensorimotor postural control in healthy and pathological stance and gait

Postural control during standing and walking is an inherently unstable task requiring the interaction of various biomechanical, sensory, and neurophysiological mechanisms to shape stable patterns of whole-body coordination that are able to counteract postural disequilibrium. This thesis focused on the examination of central aspects of the functional roles of these mechanisms and the modes of interaction between them. A further aim was to determine the conditions of dynamic stability for healthy standing and walking performance as well as for certain balance and gait disorders. By studying movement fluctuations in the walking pattern it could be demonstrated that dynamic stability during walking depends on gait speed and is differentially regulated for the medio-lateral and the fore-aft walking planes. Stability control in the fore-aft walking plane exhibits attractor dynamics typical for a dynamical system. Accordingly, the most stable pattern of movement coordination in terms of minimal fluctuations in the order parameter (i.e., the relative phase between the two oscillating legs) can be observed at the attractor of self-paced walking. Critical fluctuations occur at increasingly non-preferred speeds, indicating a loss of dynamic gait stability close to the speed boundaries of the walking mode. Moreover, stability control during slow walking is critically dependent on sensory feedback control, whereas dynamic stability during fast walking relies mainly on the smooth operation of cerebellar pacemaker regions. Disturbances of sensory and cerebellar locomotor control in certain gait disorders could be further linked to a loss of dynamic gait stability, in particular an increased risk of falls. Furthermore, this thesis examined alterations in the sensorimotor postural control scheme that may trigger the experience of subjective imbalance and vertigo in the conditions of phobic postural vertigo and visual height intolerance. Both conditions are characterized by an inadequate mode of balance regulation featuring increased levels of open-loop balance control and a precipitate integration of closed-loop sensory feedback into the postural control scheme. This inadequate balance control strategy is accompanied by a stiffening of the anti-gravity musculature and is elicited by specific influences of attention and sensory feedback control. The findings of this thesis contribute to the understanding of central sensorimotor mechanisms involved in the control of dynamic postural stability during standing and walking. They further provide relevant information for the differential diagnosis and fall risk estimation of certain balance and gait disorders

協調的な人間のバランス回復モダリティ再現のための予測と学習制御戦略

Tohoku University林部充宏課

Human-Inspired Balancing and Recovery Stepping for Humanoid Robots

Robustly maintaining balance on two legs is an important challenge for humanoid robots. The work presented in this book represents a contribution to this area. It investigates efficient methods for the decision-making from internal sensors about whether and where to step, several improvements to efficient whole-body postural balancing methods, and proposes and evaluates a novel method for efficient recovery step generation, leveraging human examples and simulation-based reinforcement learning