Modelling visual-vestibular integration and behavioural adaptation in the driving simulator

It is well established that not only vision but also other sensory modalities affect drivers’ control of their vehicles, and that drivers adapt over time to persistent changes in sensory cues (for example in driving simulators), but the mechanisms underlying these behavioural phenomena are poorly understood. Here, we consider the existing literature on how driver steering in slalom tasks is affected by down-scaling of vestibular cues, and propose, for the first time, a computational model of driver behaviour that can, based on neurobiologically plausible mechanisms, explain the empirically observed effects, namely: decreased task performance and increased steering effort during initial exposure, followed by a partial reversal of these effects as task exposure is prolonged. Unexpectedly, the model also reproduced another previously unexplained empirical finding: a local optimum for motion down-scaling, where path-tracking is better than when one-to-one motion cues are available. Overall, our findings suggest that: (1) drivers make direct use of vestibular information as part of determining appropriate steering actions, and (2) motion down-scaling causes a yaw rate underestimation phenomenon, where drivers behave as if the simulated vehicle is rotating more slowly than it is. However, (3) in the slalom task, a certain degree of such underestimation brings a path-tracking performance benefit. Furthermore, (4) behavioural adaptation in simulated slalom driving tasks may occur due to (a) down-weighting of vestibular cues, and/or (b) increased sensitivity in timing and magnitude of steering corrections, but (c) seemingly not in the form of a full compensatory rescaling of the received vestibular input. The analyses presented here provide new insights and hypotheses about simulated driving and simulator design, and the developed models can be used to support research on multisensory integration and behavioural adaptation in both driving and other task domains

A MECHANISTIC APPROACH TO POSTURAL DEVELOPMENT IN CHILDREN

Upright standing is intrinsically unstable and requires active control. The central nervous system's feedback process is the active control that integrates multi-sensory information to generate appropriate motor commands to control the plant (the body with its musculotendon actuators). Maintaining standing balance is not trivial for a developing child because the feedback and the plant are both developing and the sensory inputs used for feedback are continually changing. Knowledge gaps exist in characterizing the critical ability of adaptive multi-sensory reweighting for standing balance control in children. Furthermore, the separate contributions of the plant and feedback and their relationship are poorly understood in children, especially when considering that the body is multi-jointed and feedback is multi-sensory. The purposes of this dissertation are to use a mechanistic approach to study multi-sensory abilities of typically developing (TD) children and children with Developmental Coordination Disorder (DCD). The specific aims are: 1) to characterize postural control under different multi-sensory conditions in TD children and children with DCD; 2) to characterize the development of adaptive multi-sensory reweighting in TD children and children with DCD; and, 3) to identify the plant and feedback for postural control in TD children and how they change in response to visual reweighting. In the first experiment (Aim 1), TD children, adults, and 7-year-old children with DCD are tested under four sensory conditions (no touch/no vision, with touch/no vision, no touch/with vision, and with touch/with vision). We found that touch robustly attenuated standing sway in all age groups. Children with DCD used touch less effectively than their TD peers and they also benefited from using vision to reduce sway. In the second experiment (Aim 2), TD children (4- to 10-year-old) and children with DCD (6- to 11-year-old) were presented with simultaneous small-amplitude touch bar and visual scene movement at 0.28 and 0.2 Hz, respectively, within five conditions that independently varied the amplitude of the stimuli. We found that TD children can reweight to both touch and vision from 4 years on and the amount of reweighting increased with age. However, multi-sensory fusion (i.e., inter-modal reweighting) was only observed in the older children. Children with DCD reweight to both touch and vision at a later age (10.8 years) than their TD peers. Even older children with DCD do not show advanced multisensory fusion. Two signature deficits of multisensory reweighting are a weak vision reweighting and a general phase lag to both sensory modalities. The final aim involves closed-loop system identification of the plant and feedback using electromyography (EMG) and kinematic responses to a high- or low-amplitude visual perturbation and two mechanical perturbations in children ages six and ten years and adults. We found that the plant is different between children and adults. Children demonstrate a smaller phase difference between trunk and leg than adults at higher frequencies. Feedback in children is qualitatively similar to adults. Quantitatively, children show less phase advance at the peak of the feedback curve which may be due to a longer time delay. Under the high and low visual amplitude conditions, children show less gain change (interpreted as reweighting) than adults in the kinematic and EMG responses. The observed kinematic and EMG reweighting are mainly due to the different use of visual information by the central nervous system as measured by the open-loop mapping from visual scene angle to EMG activity. The plant and the feedback do not contribute to reweighting

Doctor of Philosophy

dissertationIntegration of sensory inputs by the central nervous system (CNS) is necessary for adequate postural stability, but diminishes with age and is further impaired in Parkinson disease (PD). As a result, the CNS cannot appropriately weight sensory stimuli to facilitate postural responses to sudden changes in sensory input. Training the sensorimotor system to ignore or rapidly adapt to aberrant postural cues may improve postural control in PD. We evaluated the influence of acute and repeated exposure to galvanic vestibular stimulation (GVS) on postural responses during static and dynamic tasks to determine whether training improved these responses. We hypothesized that individuals with PD would demonstrate impaired postural recovery responses to acute GVS relative to healthy controls and that individuals with PD and healthy elders would demonstrate diminished adaptive responses to repeated GVS compared to young adults. Twelve individuals with PD (PD group), 15 healthy young adults (HY group), and 11 healthy elders (HE group) participated. Timing of GVS was randomly applied during each task. Fifteen acquisition and nine retention trials with GVS were compared to assess learning. The PD group took longer to stabilize their center of pressure (COP) in quiet stance following GVS acutely compared to controls. The PD and HE groups had lower sample entropy (SaEn) compared to the HY. Neither the PD nor HE groups demonstrated changes in SaEn or meaningful improvements in postural control during acquisition or retention. SaEn in the HY group acutely decreased and then increased at retention which coincided with a meaningful improvement in postural control. The PD group had impaired motor planning, postural preparation, and postural stability during a rise to toes task following acute GVS, but these constructs returned to baseline at later acquisition and retention time points. Controls suppressed GVS acutely Postural coordination decreased acutely in the PD group during tether release. This persisted and an adaptive trend in BOS transition was noted with repeated GVS exposure in this group. No changes were observed in the control groups. Taken together, these results demonstrated that acute GVS differentially affects postural control in individuals with PD. Our results support the hypothesis that reweighting of sensory stimuli is impaired in PD. We also show that individuals with PD are able to suppress attention to a vestibular illusion and demonstrate adaptive responses to a postural threat

Identifying Plant and Feedback in Human Posture Control

Human upright bipedal stance is a classic example of a control system consisting of a plant (i.e., the physical body and its actuators) and feedback (i.e., neural control) operating continuously in a closed loop. Determining the mechanistic basis of behavior in a closed loop control system is problematic because experimental manipulations or deficits due to trauma/injury influence all parts of the loop. Moreover, experimental techniques to open the loop (e.g., isolate the plant) are not viable because bipedal upright stance is not possible without feedback. The goal of the proposed study is to use a technique called closed loop system identification (CLSI) to investigate properties of the plant and feedback separately. Human upright stance has typically been approximated as a single-joint inverted pendulum, simplifying not only the control of a multi-linked body but also how sensory information is processed relative to body dynamics. However, a recent study showed that a single-joint approximation is inadequate. Trunk and leg segments are in-phase at frequencies below 1 Hz of body sway and simultaneously anti-phase at frequencies above 1 Hz during quiet stance. My dissertation studies have investigated the coordination between the leg and trunk segments and how sensory information is processed relative to that coordination. For example, additional sensory information provided through visual or light touch information led to a change of the in-phase pattern but not the anti-phase pattern, indicating that the anti-phase pattern may not be neurally controlled, but more a function of biomechanical properties of a two-segment body. In a subsequent study, I probed whether an internal model of the body processes visual information relative to a single or double-linked body. The results suggested a simple control strategy that processes sensory information relative to a single-joint internal model providing further evidence that the anti-phase pattern is biomechanically driven. These studies suggest potential mechanisms but cannot rule out alternative hypotheses because the source of behavioral changes can be attributed to properties of the plant and/or feedback. Here I adopt the CLSI approach using perturbations to probe separate processes within the postural control loop. Mechanical perturbations introduce sway as an input to the feedback, which in turn generates muscle activity as an output. Visual perturbations elicit muscle activity (a motor command) as an input to the plant, which then triggers body sway as an output. Mappings of muscle activity to body sway and body sway to muscle activity are used to identify properties of the plant and feedback, respectively. The results suggest that feedback compensates for the low-pass properties of the plant, except at higher frequencies. An optimal control model minimizing the amount of muscle activation suggests that the mechanism underlying this lack of compensation may be due to an uncompensated time delay. These techniques have the potential for more precise identification of the source of deficits in the postural control loop, leading to improved rehabilitation techniques and treatment of balance deficits, which currently contributes to 40% of nursing home admissions and costs the US health care system over $20B per year

High reward enhances perceptual learning.

Studies of perceptual learning have revealed a great deal of plasticity in adult humans. In this study, we systematically investigated the effects and mechanisms of several forms (trial-by-trial, block, and session rewards) and levels (no, low, high, subliminal) of monetary reward on the rate, magnitude, and generalizability of perceptual learning. We found that high monetary reward can greatly promote the rate and boost the magnitude of learning and enhance performance in untrained spatial frequencies and eye without changing interocular, interlocation, and interdirection transfer indices. High reward per se made unique contributions to the enhanced learning through improved internal noise reduction. Furthermore, the effects of high reward on perceptual learning occurred in a range of perceptual tasks. The results may have major implications for the understanding of the nature of the learning rule in perceptual learning and for the use of reward to enhance perceptual learning in practical applications

The role of vestibular cues in postural sway

Controlling posture requires continuous sensory feedback about body motion and orientation, including from the vestibular organs. Little is known about the role of tilt vs. translation vs. rotation vestibular cues. We examined whether intersubject differences in vestibular function were correlated with intersubject differences in postural control. Vestibular function was assayed using vestibular direction-recognition perceptual thresholds, which determine the smallest motion that can be reliably perceived by a subject seated on a motorized platform in the dark. In study A, we measured thresholds for lateral translation, vertical translation, yaw rotation, and head-centered roll tilts. In study B, we measured thresholds for roll, pitch, and left anterior-right posterior and right anterior-left posterior tilts. Center-of-pressure (CoP) sway was measured in sensory organization tests (study A) and Romberg tests (study B). We found a strong positive relationship between CoP sway and lateral translation thresholds but not CoP sway and other thresholds. This finding suggests that the vestibular encoding of lateral translation may contribute substantially to balance control. Since thresholds assay sensory noise, our results support the hypothesis that vestibular noise contributes to spontaneous postural sway. Specifically, we found that lateral translation thresholds explained more of the variation in postural sway in postural test conditions with altered proprioceptive cues (vs. a solid surface), consistent with postural sway being more dependent on vestibular noise when the vestibular contribution to balance is higher. These results have potential implications for vestibular implants, balance prostheses, and physical therapy exercises.NEW & NOTEWORTHY Vestibular feedback is important for postural control, but little is known about the role of tilt cues vs. translation cues vs. rotation cues. We studied healthy human subjects with no known vestibular pathology or symptoms. Our findings showed that vestibular encoding of lateral translation correlated with medial-lateral postural sway, consistent with lateral translation cues contributing to balance control. This adds support to the hypothesis that vestibular noise contributes to spontaneous postural sway

Combining Model-Based and Model-Free Updates for Trajectory-Centric Reinforcement Learning

Reinforcement learning (RL) algorithms for real-world robotic applications need a data-efficient learning process and the ability to handle complex, unknown dynamical systems. These requirements are handled well by model-based and model-free RL approaches, respectively. In this work, we aim to combine the advantages of these two types of methods in a principled manner. By focusing on time-varying linear-Gaussian policies, we enable a model-based algorithm based on the linear quadratic regulator (LQR) that can be integrated into the model-free framework of path integral policy improvement (PI2). We can further combine our method with guided policy search (GPS) to train arbitrary parameterized policies such as deep neural networks. Our simulation and real-world experiments demonstrate that this method can solve challenging manipulation tasks with comparable or better performance than model-free methods while maintaining the sample efficiency of model-based methods. A video presenting our results is available at https://sites.google.com/site/icml17pilqrComment: Paper accepted to the International Conference on Machine Learning (ICML) 201

Analysis of Human Push Recovery Motions Based on Optimization

The ability to cope with large perturbations is essential to avoid falling for humans as well as for humanoid robots. Every day millions of people are affected by injuries due to falling. This is a huge problem not only for the individuum but also for the society as it costs the health care systems billions of euros. Also in the field of humanoid robots fall avoidance is very important as it protects robots against breakage. In this thesis, the problem of fall avoidance is addressed using a combination of optimization, human-modeling and recorded push recovery motions. The aim is to identify the principles that lead to human-like push recovery motions. The human is modeled by rigid segments combined by joints leading to an underactuated multi-body representation. These models are included in multiple stage optimal control problems to reconstruct and sythesize human push recovery motions considering the dynamics of a human over the whole time horizon. Due to the high nonlinearity, the optimization problem is solved based on a direct multiple shooting method. To analyze the human push recovery motions, dynamically-consistent motions for the model that closely track experimental data are produced. The joint angles and joint torques for the human model controlled by joint torque derivatives are compared for perturbed and unperturbed motions from two subjects. The results verify the assumption that the heavier the perturbation is and the higher it is applied at the upper body, the larger are the resulting joint torques. We show that including optimally chosen spring-damper elements in the joints can reduce the active joint torques significantly. We further exploit our motion reconstruction approach to determine the states that are most affected during a perturbation. Relevant parameters such as the orientation and position of the head and body, joint angles and torques of the perturbed motions are analyzed for deviations to the unperturbed motions at the point in time when the push occurs. Identifying the point in time when the model states of the perturbed motions differ from the unperturbed motions, the reaction times are determined. To better understand human push recovery motions, we also investigate in a motion sythesis approach. This approach enables a control hypothesis, in the form of a specific objective function, to be formed. The minimization of effort combined with a periodicity formulation results in human-like motions and the influence of the push strength is analyzed. Formulating the objective function as a weighted linear combination of possible optimality criteria provides the possibility to analyze different optimality criteria and their resulting motion. The difficulty is, that for a given motion, it is not known, which criteria lead to that specific motion. In this thesis, the results for different basal objective functions are analyzed. These studies prepare to determine the optimal weights of the criteria by including the presented motion generation formulation in an inverse optimal control problem. Having analyzed general weights that lead to a good approximation of the human recovery motions, the resulting objective function can be used to generate push recovery motions also for humanoid robots or assistive devices such as exoskeletons. To show another application in the improvement of technical assistive devices, we include two combined human exoskeleton models of different weights in our calculations. This allows us to analyze the joint torques for these models including the exoskeletons and compare the results to a human model. As the resulting joint torques are quite large, we also formulate combined human exoskeleton models with passive spring-damper elements that act in parallel to the active torques. This compliant formulation leads to a significant reduction of the active joint torque needed for the recovery motion. The reduction of the active joint torques allows the reduction of energy needed for the recovery motion or can enable the recovery from stronger perturbations