Uncertainty of Feedback and State Estimation Determines the Speed of Motor Adaptation

Humans can adapt their motor behaviors to deal with ongoing changes. To achieve this, the nervous system needs to estimate central variables for our movement based on past knowledge and new feedback, both of which are uncertain. In the Bayesian framework, rates of adaptation characterize how noisy feedback is in comparison to the uncertainty of the state estimate. The predictions of Bayesian models are intuitive: the nervous system should adapt slower when sensory feedback is more noisy and faster when its state estimate is more uncertain. Here we want to quantitatively understand how uncertainty in these two factors affects motor adaptation. In a hand reaching experiment we measured trial-by-trial adaptation to a randomly changing visual perturbation to characterize the way the nervous system handles uncertainty in state estimation and feedback. We found both qualitative predictions of Bayesian models confirmed. Our study provides evidence that the nervous system represents and uses uncertainty in state estimate and feedback during motor adaptation

Chance, long tails, and inference: a non-Gaussian, Bayesian theory of vocal learning in songbirds

Traditional theories of sensorimotor learning posit that animals use sensory error signals to find the optimal motor command in the face of Gaussian sensory and motor noise. However, most such theories cannot explain common behavioral observations, for example that smaller sensory errors are more readily corrected than larger errors and that large abrupt (but not gradually introduced) errors lead to weak learning. Here we propose a new theory of sensorimotor learning that explains these observations. The theory posits that the animal learns an entire probability distribution of motor commands rather than trying to arrive at a single optimal command, and that learning arises via Bayesian inference when new sensory information becomes available. We test this theory using data from a songbird, the Bengalese finch, that is adapting the pitch (fundamental frequency) of its song following perturbations of auditory feedback using miniature headphones. We observe the distribution of the sung pitches to have long, non-Gaussian tails, which, within our theory, explains the observed dynamics of learning. Further, the theory makes surprising predictions about the dynamics of the shape of the pitch distribution, which we confirm experimentally

스티어 바이 와이어 시스템의 목표 조향감 재현을 위한 조향 반력 제어

학위논문(박사)--서울대학교 대학원 :공과대학 기계공학부,2020. 2. 이경수.This dissertation focused on the development of and steering assist torque control algorithm of Electric-Power-Steering (EPS) system from the conventional steering system perspective and Steer-by-Wire (SBW) system. The steering assist torque control algorithm has been developed to overcome the major disadvantage of the conventional method of time-consuming tuning to achieve the desired steering feel. A reference steering wheel torque map was designed by post-processing data obtained from target performance vehicle tests with a highly-rated steering feel for both sinusoidal and transition steering inputs. Adaptive sliding-mode control was adopted to ensure robustness against uncertainty in the steering system, and the equivalent moment of inertia damping coefficient and effective compliance were adapted to improve tracking performance. Effective compliance played a role in compensating the error between the nominal rack force and the actual rack force. For the SBW system, the previously proposed EPS assist torque algorithm has been also enhanced using impedance model and applied to steering feedback system. Stable execution and how to give the person the proper steering feedback torque of contact tasks by steering wheel system interaction with human has been identified as one of the major challenges in SBW system. Thus, the problem was solved by utilizing the target steering torque map proposed above. The impedance control consists of impedance model (Reference model with the target steering wheel torque map) and controller (Adaptive sliding mode control). The performance of the proposed controller was evaluated by conducting computer simulations and a hardware-in-the-loop simulation (HILS) under various steering conditions. Optimal steering wheel torque tracking performances were successfully achieved by the proposed EPS and SBW control algorithm.본 논문은 종래의 조향 시스템 관점에서 전동식 동력 조향 (EPS) 시스템과 스티어 바이 와이어 (SBW) 조향 보조 토크 제어 알고리즘의 개발을 중점으로 하였습니다. 기존 조향 보조 토크 제어 알고리즘은 원하는 조향감을 구현하기 위해 종래의 시간 소모적 인 튜닝 방법을 사용합니다. 이러한 주요 단점을 극복하기 위해 새로운 조향 보조 제어 알고리즘을 개발하였습니다. 목표 스티어링 휠 토크 맵은 정현파(Weave test) 및 등속도 스티어링 입력 (Transition test) 모두에 대해 높은 등급의 조향감을 차량 테스트에서 얻은 후 데이터 처리를 하여 설계되었습니다. 스티어링 시스템의 불확실성에 대한 강건성을 보장하기 위해 적응 형 슬라이딩 모드 제어가 채택되었으며, 관성 모멘트 감쇠 계수와 컴플라이언스 계수(Effective compliance)가 제어기 성능을 개선하도록 적응형 파라미터로 선정되었습니다. 컴플라이언스 계수는 계산된 랙 힘과 실제 랙 힘 사이의 차이를 보상하는 역할을 했습니다. SBW 시스템의 경우, 이전에 제안 된 EPS 지원 토크 알고리즘을 개선하고 향상시키기 위해 임피던스 모델을 사용하였으며 스티어링 피드백 시스템에 적용되었습니다. SBW 시스템의 주요 과제 중 하나는 사람과 스티어링 휠 시스템 상호 작용에 의해 안정적인 작동과 사람에게 적절한 스티어링 피드백 토크를 제공하는 방법입니다. 임피던스 제어는 임피던스 모델 (타겟 스티어링 휠 토크 맵)과 컨트롤러 (적응 슬라이딩 모드 제어)로 구성됩니다. 따라서, 상기 제안 된 목표 조향 토크 맵을 이용함으로써 스티어 바이 와이어에서 스티어링 피드백 토크를 절절히 적용 됨을 확인 하였습니다. 제안 된 컨트롤러의 성능은 다양한 조향 조건에서 컴퓨터 시뮬레이션 및 HILS (Hardware-in-the-loop) 시뮬레이션을 수행하여 평가되었습니다. 제안 된 EPS 및 SBW 제어 알고리즘을 통해 최적의 스티어링 휠 토크 추적 성능을 달성했습니다.Chapter 1 Introduction 1 1.1. Background and Motivation 1 1.2. Previous Researches 4 1.3. Thesis Objectives 9 1.4. Thesis Outline 10 Chapter 2 Dynamic Model of Steering Systems 11 2.1. Dynamic model of Hydraulic/Electrohydraulic Power-Assisted Steering Model 11 2.2. Dynamic model of Electric-Power-Assisted-Steering Model 17 2.3. Dynamic model of Steer-by-Wire Model 21 2.4. Rack force characteristic of steering system 23 Chapter 3 Target steering wheel torque tracking control 28 3.1. Target steering torque map generation 28 3.2. Adaptive sliding mode control design for target steering wheel torque tracking with EPS 30 3.2.1. Steering states estimation with a kalman filter 38 3.3. Impedance Control Design for Target Steering Wheel Torque Tracking with SBW 43 Chapter 4 Validation with Simulation and Hardware-in-the-Loops Simulation 49 4.1. Computer Simulation Results for EPS system 49 4.2. Hardware-in-the-Loops Simulation Results for EPS system 61 4.3. Computer Simulation Results for SBW system 77 4.4. Hardware-in-the-Loops Simulation Results for SBW system 82 Chapter 5 Conclusion and Future works 89 Bibliography 91 Abstract in Korean 97Docto

How does our motor system determine its learning rate?

Motor learning is driven by movement errors. The speed of learning can be quantified by the learning rate, which is the proportion of an error that is corrected for in the planning of the next movement. Previous studies have shown that the learning rate depends on the reliability of the error signal and on the uncertainty of the motor system’s own state. These dependences are in agreement with the predictions of the Kalman filter, which is a state estimator that can be used to determine the optimal learning rate for each movement such that the expected movement error is minimized. Here we test whether not only the average behaviour is optimal, as the previous studies showed, but if the learning rate is chosen optimally in every individual movement. Subjects made repeated movements to visual targets with their unseen hand. They received visual feedback about their endpoint error immediately after each movement. The reliability of these error-signals was varied across three conditions. The results are inconsistent with the predictions of the Kalman filter because correction for large errors in the beginning of a series of movements to a fixed target was not as fast as predicted and the learning rates for the extent and the direction of the movements did not differ in the way predicted by the Kalman filter. Instead, a simpler model that uses the same learning rate for all movements with the same error-signal reliability can explain the data. We conclude that our brain does not apply state estimation to determine the optimal planning correction for every individual movement, but it employs a simpler strategy of using a fixed learning rate for all movements with the same level of error-signal reliability

Signal-Injection-Assisted Full-Order Observer With Parameter Adaptation for Synchronous Reluctance Motor Drives

A back-EMF-based position observer for motion-sensorless synchronous reluctance motor (SyRM) drives is augmented with parameter-adaptation laws for improved operation at all speeds, including standstill. The augmented observer is theoretically analyzed under various operation conditions. The analysis indicates that the stator-resistance adaptation should be enabled only at low speeds, the d-axis inductance adaptation should be enabled only at medium and high speeds near no load, and the q-axis inductance adaptation should be enabled only at high speeds under high load. The augmented observer is experimentally evaluated using a 6.7-kW SyRM drive.Peer reviewe

Bayesian Optimization Using Domain Knowledge on the ATRIAS Biped

Controllers in robotics often consist of expert-designed heuristics, which can be hard to tune in higher dimensions. It is typical to use simulation to learn these parameters, but controllers learned in simulation often don't transfer to hardware. This necessitates optimization directly on hardware. However, collecting data on hardware can be expensive. This has led to a recent interest in adapting data-efficient learning techniques to robotics. One popular method is Bayesian Optimization (BO), a sample-efficient black-box optimization scheme, but its performance typically degrades in higher dimensions. We aim to overcome this problem by incorporating domain knowledge to reduce dimensionality in a meaningful way, with a focus on bipedal locomotion. In previous work, we proposed a transformation based on knowledge of human walking that projected a 16-dimensional controller to a 1-dimensional space. In simulation, this showed enhanced sample efficiency when optimizing human-inspired neuromuscular walking controllers on a humanoid model. In this paper, we present a generalized feature transform applicable to non-humanoid robot morphologies and evaluate it on the ATRIAS bipedal robot -- in simulation and on hardware. We present three different walking controllers; two are evaluated on the real robot. Our results show that this feature transform captures important aspects of walking and accelerates learning on hardware and simulation, as compared to traditional BO.Comment: 8 pages, submitted to IEEE International Conference on Robotics and Automation 201

Learning Task Constraints from Demonstration for Hybrid Force/Position Control

We present a novel method for learning hybrid force/position control from demonstration. We learn a dynamic constraint frame aligned to the direction of desired force using Cartesian Dynamic Movement Primitives. In contrast to approaches that utilize a fixed constraint frame, our approach easily accommodates tasks with rapidly changing task constraints over time. We activate only one degree of freedom for force control at any given time, ensuring motion is always possible orthogonal to the direction of desired force. Since we utilize demonstrated forces to learn the constraint frame, we are able to compensate for forces not detected by methods that learn only from the demonstrated kinematic motion, such as frictional forces between the end-effector and the contact surface. We additionally propose novel extensions to the Dynamic Movement Primitive (DMP) framework that encourage robust transition from free-space motion to in-contact motion in spite of environment uncertainty. We incorporate force feedback and a dynamically shifting goal to reduce forces applied to the environment and retain stable contact while enabling force control. Our methods exhibit low impact forces on contact and low steady-state tracking error.Comment: Under revie