34 research outputs found
Safe and Robust Learning Control with Gaussian Processes
Abstract-This paper introduces a learning-based robust control algorithm that provides robust stability and performance guarantees during learning. The approach uses Gaussian process (GP) regression based on data gathered during operation to update an initial model of the system and to gradually decrease the uncertainty related to this model. Embedding this data-based update scheme in a robust control framework guarantees stability during the learning process. Traditional robust control approaches have not considered online adaptation of the model and its uncertainty before. As a result, their controllers do not improve performance during operation. Typical machine learning algorithms that have achieved similar high-performance behavior by adapting the model and controller online do not provide the guarantees presented in this paper. In particular, this paper considers a stabilization task, linearizes the nonlinear, GP-based model around a desired operating point, and solves a convex optimization problem to obtain a linear robust controller. The resulting performance improvements due to the learning-based controller are demonstrated in experiments on a quadrotor vehicle
Safe Controller Optimization for Quadrotors with Gaussian Processes
One of the most fundamental problems when designing controllers for dynamic
systems is the tuning of the controller parameters. Typically, a model of the
system is used to obtain an initial controller, but ultimately the controller
parameters must be tuned manually on the real system to achieve the best
performance. To avoid this manual tuning step, methods from machine learning,
such as Bayesian optimization, have been used. However, as these methods
evaluate different controller parameters on the real system, safety-critical
system failures may happen. In this paper, we overcome this problem by
applying, for the first time, a recently developed safe optimization algorithm,
SafeOpt, to the problem of automatic controller parameter tuning. Given an
initial, low-performance controller, SafeOpt automatically optimizes the
parameters of a control law while guaranteeing safety. It models the underlying
performance measure as a Gaussian process and only explores new controller
parameters whose performance lies above a safe performance threshold with high
probability. Experimental results on a quadrotor vehicle indicate that the
proposed method enables fast, automatic, and safe optimization of controller
parameters without human intervention.Comment: IEEE International Conference on Robotics and Automation, 2016. 6
pages, 4 figures. A video of the experiments can be found at
http://tiny.cc/icra16_video . A Python implementation of the algorithm is
available at https://github.com/befelix/SafeOp
Episodic Learning with Control Lyapunov Functions for Uncertain Robotic Systems
Many modern nonlinear control methods aim to endow systems with guaranteed
properties, such as stability or safety, and have been successfully applied to
the domain of robotics. However, model uncertainty remains a persistent
challenge, weakening theoretical guarantees and causing implementation failures
on physical systems. This paper develops a machine learning framework centered
around Control Lyapunov Functions (CLFs) to adapt to parametric uncertainty and
unmodeled dynamics in general robotic systems. Our proposed method proceeds by
iteratively updating estimates of Lyapunov function derivatives and improving
controllers, ultimately yielding a stabilizing quadratic program model-based
controller. We validate our approach on a planar Segway simulation,
demonstrating substantial performance improvements by iteratively refining on a
base model-free controller
Pseudospectral Model Predictive Control under Partially Learned Dynamics
Trajectory optimization of a controlled dynamical system is an essential part
of autonomy, however many trajectory optimization techniques are limited by the
fidelity of the underlying parametric model. In the field of robotics, a lack
of model knowledge can be overcome with machine learning techniques, utilizing
measurements to build a dynamical model from the data. This paper aims to take
the middle ground between these two approaches by introducing a semi-parametric
representation of the underlying system dynamics. Our goal is to leverage the
considerable information contained in a traditional physics based model and
combine it with a data-driven, non-parametric regression technique known as a
Gaussian Process. Integrating this semi-parametric model with model predictive
pseudospectral control, we demonstrate this technique on both a cart pole and
quadrotor simulation with unmodeled damping and parametric error. In order to
manage parametric uncertainty, we introduce an algorithm that utilizes Sparse
Spectrum Gaussian Processes (SSGP) for online learning after each rollout. We
implement this online learning technique on a cart pole and quadrator, then
demonstrate the use of online learning and obstacle avoidance for the dubin
vehicle dynamics.Comment: Accepted but withdrawn from AIAA Scitech 201
Multi-Robot Transfer Learning: A Dynamical System Perspective
Multi-robot transfer learning allows a robot to use data generated by a
second, similar robot to improve its own behavior. The potential advantages are
reducing the time of training and the unavoidable risks that exist during the
training phase. Transfer learning algorithms aim to find an optimal transfer
map between different robots. In this paper, we investigate, through a
theoretical study of single-input single-output (SISO) systems, the properties
of such optimal transfer maps. We first show that the optimal transfer learning
map is, in general, a dynamic system. The main contribution of the paper is to
provide an algorithm for determining the properties of this optimal dynamic map
including its order and regressors (i.e., the variables it depends on). The
proposed algorithm does not require detailed knowledge of the robots' dynamics,
but relies on basic system properties easily obtainable through simple
experimental tests. We validate the proposed algorithm experimentally through
an example of transfer learning between two different quadrotor platforms.
Experimental results show that an optimal dynamic map, with correct properties
obtained from our proposed algorithm, achieves 60-70% reduction of transfer
learning error compared to the cases when the data is directly transferred or
transferred using an optimal static map.Comment: 7 pages, 6 figures, accepted at the 2017 IEEE/RSJ International
Conference on Intelligent Robots and System
On the Role of Models in Learning Control: Actor-Critic Iterative Learning Control
Learning from data of past tasks can substantially improve the accuracy of
mechatronic systems. Often, for fast and safe learning a model of the system is
required. The aim of this paper is to develop a model-free approach for fast
and safe learning for mechatronic systems. The developed actor-critic iterative
learning control (ACILC) framework uses a feedforward parameterization with
basis functions. These basis functions encode implicit model knowledge and the
actor-critic algorithm learns the feedforward parameters without explicitly
using a model. Experimental results on a printer setup demonstrate that the
developed ACILC framework is capable of achieving the same feedforward signal
as preexisting model-based methods without using explicit model knowledge.Comment: 6 pages, 4 figures, 21st IFAC World Congress 202