69 research outputs found
Neural Lyapunov Control
We propose new methods for learning control policies and neural network
Lyapunov functions for nonlinear control problems, with provable guarantee of
stability. The framework consists of a learner that attempts to find the
control and Lyapunov functions, and a falsifier that finds counterexamples to
quickly guide the learner towards solutions. The procedure terminates when no
counterexample is found by the falsifier, in which case the controlled
nonlinear system is provably stable. The approach significantly simplifies the
process of Lyapunov control design, provides end-to-end correctness guarantee,
and can obtain much larger regions of attraction than existing methods such as
LQR and SOS/SDP. We show experiments on how the new methods obtain high-quality
solutions for challenging control problems.Comment: NeurIPS 201
Safe Learning of Quadrotor Dynamics Using Barrier Certificates
To effectively control complex dynamical systems, accurate nonlinear models
are typically needed. However, these models are not always known. In this
paper, we present a data-driven approach based on Gaussian processes that
learns models of quadrotors operating in partially unknown environments. What
makes this challenging is that if the learning process is not carefully
controlled, the system will go unstable, i.e., the quadcopter will crash. To
this end, barrier certificates are employed for safe learning. The barrier
certificates establish a non-conservative forward invariant safe region, in
which high probability safety guarantees are provided based on the statistics
of the Gaussian Process. A learning controller is designed to efficiently
explore those uncertain states and expand the barrier certified safe region
based on an adaptive sampling scheme. In addition, a recursive Gaussian Process
prediction method is developed to learn the complex quadrotor dynamics in
real-time. Simulation results are provided to demonstrate the effectiveness of
the proposed approach.Comment: Submitted to ICRA 2018, 8 page
Episodic Learning with Control Lyapunov Functions for Uncertain Robotic Systems
Many modern nonlinear control methods aim to endow systems with guaranteed
properties, such as stability or safety, and have been successfully applied to
the domain of robotics. However, model uncertainty remains a persistent
challenge, weakening theoretical guarantees and causing implementation failures
on physical systems. This paper develops a machine learning framework centered
around Control Lyapunov Functions (CLFs) to adapt to parametric uncertainty and
unmodeled dynamics in general robotic systems. Our proposed method proceeds by
iteratively updating estimates of Lyapunov function derivatives and improving
controllers, ultimately yielding a stabilizing quadratic program model-based
controller. We validate our approach on a planar Segway simulation,
demonstrating substantial performance improvements by iteratively refining on a
base model-free controller
Safe Control Under Input Limits with Neural Control Barrier Functions
We propose new methods to synthesize control barrier function (CBF)-based
safe controllers that avoid input saturation, which can cause safety
violations. In particular, our method is created for high-dimensional, general
nonlinear systems, for which such tools are scarce. We leverage techniques from
machine learning, like neural networks and deep learning, to simplify this
challenging problem in nonlinear control design. The method consists of a
learner-critic architecture, in which the critic gives counterexamples of input
saturation and the learner optimizes a neural CBF to eliminate those
counterexamples. We provide empirical results on a 10D state, 4D input
quadcopter-pendulum system. Our learned CBF avoids input saturation and
maintains safety over nearly 100% of trials.Comment: CORL 202
Automated and Sound Synthesis of Lyapunov Functions with SMT Solvers
In this paper we employ SMT solvers to soundly synthesise Lyapunov functions
that assert the stability of a given dynamical model. The search for a Lyapunov
function is framed as the satisfiability of a second-order logical formula,
asking whether there exists a function satisfying a desired specification
(stability) for all possible initial conditions of the model. We synthesise
Lyapunov functions for linear, non-linear (polynomial), and for parametric
models. For non-linear models, the algorithm also determines a region of
validity for the Lyapunov function. We exploit an inductive framework to
synthesise Lyapunov functions, starting from parametric templates. The
inductive framework comprises two elements: a learner proposes a Lyapunov
function, and a verifier checks its validity - its lack is expressed via a
counterexample (a point over the state space), for further use by the learner.
Whilst the verifier uses the SMT solver Z3, thus ensuring the overall soundness
of the procedure, we examine two alternatives for the learner: a numerical
approach based on the optimisation tool Gurobi, and a sound approach based
again on Z3. The overall technique is evaluated over a broad set of benchmarks,
which shows that this methodology not only scales to 10-dimensional models
within reasonable computational time, but also offers a novel soundness proof
for the generated Lyapunov functions and their domains of validity
Automated Stability Analysis of Piecewise Affine Dynamics Using Vertices
This paper presents an automated algorithm to analyze the stability of
piecewise affine (PWA) dynamical systems due to their broad applications. We
parametrize the Lyapunov function as a PWA function, with polytopic regions
defined by the PWA dynamics. Using this parametrization, Stability conditions
can be expressed as linear constraints restricted to polytopes so that the
search for a Lyapunov function involves solving a linear program. However, a
valid Lyapunov function might not be found given these polytopic regions. A
natural response is to increase the size of the parametrization of the Lyapunov
function by dividing regions and solving the new linear program. This paper
proposes two new methods to divide each polytope into smaller ones. The first
approach divides a polytope based on the sign of the derivative of the
candidate Lyapunov function, while the second divides it based on the change in
the vector field of the PWA dynamical system. In addition, we propose using
Delaunay triangulation to achieve automated division of regions and preserve
the continuity of the PWA Lyapunov function. Examples involving learned models
and explicit MPC controllers demonstrate that the proposed method of dividing
regions leads to valid Lyapunov functions with fewer regions than existing
methods, reducing the computational time taken for stability analysisComment: 11 pages, 11 figure
- …