Provably Safe Reinforcement Learning via Action Projection using Reachability Analysis and Polynomial Zonotopes

While reinforcement learning produces very promising results for many applications, its main disadvantage is the lack of safety guarantees, which prevents its use in safety-critical systems. In this work, we address this issue by a safety shield for nonlinear continuous systems that solve reach-avoid tasks. Our safety shield prevents applying potentially unsafe actions from a reinforcement learning agent by projecting the proposed action to the closest safe action. This approach is called action projection and is implemented via mixed-integer optimization. The safety constraints for action projection are obtained by applying parameterized reachability analysis using polynomial zonotopes, which enables to accurately capture the nonlinear effects of the actions on the system. In contrast to other state-of-the-art approaches for action projection, our safety shield can efficiently handle input constraints and dynamic obstacles, eases incorporation of the spatial robot dimensions into the safety constraints, guarantees robust safety despite process noise and measurement errors, and is well suited for high-dimensional systems, as we demonstrate on several challenging benchmark systems

Distributed Set-Based Observers Using Diffusion Strategy

Distributed estimation is more robust against single points of failure and requires less communication overhead compared to the centralized version. Among distributed estimation techniques, set-based estimation has gained much attention as it provides estimation guarantees for safety-critical applications and copes with unknown but bounded uncertainties. We propose two distributed set-based observers using interval-based and set-membership approaches for a linear discrete-time dynamical system with bounded modeling and measurement uncertainties. Both algorithms utilize a new over-approximating zonotopes intersection step named the set-based diffusion step. We use the term diffusion since our intersection of zonotopes formula resembles the traditional diffusion step in the stochastic Kalman filter. Our new zonotopes intersection takes linear time. Our set-based diffusion step decreases the estimation errors and the size of estimated sets and can be seen as a lightweight approach to achieve partial consensus between the distributed estimated sets. Every node shares its measurement with its neighbor in the measurement update step. The neighbors intersect their estimated sets constituting our proposed set-based diffusion step. We represent sets as zonotopes since they compactly represent high-dimensional sets, and they are closed under linear mapping and Minkowski addition. The applicability of our algorithms is demonstrated by a localization example. All used data and code to recreate our findings are publicly availabl

JuliaReach: a Toolbox for Set-Based Reachability

We present JuliaReach, a toolbox for set-based reachability analysis of dynamical systems. JuliaReach consists of two main packages: Reachability, containing implementations of reachability algorithms for continuous and hybrid systems, and LazySets, a standalone library that implements state-of-the-art algorithms for calculus with convex sets. The library offers both concrete and lazy set representations, where the latter stands for the ability to delay set computations until they are needed. The choice of the programming language Julia and the accompanying documentation of our toolbox allow researchers to easily translate set-based algorithms from mathematics to software in a platform-independent way, while achieving runtime performance that is comparable to statically compiled languages. Combining lazy operations in high dimensions and explicit computations in low dimensions, JuliaReach can be applied to solve complex, large-scale problems.Comment: Accepted in Proceedings of HSCC'19: 22nd ACM International Conference on Hybrid Systems: Computation and Control (HSCC'19

LazySets.jl: Scalable symbolic-numeric set computations

LazySets.jl is a Julia library that provides ways to symbolically represent sets of points as geometric shapes, with a special focus on convex sets and polyhedral approximations. LazySets provides methods to apply common set operations, convert between different set representations, and efficiently compute with sets in high dimensions using specialized algorithms based on the set types. LazySets is the core library of JuliaReach, a cutting-edge software addressing the fundamental problem of reachability analysis: computing the set of states that are reachable by a dynamical system from all initial states and for all admissible inputs and parameters. While the library was originally designed for reachability and formal verification, its scope goes beyond such topics. LazySets is an easy-to-use, general-purpose and scalable library for computations that mix symbolics and numerics. In this article we showcase the basic functionality, highlighting some of the key design choices.Comment: published in the Proceedings of the JuliaCon Conferences 202

Reachability-based Trajectory Design

Autonomous mobile robots have the potential to increase the availability and accessibility of goods and services throughout society. However, to enable public trust in such systems, it is critical to certify that they are safe. This requires formally specifying safety, and designing motion planning methods that can guarantee safe operation (note, this work is only concerned with planning, not perception). The typical paradigm to attempt to ensure safety is receding-horizon planning, wherein a robot creates a short plan, then executes it while creating its next short plan in an iterative fashion, allowing a robot to incorporate new sensor information over time. However, this requires a robot to plan in real time. Therefore, the key challenge in making safety guarantees lies in balancing performance (how quickly a robot can plan) and conservatism (how cautiously a robot behaves). Existing methods suffer from a tradeoff between performance and conservatism, which is rooted in the choice of model used describe a robot; accuracy typically comes at the price of computation speed. To address this challenge, this dissertation proposes Reachability-based Trajectory Design (RTD), which performs real-time, receding-horizon planning with a simplified planning model, and ensures safety by describing the model error using a reachable set of the robot. RTD begins with the offline design of a continuum of parameterized trajectories for the plan- ning model; each trajectory ends with a fail-safe maneuver such as braking to a stop. RTD then computes the robot’s Forward Reachable Set (FRS), which contains all points in workspace reach- able by the robot for each parameterized trajectory. Importantly, the FRS also contains the error model, since a robot can typically never track planned trajectories perfectly. Online (at runtime), the robot intersects the FRS with sensed obstacles to provably determine which trajectory plans could cause collisions. Then, the robot performs trajectory optimization over the remaining safe trajectories. If no new safe plan can be found, the robot can execute its previously-found fail-safe maneuver, enabling perpetual safety. This dissertation begins by presenting RTD as a theoretical framework, then presents three representations of a robot’s FRS, using (1) sums-of-squares (SOS) polynomial programming, (2) zonotopes (a special type of convex polytope), and (3) rotatotopes (a generalization of zonotopes that enable representing a robot’s swept volume). To enable real-time planning, this work also de- velops an obstacle representation that enables provable safety while treating obstacles as discrete, finite sets of points. The practicality of RTD is demonstrated on four different wheeled robots (using the SOS FRS), two quadrotor aerial robots (using the zonotope FRS), and one manipulator robot (using the rotatotope FRS). Over thousands of simulations and dozens of hardware trials, RTD performs safe, real-time planning in arbitrary and challenging environments. In summary, this dissertation proposes RTD as a general purpose, practical framework for provably safe, real-time robot motion planning.PHDMechanical EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/162884/1/skousik_1.pd