Stability Verification of Neural Network Controllers using Mixed-Integer Programming

We propose a framework for the stability verification of Mixed-Integer Linear Programming (MILP) representable control policies. This framework compares a fixed candidate policy, which admits an efficient parameterization and can be evaluated at a low computational cost, against a fixed baseline policy, which is known to be stable but expensive to evaluate. We provide sufficient conditions for the closed-loop stability of the candidate policy in terms of the worst-case approximation error with respect to the baseline policy, and we show that these conditions can be checked by solving a Mixed-Integer Quadratic Program (MIQP). Additionally, we demonstrate that an outer and inner approximation of the stability region of the candidate policy can be computed by solving an MILP. The proposed framework is sufficiently general to accommodate a broad range of candidate policies including ReLU Neural Networks (NNs), optimal solution maps of parametric quadratic programs, and Model Predictive Control (MPC) policies. We also present an open-source toolbox in Python based on the proposed framework, which allows for the easy verification of custom NN architectures and MPC formulations. We showcase the flexibility and reliability of our framework in the context of a DC-DC power converter case study and investigate its computational complexity

Robust online motion planning with reachable sets

Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2013.Cataloged from PDF version of thesis.Includes bibliographical references (p. 51-55).In this thesis we consider the problem of generating motion plans for a nonlinear dynamical system that are guaranteed to succeed despite uncertainty in the environment, parametric model uncertainty, disturbances, and/or errors in state estimation. Furthermore, we consider the case where these plans must be generated online, because constraints such as obstacles in the environment may not be known until they are perceived (with a noisy sensor) at runtime. Previous work on feedback motion planning for nonlinear systems was limited to offline planning due to the computational cost of safety verification. Here we augment the traditional trajectory library approach by designing locally stabilizing controllers for each nominal trajectory in the library and providing guarantees on the resulting closed loop systems. We leverage sums-of-squares programming to design these locally stabilizing controllers by explicitly attempting to minimize the size of the worst case reachable set of the closed-loop system subjected to bounded disturbances and uncertainty. The reachable sets associated with each trajectory in the library can be thought of as "funnels" that the system is guaranteed to remain within. The resulting funnel library is then used to sequentially compose motion plans at runtime while ensuring the safety of the robot. A major advantage of the work presented here is that by explicitly taking into account the effect of uncertainty, the robot can evaluate motion plans based on how vulnerable they are to disturbances. We demonstrate our method on a simulation of a plane flying through a two dimensional forest of polygonal trees with parametric uncertainty and disturbances in the form of a bounded "cross-wind". We further validate our approach by carefully evaluating the guarantees on invariance provided by funnels on two challenging underactuated systems (the "Acrobot" and a small-sized airplane).by Anirudha Majumdar.S.M

Correct-By-Construction Control Synthesis for Systems with Disturbance and Uncertainty

This dissertation focuses on correct-by-construction control synthesis for Cyber-Physical Systems (CPS) under model uncertainty and disturbance. CPSs are systems that interact with the physical world and perform complicated dynamic tasks where safety is often the overriding factor. Correct-by-construction control synthesis is a concept that provides formal performance guarantees to closed-loop systems by rigorous mathematic reasoning. Since CPSs interact with the environment, disturbance and modeling uncertainty are critical to the success of the control synthesis. Disturbance and uncertainty may come from a variety of sources, such as exogenous disturbance, the disturbance caused by co-existing controllers and modeling uncertainty. To better accommodate the different types of disturbance and uncertainty, the verification and control synthesis methods must be chosen accordingly. Four approaches are included in this dissertation. First, to deal with exogenous disturbance, a polar algorithm is developed to compute an avoidable set for obstacle avoidance. Second, a supervised learning based method is proposed to design a good student controller that has safety built-in and rarely triggers the intervention of the supervisory controller, thus targeting the design of the student controller. Third, to deal with the disturbance caused by co-existing controllers, a Lyapunov verification method is proposed to formally verify the safety of coexisting controllers while respecting the confidentiality requirement. Finally, a data-driven approach is proposed to deal with model uncertainty. A minimal robust control invariant set is computed for an uncertain dynamic system without a given model by first identifying the set of admissible models and then simultaneously computing the invariant set while selecting the optimal model. The proposed methods are applicable to many real-world applications and reflect the notion of using the structure of the system to achieve performance guarantees without being overly conservative.PHDMechanical EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/145933/1/chenyx_1.pd

Learning a feasible and stabilizing explicit model predictive control law by robust optimization

Fast model predictive control on embedded sys- tems has been successfully applied to plants with microsecond sampling times employing a precomputed state-to-input map. However, the complexity of this so-called explicit MPC can be prohibitive even for low-dimensional systems. In this pa- per, we introduce a new synthesis method for low-complexity suboptimal MPC controllers based on function approximation from randomly chosen point-wise sample values. In addition to standard machine learning algorithms formulated as convex programs, we provide sufficient conditions on the learning algo- rithm in the form of tractable convex constraints that guarantee input and state constraint satisfaction, recursive feasibility and stability of the closed loop system. The resulting control law can be fully parallelized, which renders the approach particularly suitable for highly concurrent embedded platforms such as FPGAs. A numerical example shows the effectiveness of the proposed method

Approximate Dynamic Programming for Constrained Piecewise Affine Systems with Stability and Safety Guarantees

Infinite-horizon optimal control of constrained piecewise affine (PWA) systems has been approximately addressed by hybrid model predictive control (MPC), which, however, has computational limitations, both in offline design and online implementation. In this paper, we consider an alternative approach based on approximate dynamic programming (ADP), an important class of methods in reinforcement learning. We accommodate non-convex union-of-polyhedra state constraints and linear input constraints into ADP by designing PWA penalty functions. PWA function approximation is used, which allows for a mixed-integer encoding to implement ADP. The main advantage of the proposed ADP method is its online computational efficiency. Particularly, we propose two control policies, which lead to solving a smaller-scale mixed-integer linear program than conventional hybrid MPC, or a single convex quadratic program, depending on whether the policy is implicitly determined online or explicitly computed offline. We characterize the stability and safety properties of the closed-loop systems, as well as the sub-optimality of the proposed policies, by quantifying the approximation errors of value functions and policies. We also develop an offline mixed-integer linear programming-based method to certify the reliability of the proposed method. Simulation results on an inverted pendulum with elastic walls and on an adaptive cruise control problem validate the control performance in terms of constraint satisfaction and CPU time

Towards a safe and responsive control framework for human-centered robots

Human-centered robots are a specific kind of service robot, which interact with humans physically or cognitively and help humans with tasks in uncertain environments. They can be humanoid robots, exoskeletons, or manipulators and mobile platforms that provide us good services. However, human-centered robots are still not perfect enough for us to use nowadays. On the one hand, human-centered robots are still slow and inefficient for their tasks because the human inputs and dynamics that they react to are uncertain, immeasurable, or even completely unknown. On the other hand, human-centered robots face much more complicated safety requirements than other kinds of robots because humans are dynamic and vulnerable during physical human-robot interaction. To resolve these issues of human-centered robots, the work in this dissertation explores new models for reducing human uncertainty and new control algorithms for improving safety warranty. The first half of this dissertation introduces a complex stiffness model for describing the uncertain human impedance. The discovery of this new model is motivated to explain the observation of a consistent damping ratio of a human versus different environmental dynamics. It replaces the linear damping term in a conventional mass-spring-damping model with a hysteretic damping term, an imaginary value in the frequency domain. Because of the correlation between the stiffness term and the newly discovered hysteretic damping term in the complex stiffness model, we can significantly reduce the human impedance uncertainty. Based on the complex stiffness model, we can adopt nonlinear control strategies for improving the responsiveness and the human-friendliness of human-centered robots. The second half of this dissertation introduces the concept of a barrier pair, which consists of a barrier function and a controller for the safety verification and warranty of a human-centered robot. We obtain a barrier pair by solving an optimization problem subject to a series of linear matrix inequalities representing the state-space, input, and stability constraints of a human-centered robot. By incorporating sampling-based methods into the synthesis of barrier pairs, human-centered robots can guarantee safe operation with non-convex state-space constraints. The sampling-based barrier pair method helps us construct a control framework of human-robot shared autonomy. A human-centered robot in this control framework uses an inference of a human's objective to figure out how to assist the human and prevent the human from potential accidents.Mechanical Engineerin