Output Reachable Set Estimation and Verification for Multi-Layer Neural Networks

In this paper, the output reachable estimation and safety verification problems for multi-layer perceptron neural networks are addressed. First, a conception called maximum sensitivity in introduced and, for a class of multi-layer perceptrons whose activation functions are monotonic functions, the maximum sensitivity can be computed via solving convex optimization problems. Then, using a simulation-based method, the output reachable set estimation problem for neural networks is formulated into a chain of optimization problems. Finally, an automated safety verification is developed based on the output reachable set estimation result. An application to the safety verification for a robotic arm model with two joints is presented to show the effectiveness of proposed approaches.Comment: 8 pages, 9 figures, to appear in TNNL

Forward Invariant Cuts to Simplify Proofs of Safety

The use of deductive techniques, such as theorem provers, has several advantages in safety verification of hybrid sys- tems; however, state-of-the-art theorem provers require ex- tensive manual intervention. Furthermore, there is often a gap between the type of assistance that a theorem prover requires to make progress on a proof task and the assis- tance that a system designer is able to provide. This paper presents an extension to KeYmaera, a deductive verification tool for differential dynamic logic; the new technique allows local reasoning using system designer intuition about per- formance within particular modes as part of a proof task. Our approach allows the theorem prover to leverage for- ward invariants, discovered using numerical techniques, as part of a proof of safety. We introduce a new inference rule into the proof calculus of KeYmaera, the forward invariant cut rule, and we present a methodology to discover useful forward invariants, which are then used with the new cut rule to complete verification tasks. We demonstrate how our new approach can be used to complete verification tasks that lie out of the reach of existing deductive approaches us- ing several examples, including one involving an automotive powertrain control system.Comment: Extended version of EMSOFT pape

A New Simulation Metric to Determine Safe Environments and Controllers for Systems with Unknown Dynamics

We consider the problem of extracting safe environments and controllers for reach-avoid objectives for systems with known state and control spaces, but unknown dynamics. In a given environment, a common approach is to synthesize a controller from an abstraction or a model of the system (potentially learned from data). However, in many situations, the relationship between the dynamics of the model and the \textit{actual system} is not known; and hence it is difficult to provide safety guarantees for the system. In such cases, the Standard Simulation Metric (SSM), defined as the worst-case norm distance between the model and the system output trajectories, can be used to modify a reach-avoid specification for the system into a more stringent specification for the abstraction. Nevertheless, the obtained distance, and hence the modified specification, can be quite conservative. This limits the set of environments for which a safe controller can be obtained. We propose SPEC, a specification-centric simulation metric, which overcomes these limitations by computing the distance using only the trajectories that violate the specification for the system. We show that modifying a reach-avoid specification with SPEC allows us to synthesize a safe controller for a larger set of environments compared to SSM. We also propose a probabilistic method to compute SPEC for a general class of systems. Case studies using simulators for quadrotors and autonomous cars illustrate the advantages of the proposed metric for determining safe environment sets and controllers.Comment: 22nd ACM International Conference on Hybrid Systems: Computation and Control (2019

Approximating Reachable Sets for Neural Network based Models in Real-Time via Optimal Control

In this paper, we present a data-driven framework for real-time estimation of reachable sets for control systems where the plant is modeled using neural networks (NNs). We utilize a running example of a quadrotor model that is learned using trajectory data via NNs. The NN learned offline, can be excited online to obtain linear approximations for reachability analysis. We use a dynamic mode decomposition based approach to obtain linear liftings of the NN model. The linear models thus obtained can utilize optimal control theory to obtain polytopic approximations to the reachable sets in real-time. The polytopic approximations can be tuned to arbitrary degrees of accuracy. The proposed framework can be extended to other nonlinear models that utilize NNs to estimate plant dynamics. We demonstrate the effectiveness of the proposed framework using an illustrative simulation of quadrotor dynamics.Comment: 14 pages, 11 figures, journal paper that has been conditionally accepte