Falsification of Cyber-Physical Systems with Robustness-Guided Black-Box Checking

For exhaustive formal verification, industrial-scale cyber-physical systems (CPSs) are often too large and complex, and lightweight alternatives (e.g., monitoring and testing) have attracted the attention of both industrial practitioners and academic researchers. Falsification is one popular testing method of CPSs utilizing stochastic optimization. In state-of-the-art falsification methods, the result of the previous falsification trials is discarded, and we always try to falsify without any prior knowledge. To concisely memorize such prior information on the CPS model and exploit it, we employ Black-box checking (BBC), which is a combination of automata learning and model checking. Moreover, we enhance BBC using the robust semantics of STL formulas, which is the essential gadget in falsification. Our experiment results suggest that our robustness-guided BBC outperforms a state-of-the-art falsification tool.Comment: Accepted to HSCC 202

Case Study: Verifying the Safety of an Autonomous Racing Car with a Neural Network Controller

This paper describes a verification case study on an autonomous racing car with a neural network (NN) controller. Although several verification approaches have been recently proposed, they have only been evaluated on low-dimensional systems or systems with constrained environments. To explore the limits of existing approaches, we present a challenging benchmark in which the NN takes raw LiDAR measurements as input and outputs steering for the car. We train a dozen NNs using reinforcement learning (RL) and show that the state of the art in verification can handle systems with around 40 LiDAR rays. Furthermore, we perform real experiments to investigate the benefits and limitations of verification with respect to the sim2real gap, i.e., the difference between a system’s modeled and real performance. We identify cases, similar to the modeled environment, in which verification is strongly correlated with safe behavior. Finally, we illustrate LiDAR fault patterns that can be used to develop robust and safe RL algorithms

A Deontic Logic Analysis of Autonomous Systems' Safety

We consider the pressing question of how to model, verify, and ensure that autonomous systems meet certain \textit{obligations} (like the obligation to respect traffic laws), and refrain from impermissible behavior (like recklessly changing lanes). Temporal logics are heavily used in autonomous system design; however, as we illustrate here, temporal (alethic) logics alone are inappropriate for reasoning about obligations of autonomous systems. This paper proposes the use of Dominance Act Utilitarianism (DAU), a deontic logic of agency, to encode and reason about obligations of autonomous systems. We use DAU to analyze Intel's Responsibility-Sensitive Safety (RSS) proposal as a real-world case study. We demonstrate that DAU can express well-posed RSS rules, formally derive undesirable consequences of these rules, illustrate how DAU could help design systems that have specific obligations, and how to model-check DAU obligations.Comment: 11 pages, 4 figures, In 23rd ACM International Conference on Hybrid Systems: Computation and Contro

Compositional Synthesis via a Convex Parameterization of Assume-Guarantee Contracts

We develop an assume-guarantee framework for control of large scale linear (time-varying) systems from finite-time reach and avoid or infinite-time invariance specifications. The contracts describe the admissible set of states and controls for individual subsystems. A set of contracts compose correctly if mutual assumptions and guarantees match in a way that we formalize. We propose a rich parameterization of contracts such that the set of parameters that compose correctly is convex. Moreover, we design a potential function of parameters that describes the distance of contracts from a correct composition. Thus, the verification and synthesis for the aggregate system are broken to solving small convex programs for individual subsystems, where correctness is ultimately achieved in a compositional way. Illustrative examples demonstrate the scalability of our method

Sufficient conditions for satisfaction of formulas with until operators in hybrid systems

In this paper, we introduce tools to verify the satisfaction of temporal logic specifications using the until operator for hybrid dynamical systems. Hybrid dynamical systems are given in terms of differential and difference inclusions, which capture the continuous and discrete dynamics (or events), respectively. For such systems, conditional invariance and eventual conditional invariance are employed to characterize dynamical properties associated with the until operators. Sufficient conditions for the satisfaction of temporal logic specifications involving the until operator are provided by guaranteeing properties of the data defining the systems and the existence of barrier functions or Lyapunov-like functions. Examples illustrate the results throughout the paper

Local lipschitzness of reachability maps for hybrid systems with applications to safety

Motivated by the safety problem, several definitions of reachability maps, for hybrid dynamical systems, are introduced. It is well established that, under certain conditions, the solutions to continuous-time systems depend continuously with respect to initial conditions. In such setting, the reachability maps considered in this paper are locally Lipschitz (in the Lipschitz sense for set-valued maps) when the right-hand side of the continuous-time system is locally Lipschitz. However, guaranteeing similar properties for reachability maps for hybrid systems is much more challenging. Examples of hybrid systems for which the reachability maps do not depend nicely with respect to their arguments, in the Lipschitz sense, are introduced. With such pathological cases properly identified, sufficient conditions involving the data defining a hybrid system assuring Lipschitzness of the reachability maps are formulated. As an application, the proposed conditions are shown to be useful to significantly improve an existing converse theorem for safety given in terms of barrier functions. Namely, for a class of safe hybrid systems, we show that safety is equivalent to the existence of a locally Lipschitz barrier function. Examples throughout the paper illustrate the results

Tools and Algorithms for the Construction and Analysis of Systems

This open access two-volume set constitutes the proceedings of the 27th International Conference on Tools and Algorithms for the Construction and Analysis of Systems, TACAS 2021, which was held during March 27 – April 1, 2021, as part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2021. The conference was planned to take place in Luxembourg and changed to an online format due to the COVID-19 pandemic. The total of 41 full papers presented in the proceedings was carefully reviewed and selected from 141 submissions. The volume also contains 7 tool papers; 6 Tool Demo papers, 9 SV-Comp Competition Papers. The papers are organized in topical sections as follows: Part I: Game Theory; SMT Verification; Probabilities; Timed Systems; Neural Networks; Analysis of Network Communication. Part II: Verification Techniques (not SMT); Case Studies; Proof Generation/Validation; Tool Papers; Tool Demo Papers; SV-Comp Tool Competition Papers