Building DryVR: A verification and controller synthesis engine for cyber-physical systems and safety-critical autonomous vehicle features

To test safety of autonomous vehicles, large corporations have raced to log millions of miles of test driving on public roads. While this can improve confidence in such systems, testing alone cannot establish of absence of failure scenarios. In fact, it has also been reported that the amount of data required to guarantee a probability of 10^-9 fatality per hour of driving would require 10^9 hours of driving [1] [2], which is roughly in the order of thirty billion miles. Formal verification can give guarantees about absence of failures and potentially reduce the amount of testing needed significantly. Simulation based verification is a promising approach to provide formal safety guarantees to Cyber-Physical Systems (CPS). However, existing verification tools rely on the explicit mathematical models of the system. Detailed mathematical models are often not available or are too complex for formal verification tools. To address this issue, the DryVR approach for verification is presented in [3]. DryVR views a cyber-physical system as a combination of a white-box transition graph and a black-box simulator. This alleviates the need for complete mathematical models, but at the same time exploits models when they are available. A verification algorithm for directed acyclic time-dependent transition graph is also presented in [3]. In this thesis, we present the detailed construction of the DryVR tool with several new functionalities, which includes: (a) verification on state-dependent cyclic transition graph with guard and reset functions; (b) controller synthesis that searches transition graph for given reach-avoid specification; (c) interface that allows user to connect DryVR with arbitrary black-box simulators, and (d) integration with Jupyter Notebook [4]. We also present a case study for autonomous vehicle system in this thesis, and DryVR comes with verification and controller synthesis examples to illustrate its capabilities. The evaluation of included examples is presented in later chapter shows that both verification and controller synthesis are promising starting point for DryVR to become a comprehensive verification and synthesis toolbox for practical CPS

Formal Approaches to Control System Security From Static Analysis to Runtime Enforcement

With the advent of Industry 4.0, industrial facilities and critical infrastructures are transforming into an ecosystem of heterogeneous physical and cyber components, such as programmable logic controllers, increasingly interconnected and therefore exposed to cyber-physical attacks, i.e., security breaches in cyberspace that may adversely affect the physical processes underlying industrial control systems. The main contributions of this thesis follow two research strands that address the security concerns of industrial control systems via formal methodologies. As our first contribution, we propose a formal approach based on model checking and statistical model checking, within the MODEST TOOLSET, to analyse the impact of attacks targeting nontrivial control systems equipped with an intrusion detection system (IDS) capable of detecting and mitigating attacks. Our goal is to evaluate the impact of cyber-physical attacks, i.e., attacks targeting sensors and/or actuators of the system with potential consequences on the safety of the inner physical process. Our security analysis estimates both the physical impact of the attacks and the performance of the IDS. As our second contribution, we propose a formal approach based on runtime enforcement to ensure specification compliance in networks of controllers, possibly compromised by colluding malware that may tamper with actuator commands, sensor readings, and inter-controller communications. Our approach relies on an ad-hoc sub-class of Ligatti et al.’s edit automata to enforce controllers represented in Hennessy and Regan’s Timed Process Language. We define a synthesis algorithm that, given an alphabet P of observable actions and a timed correctness property e, returns a monitor that enforces the property e during the execution of any (potentially corrupted) controller with alphabet P, and complying with the property e. Our monitors correct and suppress incorrect actions coming from corrupted controllers and emit actions in full autonomy when the controller under scrutiny is not able to do so in a correct manner. Besides classical requirements, such as transparency and soundness, the proposed enforcement enjoys deadlock- and diverge-freedom of monitored controllers, together with compositionality when dealing with networks of controllers. Finally, we test the proposed enforcement mechanism on a non-trivial case study, taken from the context of industrial water treatment systems, in which the controllers are injected with different malware with different malicious goals

Input Synthesis for Sampled Data Systems by Program Logic

Inspired by a concrete industry problem we consider the input synthesis problem for hybrid systems: given a hybrid system that is subject to input from outside (also called disturbance or noise), find an input sequence that steers the system to the desired postcondition. In this paper we focus on sampled data systems--systems in which a digital controller interrupts a physical plant in a periodic manner, a class commonly known in control theory--and furthermore assume that a controller is given in the form of an imperative program. We develop a structural approach to input synthesis that features forward and backward reasoning in program logic for the purpose of reducing a search space. Although the examples we cover are limited both in size and in structure, experiments with a prototype implementation suggest potential of our program logic based approach.Comment: In Proceedings HAS 2014, arXiv:1501.0540

Sciduction: Combining Induction, Deduction, and Structure for Verification and Synthesis

Even with impressive advances in automated formal methods, certain problems in system verification and synthesis remain challenging. Examples include the verification of quantitative properties of software involving constraints on timing and energy consumption, and the automatic synthesis of systems from specifications. The major challenges include environment modeling, incompleteness in specifications, and the complexity of underlying decision problems. This position paper proposes sciduction, an approach to tackle these challenges by integrating inductive inference, deductive reasoning, and structure hypotheses. Deductive reasoning, which leads from general rules or concepts to conclusions about specific problem instances, includes techniques such as logical inference and constraint solving. Inductive inference, which generalizes from specific instances to yield a concept, includes algorithmic learning from examples. Structure hypotheses are used to define the class of artifacts, such as invariants or program fragments, generated during verification or synthesis. Sciduction constrains inductive and deductive reasoning using structure hypotheses, and actively combines inductive and deductive reasoning: for instance, deductive techniques generate examples for learning, and inductive reasoning is used to guide the deductive engines. We illustrate this approach with three applications: (i) timing analysis of software; (ii) synthesis of loop-free programs, and (iii) controller synthesis for hybrid systems. Some future applications are also discussed

Model checking embedded system designs

We survey the basic principles behind the application of model checking to controller verification and synthesis. A promising development is the area of guided model checking, in which the state space search strategy of the model checking algorithm can be influenced to visit more interesting sets of states first. In particular, we discuss how model checking can be combined with heuristic cost functions to guide search strategies. Finally, we list a number of current research developments, especially in the area of reachability analysis for optimal control and related issues

Sound and Automated Synthesis of Digital Stabilizing Controllers for Continuous Plants

Modern control is implemented with digital microcontrollers, embedded within a dynamical plant that represents physical components. We present a new algorithm based on counter-example guided inductive synthesis that automates the design of digital controllers that are correct by construction. The synthesis result is sound with respect to the complete range of approximations, including time discretization, quantization effects, and finite-precision arithmetic and its rounding errors. We have implemented our new algorithm in a tool called DSSynth, and are able to automatically generate stable controllers for a set of intricate plant models taken from the literature within minutes.Comment: 10 page