Automated Validation for Synchronous Reactive Embedded Systems

Proper functionality is a necessity for systems used in safety-critical applications; consequently, software in these systems is often subject to rigorous validation and formal verification that aims at ensuring expected behavior. To aid in the design of these systems, several synchronous programming languages exist for describing deterministic system models suitable for formal verification and validation. Examples of such synchronous languages include SIGNAL, Lustre, MRICDF, and Esterel. Common application domains for synchronous programs include avionics, automotive control, process control, and defense systems. In many cases, rigorous formal verification of these systems is unfeasible because the methods, such as theorem proving and model checking, are too expensive. A theorem proving approach requires a great deal of user involvement and expertise, and a model checking approach may not be feasible on systems of substantial complexity due to computation constraints. This thesis presents the design, implementation, and evaluation of SAGA, a prototype tool for the automated validation of synchronous reactive embedded systems. SAGA shifts the testing effort associated with critical systems from creating individual test cases manually to reasoning about the safety and environment properties of a system. The approach SAGA takes is to generate relevant inputs to the system-under-test from a user-specified environment description, and to validate the resulting system behavior against user-specified safety properties. This overview of SAGA includes a thorough user's guide and important implementation details. Additionally, the validation process with SAGA is qualitatively assessed. The assessment is done through a case study involving the celebrated steam boiler control specification problem. Results from this case study reveal the utility of SAGA in exposing non-trivial system errors.College of Engineering Undergraduate Research ScholarshipNo embarg

Collaborative Verification-Driven Engineering of Hybrid Systems

Hybrid systems with both discrete and continuous dynamics are an important model for real-world cyber-physical systems. The key challenge is to ensure their correct functioning w.r.t. safety requirements. Promising techniques to ensure safety seem to be model-driven engineering to develop hybrid systems in a well-defined and traceable manner, and formal verification to prove their correctness. Their combination forms the vision of verification-driven engineering. Often, hybrid systems are rather complex in that they require expertise from many domains (e.g., robotics, control systems, computer science, software engineering, and mechanical engineering). Moreover, despite the remarkable progress in automating formal verification of hybrid systems, the construction of proofs of complex systems often requires nontrivial human guidance, since hybrid systems verification tools solve undecidable problems. It is, thus, not uncommon for development and verification teams to consist of many players with diverse expertise. This paper introduces a verification-driven engineering toolset that extends our previous work on hybrid and arithmetic verification with tools for (i) graphical (UML) and textual modeling of hybrid systems, (ii) exchanging and comparing models and proofs, and (iii) managing verification tasks. This toolset makes it easier to tackle large-scale verification tasks

Robust Computer Algebra, Theorem Proving, and Oracle AI

In the context of superintelligent AI systems, the term "oracle" has two meanings. One refers to modular systems queried for domain-specific tasks. Another usage, referring to a class of systems which may be useful for addressing the value alignment and AI control problems, is a superintelligent AI system that only answers questions. The aim of this manuscript is to survey contemporary research problems related to oracles which align with long-term research goals of AI safety. We examine existing question answering systems and argue that their high degree of architectural heterogeneity makes them poor candidates for rigorous analysis as oracles. On the other hand, we identify computer algebra systems (CASs) as being primitive examples of domain-specific oracles for mathematics and argue that efforts to integrate computer algebra systems with theorem provers, systems which have largely been developed independent of one another, provide a concrete set of problems related to the notion of provable safety that has emerged in the AI safety community. We review approaches to interfacing CASs with theorem provers, describe well-defined architectural deficiencies that have been identified with CASs, and suggest possible lines of research and practical software projects for scientists interested in AI safety.Comment: 15 pages, 3 figure

Sciduction: Combining Induction, Deduction, and Structure for Verification and Synthesis

Even with impressive advances in automated formal methods, certain problems in system verification and synthesis remain challenging. Examples include the verification of quantitative properties of software involving constraints on timing and energy consumption, and the automatic synthesis of systems from specifications. The major challenges include environment modeling, incompleteness in specifications, and the complexity of underlying decision problems. This position paper proposes sciduction, an approach to tackle these challenges by integrating inductive inference, deductive reasoning, and structure hypotheses. Deductive reasoning, which leads from general rules or concepts to conclusions about specific problem instances, includes techniques such as logical inference and constraint solving. Inductive inference, which generalizes from specific instances to yield a concept, includes algorithmic learning from examples. Structure hypotheses are used to define the class of artifacts, such as invariants or program fragments, generated during verification or synthesis. Sciduction constrains inductive and deductive reasoning using structure hypotheses, and actively combines inductive and deductive reasoning: for instance, deductive techniques generate examples for learning, and inductive reasoning is used to guide the deductive engines. We illustrate this approach with three applications: (i) timing analysis of software; (ii) synthesis of loop-free programs, and (iii) controller synthesis for hybrid systems. Some future applications are also discussed