Maximum Resilience of Artificial Neural Networks

The deployment of Artificial Neural Networks (ANNs) in safety-critical applications poses a number of new verification and certification challenges. In particular, for ANN-enabled self-driving vehicles it is important to establish properties about the resilience of ANNs to noisy or even maliciously manipulated sensory input. We are addressing these challenges by defining resilience properties of ANN-based classifiers as the maximal amount of input or sensor perturbation which is still tolerated. This problem of computing maximal perturbation bounds for ANNs is then reduced to solving mixed integer optimization problems (MIP). A number of MIP encoding heuristics are developed for drastically reducing MIP-solver runtimes, and using parallelization of MIP-solvers results in an almost linear speed-up in the number (up to a certain limit) of computing cores in our experiments. We demonstrate the effectiveness and scalability of our approach by means of computing maximal resilience bounds for a number of ANN benchmark sets ranging from typical image recognition scenarios to the autonomous maneuvering of robots.Comment: Timestamp research work conducted in the project. version 2: fix some typos, rephrase the definition, and add some more existing wor

Modular Verification of Interrupt-Driven Software

Interrupts have been widely used in safety-critical computer systems to handle outside stimuli and interact with the hardware, but reasoning about interrupt-driven software remains a difficult task. Although a number of static verification techniques have been proposed for interrupt-driven software, they often rely on constructing a monolithic verification model. Furthermore, they do not precisely capture the complete execution semantics of interrupts such as nested invocations of interrupt handlers. To overcome these limitations, we propose an abstract interpretation framework for static verification of interrupt-driven software that first analyzes each interrupt handler in isolation as if it were a sequential program, and then propagates the result to other interrupt handlers. This iterative process continues until results from all interrupt handlers reach a fixed point. Since our method never constructs the global model, it avoids the up-front blowup in model construction that hampers existing, non-modular, verification techniques. We have evaluated our method on 35 interrupt-driven applications with a total of 22,541 lines of code. Our results show the method is able to quickly and more accurately analyze the behavior of interrupts.Comment: preprint of the ASE 2017 pape

Partitioning Interpolant-Based Verificationfor effective Unbounded Model Checking

Interpolant-based model checking has been shown to be effective on large verification instances, as it efficiently combines automated abstraction and reachability fixed-point checks. On the other hand, methods based on variable quantification have proved their ability to remove free inputs, thus projecting the search space over state variables. In this paper we propose an integrated approach which combines the abstraction power of interpolation with techniques that rely on AIG and/or BDD representations of states, directly supporting variable quantification and fixed-point checks. The underlying idea of this combination is to adopt AIG- or BDD-based quantifications to limit and restrict the search space and the complexity of the interpolant-based approach. The exploited strategies, most of which are individually well-known, are integrated with a new flavor, specifically designed to improve their effectiveness on difficult verification instances. Experimental results, specifically oriented to hard-to-solve verification problems, show the robustness of our approach

Generalized Strong Preservation by Abstract Interpretation

Standard abstract model checking relies on abstract Kripke structures which approximate concrete models by gluing together indistinguishable states, namely by a partition of the concrete state space. Strong preservation for a specification language L encodes the equivalence of concrete and abstract model checking of formulas in L. We show how abstract interpretation can be used to design abstract models that are more general than abstract Kripke structures. Accordingly, strong preservation is generalized to abstract interpretation-based models and precisely related to the concept of completeness in abstract interpretation. The problem of minimally refining an abstract model in order to make it strongly preserving for some language L can be formulated as a minimal domain refinement in abstract interpretation in order to get completeness w.r.t. the logical/temporal operators of L. It turns out that this refined strongly preserving abstract model always exists and can be characterized as a greatest fixed point. As a consequence, some well-known behavioural equivalences, like bisimulation, simulation and stuttering, and their corresponding partition refinement algorithms can be elegantly characterized in abstract interpretation as completeness properties and refinements

Parameterized verification and repair of concurrent systems

In this thesis, we present novel approaches for model checking, repair and synthesis of systems that may be parameterized in their number of components. The parameterized model checking problem (PMCP) is in general undecidable, and therefore the focus is on restricted classes of parameterized concurrent systems where the problem is decidable. Under certain conditions, the problem is decidable for guarded protocols, and for systems that communicate via a token, a pairwise, or a broadcast synchronization. In this thesis we improve existing results for guarded protocols and we show that the PMCP of guarded protocols and token passing systems is decidable for specifications that add a quantitative aspect to LTL, called Prompt-LTL. Furthermore, we present, to our knowledge, the first parameterized repair algorithm. The parameterized repair problem is to find a refinement of a process implementation p such that the concurrent system with an arbitrary number of instances of p is correct. We show how this algorithm can be used on classes of systems that can be represented as well structured transition systems (WSTS). Additionally we present two safety synthesis algorithms that utilize a lazy approach. Given a faulty system, the algorithms first symbolically model check the system, then the obtained error traces are analyzed to synthesize a candidate that has no such traces. Experimental results show that our algorithm solves a number of benchmarks that are intractable for existing tools. Furthermore, we introduce our tool AIGEN for generating random Boolean functions and transition systems in a symbolic representation.In dieser Arbeit stellen wir neuartige Ans atze für das Model-Checking, die Reparatur und die Synthese von Systemen vor, die in ihrer Anzahl von Komponenten parametrisiert sein können. Das Problem des parametrisierten Model-Checking (PMCP) ist im Allgemeinen unentscheidbar, und daher liegt der Fokus auf eingeschränkten Klassen parametrisierter synchroner Systeme, bei denen das Problem entscheidbar ist. Unter bestimmten Bedingungen ist das Problem für Guarded Protocols und für Systeme, die über ein Token, eine Pairwise oder eine Broadcast-Synchronisation kommunizieren, entscheidbar. In dieser Arbeit verbessern wir bestehende Ergebnisse für Guarded Protocols und zeigen die Entscheidbarkeit des PMCP für Guarded Protocols und Token-Passing Systeme mit Spezifikationen in der temporalen Logik Prompt-LTL, die LTL einen quantitativen Aspekt hinzufügt. Darüber hinaus präsentieren wir unseres Wissens den ersten parametrisierten Reparaturalgorithmus. Das parametrisierte Reparaturproblem besteht darin, eine Verfeinerung einer Prozessimplementierung p zu finden, so dass das synchrone Systeme mit einer beliebigen Anzahl von Instanzen von p korrekt ist. Wir zeigen, wie dieser Algorithmus auf Klassen von Systemen angewendet werden kann, die als Well Structured Transition Systems (WSTS) dargestellt werden können. Außerdem präsentieren wir zwei Safety-Synthesis Algorithmen, die einen "lazy" Ansatz verwenden. Bei einem fehlerhaften System überprüfen die Algorithmen das System symbolisch, dann werden die erhaltenen "Gegenbeispiel" analysiert, um einen Kandidaten zu synthetisieren der keine solchen Fehlerpfade hat. Versuchsergebnisse zeigen, dass unser Algorithmus eine Reihe von Benchmarks löst, die für bestehende Tools nicht lösbar sind. Darüber hinaus stellen wir unser Tool AIGEN zur Erzeugung zufälliger Boolescher Funktionen und Transitionssysteme in einer symbolischen Darstellung vor

A Static Analyzer for Large Safety-Critical Software

We show that abstract interpretation-based static program analysis can be made efficient and precise enough to formally verify a class of properties for a family of large programs with few or no false alarms. This is achieved by refinement of a general purpose static analyzer and later adaptation to particular programs of the family by the end-user through parametrization. This is applied to the proof of soundness of data manipulation operations at the machine level for periodic synchronous safety critical embedded software. The main novelties are the design principle of static analyzers by refinement and adaptation through parametrization, the symbolic manipulation of expressions to improve the precision of abstract transfer functions, the octagon, ellipsoid, and decision tree abstract domains, all with sound handling of rounding errors in floating point computations, widening strategies (with thresholds, delayed) and the automatic determination of the parameters (parametrized packing)

A simple abstraction of arrays and maps by program translation

We present an approach for the static analysis of programs handling arrays, with a Galois connection between the semantics of the array program and semantics of purely scalar operations. The simplest way to implement it is by automatic, syntactic transformation of the array program into a scalar program followed analysis of the scalar program with any static analysis technique (abstract interpretation, acceleration, predicate abstraction,.. .). The scalars invariants thus obtained are translated back onto the original program as universally quantified array invariants. We illustrate our approach on a variety of examples, leading to the " Dutch flag " algorithm