Robustness Testing of Intermediate Verifiers

Program verifiers are not exempt from the bugs that affect nearly every piece of software. In addition, they often exhibit brittle behavior: their performance changes considerably with details of how the input program is expressed-details that should be irrelevant, such as the order of independent declarations. Such a lack of robustness frustrates users who have to spend considerable time figuring out a tool's idiosyncrasies before they can use it effectively. This paper introduces a technique to detect lack of robustness of program verifiers; the technique is lightweight and fully automated, as it is based on testing methods (such as mutation testing and metamorphic testing). The key idea is to generate many simple variants of a program that initially passes verification. All variants are, by construction, equivalent to the original program; thus, any variant that fails verification indicates lack of robustness in the verifier. We implemented our technique in a tool called "mugie", which operates on programs written in the popular Boogie language for verification-used as intermediate representation in numerous program verifiers. Experiments targeting 135 Boogie programs indicate that brittle behavior occurs fairly frequently (16 programs) and is not hard to trigger. Based on these results, the paper discusses the main sources of brittle behavior and suggests means of improving robustness

Automated Theorem Proving with Extensions of First-Order Logic

Automated theorem provers are computer programs that check whether a logical conjecture follows from a set of logical statements. The conjecture and the statements are expressed in the language of some formal logic, such as first-order logic. Theorem provers for first-order logic have been used for automation in proof assistants, verification of programs, static analysis of networks, and other purposes. However, the efficient usage of these provers remains challenging. One of the challenges is the complexity of translating domain problems to first-order logic. Not only can such translation be cumbersome due to semantic differences between the domain and the logic, but it might inadvertently result in problems that provers cannot easily handle.The work presented in the thesis addresses this challenge by developing an extension of first-order logic named FOOL. FOOL contains syntactical features of programming languages and more expressive logics, is friendly for translation of problems from various domains, and can be efficiently supported by existing theorem provers. We describe the syntax and semantics of FOOL and present a simple translation from FOOL to plain first-order logic. We describe an efficient clausal normal form transformation algorithm for FOOL and based on it implement a support for FOOL in the Vampire theorem prover. We illustrate the efficient use of FOOL for program verification by describing a concise encoding of next state relations of imperative programs in FOOL. We show a usage of features of FOOL in problems of static analysis of networks. We demonstrate the efficiency of automated theorem proving in FOOL with an extensive set of experiments. In these experiments we compare the performance of Vampire on a large collection of problems from various sources translated to FOOL and ordinary first-order logic. Finally, we fix the syntax for FOOL in TPTP, the standard language of first-order theorem provers

Towards the Correctness of Software Behavior in UML: A Model Checking Approach Based on Slicing

Embedded systems are systems which have ongoing interactions with their environments, accepting requests and producing responses. Such systems are increasingly used in applications where failure is unacceptable: traffic control systems, avionics, automobiles, etc. Correct and highly dependable construction of such systems is particularly important and challenging. A very promising and increasingly attractive method for achieving this goal is using the approach of formal verification. A formal verification method consists of three major components: a model for describing the behavior of the system, a specification language to embody correctness requirements, and an analysis method to verify the behavior against the correctness requirements. This Ph.D. addresses the correctness of the behavioral design of embedded systems, using model checking as the verification technology. More precisely, we present an UML-based verification method that checks whether the conditions on the evolution of the embedded system are met by the model. Unfortunately, model checking is limited to medium size systems because of its high space requirements. To overcome this problem, this Ph.D. suggests the integration of the slicing (reduction) technique

Triggerless happy: Intermediate verification with a first-order prover

SMT solvers have become de rigueur in deductive verification to automatically prove the validity of verification conditions. While these solvers provide an effective support for theories—such as arithmetic—that feature strongly in program verification, they tend to be more limited in dealing with first-order quantification, for which they have to rely on special annotations—known as triggers—to guide the instantiation of quantifiers. Writing effective triggers is necessary to achieve satisfactory performance with SMT solvers, but remains a tricky endeavor—beyond the purview of non-highly trained experts. In this paper, we experiment with the idea of using first-order provers instead of SMT solvers to prove the validity of verification conditions. First-order provers offer a native support for unrestricted quantification, but have been traditionally limited in theory reasoning. By leveraging some recent extensions to narrow this gap in the Vampire first-order prover, we describe a first-order encoding of verification conditions of programs written in the Boogie intermediate verification language. Experiments with a prototype implementation on a variety of Boogie programs suggest that first-order provers can help achieve more flexible and robust performance in program verification, while avoiding the pitfalls of having to manually guide instantiations by means of triggers