LMS-Verify: abstraction without regret for verified systems programming

Performance critical software is almost always developed in C, as programmers do not trust high-level languages to deliver the same reliable performance. This is bad because low-level code in unsafe languages attracts security vulnerabilities and because development is far less productive, with PL advances mostly lost on programmers operating under tight performance constraints. High-level languages provide memory safety out of the box, but they are deemed too slow and unpredictable for serious system software. Recent years have seen a surge in staging and generative programming: the key idea is to use high-level languages and their abstraction power as glorified macro systems to compose code fragments in first-order, potentially domain-specific, intermediate languages, from which fast C can be emitted. But what about security? Since the end result is still C code, the safety guarantees of the high-level host language are lost. In this paper, we extend this generative approach to emit ACSL specifications along with C code. We demonstrate that staging achieves ``abstraction without regret'' for verification: we show how high-level programming models, in particular higher-order composable contracts from dynamic languages, can be used at generation time to compose and generate first-order specifications that can be statically checked by existing tools. We also show how type classes can automatically attach invariants to data types, reducing the need for repetitive manual annotations. We evaluate our system on several case studies that varyingly exercise verification of memory safety, overflow safety, and functional correctness. We feature an HTTP parser that is (1) fast (2) high-level: implemented using staged parser combinators (3) secure: with verified memory safety. This result is significant, as input parsing is a key attack vector, and vulnerabilities related to HTTP parsing have been documented in all widely-used web servers.</jats:p

Verification by Reduction to Functional Programs

In this thesis, we explore techniques for the development and verification of programs in a high-level, expressive, and safe programming language. Our programs can express problems over unbounded domains and over recursive and mutable data structures. We present an implementation language flexible enough to build interesting and useful systems. We mostly maintain a core shared language for the specifications and the implementation, with only a few extensions specific to expressing the specifications. Extensions of the core shared language include imperative features with state and side effects, which help when implementing efficient systems. Our language is a subset of the Scala programming language. Once verified, programs can be compiled and executed using the existing Scala tools. We present algorithms for verifying programs written in this language. We take a layer-based approach, where we reduce, at each step, the program to an equivalent program in a simpler language. We first purify functions by transforming away mutations into explicit return types in the functions' signatures. This step rewrites all mutations of data structures into cloning operations. We then translate local state into a purely functional code, hence eliminating all traces of imperative programming. The final language is a functional subset of Scala, on which we apply verification. We integrate our pipeline of translations into Leon, a verifier for Scala. We verify the core functional language by using an algorithm already developed inside Leon. The program is encoded into equivalent first-order logic formulas over a combination of theories and recursive functions. The formulas are eventually discharged to an external SMT solver. We extend this core language and the solving algorithm with support for both infinite-precision integers and bit-vectors. The algorithm takes into account the semantics gap between the two domains, and the programmer is ultimately responsible to use the proper type to represent the data. We build a reusable interface for SMT-LIB that enables us to swap solvers transparently in order to validate the formulas emitted by Leon. We experiment with writing solvers in Scala; they could offer both a better and safer integration with the rest of the system. We evaluate the cost of using a higher-order language to implement such solvers, traditionally written in C/C++. Finally, we experiment with the system by building fully working and verified applications. We rely on the intersection of many features including higher-order functions, mutable data structures, recursive functions, and nondeterministic environment dependencies, to build concise and verified applications

Covering All the Bases: Type-Based Verification of Test Input Generators

Test input generators are an important part of property-based testing (PBT) frameworks. Because PBT is intended to test deep semantic and structural properties of a program, the outputs produced by these generators can be complex data structures, constrained to satisfy properties the developer believes is most relevant to testing the function of interest. An important feature expected of these generators is that they be capable of producing all acceptable elements that satisfy the function's input type and generator-provided constraints. However, it is not readily apparent how we might validate whether a particular generator's output satisfies this coverage requirement. Typically, developers must rely on manual inspection and post-mortem analysis of test runs to determine if the generator is providing sufficient coverage; these approaches are error-prone and difficult to scale as generators become more complex. To address this important concern, we present a new refinement type-based verification procedure for validating the coverage provided by input test generators, based on a novel interpretation of types that embeds ``must-style'' underapproximate reasoning principles as a fundamental part of the type system. The types associated with expressions now capture the set of values guaranteed to be produced by the expression, rather than the typical formulation that uses types to represent the set of values an expression may produce. Beyond formalizing the notion of coverage types in the context of a rich core language with higher-order procedures and inductive datatypes, we also present a detailed evaluation study to justify the utility of our ideas

Deductive Synthesis and Repair

In this thesis, we explore techniques for the development of recursive functional programs over unbounded domains that are proved correct according to their high-level specifications. We present algorithms for automatically synthesizing executable code, starting from the speci- fication alone. We implement these algorithms in the Leon system. We augment relational specifications with a concise notation for symbolic tests, which are are helpful to characterize fragments of the functionsâ behavior. We build on our synthesis procedure to automatically repair invalid functions by generating alternative implementations. Our approach therefore formulates program repair in the framework of deductive synthesis and uses the existing program structure as a hint to guide synthesis. We rely on user-specified tests as well as automatically generated ones to localize the fault. This localization enables our procedure to repair functions that would otherwise be out of reach of our synthesizer, and ensures that most of the original behavior is preserved. We also investigate multiple ways of enabling Leon programs to interact with external, un- trusted code. For that purpose, we introduce a precise inter-procedural effect analysis for arbitrary Scala programs with mutable state, dynamic object allocation, and dynamic dispatch. We analyzed the Scala standard library containing 58000 methods and classified them into sev- eral categories according to their effects. Our analysis proves that over one half of all methods are pure, identifies a number of conditionally pure methods, and computes summary graphs and regular expressions describing the side effects of non-pure methods. We implement the synthesis and repair algorithms within the Leon system and deploy them as part of a novel interactive development environment available as a web interface. Our implementation is able to synthesize, within seconds, a number of useful recursive functions that manipulate unbounded numbers and data structures. Our repair procedure automatically locates various kinds of errors in recursive functions and fixes them by synthesizing alternative implementations

Modular implicits

We present modular implicits, an extension to the OCaml language for ad-hoc polymorphism inspired by Scala implicits and modular type classes. Modular implicits are based on type-directed implicit module parameters, and elaborate straightforwardly into OCaml's first-class functors. Basing the design on OCaml's modules leads to a system that naturally supports many features from other languages with systematic ad-hoc overloading, including inheritance, instance constraints, constructor classes and associated types

A type-theoretic framework for software component synthesis

A language-agnostic approach for type-based component-oriented software synthesis is developed from the fundamental principles of abstract algebra and Combinatory Logic. It relies on an enumerative type inhabitation algorithm for Finite Combinatory Logic with Intersection Types (FCL) and a universal algebraic construction to translate terms of Combinatory Logic into any given target language. New insights are gained on the combination of semantic domains of discourse with intersection types. Long standing gaps in the algorithmic understanding of the type inhabitation question of FCL are closed. A practical implementation is developed and its applications by the author and other researchers are discussed. They include, but are not limited to, vast improvements in the context of synthesis of software product line members. An interactive theorem prover, Coq, is used to formalize and check all the theoretical results. This makes them more reusable for other developments and enhances confidence in their correctness.Es wird ein sprachunabhängiger Ansatz für die typbasierte und komponentenorientierte Synthese von Software entwickelt. Hierzu werden grundlegende Erkenntnisse über abstrakte Algebra und kombinatorische Logik verwendet. Der Ansatz beruht auf dem enumerativen Typinhabitationsproblem der endlichen kombinatorischen Logik mit Intersektionstypen, sowie einer universellen algebraischen Konstruktion, um Ergebnisterme in jede beliebe Zielsprache übersetzen zu können. Es werden neue Einblicke gewonnen, wie verschiedene semantische Domänen des Diskurses über Softwareeigenschaften miteinander verbunden werden können. Offene Fragestellungen im Zusammenhand mit der Algorithmik des Typinhabitationsproblems für Intersektionstypen werden beantwortet. Eine praktische Implementierung des Ansatzes wird entwickelt und ihre bisherigen Anwendungen durch den Autor und andere Wissenschaftler werden diskutiert. Diese beinhalten starke Verbesserungen im Zusammenhang mit der Synthese von Ausprägungen von Software Produktlinien. Ein interaktiver Theorembeweiser wir genutzt, um alle Ergebnisse der Arbeit zu formalisieren und mechanisch zu überprüfen. Dies trägt zum einen zur Wiederverwendbarkeit der theoretischen Ergebnisse in anderen Kontexten bei, und erhöht zum andern das Vertrauen in ihre Korrektheit

Cautiously Optimistic Program Analyses for Secure and Reliable Software

Modern computer systems still have various security and reliability vulnerabilities. Well-known dynamic analyses solutions can mitigate them using runtime monitors that serve as lifeguards. But the additional work in enforcing these security and safety properties incurs exorbitant performance costs, and such tools are rarely used in practice. Our work addresses this problem by constructing a novel technique- Cautiously Optimistic Program Analysis (COPA). COPA is optimistic- it infers likely program invariants from dynamic observations, and assumes them in its static reasoning to precisely identify and elide wasteful runtime monitors. The resulting system is fast, but also ensures soundness by recovering to a conservatively optimized analysis when a likely invariant rarely fails at runtime. COPA is also cautious- by carefully restricting optimizations to only safe elisions, the recovery is greatly simplified. It avoids unbounded rollbacks upon recovery, thereby enabling analysis for live production software. We demonstrate the effectiveness of Cautiously Optimistic Program Analyses in three areas: Information-Flow Tracking (IFT) can help prevent security breaches and information leaks. But they are rarely used in practice due to their high performance overhead (>500% for web/email servers). COPA dramatically reduces this cost by eliding wasteful IFT monitors to make it practical (9% overhead, 4x speedup). Automatic Garbage Collection (GC) in managed languages (e.g. Java) simplifies programming tasks while ensuring memory safety. However, there is no correct GC for weakly-typed languages (e.g. C/C++), and manual memory management is prone to errors that have been exploited in high profile attacks. We develop the first sound GC for C/C++, and use COPA to optimize its performance (16% overhead). Sequential Consistency (SC) provides intuitive semantics to concurrent programs that simplifies reasoning for their correctness. However, ensuring SC behavior on commodity hardware remains expensive. We use COPA to ensure SC for Java at the language-level efficiently, and significantly reduce its cost (from 24% down to 5% on x86). COPA provides a way to realize strong software security, reliability and semantic guarantees at practical costs.PHDComputer Science & EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/170027/1/subarno_1.pd