Testing noninterference, quickly

Information-flow control mechanisms are difficult to design and labor intensive to prove correct. To reduce the time wasted on proof attempts doomed to fail due to broken definitions, we advocate modern random testing techniques for finding counterexamples during the design process. We show how to use QuickCheck, a property-based random-testing tool, to guide the design of a simple information-flow abstract machine. We find that both sophisticated strategies for generating well-distributed random programs and readily falsifiable formulations of noninterference properties are critically important. We propose several approaches and evaluate their effectiveness on a collection of injected bugs of varying subtlety. We also present an effective technique for shrinking large counterexamples to minimal, easily comprehensible ones. Taken together, our best methods enable us to quickly and automatically generate simple counterexamples for all these bugs

A Verified Information-Flow Architecture

SAFE is a clean-slate design for a highly secure computer system, with pervasive mechanisms for tracking and limiting information flows. At the lowest level, the SAFE hardware supports fine-grained programmable tags, with efficient and flexible propagation and combination of tags as instructions are executed. The operating system virtualizes these generic facilities to present an information-flow abstract machine that allows user programs to label sensitive data with rich confidentiality policies. We present a formal, machine-checked model of the key hardware and software mechanisms used to dynamically control information flow in SAFE and an end-to-end proof of noninterference for this model. We use a refinement proof methodology to propagate the noninterference property of the abstract machine down to the concrete machine level. We use an intermediate layer in the refinement chain that factors out the details of the information-flow control policy and devise a code generator for compiling such information-flow policies into low-level monitor code. Finally, we verify the correctness of this generator using a dedicated Hoare logic that abstracts from low-level machine instructions into a reusable set of verified structured code generators

Understanding and Enforcing Opacity

Abstract—This paper puts a spotlight on the specification and enforcement of opacity, a security policy for protecting sensitive properties of system behavior. We illustrate the fine granularity of the opacity policy by location privacy and privacy-preserving aggregation scenarios. We present a frame-work for opacity and explore its key differences and formal connections with such well-known information-flow models as noninterference, knowledge-based security, and declassifica-tion. Our results are machine-checked and parameterized in the observational power of the attacker, including progress-insensitive, progress-sensitive, and timing-sensitive attackers. We present two approaches to enforcing opacity: a whitebox monitor and a blackbox sampling-based enforcement. We report on experiments with prototypes that utilize state-of-the-art Satisfiability Modulo Theories (SMT) solvers and the random testing tool QuickCheck to establish opacity for the location and aggregation-based scenarios. I

Integration of Static and Dynamic Analysis Techniques for Checking Noninterference

In this article, we present an overview of recent combinations of deductive program verification and automatic test generation on the one hand and static analysis on the other hand, with the goal of checking noninterference. Noninterference is the non-functional property that certain confidential information cannot leak to certain public output, i.e., the confidentiality of that information is always preserved. We define the noninterference properties that are checked along with the individual approaches that we use in different combinations. In one use case, our framework for checking noninterference employs deductive verification to automatically generate tests for noninterference violations with an improved test coverage. In another use case, the framework provides two combinations of deductive verification with static analysis based on system dependence graphs to prove noninterference, thereby reducing the effort for deductive verification

Relational Symbolic Execution

Symbolic execution is a classical program analysis technique used to show that programs satisfy or violate given specifications. In this work we generalize symbolic execution to support program analysis for relational specifications in the form of relational properties - these are properties about two runs of two programs on related inputs, or about two executions of a single program on related inputs. Relational properties are useful to formalize notions in security and privacy, and to reason about program optimizations. We design a relational symbolic execution engine, named RelSym which supports interactive refutation, as well as proving of relational properties for programs written in a language with arrays and for-like loops

Information flow analysis for a dynamically typed language with staged metaprogramming

Web applications written in JavaScript are regularly used for dealing with sensitive or personal data. Consequently, reasoning about their security properties has become an important problem, which is made very difficult by the highly dynamic nature of the language, particularly its support for runtime code generation via eval. In order to deal with this, we propose to investigate security analyses for languages with more principled forms of dynamic code generation. To this end, we present a static information flow analysis for a dynamically typed functional language with prototype-based inheritance and staged metaprogramming. We prove its soundness, implement it and test it on various examples designed to show its relevance to proving security properties, such as noninterference, in JavaScript. To demonstrate the applicability of the analysis, we also present a general method for transforming a program using eval into one using staged metaprogramming. To our knowledge, this is the first fully static information flow analysis for a language with staged metaprogramming, and the first formal soundness proof of a CFA-based information flow analysis for a functional programming language

Random Testing For Language Design

Property-based random testing can facilitate formal verification, exposing errors early on in the proving process and guiding users towards correct specifications and implementations. However, effective random testing often requires users to write custom generators for well-distributed random data satisfying complex logical predicates, a task which can be tedious and error prone. In this work, I aim to reduce the cost of property-based testing by making such generators easier to write, read and maintain. I present a domain-specific language, called Luck, in which generators are conveniently expressed by decorating predicates with lightweight annotations to control both the distribution of generated values and the amount of constraint solving that happens before each variable is instantiated. I also aim to increase the applicability of testing to formal verification by bringing advanced random testing techniques to the Coq proof assistant. I describe QuickChick, a QuickCheck clone for Coq, and improve it by incorporating ideas explored in the context of Luck to automatically derive provably correct generators for data constrained by inductive relations. Finally, I evaluate both QuickChick and Luck in a variety of complex case studies from programming languages literature, such as information-flow abstract machines and type systems for lambda calculi

Hyperfuzzing: black-box security hypertesting with a grey-box fuzzer

Information leakage is a class of error that can lead to severe consequences. However unlike other errors, it is rarely explicitly considered during the software testing process. LeakFuzzer advances the state of the art by using a noninterference security property together with a security flow policy as an oracle. As the tool extends the state of the art fuzzer, AFL++, LeakFuzzer inherits the advantages of AFL++ such as scalability, automated input generation, high coverage and low developer intervention. The tool can detect the same set of errors that a normal fuzzer can detect, with the addition of being able to detect violations of secure information flow policies. We evaluated LeakFuzzer on a diverse set of 10 C and C++ benchmarks containing known information leaks, ranging in size from just 80 to over 900k lines of code. Seven of these are taken from real-world CVEs including Heartbleed and a recent error in PostgreSQL. Given 20 24-hour runs, LeakFuzzer can find 100% of the leaks in the SUTs whereas existing techniques using such as the CBMC model checker and AFL++ augmented with different sanitizers can only find 40% at best.Comment: 11 pages, 4 figure

Hypertesting:The Case for Automated Testing of Hyperproperties

Abstract. Proof systems give absolute guarantees but are notoriously difficult to use for non-experts. Bug-finding tools make no completeness guarantees but offer a high degree of automation and are relatively easy to use for developers. For safety properties, the effectiveness of automatic test generation and bug finding is well established. For security properties like non-interference, which cannot be expressed as properties of a single program execution (i.e., hyperproperties), methods for testing and bug finding are in their infancy. In general, violations of hyperproperties cannot be expressed with a single test case like safety properties, so existing bug finding methods do not apply. This paper takes the position that we should fill this gap in the arsenal of ver-ification technology and outlines concepts and tools for the next generation of bug finding systems. In particular, we aim to establish a generalized concept for the generation of “hypertests”, sets of tests that either provide some level of con-fidence in the system or give counterexamples to hyperproperties. As concrete instances of hypertesting, we foresee automated testing for violations of secure information flow and of numeric and cryptographic properties of programs.