Proving Memory Safety of the ANI Windows Image Parser Using Compositional Exhaustive Testing

We report in this paper how we proved memory safety of a complex Windows image parser written in low-level C in only three months of work and using only three core tech-niques, namely (1) symbolic execution at the x86 binary level, (2) exhaustive program path enumeration and testing, and (3) user-guided program decomposition and summariza-tion. We also used a new tool, named MicroX, for executing code fragments in isolation using a custom virtual machine designed for testing purposes. As a result of this work, we are able to prove, for the first time, that a Windows image parser is memory safe, i.e., free of any buffer-overflow secu-rity vulnerabilities, modulo the soundness of our tools and several additional assumptions regarding bounding input-dependent loops, fixing a few buffer-overflow bugs, and ex-cluding some code parts that are not memory safe by design. In the process, we also discovered and fixed several limita-tions in our tools, and narrowed the gap between systematic testing and verification. 1

IC-Cut: A Compositional Search Strategy for Dynamic Test Generation

Abstract. We present IC-Cut, short for “Interface-Complexity-based Cut”, a new compositional search strategy for systematically testing large programs. IC-Cut dynamically detects function interfaces that are simple enough to be cost-effective for summarization. IC-Cut then hierarchically decomposes the program into units defined by such functions and their sub-functions in the call graph. These units are tested independently, their test results are recorded as low-complexity function summaries, and the summaries are reused when testing higher-level functions in the call graph, thus limiting overall path explosion. When the decomposed units are tested exhaustively, they constitute verified components of the program. IC-Cut is run dynamically and on-the-fly during the search, typically refining cuts as the search advances. We have implemented this algorithm as a new search strategy in the whitebox fuzzer SAGE, and present detailed experimental results ob-tained when fuzzing the ANI Windows image parser. Our results show that IC-Cut alleviates path explosion while preserving or even increasing code coverage and bug finding, compared to the current generational-search strategy used in SAGE.

Automated Black Box Generation of Structured Inputs for Use in Software Testing

A common problem in automated software testing is the need to generate many inputs with complex structure in a black-box fashion. For example, a library for manipulating red-black trees may require that inputs are themselves valid red-black trees, meaning anything invalid is not suitable for testing. As another example, in order to test code generation in a compiler, it is necessary to use input programs which are both syntactically valid and well-typed. Despite the importance of this problem, we observe that existing solutions are few in number and have severe drawbacks, including unreasonably slow performance and a lack of generality to testing different systems.This thesis presents a solution to this problem of black-box structured input generation. I observe that test inputs can be described as solutions to systems of logical constraints, and that more expressive constraints can lead to more complex tests. In order to test effectively and generate many tests, we need high-performance constraint solvers capable of finding many solutions to these constraints. I observe that constraint logic programming (CLP) offers an expressive constraint language paired with a high-performance constraint solver, and thus serves as a potential solution to this problem. Via a series of case studies, I have found that CLP (1) is applicable to testing a wide variety of systems; (2) can scale to more complex constraints than ever previously described; and (3) is often orders of magnitude faster than competing solutions. These case studies have also exposed dozens of bugs in high-profile software, including the Rust compiler and the Z3 SMT solver

Automatic techniques for detecting and exploiting symmetry in model checking

The application of model checking is limited due to the state-space explosion problem – as the number of components represented by a model increase, the worst case size of the associated state-space grows exponentially. Current techniques can handle limited kinds of symmetry, e.g. full symmetry between identical components in a concurrent system. They avoid the problem of automatic symmetry detection by requiring the user to specify the presence of symmetry in a model (explicitly, or by annotating the associated specification using additional language keywords), or by restricting the input language of a model checker so that only symmetric systems can be specified. Additionally, computing unique representatives for each symmetric equivalence class is easy for these limited kinds of symmetry. We present a theoretical framework for symmetry reduction which can be applied to explicit state model checking. The framework includes techniques for automatic symmetry detection using computational group theory, which can be applied with no additional user input. These techniques detect structural symmetries induced by the topology of a concurrent system, so our framework includes exact and approximate techniques to efficiently exploit arbitrary symmetry groups which may arise in this way. These techniques are also based on computational group theoretic methods. We prove that our framework is logically sound, and demonstrate its general applicability to explicit state model checking. By providing a new symmetry reduction package for the SPIN model checker, we show that our framework can be feasibly implemented as part of a system which is widely used in both industry and academia. Through a study of SPIN users, we assess the usability of our automatic symmetry detection techniques in practice

Fuzz Driver Generation

Poor software quality has led to tremendous costs and safety disasters, thus, software defects make the news with alarming regularity. Fuzzing is a bug detection technique. In particular, it is a software testing method where a stream of random input is sent to an application to stress the application and cause unexpected behaviour, resource leaks or crashes. When it comes to fuzzing software libraries, a fuzz driver plays an important role because it is the binder between the fuzzer and the target program. Traditionally fuzzing was used in closed-source platforms and also it is used to find vulnerabilities in kernels. However, recent developments show that fuzzing is now applied to open-source libraries. This research study analyses the role of a fuzz driver in the domain of fuzzing to recognise its importance, applications, techniques, challenges and future directions. This study intends to explore the state-of-the-art fuzz driver development strategies and identify trends in research and areas of potential improvements. We identified that fuzz driver generation is mainly seen as a minor activity in fuzzing research. It was evident that the development of a fuzz driver is laborious and time-consuming in nature but multiple innovative methodologies have been adopted in recent years to ease this task There are three main techniques to develop a fuzz driver: software developers manually writing a fuzz driver, semi-automatic generation of a fuzz driver through human-in-the-loop approaches and fully automatic generation of a fuzz driver. This research study evaluates these techniques through case studies and empirical analysis to recognise the best state-of-the-art fuzz driver generation strategy available for researchers and software testers. Our results show that manually developed fuzz drivers still outperform other methodologies in terms of performance but our results show how other methodologies could surpass their performance levels. Furthermore, this study analyses the effect of varying complexity levels of target functions on the performance of the fuzzing campaigns initiated through multiple fuzz driver generation techniques.Thesis (MPhil) -- University of Adelaide, School of Computer Science , 202

Fifth Conference on Artificial Intelligence for Space Applications

The Fifth Conference on Artificial Intelligence for Space Applications brings together diverse technical and scientific work in order to help those who employ AI methods in space applications to identify common goals and to address issues of general interest in the AI community. Topics include the following: automation for Space Station; intelligent control, testing, and fault diagnosis; robotics and vision; planning and scheduling; simulation, modeling, and tutoring; development tools and automatic programming; knowledge representation and acquisition; and knowledge base/data base integration

Tools and Algorithms for the Construction and Analysis of Systems

This open access book constitutes the proceedings of the 28th International Conference on Tools and Algorithms for the Construction and Analysis of Systems, TACAS 2022, which was held during April 2-7, 2022, in Munich, Germany, as part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2022. The 46 full papers and 4 short papers presented in this volume were carefully reviewed and selected from 159 submissions. The proceedings also contain 16 tool papers of the affiliated competition SV-Comp and 1 paper consisting of the competition report. TACAS is a forum for researchers, developers, and users interested in rigorously based tools and algorithms for the construction and analysis of systems. The conference aims to bridge the gaps between different communities with this common interest and to support them in their quest to improve the utility, reliability, exibility, and efficiency of tools and algorithms for building computer-controlled systems