Towards cooperative software verification with test generation and formal verification

There are two major methods for software verification: testing and formal verification. To increase our confidence in software on a large scale, we require tools that apply these methods automatically and reliably. Testing with manually written tests is widespread, but for automatically generated tests for the C programming language there is no standardized format. This makes the use and comparison of automated test generators expensive. In addition, testing can never provide full confidence in software—it can show the presence of bugs, but not their absence. In contrast, formal verification uses established, standardized formats and can prove the absence of bugs. Unfortunately, even successful formal-verification techniques suffer from different weaknesses. Compositions of multiple techniques try to combine the strengths of complementing techniques, but such combinations are often designed as cohesive, monolithic units. This makes them inflexible and it is costly to replace components. To improve on this state of the art, we work towards an off-the-shelf cooperation between verification tools through standardized exchange formats. First, we work towards standardization of automated test generation for C. We increase the comparability of test generators through a common benchmarking framework and reliable tooling, and provide means to reliably compare the bug-finding capabilities of test generators and formal verifiers. Second, we introduce new concepts for the off-the-shelf cooperation between verifiers (both test generators and formal verifiers). We show the flexibility of these concepts through an array of combinations and through an application to incremental verification. We also show how existing, strongly coupled techniques in software verification can be decomposed into stand-alone components that cooperate through clearly defined interfaces and standardized exchange formats. All our work is backed by rigorous implementation of the proposed concepts and thorough experimental evaluations that demonstrate the benefits of our work. Through these means we are able to improve the comparability of automated verifiers, allow the cooperation between a large array of existing verifiers, increase the effectiveness of software verification, and create new opportunities for further research on cooperative verification

Proceedings of the First NASA Formal Methods Symposium

Topics covered include: Model Checking - My 27-Year Quest to Overcome the State Explosion Problem; Applying Formal Methods to NASA Projects: Transition from Research to Practice; TLA+: Whence, Wherefore, and Whither; Formal Methods Applications in Air Transportation; Theorem Proving in Intel Hardware Design; Building a Formal Model of a Human-Interactive System: Insights into the Integration of Formal Methods and Human Factors Engineering; Model Checking for Autonomic Systems Specified with ASSL; A Game-Theoretic Approach to Branching Time Abstract-Check-Refine Process; Software Model Checking Without Source Code; Generalized Abstract Symbolic Summaries; A Comparative Study of Randomized Constraint Solvers for Random-Symbolic Testing; Component-Oriented Behavior Extraction for Autonomic System Design; Automated Verification of Design Patterns with LePUS3; A Module Language for Typing by Contracts; From Goal-Oriented Requirements to Event-B Specifications; Introduction of Virtualization Technology to Multi-Process Model Checking; Comparing Techniques for Certified Static Analysis; Towards a Framework for Generating Tests to Satisfy Complex Code Coverage in Java Pathfinder; jFuzz: A Concolic Whitebox Fuzzer for Java; Machine-Checkable Timed CSP; Stochastic Formal Correctness of Numerical Algorithms; Deductive Verification of Cryptographic Software; Coloured Petri Net Refinement Specification and Correctness Proof with Coq; Modeling Guidelines for Code Generation in the Railway Signaling Context; Tactical Synthesis Of Efficient Global Search Algorithms; Towards Co-Engineering Communicating Autonomous Cyber-Physical Systems; and Formal Methods for Automated Diagnosis of Autosub 6000

Fundamental Approaches to Software Engineering

computer software maintenance; computer software selection and evaluation; formal logic; formal methods; formal specification; programming languages; semantics; software engineering; specifications; verificatio

Hardware-Assisted Processor Tracing for Automated Bug Finding and Exploit Prevention

The proliferation of binary-only program analysis techniques like fuzz testing and symbolic analysis have lead to an acceleration in the number of publicly disclosed vulnerabilities. Unfortunately, while bug finding has benefited from recent advances in automation and a decreasing barrier to entry, bug remediation has received less attention. Consequently, analysts are publicly disclosing bugs faster than developers and system administrators can mitigate them. Hardware-supported processor tracing within commodity processors opens new doors to observing low-level behaviors with efficiency, transparency, and integrity that can close this automation gap. Unfortunately, several trade-offs in its design raise serious technical challenges that have limited widespread adoption. Specifically, modern processor traces only capture control flow behavior, yield high volumes of data that can incur overhead to sift through, and generally introduce a semantic gap between low-level behavior and security relevant events. To solve the above challenges, I propose control-oriented record and replay, which combines concrete traces with symbolic analysis to uncover vulnerabilities and exploits. To demonstrate the efficacy and versatility of my approach, I first present a system called ARCUS, which is capable of analyzing processor traces flagged by host-based monitors to detect, localize, and provide preliminary patches to developers for memory corruption vulnerabilities. ARCUS has detected 27 previously known vulnerabilities alongside 4 novel cases, leading to the issuance of several advisories and official developer patches. Next, I present MARSARA, a system that protects the integrity of execution unit partitioning in data provenance-based forensic analysis. MARSARA prevents several expertly crafted exploits from corrupting partitioned provenance graphs while incurring little overhead compared to prior work. Finally, I present Bunkerbuster, which extends the ideas from ARCUS and MARSARA into a system capable of proactively hunting for bugs across multiple end-hosts simultaneously, resulting in the discovery and patching of 4 more novel bugs.Ph.D

Fundamental Approaches to Software Engineering

This open access book constitutes the proceedings of the 23rd International Conference on Fundamental Approaches to Software Engineering, FASE 2020, which took place in Dublin, Ireland, in April 2020, and was held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2020. The 23 full papers, 1 tool paper and 6 testing competition papers presented in this volume were carefully reviewed and selected from 81 submissions. The papers cover topics such as requirements engineering, software architectures, specification, software quality, validation, verification of functional and non-functional properties, model-driven development and model transformation, software processes, security and software evolution

Combining over- and under-approximating program analyses for automatic software testing

This dissertation attacks the well-known problem of path-imprecision in static program analysis. Our starting point is an existing static program analysis that over-approximates the execution paths of the analyzed program. We then make this over-approximating program analysis more precise for automatic testing in an object-oriented programming language. We achieve this by combining the over-approximating program analysis with usage-observing and under-approximating analyses. More specifically, we make the following contributions. We present a technique to eliminate language-level unsound bug warnings produced by an execution-path-over-approximating analysis for object-oriented programs that is based on the weakest precondition calculus. Our technique post-processes the results of the over-approximating analysis by solving the produced constraint systems and generating and executing concrete test-cases that satisfy the given constraint systems. Only test-cases that confirm the results of the over-approximating static analysis are presented to the user. This technique has the important side-benefit of making the results of a weakest-precondition based static analysis easier to understand for human consumers. We show examples from our experiments that visually demonstrate the difference between hundreds of complicated constraints and a simple corresponding JUnit test-case. Besides eliminating language-level unsound bug warnings, we present an additional technique that also addresses user-level unsound bug warnings. This technique pre-processes the testee with a dynamic analysis that takes advantage of actual user data. It annotates the testee with the knowledge obtained from this pre-processing step and thereby provides guidance for the over-approximating analysis. We also present an improvement to dynamic invariant detection for object-oriented programming languages. Previous approaches do not take behavioral subtyping into account and therefore may produce inconsistent results, which can throw off automated analyses such as the ones we are performing for bug-finding. Finally, we address the problem of unwanted dependencies between test-cases caused by global state. We present two techniques for efficiently re-initializing global state between test-case executions and discuss their trade-offs. We have implemented the above techniques in the JCrasher, Check 'n' Crash, and DSD-Crasher tools and present initial experience in using them for automated bug finding in real-world Java programs.Ph.D.Committee Chair: Smaragdakis, Yannis; Committee Member: Dwyer, Matthew; Committee Member: Orso, Alessandro; Committee Member: Pande, Santosh; Committee Member: Rugaber, Spence

RUN-TIME ANALYSIS AND SECURITY OF MULTI-LANGUAGE SYSTEMS

The contemporary software development landscape has witnessed a widespread integration of diverse programming languages, leveraging the specific advantages of each, such as the efficiency of C and the programmability of Python. This trend finds notable applications in prominent domains, including the Android operating system and advanced machine learning frameworks like PyTorch. However, adopting this multi-language approach has ushered in aseries of great challenges for developers, necessitating the identification of robust solutions to tackle potential security vulnerabilities.Traditional techniques such as program analysis and fuzzing, initially designed for single-language software, face limitations in effectively uncovering vulnerabilities in multi-language systems. Program analysis grapples with challenges in comprehending the intricate control and data flows across diverse languages, often resulting in incomplete vulnerability detection. Conversely, greybox fuzzing encounters difficulties adapting to the nuances of various languages, leading to incomplete code coverage and complications in reproducing identified vulnerabilities. The intricacies within runtime systems supporting multilingual software exacerbate the security clearance predicament, as these systems are often constructed using multiple languages. This complexity adds an additional layer of difficulty for conventional security techniques, emphasizing the need for more adaptive and comprehensive approachestailored to the unique challenges posed by the multifaceted nature of multi-language systems.Within the scope of my dissertation, I endeavored to tackle the intricate challenges posed by vulnerabilities in multi-language software through a comprehensive and multifaceted approach. This approach entailed conducting extensive empirical investigations into vulnerability susceptibility, facilitating the development of dynamic cross-language information flow analysis. Recognizing the pivotal significance of comprehensive test input coverage, I devisedan integrated greybox fuzzing methodology. This innovative approach integrates sensitivity analysis and comprehensive whole-system coverage measurements, significantly enhancing the efficiency of the fuzzing process and vulnerability identification. Furthermore, I focused on fortifying runtime security by proposing a novel two-level collaborative fuzzing framework tailored explicitly for Python language runtime. This contribution was pivotal in reinforcing the software system’s foundational safeguards, ensuring a robust defense mechanism against potential security threats

Tools and Algorithms for the Construction and Analysis of Systems

This open access two-volume set constitutes the proceedings of the 27th International Conference on Tools and Algorithms for the Construction and Analysis of Systems, TACAS 2021, which was held during March 27 – April 1, 2021, as part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2021. The conference was planned to take place in Luxembourg and changed to an online format due to the COVID-19 pandemic. The total of 41 full papers presented in the proceedings was carefully reviewed and selected from 141 submissions. The volume also contains 7 tool papers; 6 Tool Demo papers, 9 SV-Comp Competition Papers. The papers are organized in topical sections as follows: Part I: Game Theory; SMT Verification; Probabilities; Timed Systems; Neural Networks; Analysis of Network Communication. Part II: Verification Techniques (not SMT); Case Studies; Proof Generation/Validation; Tool Papers; Tool Demo Papers; SV-Comp Tool Competition Papers

Performant Software Hardening under Hardware Support

With a booming number of applications and end-users in the past decade, software security has been emphasized more than ever. Nonetheless, a consistent increase of security-critical bugs has been observed along the way, mainly due to the variety and complexity of existing software pieces. To mitigate the situation, software hardening in the daily development cycle typically involves three phases, including bug finding, runtime security enforcement, and fault analyses in case the prior steps have failed. Among the various software hardening techniques proposed, a considerable number of works have relied on available hardware support to achieve their goals. The reasons behind the noticeable trend are three-folded. First, the performance benefit from hardware can be substantial compared to a purely software-based solution. Second, compatibility and ease of use are also keys for more solutions to adopt hardware features besides the performance gain. Last, implementation with hardware support can consequentially present a smaller codebase, thus introducing less attack surface for attackers. In this dissertation, I present three hardware-assisted solutions for performant software hardening. The first one is PITTYPAT, a runtime enforcement for path-sensitive control-flow integrity. By utilizing Intel PT, it computes branch targets with points-to analyses in an efficient and precise manner. The second one is SNAP, a customized hardware platform that implements hardware primitives to enhance the performance of coverage-guided fuzzing. Given the program states originated from the existing CPU pipeline, our prototype on the FPGA platform enables a transparent support of fuzzing with near-zero tracing overhead. Finally, I will present a nested virtualization framework for fuzzing non-user applications, such as hypervisors. With a snapshot mechanism supported by the x86 virtualization extension and a customized kernel for fuzzing execution, our system demonstrates a 72x improvement on the fuzzing throughput compared to the prior solutions, and finds 14 zero-day bugs among the real-world hypervisors.Ph.D