Specifying and Verifying Concurrent Algorithms with Histories and Subjectivity

We present a lightweight approach to Hoare-style specifications for fine-grained concurrency, based on a notion of time-stamped histories that abstractly capture atomic changes in the program state. Our key observation is that histories form a partial commutative monoid, a structure fundamental for representation of concurrent resources. This insight provides us with a unifying mechanism that allows us to treat histories just like heaps in separation logic. For example, both are subject to the same assertion logic and inference rules (e.g., the frame rule). Moreover, the notion of ownership transfer, which usually applies to heaps, has an equivalent in histories. It can be used to formally represent helping---an important design pattern for concurrent algorithms whereby one thread can execute code on behalf of another. Specifications in terms of histories naturally abstract granularity, in the sense that sophisticated fine-grained algorithms can be given the same specifications as their simplified coarse-grained counterparts, making them equally convenient for client-side reasoning. We illustrate our approach on a number of examples and validate all of them in Coq.Comment: 17 page

Automatic verification of multi-threaded programs by inference of rely-guarantee specifications

Ministry of Education, Singapore under its Academic Research Funding Tier 2; National Research Foundation (NRF) Singapor

Correctness and Progress Verification of Non-Blocking Programs

The progression of multi-core processors has inspired the development of concurrency libraries that guarantee safety and liveness properties of multiprocessor applications. The difficulty of reasoning about safety and liveness properties in a concurrent environment has led to the development of tools to verify that a concurrent data structure meets a correctness condition or progress guarantee. However, these tools possess shortcomings regarding the ability to verify a composition of data structure operations. Additionally, verification techniques for transactional memory evaluate correctness based on low-level read/write histories, which is not applicable to transactional data structures that use a high-level semantic conflict detection. In my dissertation, I present tools for checking the correctness of multiprocessor programs that overcome the limitations of previous correctness verification techniques. Correctness Condition Specification (CCSpec) is the first tool that automatically checks the correctness of a composition of concurrent multi-container operations performed in a non-atomic manner. Transactional Correctness tool for Abstract Data Types (TxC-ADT) is the first tool that can check the correctness of transactional data structures. TxC-ADT elevates the standard definitions of transactional correctness to be in terms of an abstract data type, an essential aspect for checking correctness of transactions that synchronize only for high-level semantic conflicts. Many practical concurrent data structures, transactional data structures, and algorithms to facilitate non-blocking programming all incorporate helping schemes to ensure that an operation comprising multiple atomic steps is completed according to the progress guarantee. The helping scheme introduces additional interference by the active threads in the system to achieve the designed progress guarantee. Previous progress verification techniques do not accommodate loops whose termination is dependent on complex behaviors of the interfering threads, making these approaches unsuitable. My dissertation presents the first progress verification technique for non-blocking algorithms that are dependent on descriptor-based helping mechanisms

Verifikation Nicht-blockierender Datenstrukturen mit Manueller Speicherverwaltung

Verification of concurrent data structures is one of the most challenging tasks in software verification. The topic has received considerable attention over the course of the last decade. Nevertheless, human-driven techniques remain cumbersome and notoriously difficult while automated approaches suffer from limited applicability. This is particularly true in the absence of garbage collection. The intricacy of non-blocking manual memory management (manual memory reclamation) paired with the complexity of concurrent data structures has so far made automated verification prohibitive. We tackle the challenge of automated verification of non-blocking data structures which manually manage their memory. To that end, we contribute several insights that greatly simplify the verification task. The guiding theme of those simplifications are semantic reductions. We show that the verification of a data structure's complicated target semantics can be conducted in a simpler and smaller semantics which is more amenable to automatic techniques. Some of our reductions rely on good conduct properties of the data structure. The properties we use are derived from practice, for instance, by exploiting common programming patterns. Furthermore, we also show how to automatically check for those properties under the smaller semantics. The main contributions are: (i) A compositional verification approach that verifies the memory management and the data structure separately. (ii) A notion of weak ownership that applies when memory is reclaimed and reused, bridging the gap between garbage collection and manual memory management (iii) A notion of pointer races and harmful ABAs the absence of which ensures that the memory management does not influence the data structure, i.e., it behaves as if executed under garbage collection. Notably, we show that a check for pointer races and harmful ABAs only needs to consider executions where at most a single address is reused. (iv) A notion of strong pointer races the absence of which entails the absence of ordinary pointer races and harmful ABAs. We devise a highly-efficient type check for strong pointer races. After a successful type check, the actual verification can be performed under garbage collection using an off-the-shelf verifier. (v) Experimental evaluations of the aforementioned contributions. We are the first to fully automatically verify practical non-blocking data structures with manual memory management.Verifikation nebenläufiger Datenstrukturen ist eine der herausforderndsten Aufgaben der Programmverifikation. Trotz vieler Beiträge zu diesem Thema, bleiben die existierenden manuellen Techniken mühsam und kompliziert in der Anwendung. Auch automatisierte Verifikationsverfahren sind nur eingeschränkt anwendbar. Diese Schwächen sind besonders ausgeprägt, wenn sich Programme nicht auf einen Garbage-Collector verlassen. Die Komplexität manueller Speicherverwaltung gepaart mit komplexen nicht-blockierenden Datenstrukturen macht die automatisierte Programmverifikation derzeit unmöglich. Diese Arbeit betrachtet die automatisierte Verifikation nicht-blockierender Datenstrukturen, welche ihren Speicher manuell verwalten. Dazu werden Konzepte vorgestellt, die die Verifikation stark vereinfachen. Das Leitmotiv dabei ist die semantische Reduktion, welche die Verifikation in einer leichteren Semantik erlaubt, ohne die eigentliche komplexere Semantik zu betrachten. Einige dieser Reduktion beruhen auf einem Wohlverhalten des zu verifizierenden Programms. Dabei wird das Wohlverhalten mit Bezug auf praxisnahe Eigenschaften definiert, wie sie z.B. von gängigen Programmiermustern vorgegeben werden. Ferner wird gezeigt, dass die Wohlverhaltenseigenschaften ebenfalls unter der einfacheren Semantik nachgewiesen werden können. Die Hauptresultate der vorliegenden Arbeit sind die Folgenden: (i) Ein kompositioneller Verifikationsansatz, welcher Speicherverwaltung und Datenstruktur getrennt verifiziert. (ii) Ein Begriff des Weak-Ownership, welcher selbst dann Anwendung findet, wenn Speicher wiederverwendet wird. (iii) Ein Begriff des Pointer-Race und des Harmful-ABA, deren Abwesenheit garantiert, dass die Speicherverwaltung keinen Einfluss auf die Datenstruktur ausübt und somit unter der Annahme von Garbage-Collection verifiziert werden kann. Bemerkenswerterweise genügt es diese Abwesenheit unter Reallokation nur einer fixex Speicherzelle zu prüfen. (iv) Ein Begriff des Strong-Pointer-Race, dessen Abwesenheit sowohl Pointer-Races als auch Harmful-ABA ausschließt. Um ein Programm auf Strong-Pointer-Races zu prüfen, präsentieren wir ein Typsystem. Ein erfolgreicher Typcheck erlaubt die tatsächlich zu überprüfende Eigenschaft unter der Annahme eines Garbage-Collectors nachzuweisen. (v) Experimentelle Evaluationen. Die vorgestellten Techniken sind die Ersten, die nicht-blockierende Datenstrukturen mit gängigen Speicherverwaltungen vollständig automatisch verifizieren können