Dash+: Extending Alloy with Replicated Processes for Modelling Transition Systems

Modelling systems abstractly shows great promise to uncover bugs early in system development. The formal language Alloy provides the means of writing constraints abstractly but lacks explicit constructs for describing transition systems. Extensions to Alloy, such as Electrum, DynAlloy, and Dash, provide such constructs. However, still missing are language constructs to describe easily multiple processes with the same behavior (replicated processes) running in parallel as is found in languages such as PlusCal and Promela. We propose extensions to Dash for replicated processes. The result is Dash+: an Alloy language extension for describing transition systems that include both concurrent and hierarchical states and replicated concurrent processes. The processes can communicate via buffers or exchange information through variables and events. The key contributions of our novel approach are: 1) Replicated and non-replicated components can be nested arbitrarily at any level in the state hierarchy 2) Replicated components can exchange information directly without resorting to global variables as is the case in PlusCal and Promela 3) A modeller can abstractly model the topology of the processes (ring, list, etc.) through constraints on the set indexing the processes 4) Buffers can be used to facilitate communication between replicated components Dash+ stays consistent with the semantics of Dash and uses the notion of big steps and small steps to describe changes in the system. The semantics are implemented in a translation to Alloy in a way that accommodates the following model checking options: traces-based model checking, transitive closure-based model checking (TCMC), and Electrum. Our implementation is fully integrated into the Alloy Analyzer. This thesis presents case studies to demonstrate the features of Dash+ in modelling systems with concurrent processes and the benefits that Dash+ offers over existing languages. We check for properties in each of the models in the case studies to demonstrate how different model checking options can be used

Currency management system: a distributed banking service for the grid

Market based resource allocation mechanisms require mechanisms to regulate and manage the usage of traded resources. One mechanism to control this is the definition of some kind of currency. Within this context, we have implemented a first prototype of our Currency Management System, which stands for a decentralized and scalable banking service for the Grid. Basically, our system stores user accounts within a DHT and its basic operation is the transferFunds which, as its name suggests, transfers virtual currency from an account to one another

Applying Formal Methods to Networking: Theory, Techniques and Applications

Despite its great importance, modern network infrastructure is remarkable for the lack of rigor in its engineering. The Internet which began as a research experiment was never designed to handle the users and applications it hosts today. The lack of formalization of the Internet architecture meant limited abstractions and modularity, especially for the control and management planes, thus requiring for every new need a new protocol built from scratch. This led to an unwieldy ossified Internet architecture resistant to any attempts at formal verification, and an Internet culture where expediency and pragmatism are favored over formal correctness. Fortunately, recent work in the space of clean slate Internet design---especially, the software defined networking (SDN) paradigm---offers the Internet community another chance to develop the right kind of architecture and abstractions. This has also led to a great resurgence in interest of applying formal methods to specification, verification, and synthesis of networking protocols and applications. In this paper, we present a self-contained tutorial of the formidable amount of work that has been done in formal methods, and present a survey of its applications to networking.Comment: 30 pages, submitted to IEEE Communications Surveys and Tutorial

Model-based testing of global properties on large-scale distributed systems

Context Large-scale distributed systems are becoming commonplace with the large popularity of peer-to-peer and cloud computing. The increasing importance of these systems contrasts with the lack of integrated solutions to build trustworthy software. A key concern of any large-scale distributed system is the validation of global properties, which cannot be evaluated on a single node. Thus, it is necessary to gather data from distributed nodes and to aggregate these data into a global view. This turns out to be very challenging because of the system's dynamism that imposes very frequent changes in local values that affect global properties. This implies that the global view has to be frequently updated to ensure an accurate validation of global properties. Objective In this paper, we present a model-based approach to define a dynamic oracle for checking global properties. Our objective is to abstract relevant aspects of such systems into models. These models are updated at runtime, by monitoring the corresponding distributed system. Method We conduce real-scale experimental validation to evaluate the ability of our approach to check global properties. In this validation, we apply our approach to test two open-source implementations of distributed hash tables. The experiments are deployed on two clusters of 32 nodes. Results The experiments reveal an important defect on one implementation and show clear performance differences between the two implementations. The defect would not be detected without a global view of the system. Conclusion Testing global properties on distributed software consists of gathering data from different nodes and building a global view of the system, where properties are validated. This process requires a distributed test architecture and tools for representing and validating global properties. Model-based techniques are an expressive mean for building oracles that validate global properties on distributed systems. © 2014 Elsevier B.V. All rights reserved

Formal verification of the pastry protocol

Pastry is a structured P2P algorithm realizing a Distributed Hash Table (DHT) over an underlying virtual ring of nodes. Hash keys are assigned to the numerically closest node, according to their Ids that both keys and nodes share from the same Id space. Nodes join and leave the ring dynamically and it is desired that a lookup request from arbitrary node for a key is routed to the responsible node for that key which then delivers the message as answer. Several implementations of Pastry are available and have been applied in practice, but no attempt has so far been made to formally describe the algorithm or to verify its properties. Since Pastry combines rather complex data structures, asynchronous communication, concurrency, resilience to churn, i.e. spontaneous join and departure of nodes, it makes an interesting target for verification. This thesis formally models and improves Pastry\u27s core algorithms, such that they provide the correct lookup service in the presence of churn and maintain a local data structures to adapt the dynamic updates of neighborhood. This thesis focuses on Join protocol of Pastry and formally defines different statuses (from "dead" to "ready") of a node according to its stage during join. Only "ready" nodes are suppose to have consistent key mapping among each other and are allowed to deliver the answer message. The correctness property is identified by this thesis tobe CorrectDelivery, stating that there is always at most one node that can deliver an answer to a lookup request for a key and this node is the numerically closest "ready" node to that key. This property is non-trivial to preserve in the presence of churn. The specification language TLA+ is used to model different versions of Pastry algorithm starting with CastroPastry, followed by HaeberlenPastry, IdealPastry and finally LuPastry. The TLA+ model checker TLC is employed to validate the models and to search for bugs. Models are simplified for more efficient checking with TLC and consequently mitigating the state explosion problem. Through this thesis, unexpected violations of CorrectDelivery in CastroPastry and HaeberlenPastry are discovered and analyzed. Based on the analysis, Haeberlen-Pastry is improved to a new design of the Pastry protocol IdealPastry, which is first verified using the interactive theorem prover TLAPS for TLA+. IdealPastry assumes that a "ready" node handles one joining node at a time and it assumes that (1) no departure of nodes (2) no concurrent join between two "ready" nodes closed to each other. The last assumption of IdealPastry is removed by its improved version LuPastry. In LuPastry, a "ready" node adds the joining node directly when it receives the join request and does not accepts any further join request until it gets the confirmation from the current joining node that it is "ready". LuPastry is proved to be correct w.r.t. CorrectDelivery under the assumption that no nodes leave the network, which cannot be further relaxed due to possible network separation when particular nodes simultaneously leave the network. The most subtle part of the deductive system verification is the search for an appropriate inductive invariant which implies the required safety property and is inductively preserved by all possible actions. The search is guided by the construction of the proof, where TLC is used to discover unexpected violations of a hypothetical invariant postulated in an earlier stage. The final proof of LuPastry consists of more than 10,000 proof steps, which are interactively checked in time by using TLAPS launching different back-end automated theorem provers. This thesis serves also as a case study giving the evidence of possibility and the methodology of how to formally model, to analyze and to manually conduct a formal proof of complex transition system for its safety property. Using LuPastry as template, a more general framework on verification of DHT can be constructed.Pastry ist ein P2P ( peer-to-peer) Algorithmus, der eine verteilte Hashtabelle (DHT) über einem als virtuellen Ring strukturierten Netzwerk realisiert. Knoten-Identifikatoren und Hash-Schlüssel entstammen derselben Menge, und jeder Knoten verwaltet die Schlüssel, die seinem Identifikator am nächsten liegen. Knoten können sich zur Laufzeit ins Netz einfügen bzw. es verlassen. Dennoch sollen Anfragen nach einem Schlüssel von beliebigen Knoten immer zu demjenigen Knoten weitergeleitet werden, der den Schlüssel verwaltet, und der die Anfrage dann beantwortet. Pastry wurde mehrfach implementiert und praktisch eingesetzt, aber der Algorithmus wurde bisher noch nie mathematisch präzise modelliert und auf Korrektheit untersucht. Da bei Pastry komplexe Datenstrukturen, asynchrone Kommunikation in einem verteilten Netzwerk und Robustheit gegen churn, d.h. spontanes Einfügen oder Verlassen von Knoten zusammenkommen, stellt das Protokoll eine interessante Fallstudie für formale Verifikationstechniken dar. In dieser Arbeit werden die Kernalgorithmen von Pastry modelliert, die Anfragen nach Schlüsseln in Gegenwart von churn behandeln und lokale Datenstrukturen verwalten, welche die jeweilige Nachbarschaftsbeziehungen zwischen Knoten zur Laufzeit widerspiegeln. Diese Dissertation behandelt insbesondere das Join-Protokoll von Pastry zum Einfügen neuer Knoten ins Netz, das jedem Knoten seinen Status (von dead\u27\u27 bis ready\u27\u27) zuweist. Knoten mit Status ready\u27\u27 müssen untereinander konsistente Modelle der Zuständigkeit für Schlüssel aufweisen und dürfen Anfragen nach Schlüsseln beantworten. Als zentrale Korrektheitseigenschaft wird in dieser Arbeit CorrectDelivery untersucht, die ausdrückt, dass zu jeder Zeit höchstens ein Knoten Anfragen nach einem Schlüssel beantworten darf, und dass es sich dabei um den Knoten mit Status ready\u27\u27 handelt, dessen Identifikator dem Schlüssel numerisch am nâchsten liegt. In Gegenwart von churn ist es nicht einfach diese Eigenschaft sicherzustellen. Wir benutzen die Spezifikationssprache TLA+, um verschiedene Versionen des Pastry-Protokolls zu modellieren: zunächst CastroPastry, gefolgt von HaeberlenPastry und IdealPastry, und schließlich LuPastry. Mit Hilfe des Modelcheckers TLC für TLA+ werden verschiedene qualitative Eigenschaften untersucht, um die Modelle zu validieren und Fehler zu finden. Dafür werden die Modelle zum Teil vereinfacht, um das Problem der Zustandsexplosion zu mindern und so die Effizienz des Modelchecking zu verbessern. Diese Arbeit konnte unerwartete Abläufe von CastroPastry und HaeberlenPastry aufdecken, bei denen die Eigenschaft CorrectDelivery verletzt wird. Auf der Grundlage dieser Analyse und einiger Verbesserungen von HaeberlenPastry wird das Protokoll IdealPastry entwickelt und seine Korrektheit mit Hilfe des interaktiven Theorembeweisers TLAPS für TLA+ gezeigt. IdealPastry stellt sicher, dass ein ready\u27\u27 Knoten zu jeder Zeit höchstens einen neuen Knoten ins Netz einfügt, und es nimmt an, dass (1) kein Knoten je das Netz verlässt und (2) keine zwei Knoten zwischen benachbarten ready\u27\u27 Knoten eingefügt werden. Der Algorithmus LuPastry verbessert IdealPastry und beseitigt Annahme (2) von IdealPastry. In dieser Version nimmt ein ready\u27\u27 Knoten den neu einzufügenden Knoten unmittelbar in seine Nachbarschaft auf und akzeptiert dann solange keinen weiteren neu hinzukommenden Knoten, bis der erste Knoten bestätigt, dass er Status ready\u27\u27 erreicht hat. LuPastry wird als korrekt bezüglich der Eigenschaft CorrectDelivery nachgewiesen, unter der Annahme, dass keine Knoten das Netz verlassen. Diese Annahme kann im allgemeinen nicht vermieden werden, da der Ring in separate Teilnetze zerfallen könnte, wenn bestimmte Knoten gleichzeitig das Netz verlassen. Die größte Herausforderung bei deduktiven Ansätzen zur Systemverifikation ist es, eine geeignete Invariante zu finden, die sowohl die angestrebte Sicherheitseigenschaft impliziert als auch induktiv von allen Systemaktionen erhalten wird. Während der Konstruktion des Korrektheitsbeweises wird TLC benutzt, um unerwartete Gegenbeispiele zu hypothetischen Invarianten zu finden, die zuvor postuliert wurden. Der Beweis des LuPastry-Protokolls besteht aus circa 10000 Beweisschritten, die von TLAPS und seinen integrierten automatischen Theorembeweisern verifiziert werden. Die vorliegende Arbeit dient auch als Fallstudie, welche die Möglichkeit der formalen Modellierung, Analyse und Korrektheitsbeweises von komplexen Transitionssystemen aufzeigt und die dabei notwendigen Einzelschritte und -techniken behandelt. LuPastry kann als Vorlage benutzt werden, um einen allgemeineren Rahmen für die Verifikation von DHT-Protokollen zu schaffen

Analysis of the Blockchain Protocol in Asynchronous Networks

Nakamoto\u27s famous blockchain protocol enables achieving consensus in a so-called \emph{permissionless setting}---anyone can join (or leave) the protocol execution, and the protocol instructions do not depend on the identities of the players. His ingenious protocol prevents ``sybil attacks\u27\u27 (where an adversary spawns any number of new players) by relying on computational puzzles (a.k.a. ``moderately hard functions\u27\u27) introduced by Dwork and Naor (Crypto\u2792). The analysis of the blockchain consensus protocol (a.k.a. Nakamoto consensus) has been a notoriously difficult task. Prior works that analyze it either make the simplifying assumption that network channels are fully synchronous (i.e. messages are instantly delivered without delays) (Garay et al, Eurocrypt\u2715) or only consider specific attacks (Nakamoto\u2708; Sampolinsky and Zohar, FinancialCrypt\u2715); additionally, as far as we know, none of them deal with players joining or leaving the protocol. In this work we prove that the blockchain consensus mechanism satisfies a strong forms of *consistency* and *liveness* in an asynchronous network with adversarial delays that are a-priori bounded, within a formal model allowing for adaptive corruption and spawning of new players, assuming that the computational puzzle is modeled as a random oracle. (We complement this result by showing a simple attack against the blockchain protocol in a fully asynchronous setting, showing that the “puzzle-hardness” needs to be appropriately set as a function of the maximum network delay; this attack applies even for static corruption.) As an independent contribution, we define an abstract notion of a blockchain protocol and identify appropriate security properties of such protocols; we prove that Nakamoto\u27s blockchain protocol satisfies them and that these properties are sufficient for typical applications; we hope that this abstraction may simplify further applications of blockchains