343 research outputs found
Programming Language Abstractions for Modularly Verified Distributed Systems
Distributed systems are rarely developed as monolithic programs. Instead, like any software, these systems may consist of multiple program components, which are then compiled separately and linked together. Modern systems also incorporate various services interacting with each other and with client applications. However, state-of-the-art verification tools focus predominantly on verifying standalone, closed-world protocols or systems, thus failing to account for the compositional nature of distributed systems. For example, standalone verification has the drawback that when protocols and their optimized implementations evolve, one must re-verify the entire system from scratch, instead of leveraging compositionality to contain the reverification effort.
In this paper, we focus on the challenge of modular verification of distributed systems with respect to high-level protocol invariants as well as for low-level implementation safety properties. We argue that the missing link between the two is a programming paradigm that would allow one to reason about both high-level distributed protocols and low-level implementation primitives in a single verification-friendly framework. Such a link would make it possible to reap the benefits from both the vast body of research in distributed computing, focused on modular protocol decomposition and consistency properties, as well as from the recent advances in program verification, enabling construction of provably correct systems implementations. To showcase the modular verification challenges, we present some typical scenarios of decomposition between a distributed protocol and its implementations. We then describe our ongoing research agenda, in which we are attempting to address the outlined problems by providing a typing discipline and a set of domain-specific primitives for specifying, implementing and verifying distributed systems. Our approach, mechanized within a proof assistant, provides the means of decomposition necessary for modular proofs about distributed protocols and systems
From Network Interface to Multithreaded Web Applications: A Case Study in Modular Program Verification
Many verifications of realistic software systems are monolithic, in the sense that they define single global invariants over complete system state. More modular proof techniques promise to support reuse of component proofs and even reduce the effort required to verify one concrete system, just as modularity simplifies standard software development. This paper reports on one case study applying modular proof techniques in the Coq proof assistant. To our knowledge, it is the first modular verification certifying a system that combines infrastructure with an application of interest to end users. We assume a nonblocking API for managing TCP networking streams, and on top of that we work our way up to certifying multithreaded, database-backed Web applications. Key verified components include a cooperative threading library and an implementation of a domain-specific language for XML processing. We have deployed our case-study system on mobile robots, where it interfaces with off-the-shelf components for sensing, actuation, and control.National Science Foundation (U.S.) (Grant CCF-1253229)United States. Defense Advanced Research Projects Agency (Agreement FA8750-12-2-0293
From Network Interface to Multithreaded Web Applications: A Case Study in Modular Program Verification
Many verifications of realistic software systems are monolithic, in the sense that they define single global invariants over complete system state. More modular proof techniques promise to support reuse of component proofs and even reduce the effort required to verify one concrete system, just as modularity simplifies standard software development. This paper reports on one case study applying modular proof techniques in the Coq proof assistant. To our knowledge, it is the first modular verification certifying a system that combines infrastructure with an application of interest to end users. We assume a nonblocking API for managing TCP networking streams, and on top of that we work our way up to certifying multithreaded, database-backed Web applications. Key verified components include a cooperative threading library and an implementation of a domain-specific language for XML processing. We have deployed our case-study system on mobile robots, where it interfaces with off-the-shelf components for sensing, actuation, and control.National Science Foundation (U.S.) (NSF grant CCF-1253229)United States. Defense Advanced Research Projects Agency (DARPA, agreement number FA8750-12-2-0293
Shared Memory Implementations of Protocol Programming Languages:Data-Race-Free
Protocol programming languages are domain-specific languages that offer higher-level abstractions for programming of synchronization and communication protocols among participants. However, most implementations of protocol programming languages on shared memory architectures use pointer passing to exchange data in communications, so programs can still run into data races. We report on our ongoing efforts toward the first shared memory implementation of a protocol programming language that guarantees freedom of data races, without excessive copying, by leveraging the programming language Rust and its type system.</p
Paxos Consensus, Deconstructed and Abstracted (Extended Version)
Lamport's Paxos algorithm is a classic consensus protocol for state machine
replication in environments that admit crash failures. Many versions of Paxos
exploit the protocol's intrinsic properties for the sake of gaining better
run-time performance, thus widening the gap between the original description of
the algorithm, which was proven correct, and its real-world implementations. In
this work, we address the challenge of specifying and verifying complex
Paxos-based systems by (a) devising composable specifications for
implementations of Paxos's single-decree version, and (b) engineering
disciplines to reason about protocol-aware, semantics-preserving optimisations
to single-decree Paxos. In a nutshell, our approach elaborates on the
deconstruction of single-decree Paxos by Boichat et al. We provide novel
non-deterministic specifications for each module in the deconstruction and
prove that the implementations refine the corresponding specifications, such
that the proofs of the modules that remain unchanged can be reused across
different implementations. We further reuse this result and show how to obtain
a verified implementation of Multi-Paxos from a verified implementation of
single-decree Paxos, by a series of novel protocol-aware transformations of the
network semantics, which we prove to be behaviour-preserving.Comment: Accepted for publication in the 27th European Symposium on
Programming (ESOP'18
Formal Specification and Verification of Hyperledger Fabric Chaincode
Smart contracts are programs building on blockchain technology.
They implement functionality that has been agreed on between
the concerned parties on a network. However, their immutability
and exposed position make them vulnerable to programming errors,
leading to faulty behavior and possible exploits. Therefore,
smart contracts demand a particularly thorough analysis,
ideally using formal program verification. In this paper, we
present an approach for the deductive verification of
Hyperledger Fabric smart contracts using the KeY prover.
We have extended KeY to handle Fabric ledger implementations;
in particular, we have developed mechanisms for reasoning about
serialization and object persistence. The feasibility of our
approach is demonstrated with a small case study
Toward Domain-Specific Solvers for Distributed Consistency
To guard against machine failures, modern internet services store multiple replicas of the same application data within and across data centers, which introduces the problem of keeping geo-distributed replicas consistent with one another in the face of network partitions and unpredictable message latency. To avoid costly and conservative synchronization protocols, many real-world systems provide only weak consistency guarantees (e.g., eventual, causal, or PRAM consistency), which permit certain kinds of disagreement among replicas.
There has been much recent interest in language support for specifying and verifying such consistency properties. Although these properties are usually beyond the scope of what traditional type checkers or compiler analyses can guarantee, solver-aided languages are up to the task. Inspired by systems like Liquid Haskell [Vazou et al., 2014] and Rosette [Torlak and Bodik, 2014], we believe that close integration between a language and a solver is the right path to consistent-by-construction distributed applications. Unfortunately, verifying distributed consistency properties requires reasoning about transitive relations (e.g., causality or happens-before), partial orders (e.g., the lattice of replica states under a convergent merge operation), and properties relevant to message processing or API invocation (e.g., commutativity and idempotence) that cannot be easily or efficiently carried out by general-purpose SMT solvers that lack native support for this kind of reasoning.
We argue that domain-specific SMT-based tools that exploit the mathematical foundations of distributed consistency would enable both more efficient verification and improved ease of use for domain experts. The principle of exploiting domain knowledge for efficiency and expressivity that has borne fruit elsewhere - such as in the development of high-performance domain-specific languages that trade off generality to gain both performance and productivity - also applies here. Languages augmented with domain-specific, consistency-aware solvers would support the rapid implementation of formally verified programming abstractions that guarantee distributed consistency. In the long run, we aim to democratize the development of such domain-specific solvers by creating a framework for domain-specific solver development that brings new theory solver implementation within the reach of programmers who are not necessarily SMT solver internals experts
Programming and Proving with Distributed Protocols
Distributed systems play a crucial role in modern infrastructure, but are notoriously difficult to
implement correctly. This difficulty arises from two main challenges: (a) correctly implementing
core system components (e.g., two-phase commit), so all their internal invariants hold, and (b)
correctly composing standalone system components into functioning trustworthy applications (e.g.,
persistent storage built on top of a two-phase commit instance). Recent work has developed several
approaches for addressing (a) by means of mechanically verifying implementations of core distributed
components, but no methodology exists to address (b) by composing such verified components into
larger verified applications. As a result, expensive verification efforts for key system components are
not easily reusable, which hinders further verification efforts.
In this paper, we present Disel, the first framework for implementation and compositional
verification of distributed systems and their clients, all within the mechanized, foundational context
of the Coq proof assistant. In Disel, users implement distributed systems using a domain specific
language shallowly embedded in Coq and providing both high-level programming constructs as well
as low-level communication primitives. Components of composite systems are specified in Disel as
protocols, which capture system-specific logic and disentangle system definitions from implementation
details. By virtue of Diselâs dependent type system, well-typed implementations always satisfy
their protocolsâ invariants and never go wrong, allowing users to verify system implementations
interactively using Diselâs Hoare-style program logic, which extends state-of-the-art techniques for
concurrency verification to the distributed setting. By virtue of the substitution principle and frame
rule provided by Diselâs logic, system components can be composed leading to modular, reusable
verified distributed systems.
We describe Disel, illustrate its use with a series of examples, outline its logic and metatheory,
and report on our experience using it as a framework for implementing, specifying, and verifying
distributed systems
- âŠ