Maintaining consistency in distributed systems

In systems designed as assemblies of independently developed components, concurrent access to data or data structures normally arises within individual programs, and is controlled using mutual exclusion constructs, such as semaphores and monitors. Where data is persistent and/or sets of operation are related to one another, transactions or linearizability may be more appropriate. Systems that incorporate cooperative styles of distributed execution often replicate or distribute data within groups of components. In these cases, group oriented consistency properties must be maintained, and tools based on the virtual synchrony execution model greatly simplify the task confronting an application developer. All three styles of distributed computing are likely to be seen in future systems - often, within the same application. This leads us to propose an integrated approach that permits applications that use virtual synchrony with concurrent objects that respect a linearizability constraint, and vice versa. Transactional subsystems are treated as a special case of linearizability

Fault tolerant software technology for distributed computing system

Issued as Monthly reports [nos. 1-23], Interim technical report, Technical guide books [nos. 1-2], and Final report, Project no. G-36-64

Design and Implementation of a Distributed Middleware for Parallel Execution of Legacy Enterprise Applications

A typical enterprise uses a local area network of computers to perform its business. During the off-working hours, the computational capacities of these networked computers are underused or unused. In order to utilize this computational capacity an application has to be recoded to exploit concurrency inherent in a computation which is clearly not possible for legacy applications without any source code. This thesis presents the design an implementation of a distributed middleware which can automatically execute a legacy application on multiple networked computers by parallelizing it. This middleware runs multiple copies of the binary executable code in parallel on different hosts in the network. It wraps up the binary executable code of the legacy application in order to capture the kernel level data access system calls and perform them distributively over multiple computers in a safe and conflict free manner. The middleware also incorporates a dynamic scheduling technique to execute the target application in minimum time by scavenging the available CPU cycles of the hosts in the network. This dynamic scheduling also supports the CPU availability of the hosts to change over time and properly reschedule the replicas performing the computation to minimize the execution time. A prototype implementation of this middleware has been developed as a proof of concept of the design. This implementation has been evaluated with a few typical case studies and the test results confirm that the middleware works as expected

Deducing Operation Commutativity from Replicated Data Declaration

Distributed systems often resort to data replication not only to enhance their availability but also to reduce user-perceived latency by balancing the load between replicas and routing their requests accordingly. The choice of which consistency level that should be adopted by these replicated systems is critical for the fulfilment of their performance and correctness requirements. However, defining a strategy that strikes the right balance between these concerns in this type of environments is far from being a trivial task due to the related overheads that are amplified in distributed scenarios. Recognising the tension between latency and consistency, many systems allow multiple consistency levels to coexist. Nevertheless, the performance fine-tuning mechanisms supported by the existing hybrid solutions place a high burden on the programmer since the necessary input can be somehow complex requiring him to understand the semantics of each operation of the service he is developing in order to correctly instruct the system on how to handle concurrent updates. Thus, specifying operation dependencies, orderings and invariants to be preserved or even picking the right consistency level to be assigned to a certain data item is, generally, an error-prone task that hinders reasoning. To overcome this adversity, this work aims to reduce the effort spent by the programmer by only requiring the latter to introduce a simple and intuitive input at data declaration. Following this approach, reasoning is centralised and all accesses to replicated data are identified automatically. With all data accesses identified, it is then possible to deduce the side effects of each operation and determine, for each one of them, those with which it conflicts. In this context, this thesis also presents a compile-time analysis applied to the Java language able to evaluate operation pairwise commutativity from the input given at data declaration

A bunch of sessions:a propositions-as-sessions interpretation of bunched implications in channel-based concurrency

The emergence of propositions-as-sessions, a Curry-Howard correspondence between propositions of Linear Logic and session types for concurrent processes, has settled the logical foundations of message-passing concurrency. Central to this approach is the resource consumption paradigm heralded by Linear Logic. In this paper, we investigate a new point in the design space of session type systems for message-passing concurrent programs. We identify O’Hearn and Pym’s Logic of Bunched Implications (BI) as a fruitful basis for an interpretation of the logic as a concurrent programming language. This leads to a treatment of non-linear resources that is radically different from existing approaches based on Linear Logic. We introduce a new π-calculus with sessions, called πBI; its most salient feature is a construct called spawn, which expresses new forms of sharing that are induced by structural principles in BI. We illustrate the expressiveness of πBI and lay out its fundamental theory: type preservation, deadlock-freedom, and weak normalization results for well-typed processes; an operationally sound and complete typed encoding of an affine λ-calculus; and a non-interference result for access of resources

Fast Lean Erasure-Coded Atomic Memory Object

In this work, we propose FLECKS, an algorithm which implements atomic memory objects in a multi-writer multi-reader (MWMR) setting in asynchronous networks and server failures. FLECKS substantially reduces storage and communication costs over its replication-based counterparts by employing erasure-codes. FLECKS outperforms the previously proposed algorithms in terms of the metrics that to deliver good performance such as storage cost per object, communication cost a high fault-tolerance of clients and servers, guaranteed liveness of operation, and a given number of communication rounds per operation, etc. We provide proofs for liveness and atomicity properties of FLECKS and derive worst-case latency bounds for the operations. We implemented and deployed FLECKS in cloud-based clusters and demonstrate that FLECKS has substantially lower storage and bandwidth costs, and significantly lower latency of operations than the replication-based mechanisms

Chainspace: A Sharded Smart Contracts Platform

Chainspace is a decentralized infrastructure, known as a distributed ledger, that supports user defined smart contracts and executes user-supplied transactions on their objects. The correct execution of smart contract transactions is verifiable by all. The system is scalable, by sharding state and the execution of transactions, and using S-BAC, a distributed commit protocol, to guarantee consistency. Chainspace is secure against subsets of nodes trying to compromise its integrity or availability properties through Byzantine Fault Tolerance (BFT), and extremely high-auditability, non-repudiation and `blockchain' techniques. Even when BFT fails, auditing mechanisms are in place to trace malicious participants. We present the design, rationale, and details of Chainspace; we argue through evaluating an implementation of the system about its scaling and other features; we illustrate a number of privacy-friendly smart contracts for smart metering, polling and banking and measure their performance

Acquisition of computer research equipment

Issued as Final report, Project no. G-36-61

On computing serial dependency relations

In a database system where the data is a collection of shared objects there can be concurrent access to the data by several transactions. Traditional concurrency-controls use a notion of conflict between pairs of operations (of which the transactions are composed) to ensure correctness. Herlihy has proposed the notion of a serial dependency relation over the set of operations to capture this idea of conflict. Since the smaller the conflict relation the more the concurrency, it is of interest to construct minimal serial dependency relations. In this paper, we give necessary and sufficient conditions for a pair of operations to be related by a minimal serial dependency relation. However, we go on to show that, in general, the problem of constructing a minimal relation is undecidable. We also show that some approaches advocated for constructing serial dependency relations are not feasible in general. We then provide a sufficiency condition for cases where the minimal relations are computable