Optimistic replication

Data replication is a key technology in distributed data sharing systems, enabling higher availability and performance. This paper surveys optimistic replication algorithms that allow replica contents to diverge in the short term, in order to support concurrent work practices and to tolerate failures in low-quality communication links. The importance of such techniques is increasing as collaboration through wide-area and mobile networks becomes popular. Optimistic replication techniques are different from traditional “pessimistic ” ones. Instead of synchronous replica coordination, an optimistic algorithm propagates changes in the background, discovers conflicts after they happen and reaches agreement on the final contents incrementally. We explore the solution space for optimistic replication algorithms. This paper identifies key challenges facing optimistic replication systems — ordering operations, detecting and resolving conflicts, propagating changes efficiently, and bounding replica divergence — and provides a comprehensive survey of techniques developed for addressing these challenges

Eventual Consistency: Origin and Support

Eventual consistency is demanded nowadays in geo-replicated services that need to be highly scalable and available. According to the CAP constraints, when network partitions may arise, a distributed service should choose between being strongly consistent or being highly available. Since scalable services should be available, a relaxed consistency (while the network is partitioned) is the preferred choice. Eventual consistency is not a common data-centric consistency model, but only a state convergence condition to be added to a relaxed consistency model. There are still several aspects of eventual consistency that have not been analysed in depth in previous works: 1. which are the oldest replication proposals providing eventual consistency, 2. which replica consistency models provide the best basis for building eventually consistent services, 3. which mechanisms should be considered for implementing an eventually consistent service, and 4. which are the best combinations of those mechanisms for achieving different concrete goals. This paper provides some notes on these important topics

Performance characteristics of semantics-based concurrency control protocols.

by Keith, Hang-kwong Mak.Thesis (M.Phil.)--Chinese University of Hong Kong, 1995.Includes bibliographical references (leaves 122-127).Abstract --- p.iAcknowledgement --- p.iiiChapter 1 --- Introduction --- p.1Chapter 2 --- Background --- p.4Chapter 2.1 --- Read/Write Model --- p.4Chapter 2.2 --- Abstract Data Type Model --- p.5Chapter 2.3 --- Overview of Semantics-Based Concurrency Control Protocols --- p.7Chapter 2.4 --- Concurrency Hierarchy --- p.9Chapter 2.5 --- Control Flow of the Strict Two Phase Locking Protocol --- p.11Chapter 2.5.1 --- Flow of an Operation --- p.12Chapter 2.5.2 --- Response Time of a Transaction --- p.13Chapter 2.5.3 --- Factors Affecting the Response Time of a Transaction --- p.14Chapter 3 --- Semantics-Based Concurrency Control Protocols --- p.16Chapter 3.1 --- Strict Two Phase Locking --- p.16Chapter 3.2 --- Conflict Relations --- p.17Chapter 3.2.1 --- Commutativity (COMM) --- p.17Chapter 3.2.2 --- Forward and Right Backward Commutativity --- p.19Chapter 3.2.3 --- Exploiting Context-Specific Information --- p.21Chapter 3.2.4 --- Relaxing Correctness Criterion by Allowing Bounded Inconsistency --- p.26Chapter 4 --- Related Work --- p.32Chapter 4.1 --- Exploiting Transaction Semantics --- p.32Chapter 4.2 --- Exploting Object Semantics --- p.34Chapter 4.3 --- Sacrificing Consistency --- p.35Chapter 4.4 --- Other Approaches --- p.37Chapter 5 --- Performance Study (Testbed Approach) --- p.39Chapter 5.1 --- System Model --- p.39Chapter 5.1.1 --- Main Memory Database --- p.39Chapter 5.1.2 --- System Configuration --- p.40Chapter 5.1.3 --- Execution of Operations --- p.41Chapter 5.1.4 --- Recovery --- p.42Chapter 5.2 --- Parameter Settings and Performance Metrics --- p.43Chapter 6 --- Performance Results and Analysis (Testbed Approach) --- p.46Chapter 6.1 --- Read/Write Model vs. Abstract Data Type Model --- p.46Chapter 6.2 --- Using Context-Specific Information --- p.52Chapter 6.3 --- Role of Conflict Ratio --- p.55Chapter 6.4 --- Relaxing the Correctness Criterion --- p.58Chapter 6.4.1 --- Overhead and Performance Gain --- p.58Chapter 6.4.2 --- Range Queries using Bounded Inconsistency --- p.63Chapter 7 --- Performance Study (Simulation Approach) --- p.69Chapter 7.1 --- Simulation Model --- p.70Chapter 7.1.1 --- Logical Queueing Model --- p.70Chapter 7.1.2 --- Physical Queueing Model --- p.71Chapter 7.2 --- Experiment Information --- p.74Chapter 7.2.1 --- Parameter Settings --- p.74Chapter 7.2.2 --- Performance Metrics --- p.75Chapter 8 --- Performance Results and Analysis (Simulation Approach) --- p.76Chapter 8.1 --- Relaxing Correctness Criterion of Serial Executions --- p.77Chapter 8.1.1 --- Impact of Resource Contention --- p.77Chapter 8.1.2 --- Impact of Infinite Resources --- p.80Chapter 8.1.3 --- Impact of Limited Resources --- p.87Chapter 8.1.4 --- Impact of Multiple Resources --- p.89Chapter 8.1.5 --- Impact of Transaction Type --- p.95Chapter 8.1.6 --- Impact of Concurrency Control Overhead --- p.96Chapter 8.2 --- Exploiting Context-Specific Information --- p.98Chapter 8.2.1 --- Impact of Limited Resource --- p.98Chapter 8.2.2 --- Impact of Infinite and Multiple Resources --- p.101Chapter 8.2.3 --- Impact of Transaction Length --- p.106Chapter 8.2.4 --- Impact of Buffer Size --- p.108Chapter 8.2.5 --- Impact of Concurrency Control Overhead --- p.110Chapter 8.3 --- Summary and Discussion --- p.113Chapter 8.3.1 --- Summary of Results --- p.113Chapter 8.3.2 --- Relaxing Correctness Criterion vs. Exploiting Context-Specific In- formation --- p.114Chapter 9 --- Conclusions --- p.116Bibliography --- p.122Chapter A --- Commutativity Tables for Queue Objects --- p.128Chapter B --- Specification of a Queue Object --- p.129Chapter C --- Commutativity Tables with Bounded Inconsistency for Queue Objects --- p.132Chapter D --- Some Implementation Issues --- p.134Chapter D.1 --- Important Data Structures --- p.134Chapter D.2 --- Conflict Checking --- p.136Chapter D.3 --- Deadlock Detection --- p.137Chapter E --- Simulation Results --- p.139Chapter E.l --- Impact of Infinite Resources (Bounded Inconsistency) --- p.140Chapter E.2 --- Impact of Multiple Resource (Bounded Inconsistency) --- p.141Chapter E.3 --- Impact of Transaction Type (Bounded Inconsistency) --- p.142Chapter E.4 --- Impact of Concurrency Control Overhead (Bounded Inconsistency) --- p.144Chapter E.4.1 --- Infinite Resources --- p.144Chapter E.4.2 --- Limited Resource --- p.146Chapter E.5 --- Impact of Resource Levels (Exploiting Context-Specific Information) --- p.149Chapter E.6 --- Impact of Buffer Size (Exploiting Context-Specific Information) --- p.150Chapter E.7 --- Impact of Concurrency Control Overhead (Exploiting Context-Specific In- formation) --- p.155Chapter E.7.1 --- Impact of Infinite Resources --- p.155Chapter E.7.2 --- Impact of Limited Resources --- p.157Chapter E.7.3 --- Impact of Transaction Length --- p.160Chapter E.7.4 --- Role of Conflict Ratio --- p.16

Object replication in a distributed system

PhD ThesisA number of techniques have been proposed for the construction of fault—tolerant applications. One of these techniques is to replicate vital system resources so that if one copy fails sufficient copies may still remain operational to allow the application to continue to function. Interactions with replicated resources are inherently more complex than non—replicated interactions, and hence some form of replication transparency is necessary. This may be achieved by employing replica consistency protocols to mask replica failures and maintain consistency of state between functioning replicas. To achieve consistency between replicas it is necessary to ensure that all replicas receive the same set of messages in the same order, despite failures at the senders and receivers. This can be accomplished by making use of order preserving reliable communication protocols. However, we shall show how it can be more efficient to use unordered reliable communication and to impose ordering at the application level, by making use of syntactic knowledge of the application. This thesis develops techniques for replicating objects: in general this is harder than replicating data, as objects (which can contain data) can contain calls on other objects. Handling replicated objects is essentially the same as handling replicated computations, and presents more problems than simply replicating data. We shall use the concept of the object to provide transparent replication to users: a user will interact with only a single object interface which hides the fact that the object is actually replicated. The main aspects of the replication scheme presented in this thesis have been fully implemented and tested. This includes the design and implementation of a replicated object invocation protocol and the algorithms which ensure that (replicated) atomic actions can manipulate replicated objects.Research Studentship, Science and Engineering Research Council. Esprit Project 2267 (Integrated Systems Architecture)

Remove-Win: a Design Framework for Conflict-free Replicated Data Collections

Internet-scale distributed systems often replicate data within and across data centers to provide low latency and high availability despite node and network failures. Replicas are required to accept updates without coordination with each other, and the updates are then propagated asynchronously. This brings the issue of conflict resolution among concurrent updates, which is often challenging and error-prone. The Conflict-free Replicated Data Type (CRDT) framework provides a principled approach to address this challenge. This work focuses on a special type of CRDT, namely the Conflict-free Replicated Data Collection (CRDC), e.g. list and queue. The CRDC can have complex and compound data items, which are organized in structures of rich semantics. Complex CRDCs can greatly ease the development of upper-layer applications, but also makes the conflict resolution notoriously difficult. This explains why existing CRDC designs are tricky, and hard to be generalized to other data types. A design framework is in great need to guide the systematic design of new CRDCs. To address the challenges above, we propose the Remove-Win Design Framework. The remove-win strategy for conflict resolution is simple but powerful. The remove operation just wipes out the data item, no matter how complex the value is. The user of the CRDC only needs to specify conflict resolution for non-remove operations. This resolution is destructed to three basic cases and are left as open terms in the CRDC design skeleton. Stubs containing user-specified conflict resolution logics are plugged into the skeleton to obtain concrete CRDC designs. We demonstrate the effectiveness of our design framework via a case study of designing a conflict-free replicated priority queue. Performance measurements also show the efficiency of the design derived from our design framework.Comment: revised after submissio

New hardware support transactional memory and parallel debugging in multicore processors

This thesis contributes to the area of hardware support for parallel programming by introducing new hardware elements in multicore processors, with the aim of improving the performance and optimize new tools, abstractions and applications related with parallel programming, such as transactional memory and data race detectors. Specifically, we configure a hardware transactional memory system with signatures as part of the hardware support, and we develop a new hardware filter for reducing the signature size. We also develop the first hardware asymmetric data race detector (which is also able to tolerate them), based also in hardware signatures. Finally, we propose a new module of hardware signatures that solves some of the problems that we found in the previous tools related with the lack of flexibility in hardware signatures

Hybris: Robust Hybrid Cloud Storage

International audienceBesides well-known benefits, commodity cloud storage also raises concerns that include security, reliability, and consistency. We present Hybris key-value store, the first robust hybrid cloud storage system, aiming at addressing these concerns leveraging both private and public cloud resources. Hybris robustly replicates metadata on trusted private premises (private cloud), separately from data which is dispersed (using replication or erasure coding) across multiple untrusted public clouds. Hybris maintains metadata stored on private premises at the order of few dozens of bytes per key, avoiding the scalability bottleneck at the private cloud. In turn, the hybrid design allows Hybris to efficiently and robustly tolerate cloud outages, but also potential malice in clouds without overhead. Namely, to tolerate up to f malicious clouds, in the common case of the Hybris variant with data replication, writes replicate data across f + 1 clouds, whereas reads involve a single cloud. In the worst case, only up to f additional clouds are used. This is considerably better than earlier multi-cloud storage systems that required costly 3f + 1 clouds to mask f potentially malicious clouds. Finally, Hybris leverages strong metadata consistency to guarantee to Hybris applications strong data consistency without any modifications to the eventually consistent public clouds. We implemented Hybris in Java and evaluated it using a series of micro and macro-benchmarks. Our results show that Hybris significantly outperforms comparable multi-cloud storage systems and approaches the performance of bare-bone commodity public cloud storage