17,904 research outputs found
Middleware-based Database Replication: The Gaps between Theory and Practice
The need for high availability and performance in data management systems has
been fueling a long running interest in database replication from both academia
and industry. However, academic groups often attack replication problems in
isolation, overlooking the need for completeness in their solutions, while
commercial teams take a holistic approach that often misses opportunities for
fundamental innovation. This has created over time a gap between academic
research and industrial practice.
This paper aims to characterize the gap along three axes: performance,
availability, and administration. We build on our own experience developing and
deploying replication systems in commercial and academic settings, as well as
on a large body of prior related work. We sift through representative examples
from the last decade of open-source, academic, and commercial database
replication systems and combine this material with case studies from real
systems deployed at Fortune 500 customers. We propose two agendas, one for
academic research and one for industrial R&D, which we believe can bridge the
gap within 5-10 years. This way, we hope to both motivate and help researchers
in making the theory and practice of middleware-based database replication more
relevant to each other.Comment: 14 pages. Appears in Proc. ACM SIGMOD International Conference on
Management of Data, Vancouver, Canada, June 200
Update Consistency for Wait-free Concurrent Objects
In large scale systems such as the Internet, replicating data is an essential
feature in order to provide availability and fault-tolerance. Attiya and Welch
proved that using strong consistency criteria such as atomicity is costly as
each operation may need an execution time linear with the latency of the
communication network. Weaker consistency criteria like causal consistency and
PRAM consistency do not ensure convergence. The different replicas are not
guaranteed to converge towards a unique state. Eventual consistency guarantees
that all replicas eventually converge when the participants stop updating.
However, it fails to fully specify the semantics of the operations on shared
objects and requires additional non-intuitive and error-prone distributed
specification techniques. This paper introduces and formalizes a new
consistency criterion, called update consistency, that requires the state of a
replicated object to be consistent with a linearization of all the updates. In
other words, whereas atomicity imposes a linearization of all of the
operations, this criterion imposes this only on updates. Consequently some read
operations may return out-dated values. Update consistency is stronger than
eventual consistency, so we can replace eventually consistent objects with
update consistent ones in any program. Finally, we prove that update
consistency is universal, in the sense that any object can be implemented under
this criterion in a distributed system where any number of nodes may crash.Comment: appears in International Parallel and Distributed Processing
Symposium, May 2015, Hyderabad, Indi
On Verifying Causal Consistency
Causal consistency is one of the most adopted consistency criteria for
distributed implementations of data structures. It ensures that operations are
executed at all sites according to their causal precedence. We address the
issue of verifying automatically whether the executions of an implementation of
a data structure are causally consistent. We consider two problems: (1)
checking whether one single execution is causally consistent, which is relevant
for developing testing and bug finding algorithms, and (2) verifying whether
all the executions of an implementation are causally consistent.
We show that the first problem is NP-complete. This holds even for the
read-write memory abstraction, which is a building block of many modern
distributed systems. Indeed, such systems often store data in key-value stores,
which are instances of the read-write memory abstraction. Moreover, we prove
that, surprisingly, the second problem is undecidable, and again this holds
even for the read-write memory abstraction. However, we show that for the
read-write memory abstraction, these negative results can be circumvented if
the implementations are data independent, i.e., their behaviors do not depend
on the data values that are written or read at each moment, which is a
realistic assumption.Comment: extended version of POPL 201
An efficient MPI/OpenMP parallelization of the Hartree-Fock method for the second generation of Intel Xeon Phi processor
Modern OpenMP threading techniques are used to convert the MPI-only
Hartree-Fock code in the GAMESS program to a hybrid MPI/OpenMP algorithm. Two
separate implementations that differ by the sharing or replication of key data
structures among threads are considered, density and Fock matrices. All
implementations are benchmarked on a super-computer of 3,000 Intel Xeon Phi
processors. With 64 cores per processor, scaling numbers are reported on up to
192,000 cores. The hybrid MPI/OpenMP implementation reduces the memory
footprint by approximately 200 times compared to the legacy code. The
MPI/OpenMP code was shown to run up to six times faster than the original for a
range of molecular system sizes.Comment: SC17 conference paper, 12 pages, 7 figure
Monotonic Prefix Consistency in Distributed Systems
We study the issue of data consistency in distributed systems. Specifically,
we consider a distributed system that replicates its data at multiple sites,
which is prone to partitions, and which is assumed to be available (in the
sense that queries are always eventually answered). In such a setting, strong
consistency, where all replicas of the system apply synchronously every
operation, is not possible to implement. However, many weaker consistency
criteria that allow a greater number of behaviors than strong consistency, are
implementable in available distributed systems. We focus on determining the
strongest consistency criterion that can be implemented in a convergent and
available distributed system that tolerates partitions. We focus on objects
where the set of operations can be split into updates and queries. We show that
no criterion stronger than Monotonic Prefix Consistency (MPC) can be
implemented.Comment: Submitted pape
Reconfigurable Lattice Agreement and Applications
Reconfiguration is one of the central mechanisms in distributed systems. Due to failures and connectivity disruptions, the very set of service replicas (or servers) and their roles in the computation may have to be reconfigured over time. To provide the desired level of consistency and availability to applications running on top of these servers, the clients of the service should be able to reach some form of agreement on the system configuration. We observe that this agreement is naturally captured via a lattice partial order on the system states. We propose an asynchronous implementation of reconfigurable lattice agreement that implies elegant reconfigurable versions of a large class of lattice abstract data types, such as max-registers and conflict detectors, as well as popular distributed programming abstractions, such as atomic snapshot and commit-adopt
On the Behaviour of General-Purpose Applications on Cloud Storages
Managing data over cloud infrastructures raises novel challenges with respect to existing and well studied approaches such as ACID and long running transactions. One of the main requirements is to provide availability and partition tolerance in a scenario with replicas and distributed control. This comes at the price of a weaker consistency, usually called eventual consistency. These weak memory models have proved to be suitable in a number of scenarios, such as the analysis of large data with Map-Reduce. However, due to the widespread availability of cloud infrastructures, weak storages are used not only by specialised applications but also by general purpose applications. We provide a formal approach, based on process calculi, to reason about the behaviour of programs that rely on cloud stores. For instance, one can check that the composition of a process with a cloud store ensures `strong' properties through a wise usage of asynchronous message-passing
- …