4,912 research outputs found
Middleware-based Database Replication: The Gaps between Theory and Practice
The need for high availability and performance in data management systems has
been fueling a long running interest in database replication from both academia
and industry. However, academic groups often attack replication problems in
isolation, overlooking the need for completeness in their solutions, while
commercial teams take a holistic approach that often misses opportunities for
fundamental innovation. This has created over time a gap between academic
research and industrial practice.
This paper aims to characterize the gap along three axes: performance,
availability, and administration. We build on our own experience developing and
deploying replication systems in commercial and academic settings, as well as
on a large body of prior related work. We sift through representative examples
from the last decade of open-source, academic, and commercial database
replication systems and combine this material with case studies from real
systems deployed at Fortune 500 customers. We propose two agendas, one for
academic research and one for industrial R&D, which we believe can bridge the
gap within 5-10 years. This way, we hope to both motivate and help researchers
in making the theory and practice of middleware-based database replication more
relevant to each other.Comment: 14 pages. Appears in Proc. ACM SIGMOD International Conference on
Management of Data, Vancouver, Canada, June 200
A Taxonomy of Data Grids for Distributed Data Sharing, Management and Processing
Data Grids have been adopted as the platform for scientific communities that
need to share, access, transport, process and manage large data collections
distributed worldwide. They combine high-end computing technologies with
high-performance networking and wide-area storage management techniques. In
this paper, we discuss the key concepts behind Data Grids and compare them with
other data sharing and distribution paradigms such as content delivery
networks, peer-to-peer networks and distributed databases. We then provide
comprehensive taxonomies that cover various aspects of architecture, data
transportation, data replication and resource allocation and scheduling.
Finally, we map the proposed taxonomy to various Data Grid systems not only to
validate the taxonomy but also to identify areas for future exploration.
Through this taxonomy, we aim to categorise existing systems to better
understand their goals and their methodology. This would help evaluate their
applicability for solving similar problems. This taxonomy also provides a "gap
analysis" of this area through which researchers can potentially identify new
issues for investigation. Finally, we hope that the proposed taxonomy and
mapping also helps to provide an easy way for new practitioners to understand
this complex area of research.Comment: 46 pages, 16 figures, Technical Repor
HT-Paxos: High Throughput State-Machine Replication Protocol for Large Clustered Data Centers
Paxos is a prominent theory of state machine replication. Recent data
intensive Systems those implement state machine replication generally require
high throughput. Earlier versions of Paxos as few of them are classical Paxos,
fast Paxos and generalized Paxos have a major focus on fault tolerance and
latency but lacking in terms of throughput and scalability. A major reason for
this is the heavyweight leader. Through offloading the leader, we can further
increase throughput of the system. Ring Paxos, Multi Ring Paxos and S-Paxos are
few prominent attempts in this direction for clustered data centers. In this
paper, we are proposing HT-Paxos, a variant of Paxos that one is the best
suitable for any large clustered data center. HT-Paxos further offloads the
leader very significantly and hence increases the throughput and scalability of
the system. While at the same time, among high throughput state-machine
replication protocols, HT-Paxos provides reasonably low latency and response
time
Deceit: A flexible distributed file system
Deceit, a distributed file system (DFS) being developed at Cornell, focuses on flexible file semantics in relation to efficiency, scalability, and reliability. Deceit servers are interchangeable and collectively provide the illusion of a single, large server machine to any clients of the Deceit service. Non-volatile replicas of each file are stored on a subset of the file servers. The user is able to set parameters on a file to achieve different levels of availability, performance, and one-copy serializability. Deceit also supports a file version control mechanism. In contrast with many recent DFS efforts, Deceit can behave like a plain Sun Network File System (NFS) server and can be used by any NFS client without modifying any client software. The current Deceit prototype uses the ISIS Distributed Programming Environment for all communication and process group management, an approach that reduces system complexity and increases system robustness
Object replication in a distributed system
PhD ThesisA number of techniques have been proposed for the construction of fault—tolerant
applications. One of these techniques is to replicate vital system resources so that if one
copy fails sufficient copies may still remain operational to allow the application to
continue to function. Interactions with replicated resources are inherently more complex
than non—replicated interactions, and hence some form of replication transparency is
necessary. This may be achieved by employing replica consistency protocols to mask replica
failures and maintain consistency of state between functioning replicas.
To achieve consistency between replicas it is necessary to ensure that all replicas
receive the same set of messages in the same order, despite failures at the senders and
receivers. This can be accomplished by making use of order preserving reliable
communication protocols. However, we shall show how it can be more efficient to use
unordered reliable communication and to impose ordering at the application level, by
making use of syntactic knowledge of the application.
This thesis develops techniques for replicating objects: in general this is harder than
replicating data, as objects (which can contain data) can contain calls on other objects.
Handling replicated objects is essentially the same as handling replicated computations,
and presents more problems than simply replicating data. We shall use the concept of the
object to provide transparent replication to users: a user will interact with only a single
object interface which hides the fact that the object is actually replicated.
The main aspects of the replication scheme presented in this thesis have been fully
implemented and tested. This includes the design and implementation of a replicated
object invocation protocol and the algorithms which ensure that (replicated) atomic
actions can manipulate replicated objects.Research Studentship, Science and Engineering Research Council.
Esprit Project 2267 (Integrated Systems Architecture)
QuAD: A Quorum Protocol for Adaptive Data Management in the Cloud
More and more companies move their data to the Cloud which is able to cope with the high scalability and availability demands due to its pay-as-you-go cost model. For this, databases in the Cloud are distributed and replicated across different data centers. According to the CAP theorem, distributed data management is governed by a trade-off between consistency and availability. In addition, the stronger the provided consistency level, the higher is the generated coordination overhead and thus the impact on system performance. Nevertheless, many OLTP applications demand strong consistency and use ROWA(A) for replica synchronization. ROWA(A) protocols eagerly update all (or all available) replicas and thus generate a high overhead for update transactions. In contrast, quorum-based protocols consider only a subset of sites for eager commit. This reduces the overhead for update transactions at the cost of reads, as the latter also need to access several sites. Existing quorum-based protocols do not consider the load of sites when determining the quorums; hence, they are not able to adapt at run-time to load changes. In this paper, we present QuAD, an adaptive quorum-based replication protocol that constructs quorums by dynamically selecting the optimal quorum configuration w.r.t. load and network latency. Our evaluation of QuAD based on Amazon EC2 shows that it considerably outperforms both static quorum protocols and dynamic protocols that neglect site properties in the quorum construction process
Supporting disconnected operations in mobile computing
Mobile computing has enabled users to seamlessly access databases even when they are on the move. However, in the absence of readily available high-quality communication, users are often forced to operate disconnected from the network. As a result, software applications have to be redesigned to take advantage of this environment while accommodating the new challenges posed by mobility. In particular, there is a need for replication and synchronization services in order to guarantee availability of data and functionality, (including updates) in disconnected mode. To this end we propose a scalable and highly available data replication and management service. The proposed replication technique is compared with a baseline replication technique and shown to exhibit high availability, fault tolerance and minimal access times of the data and services, which are very important in an environment with low-quality communication links.<br /
- …