55 research outputs found

    Designing the replication layer of a general-purpose datacenter key-value store

    Get PDF
    Online services and cloud applications such as graph applications, messaging systems, coordination services, HPC applications, social networks and deep learning rely on key-value stores (KVSes), in order to reliably store and quickly retrieve data. KVSes are NoSQL Databases with a read/write/read-modify-write API. KVSes replicate their dataset in a few servers, such that the KVS can continue operating in the presence of faults (availability). To allow programmers to reason about replication, KVSes specify a set of rules (consistency), which are enforced through the use of replication protocols. These rules must be intuitive to facilitate programmer productivity (programmability). A general-purpose KVS must maximize the number of operations executed per unit of time within a predetermined latency (performance) without compromising on consistency, availability or programmability. However, all three of these guarantees are at odds with performance. In this thesis, we explore the design of the replication layer of a general-purpose KVS, which is responsible for navigating this trade-off, by specifying and enforcing the consistency and availability guarantees of the KVS. We start the exploration by observing that modern, server-grade hardware with manycore servers and RDMA-capable networks, challenges conventional wisdom in protocol design. In order to investigate the impact of these advances on protocols and their design, we first create an informal taxonomy of strongly-consistent replication protocols. We focus on strong consistency semantics because they are necessary for a general-purpose KVS and they are at odds with performance. Based on this taxonomy we carefully select 10 protocols for analysis. Secondly, we present Odyssey, a frame-work tailored towards protocol implementation for multi-threaded, RDMA-enabled, in-memory, replicated KVSes. Using Odyssey, we characterize the design space of strongly-consistent replication protocols, by building, evaluating and comparing the 10 protocols. Our evaluation demonstrates that some of the protocols that were efficient in yesterday’s hardware are not so today because they cannot take advantage of the abundant parallelism and fast networking present in modern hardware. Conversely, some protocols that were inefficient in yesterday’s hardware are very attractive today. We distil our findings in a concise set of general guidelines and recommendations for protocol selection and design in the era of modern hardware. The second step of our exploration focuses on the tension between consistency and performance. The problem is that expensive strongly-consistent primitives are necessary to achieve synchronization, but in typical applications only a small fraction of accesses is actually used for synchronization. To navigate this trade-off, we advocate the adoption of Release Consistency (RC) for KVSes. We argue that RC’s one-sided barriers are ideal for capturing the ordering relationship between synchronization and non-synchronization accesses while enabling high performance. We present Kite, a general-purpose, replicated KVS that enforces RC through a novel fast/slow path mechanism that leverages the absence of failures in the typical case to maximize performance, while relying on the slow path for progress. In ad dition, Kite leverages our study of replication protocols to select the most suitable protocols for its primitives and is implemented over Odyssey to make the most out of modern hardware. Finally, Kite does not compromise on consistency, availability or programmability, as it provides sufficient primitives to implement any algorithm (consistency), does not interrupt its operation on a failure (availability), and offers the RC API that programmers are already familiar with (programmability)

    Path-Sensitive Atomic Commit: Local Coordination Avoidance for Distributed Transactions

    Get PDF
    Context: Concurrent objects with asynchronous messaging are an increasingly popular way to struc-ture highly available, high performance, large-scale software systems. To ensure data-consistency and sup-port synchronization between objects such systems often use distributed transactions with Two-Phase Locking(2pl) for concurrency control and Two-Phase commit (2pc ) as atomic commitment protocol. Inquiry: In highly available, high-throughput systems, such as large banking infrastructure, however,2plbecomes a bottleneck when objects are highly contended, when an object is queuing a lot of messages becauseof locking. <br?. Approach: In this paper we introduce Path-Sensitive Atomic Commit (psac) to address this situation. Westart from message handlers (or methods), which are decorated with pre- and post-conditions, describingtheir guards and effect. Knowledge: This allows thepsaclock mechanism to check whether the effect of two incoming messages atthe same time are independent, and to avoid locking if this is the case. As a result, more messages are directlyaccepted or rejected, and higher overall throughput is obtained. Grounding: We have implementedpsacfor a state machine-based DSL called Rebel, on top of a runtimebased on the Akka actor framework. Our performance evaluation shows thatpsacexhibits the same scalabilityand latency characteristics as standard2pl/2pc, and obtains up to 1.8 times median higher throughput incongested scenarios. Importance: We believepsacis a step towards enabling organizations to build scalable distributed applica-tions, even if their consistency requirements are not embarrassingly paralle

    A Correctness Criterion for Eager Approach Validation for Transactional Memory System

    Get PDF
    With rise of multicore systems, software transactional memory (STM) has garnered significant interest as an elegant alternative for developing concurrent code. A (memory) transaction is an unit of code in execution in memory. A software transactional memory system (STM) ensures that a transaction appears either to execute atomically (even in presence of other concurrent transactions) or to never have executed at all. To achieve this property, a commonly used approach by STM systems is eager validation approach. In this approach, when a transaction performs a write operation the update is immediately written to the memory and is visible to other transactions. If the transaction aborts, then the writes performed by it previously are undone with the aid of undo logs. In this paper, we have presented a new correctness criterion EAC for eager approach validation systems. We have developed a STM system that implements this criterion. The STM system uses database recovery criterion strictness and the notion of conflict graphs used for testing conflict serializability

    A Framework for Model-Driven Development of Mobile Applications with Context Support

    Get PDF
    Model-driven development (MDD) of software systems has been a serious trend in different application domains over the last 15 years. While technologies, platforms, and architectural paradigms have changed several times since model-driven development processes were first introduced, their applicability and usefulness are discussed every time a new technological trend appears. Looking at the rapid market penetration of smartphones, software engineers are curious about how model-driven development technologies can deal with this novel and emergent domain of software engineering (SE). Indeed, software engineering of mobile applications provides many challenges that model-driven development can address. Model-driven development uses a platform independent model as a crucial artifact. Such a model usually follows a domain-specific modeling language and separates the business concerns from the technical concerns. These platform-independent models can be reused for generating native program code for several mobile software platforms. However, a major drawback of model-driven development is that infrastructure developers must provide a fairly sophisticated model-driven development infrastructure before mobile application developers can create mobile applications in a model-driven way. Hence, the first part of this thesis deals with designing a model-driven development infrastructure for mobile applications. We will follow a rigorous design process comprising a domain analysis, the design of a domain-specific modeling language, and the development of the corresponding model editors. To ensure that the code generators produce high-quality application code and the resulting mobile applications follow a proper architectural design, we will analyze several representative reference applications beforehand. Thus, the reader will get an insight into both the features of mobile applications and the steps that are required to design and implement a model-driven development infrastructure. As a result of the domain analysis and the analysis of the reference applications, we identified context-awareness as a further important feature of mobile applications. Current software engineering tools do not sufficiently support designing and implementing of context-aware mobile applications. Although these tools (e.g., middleware approaches) support the definition and the collection of contextual information, the adaptation of the mobile application must often be implemented by hand at a low abstraction level by the mobile application developers. Thus, the second part of this thesis demonstrates how context-aware mobile applications can be designed more easily by using a model-driven development approach. Techniques such as model transformation and model interpretation are used to adapt mobile applications to different contexts at design time or runtime. Moreover, model analysis and model-based simulation help mobile application developers to evaluate a designed mobile application (i.e., app model) prior to its generation and deployment with respected to certain contexts. We demonstrate the usefulness and applicability of the model-driven development infrastructure we developed by seven case examples. These showcases demonstrate the designing of mobile applications in different domains. We demonstrate the scalability of our model-driven development infrastructure with several performance tests, focusing on the generation time of mobile applications, as well as their runtime performance. Moreover, the usability was successfully evaluated during several hands-on training sessions by real mobile application developers with different skill levels
    • …
    corecore