2,068 research outputs found
Recommended from our members
Improving DBMS performance through diverse redundancy
Database replication is widely used to improve both fault tolerance and DBMS performance. Non-diverse database replication has a significant limitation - it is effective against crash failures only. Diverse redundancy is an effective mechanism of tolerating a wider range of failures, including many non-crash failures. However it has not been adopted in practice because many see DBMS performance as the main concern. In this paper we show experimental evidence that diverse redundancy (diverse replication) can bring benefits in terms of DBMS performance, too. We report on experimental results with an optimistic architecture built with two diverse DBMSs under a load derived from TPC-C benchmark, which show that a diverse pair performs faster not only than non-diverse pairs but also than the individual copies of the DBMSs used. This result is important because it shows potential for DBMS performance better than anything achievable with the available off-the-shelf servers
Optimistic total order in wide area networks
Total order multicast greatly simplifies the implementa- tion of fault-tolerant services using the replicated state ma- chine approach. The additional latency of total ordering can be masked by taking advantage of spontaneous order- ing observed in LANs: A tentative delivery allows the ap- plication to proceed in parallel with the ordering protocol. The effectiveness of the technique rests on the optimistic as- sumption that a large share of correctly ordered tentative deliveries offsets the cost of undoing the effect of mistakes. This paper proposes a simple technique which enables the usage of optimistic delivery also in WANs with much larger transmission delays where the optimistic assumption does not normally hold. Our proposal exploits local clocks and the stability of network delays to reduce the mistakes in the ordering of tentative deliveries. An experimental evalu- ation of a modified sequencer-based protocol is presented, illustrating the usefulness of the approach in fault-tolerant database management
Recommended from our members
Fault diversity among off-the-shelf SQL database servers
Fault tolerance is often the only viable way of obtaining the required system dependability from systems built out of "off-the-shelf" (OTS) products. We have studied a sample of bug reports from four off-the-shelf SQL servers so as to estimate the possible advantages of software fault tolerance - in the form of modular redundancy with diversity - in complex off-the-shelf software. We checked whether these bugs would cause coincident failures in more than one of the servers. We found that very few bugs affected two of the four servers, and none caused failures in more than two. We also found that only four of these bugs would cause identical, undetectable failures in two servers. Therefore, a fault-tolerant server, built with diverse off-the-shelf servers, seems to have a good chance of delivering improvements in availability and failure rates compared with the individual off-the-shelf servers or their replicated, nondiverse configurations
Speculation in Parallel and Distributed Event Processing Systems
Event stream processing (ESP) applications enable the real-time processing of continuous flows of data. Algorithmic trading, network monitoring, and processing data from sensor networks are good examples of applications that traditionally rely upon ESP systems. In addition, technological advances are resulting in an increasing number of devices that are network enabled, producing information that can be automatically collected and processed. This increasing availability of on-line data motivates the development of new and more sophisticated applications that require low-latency processing of large volumes of data.
ESP applications are composed of an acyclic graph of operators that is traversed by the data. Inside each operator, the events can be transformed, aggregated, enriched, or filtered out. Some of these operations depend only on the current input events, such operations are called stateless. Other operations, however, depend not only on the current event, but also on a state built during the processing of previous events. Such operations are, therefore, named stateful.
As the number of ESP applications grows, there are increasingly strong requirements, which are often difficult to satisfy. In this dissertation, we address two challenges created by the use of stateful operations in a ESP application: (i) stateful operators can be bottlenecks because they are sensitive to the order of events and cannot be trivially parallelized by replication; and (ii), if failures are to be tolerated, the accumulated state of an stateful operator needs to be saved, saving this state traditionally imposes considerable performance costs.
Our approach is to evaluate the use of speculation to address these two issues. For handling ordering and parallelization issues in a stateful operator, we propose a speculative approach that both reduces latency when the operator must wait for the correct ordering of the events and improves throughput when the operation in hand is parallelizable. In addition, our approach does not require that user understand concurrent programming or that he or she needs to consider out-of-order execution when writing the operations.
For fault-tolerant applications, traditional approaches have imposed prohibitive performance costs due to pessimistic schemes. We extend such approaches, using speculation to mask the cost of fault tolerance.:1 Introduction 1
1.1 Event stream processing systems ......................... 1
1.2 Running example ................................. 3
1.3 Challenges and contributions ........................... 4
1.4 Outline ...................................... 6
2 Background 7
2.1 Event stream processing ............................. 7
2.1.1 State in operators: Windows and synopses ............................ 8
2.1.2 Types of operators ............................ 12
2.1.3 Our prototype system........................... 13
2.2 Software transactional memory.......................... 18
2.2.1 Overview ................................. 18
2.2.2 Memory operations............................ 19
2.3 Fault tolerance in distributed systems ...................................... 23
2.3.1 Failure model and failure detection ...................................... 23
2.3.2 Recovery semantics............................ 24
2.3.3 Active and passive replication ...................... 24
2.4 Summary ..................................... 26
3 Extending event stream processing systems with speculation 27
3.1 Motivation..................................... 27
3.2 Goals ....................................... 28
3.3 Local versus distributed speculation ....................... 29
3.4 Models and assumptions ............................. 29
3.4.1 Operators................................. 30
3.4.2 Events................................... 30
3.4.3 Failures .................................. 31
4 Local speculation 33
4.1 Overview ..................................... 33
4.2 Requirements ................................... 35
4.2.1 Order ................................... 35
4.2.2 Aborts................................... 37
4.2.3 Optimism control ............................. 38
4.2.4 Notifications ............................... 39
4.3 Applications.................................... 40
4.3.1 Out-of-order processing ......................... 40
4.3.2 Optimistic parallelization......................... 42
4.4 Extensions..................................... 44
4.4.1 Avoiding unnecessary aborts ....................... 44
4.4.2 Making aborts unnecessary........................ 45
4.5 Evaluation..................................... 47
4.5.1 Overhead of speculation ......................... 47
4.5.2 Cost of misspeculation .......................... 50
4.5.3 Out-of-order and parallel processing micro benchmarks ........... 53
4.5.4 Behavior with example operators .................... 57
4.6 Summary ..................................... 60
5 Distributed speculation 63
5.1 Overview ..................................... 63
5.2 Requirements ................................... 64
5.2.1 Speculative events ............................ 64
5.2.2 Speculative accesses ........................... 69
5.2.3 Reliable ordered broadcast with optimistic delivery .................. 72
5.3 Applications .................................... 75
5.3.1 Passive replication and rollback recovery ................................ 75
5.3.2 Active replication ............................. 80
5.4 Extensions ..................................... 82
5.4.1 Active replication and software bugs ..................................... 82
5.4.2 Enabling operators to output multiple events ........................ 87
5.5 Evaluation .................................... 87
5.5.1 Passive replication ............................ 88
5.5.2 Active replication ............................. 88
5.6 Summary ..................................... 93
6 Related work 95
6.1 Event stream processing engines ......................... 95
6.2 Parallelization and optimistic computing ................................ 97
6.2.1 Speculation ................................ 97
6.2.2 Optimistic parallelization ......................... 98
6.2.3 Parallelization in event processing .................................... 99
6.2.4 Speculation in event processing ..................... 99
6.3 Fault tolerance .................................. 100
6.3.1 Passive replication and rollback recovery ............................... 100
6.3.2 Active replication ............................ 101
6.3.3 Fault tolerance in event stream processing systems ............. 103
7 Conclusions 105
7.1 Summary of contributions ............................ 105
7.2 Challenges and future work ............................ 106
Appendices
Publications 107
Pseudocode for the consensus protocol 10
CRDTs: Consistency without concurrency control
A CRDT is a data type whose operations commute when they are concurrent.
Replicas of a CRDT eventually converge without any complex concurrency control.
As an existence proof, we exhibit a non-trivial CRDT: a shared edit buffer
called Treedoc. We outline the design, implementation and performance of
Treedoc. We discuss how the CRDT concept can be generalised, and its
limitations
- …