Practical cross-engine transactions in dual-engine database systems

With the growing DRAM capacity and core count in modern servers, database systems are becoming increasingly multi-engine to feature a heterogeneous set of engines. In particular, a memory-optimized engine and a conventional storage-centric engine may coexist to satisfy various application needs. However, handling cross-engine transactions that access more than one engine remains challenging in terms of correctness, performance and programmability. This thesis describes Skeena, an approach to cross-engine transactions with proper isolation guarantees and low overhead. Skeena adapts and integrates past concurrency control theory to provide a complete solution to supporting various isolation levels in dual-engine systems, and proposes a lightweight transaction tracking structure that captures the necessary information to guarantee correctness with low overhead. Evaluation on a 40-core server shows that Skeena only incurs minuscule overhead for cross-engine transactions, without penalizing single-engine transactions

Bringing modular concurrency control to the next level

Database users face a tension between ease-of-programming and high performance: ACID transactions can greatly simplify the programming effort of database applications by providing four useful properties—atomicity, consistency, isolation, and durability, but enforcing these properties can degrade performance. This dissertation eases this tension by improving the performance of ACID transactions for scenarios where data contention is the bottleneck. The approach that we take is federating concurrency control (CC) mechanisms. It is based on the observation that any single CC mechanism is bound to make trade-offs that cause it to perform well in some cases but poorly in others. A federation opens the opportunity of applying each mechanism only to the set of transactions or workloads where it shines, while maintaining isolation. In particular, this work builds upon Modular Concurrency Control (MCC), a recent technique that federates CCs by partitioning transactions into groups, and by applying different CC mechanisms in each group. This dissertation addresses two critical shortcomings in the current embodiment of MCC. First, cross-group data conflicts are handled with a single, unoptimized CC mechanism that can significantly limit performance. Second, configuring MCC is a complex task, which runs counter to MCC’s purpose: to improve performance without sacrificing ease-of-programming. To address these problems, this dissertation presents Tebaldi, a new transactional database that brings Modular Concurrency Control to the next level, both figuratively and literally. Tebaldi introduces a new, hierarchical model to MCC that partitions transactions recursively to compose CC mechanisms in a multi-level tree. This model increases flexibility in federating CC mechanisms, which is the key to realizing the performance potential of federation. Tebaldi reduces configuration complexity by managing the MCC federation automatically: it can detect performance issues in the current workload in real-time, and automatically adjusts its configuration to improve its performance.Computer Science

06121 Abstracts Collection -- Atomicity: A Unifying Concept in Computer Science

From 19.03.06 to 24.03.06, the Dagstuhl Seminar 06121 ``Atomicity: A Unifying Concept in Computer Science\u27\u27 was held in the International Conference and Research Center (IBFI), Schloss Dagstuhl. During the seminar, several participants presented their current research, and ongoing work and open problems were discussed. Abstracts of the presentations given during the seminar as well as abstracts of seminar results and ideas are put together in this paper. The first section describes the seminar topics and goals in general. Links to extended abstracts or full papers are provided, if available

The End of Slow Networks: It's Time for a Redesign

Next generation high-performance RDMA-capable networks will require a fundamental rethinking of the design and architecture of modern distributed DBMSs. These systems are commonly designed and optimized under the assumption that the network is the bottleneck: the network is slow and "thin", and thus needs to be avoided as much as possible. Yet this assumption no longer holds true. With InfiniBand FDR 4x, the bandwidth available to transfer data across network is in the same ballpark as the bandwidth of one memory channel, and it increases even further with the most recent EDR standard. Moreover, with the increasing advances of RDMA, the latency improves similarly fast. In this paper, we first argue that the "old" distributed database design is not capable of taking full advantage of the network. Second, we propose architectural redesigns for OLTP, OLAP and advanced analytical frameworks to take better advantage of the improved bandwidth, latency and RDMA capabilities. Finally, for each of the workload categories, we show that remarkable performance improvements can be achieved

Multi-Master Replication for Snapshot Isolation Databases

Lazy replication with snapshot isolation (SI) has emerged as a popular choice for distributed databases. However, lazy replication requires the execution of update transactions at one (master) site so that it is relatively easy for a total SI order to be determined for consistent installation of updates in the lazily replicated system. We propose a set of techniques that support update transaction execution over multiple partitioned sites, thereby allowing the master to scale. Our techniques determine a total SI order for update transactions over multiple master sites without requiring global coordination in the distributed system, and ensure that updates are installed in this order at all sites to provide consistent and scalable replication with SI. We have built our techniques into PostgreSQL and demonstrate their effectiveness through experimental evaluation.1 yea

User's and Administrator's Manual of AMGA Metadata Catalog v 2.4.0 (EMI-3)

User's and Administrator's Manual of AMGA Metadata Catalog v 2.4.0 (EMI-3

Transaktionen in föderierten Datenbanksystemen unter eingeschränkten Isolation Levels

Atomarität und Isolation von Transaktionen sind Schlüsseleigenschaften fortgeschrittener Anwendungen in föderierten Systemen, die aus verteilten, heterogenen Komponenten bestehen. Während Atomarität von praktisch allen realen Systemen durch das Zweiphasen- Commitprotokoll gewährleistet wird, unterstützt kein System eine explizite föderierte Concurrency Control. In der Literatur wurden zwar zahlreiche Lösungsansätze vorgeschlagen, doch sie haben wenig Einfluss auf Produkte genommen, weil sie die weitverbreiteten Isolation Levels nicht berücksichtigen, die Applikationen Optimierungsmöglichkeiten auf Kosten einer eingeschränkten Kontrolle über die Konsistenz der Daten erlauben. Diese Arbeit vergleicht zunächst existierende Definitionen für Isolation Levels und entwickelt eine neuartige, formale Charakterisierung für Snapshot Isolation, dem Isolation Level des Marktführers Oracle. Anschließend werden Algorithmen zur föderierten Concurrency Control vorgestellt, die beweisbar auch unter lokaler Snapshot Isolation die korrekte Ausführung föderierter Transaktionen gewährleisten, und Isolation Levels für föderierte Transaktionen diskutiert. Die Algorithmen sind in ein prototypisches föderiertes System integriert. Performancemessungen an diesem Prototyp zeigen ihre praktische Einsetzbarkeit.Atomicity and isolation of transactions are key requirements of advanced applications in federated systems consisting of distributed and heterogeneous components. While all existing federated systems support atomicity using the two-phase commit protocol, they lack support for federated concurrency control. Many possible solutions have been proposed in the literature, but they failed to make impact on real systems because they completely ignored the widely used concept of isolation levels, which offer optimization options to applications at the cost of less rigorous control over data consistency. This thesis compares existing definitions for isolation levels and develops a new characterization for Snapshot Isolation, an isolation level provided by Oracle, the market leader in the database field. We present algorithms for federated concurrency control that provably guarantee the correct execution of federated transactions even under local Snapshot Isolation, and discuss isolation levels for federated transactions. The algorithms are integrated into a federated system prototype. Performance measurements with this prototype show the practical viability of the developed methods