366,187 research outputs found
Persistent Buffer Management with Optimistic Consistency
Finding the best way to leverage non-volatile memory (NVM) on modern database
systems is still an open problem. The answer is far from trivial since the
clear boundary between memory and storage present in most systems seems to be
incompatible with the intrinsic memory-storage duality of NVM. Rather than
treating NVM either solely as memory or solely as storage, in this work we
propose how NVM can be simultaneously used as both in the context of modern
database systems. We design a persistent buffer pool on NVM, enabling pages to
be directly read/written by the CPU (like memory) while recovering corrupted
pages after a failure (like storage). The main benefits of our approach are an
easy integration in the existing database architectures, reduced costs (by
replacing DRAM with NVM), and faster peak-performance recovery
Massively Parallel Sort-Merge Joins in Main Memory Multi-Core Database Systems
Two emerging hardware trends will dominate the database system technology in
the near future: increasing main memory capacities of several TB per server and
massively parallel multi-core processing. Many algorithmic and control
techniques in current database technology were devised for disk-based systems
where I/O dominated the performance. In this work we take a new look at the
well-known sort-merge join which, so far, has not been in the focus of research
in scalable massively parallel multi-core data processing as it was deemed
inferior to hash joins. We devise a suite of new massively parallel sort-merge
(MPSM) join algorithms that are based on partial partition-based sorting.
Contrary to classical sort-merge joins, our MPSM algorithms do not rely on a
hard to parallelize final merge step to create one complete sort order. Rather
they work on the independently created runs in parallel. This way our MPSM
algorithms are NUMA-affine as all the sorting is carried out on local memory
partitions. An extensive experimental evaluation on a modern 32-core machine
with one TB of main memory proves the competitive performance of MPSM on large
main memory databases with billions of objects. It scales (almost) linearly in
the number of employed cores and clearly outperforms competing hash join
proposals - in particular it outperforms the "cutting-edge" Vectorwise parallel
query engine by a factor of four.Comment: VLDB201
Memory Power Consumption in Main-Memory Database Systems
In main-memory database systems, memory can consume a substantial amount of
power, comparable to that of the processors. However, existing memory power-saving
mechanisms are much less effective than processor power management. Unless the system
is almost idle, memory power consumption will be high.
The reason for poor memory power proportionality is that the bulk of memory power
consumption is attributable to background power, which is determined by memory power
state residency. The memory workload in existing systems is evenly distributed over the
memory modules and also in time, which precludes the occurrence of long idle intervals.
As a result, deep low-power states, which could significantly reduce background power
consumption, are rarely entered.
In this work, we aim to reduce the memory power consumption of main-memory data-
base systems. We start by investigating and explaining the patterns of memory power
consumption, under various workloads. We then propose two techniques, implemented at
the database system level, that skew memory traffic, creating long periods of idleness in a
subset of memory modules. This allows those modules to enter low-power states, reducing
overall memory power consumption. We prototyped these techniques in DimmStore, an
experimental database system.
The first technique is rate-aware data placement, which places data on memory modules
according to its access frequency. The background power in the unused or least-used
modules is reduced, without affecting background power in the most-used modules. Rate-
aware placement saves power and has little performance impact. Under a TPC-C workload,
rate-aware placement resulted in memory power savings up to 44%, with a maximum
throughput reduction of 10%.
The second technique is memory access gating, which targets background power in less-
frequently accessed memory modules by inserting periodic idle intervals. Memory gating
reduces power consumption of memory modules for which rate-aware placement alone does
not create sufficient idleness to reduce power consumption. With gating, memory accesses
to these modules become concentrated outside of the idle intervals, creating the opportunity
for low-power state use. However, because it delays memory accesses, memory gating
impacts performance. Higher memory power savings and lower performance impact occur
in workloads with lower memory access rates. Thus, in the YCSB workload with a medium
transaction rate, memory gating reduced memory power by 26%, adding 0.25 ms (30%) of
transaction latency, compared to DimmStore without gating. In the more memory intensive
TPC-C workload and low to medium transaction rate, gating can save 5% of memory power,
adding 1.5 ms (60%) of transaction latency, compared to DimmStore without gating
Energy-Utility Function-Based Resource Control for In-Memory Database Systems LIVE
The ever-increasing demand for scalable database systems is limited by their energy consumption, which is one of the major challenges in research today. While existing approaches mainly focused on transaction-oriented disk-based database systems, we are investigating and optimizing the energy consumption and performance of data-oriented scale-up in-memory database systems that make heavy use of the main power consumers, which are processors and main memory. In this demo, we present energy-utility functions as an approach for enabling the operating system to improve the energy efficiency of scalable in-memory database systems. Our highly interactive demo setup mainly allows attendees to switch between multiple DBMS workloads and watch in detail how the system responds by adapting the hardware configuration appropriately
Review on Main Memory Database
The idea of using Main Memory Database (MMDB) as physical memory is not new but is in existence quite since a decade. MMDB have evolved from a period when they were only used for caching or in high-speed data systems to a time now in twenty first century when they form a established part of the mainstream IT. Early in this century, although larger main memories were affordable but processors were not fast enough for main memory databases to be admired. However, today’s processors are faster, available in multicore and multiprocessor configurations having 64-bit memory addressability stocked with multiple gigabytes of main memory. Thus, MMDBs definitely call for a solution for meeting the requirements of next generation IT challenges. To aid this swing, database systems are reconsidered to handle implementation issues adjoining the inherent differences between disk and memory storage and gain performance benefits. This paper is a review on Main Memory Databases (MMDB)
In-memory Databases in Business Information Systems
In-memory databases are developed to keep the entire data in main memory. Compared to traditional database systems, read access is now much faster since no I/O access to a hard drive is required. In terms of write access, mechanisms are available which provide data persistence and thus secure transactions. In-memory databases have been available for a while and have proven to be suitable for particular use cases. With increasing storage density of DRAM modules, hardware systems capable of storing very large amounts of data have become affordable. In this context the question arises whether in-memory databases are suitable for business information system applications. Hasso Plattner, who developed the HANA in-memory database, is a trailblazer for this approach. He sees a lot of potential for novel concepts concerning the development of business information systems. One example is to conduct transactions and analytics in parallel and on the same database, i.e. a division into operational database systems and data warehouse systems is no longer necessary (Plattner and Zeier 2011). However, there are also voices against this approach. Larry Ellison described the idea of business information systems based on in-memory database as “wacko,” without actually making a case for his statement (cf. Bube 2010). Stonebraker (2011) sees a future for in-memory databases for business information systems but considers the division of OLTP and OLAP applications as reasonable. [From: Introduction
Parallel Database Architectures: A Simulation Study.
Parallel database systems are gaining popularity as a solution that provides scalability in large and growing databases. A parallel database system is a DBS which exploits multiprocessing systems to improve performance. Parallel database computers can be classified into three categories: shared memory, shared disk, and shared nothing. In shared memory, all resources, including main memory and disk units, are shared among several processors. In shared disk, a group of processors share a common pool of disks, but each processor has its own private main memory. In the shared-nothing system, every processor has its own memory and disk unit, that is, except for communication links, no resources are shared among the processors. In this work, we· compare the performance of the three architecture classes. Simulation models for the various architectures are introduced. Using these models, a number of experiments were conducted to compare the system performance of these architectures under different workloads and transaction models. The aim of this work is to provide a tool for evaluating the different architectures and their appropriateness for a specific database application
- …