Redesigning Transaction Processing Systems for Non-Volatile Memory

Department of Computer Science and EngineeringTransaction Processing Systems are widely used because they make the user be able to manage their data more efficiently. However, they suffer performance bottleneck due to the redundant I/O for guaranteeing data consistency. In addition to the redundant I/O, slow storage device makes the performance more degraded. Leveraging non-volatile memory is one of the promising solutions the performance bottleneck in Transaction Processing Systems. However, since the I/O granularity of legacy storage devices and non-volatile memory is not equal, traditional Transaction Processing System cannot fully exploit the performance of persistent memory. The goal of this dissertation is to fully exploit non-volatile memory for improving the performance of Transaction Processing Systems. Write amplification between Transaction Processing System is pointed out as a performance bottleneck. As first approach, we redesigned Transaction Processing Systems to minimize the redundant I/O between the Transaction Processing Systems. We present LS-MVBT that integrates recovery information into the main database file to remove temporary files for recovery. The LS-MVBT also employs five optimizations to reduce the write traffics in single fsync() calls. We also exploit the persistent memory to reduce the performance bottleneck from slow storage devices. However, since the traditional recovery method is for slow storage devices, we develop byte-addressable differential logging, user-level heap manager, and transaction-aware persistence to fully exploit the persistent memory. To minimize the redundant I/O for guarantee data consistency, we present the failure-atomic slotted paging with persistent buffer cache. Redesigning indexing structure is the second approach to exploit the non-volatile memory fully. Since the B+-tree is originally designed for block granularity, It generates excessive I/O traffics in persistent memory. To mitigate this traffic, we develop cache line friendly B+-tree which aligns its node size to cache line size. It can minimize the write traffic. Moreover, with hardware transactional memory, it can update its single node atomically without any additional redundant I/O for guaranteeing data consistency. It can also adapt Failure-Atomic Shift and Failure-Atomic In-place Rebalancing to eliminate unnecessary I/O. Furthermore, We improved the persistent memory manager that exploit traditional memory heap structure with free-list instead of segregated lists for small memory allocations to minimize the memory allocation overhead. Our performance evaluation shows that our improved version that consider I/O granularity of non-volatile memory can efficiently reduce the redundant I/O traffic and improve the performance by large of a margin.ope

Persistent Database Buffer Caching and Logging with Slotted Page Structure

Department of Computer Science and EngineeringEmerging byte-addressable persistent memory (PM) will be effective to improve the performance of computer system by reducing the redundant write operations. Traditional database management system uses recovery techniques to prevent data loss. The techniques copy the entire page into the block device storage several times for one insertion, so the amount of I/O is not negligible. In this work, we consider PM as main memory. Then, the durability of data in the buffer cache is ensured. To guarantee consistency, we exploit slotted page structure which is commonly used in database systems. We revisit that the slot header, which stores the metadata of the page in the slotted page structure, can act like a commit mark in the persistent database buffer cache. We then present two novel database management schemes using persistent buffer cache and slotted page. In-place commit scheme updates the page atomically using hardware transactional memory. It doesn't make any other copies and has optimal performance. Slot header logging scheme is needed for the case of updating pages more than one. Unlike the existing logging technique, slot header logging reduces the write operations by logging only commit mark. We implemented these schemes in SQLite and evaluate the performance compared with NVWAL, which is the state-of-the-art scheme. Our experiments show that in-place commit scheme needs only 3 cache line flush instructions for one insertion and slot header logging scheme reduces logging overhead at least 1/4.ope

Transactional filesystems

Dissertação de Mestrado em Engenharia InformáticaThe task of implementing correct software is not trivial; mainly when facing the need for supporting concurrency. To overcome this difficulty, several researchers proposed the technique of providing the well known database transactional models as an abstraction for existing programming languages, allowing a software programmer to define groups of computations as transactions and benefit from the expectable semantics of the underlying transactional model. Prototypes for this programming model are nowadays made available by many research teams but are still far from perfection due to a considerable number of operational restrictions. Mostly, these restrictions derive from the limitations on the use of input-output functions inside a transaction. These functions are frequently irreversible which disables their compatibility with a transactional engine due to its impossibility to undo their effects in the event of aborting a transaction. However, there is a group of input-output operations that are potentially reversible and that can produce a valuable tool when provided within the transactional programming model explained above: the file system operations. A programming model that would involve in a transaction not only a set of memory operations but also a set of file operations, would allow the software programmer to define algorithms in a much flexible and simple way, reaching greater stability and consistency in each application. In this document we purpose to specify and allow the use of this type of operations inside a transactional programming model, as well as studying the advantages and disadvantages of this approach

Improving the Performance and Endurance of Persistent Memory with Loose-Ordering Consistency

Persistent memory provides high-performance data persistence at main memory. Memory writes need to be performed in strict order to satisfy storage consistency requirements and enable correct recovery from system crashes. Unfortunately, adhering to such a strict order significantly degrades system performance and persistent memory endurance. This paper introduces a new mechanism, Loose-Ordering Consistency (LOC), that satisfies the ordering requirements at significantly lower performance and endurance loss. LOC consists of two key techniques. First, Eager Commit eliminates the need to perform a persistent commit record write within a transaction. We do so by ensuring that we can determine the status of all committed transactions during recovery by storing necessary metadata information statically with blocks of data written to memory. Second, Speculative Persistence relaxes the write ordering between transactions by allowing writes to be speculatively written to persistent memory. A speculative write is made visible to software only after its associated transaction commits. To enable this, our mechanism supports the tracking of committed transaction ID and multi-versioning in the CPU cache. Our evaluations show that LOC reduces the average performance overhead of memory persistence from 66.9% to 34.9% and the memory write traffic overhead from 17.1% to 3.4% on a variety of workloads.Comment: This paper has been accepted by IEEE Transactions on Parallel and Distributed System

A File System Level Snapshot In Ext4

Snapshot makes a copy of current working system. Creating a snapshot with some processing andoverheads is important to provide the reliable data service during backup. There are two types of snapshottechniques: volume-based approach and system-based approach. The volume-based Logical VolumeManager provides compact space usability, but requires some space to save snapshot and makes systemwith extra overheads. The system-based Snapshot works better than volume based. The file system basedSnapshot does not need reserve space for snapshot. Therefore, the system-based Snapshot is considered agood solution in the computer environment where large-scale space management capability is not thatcritical. However, such snapshot feature is available in Linux kernel 2.2, 2.6 in EXT3 file system. In thispaper, we proposed a system-based snapshot for the ext4 system in Linux kernel 2.6 or higher. The main concept of the system-based snapshot mainly come from old-version system based Snapshot.Keywords: Snapcreate, kernel, Inode, SnapFS