Prefetching techniques for client server object-oriented database systems

The performance of many object-oriented database applications suffers from the page fetch latency which is determined by the expense of disk access. In this work we suggest several prefetching techniques to avoid, or at least to reduce, page fetch latency. In practice no prediction technique is perfect and no prefetching technique can entirely eliminate delay due to page fetch latency. Therefore we are interested in the trade-off between the level of accuracy required for obtaining good results in terms of elapsed time reduction and the processing overhead needed to achieve this level of accuracy. If prefetching accuracy is high then the total elapsed time of an application can be reduced significantly otherwise if the prefetching accuracy is low, many incorrect pages are prefetched and the extra load on the client, network, server and disks decreases the whole system performance. Access pattern of object-oriented databases are often complex and usually hard to predict accurately. The ..

Client cache management in a distributed object database

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 1995.Includes bibliographical references (leaves 137-143).by Mark Stuart Day.Ph.D

Using khazana to support distributed application development

technical reportOne of the most important services required by most distributed applications is some form of shared data management, e.g., a directory service manages shared directory entries while groupware manages shared documents. Each such application currently must implement its own data management mechanisms, because existing runtime systems are not flexible enough to support all distributed applications efficiently. For example, groupware can be efficiently supported by a distributed object system, while a distributed database would prefer a more low-level storage abstraction. The goal of Khazana is to provide programmer's with configurable components that support the data management services required by a wide variety of distributed applications, including: consistent caching, automated replication and migration of data, persistence, access control, and fault tolerance. It does so via a carefully designed set of interfaces that supports a hierarchy of data abstractions, ranging from flat data to C++/Java objects, and that give programmers a great of control over how their data is managed. To demonstrate the effectiveness of our design, we report on our experience porting three applications to Khazana: a distributed file system, a distributed directory service, and a shared whiteboard

Providing Easy to Use and Fast Programming Support for Non-Volatile Memories

Non-Volatile Memory (NVM) technologies, such as 3D XPoint, offer DRAM-like performance and byte-addressable access to persistent data. NVMs promise an opportunity for fast, persistent data structures, and a wide range of applications stand to benefit from the performance potential of these technologies. These potential benefits are greatest when applications access NVM directly via load/store instructions rather than conventional file-based interfaces. Directly accessing NVM presents several challenges. In particular, applications need guaranteed consistency and safety semantics to protect their data structures in the face of system failures and programming errors.Implementing data structures that meet these requirements is challenging and error-prone. Existing methods for building persistent data structures require either in-depth code changes to an existing data structure or rewriting the data structure from scratch. Unfortunately, both of these methods are labor-intensive and error-prone.Failure-atomicity libraries and programming language extensions can simplify this task. However, all the proposed solutions either require pervasive changes to existing software or incur unacceptable overheads to runtime performance. As a result, porting legacy applications to leverage NVM is likely to be prohibitively difficult and time-consuming.This dissertation first presents Breeze, an NVM toolchain that minimizes the changes necessary to enable legacy code to reap the benefits of directly accessing NVM. In contrast to PMDK and NVM-Direct, Breeze reduces the programming effort of porting Memcached and MongoDB by up to 2.8×, while providing equal or superior performance.Second, it introduces NVHooks, a compiler that automatically annotates NVM accesses and avoids disruptive and error-prone changes to programs. NVHooks reduces the cost of these annotations by applying novel, NVM-specific optimizations to their placement. For our tested benchmarks, NVHooks matches the performance of hand-annotated code while minimizing programmer effort.Finally, it presents Pronto, a new NVM library that reduces the programming effort required to add persistence to volatile data structures. Pronto uses asynchronous semantic logging (ASL) to allow adding persistence to the existing volatile data structure (e.g., C++ Standard Template Library containers) with minor programming effort. ASL moves most durability code off the critical path. Our evaluation shows Pronto data structures outperform highly-optimized NVM data structures by a large margin

Simurgh: a fully decentralized and secure NVMM user space file system

The availability of non-volatile main memory (NVMM) has started a new era for storage systems and NVMM specific file systems can support extremely high data and metadata rates, which are required by many HPC and data-intensive applications. Scaling metadata performance within NVMM file systems is nevertheless often restricted by the Linux kernel storage stack, while simply moving metadata management to the user space can compromise security or flexibility. This paper introduces Simurgh, a hardware-assisted user space file system with decentralized metadata management that allows secure metadata updates from within user space. Simurgh guarantees consistency, durability, and ordering of updates without sacrificing scalability. Security is enforced by only allowing NVMM access from protected user space functions, which can be implemented through two proposed instructions. Comparisons with other NVMM file systems show that Simurgh improves metadata performance up to 18x and application performance up to 89% compared to the second-fastest file system.This work has been supported by the European Comission’s BigStorage project H2020-MSCA-ITN2014-642963. It is also supported by the Big Data in Atmospheric Physics (BINARY) project, funded by the Carl Zeiss Foundation under Grant No.: P2018-02-003.Peer ReviewedPostprint (author's final draft

Consistent, Durable, and Safe Memory Management for Byte-addressable Non Volatile Main Memory

This paper presents three building blocks for enabling the efficient and safe design of persistent data stores for emerging non-volatile memory technologies. Taking the fullest advantage of the low latency and high bandwidths of emerging memories such as phase change memory (PCM), spin torque, and memristor necessitates a serious look at placing these persistent storage technologies on the main memory bus. Doing so, however, introduces critical challenges of not sacrificing the data reliability and consistency that users demand from storage. This paper introduces techniques for (1) robust wear-aware memory allocation, (2) preventing of erroneous writes, and (3) consistency-preserving updates that are cacheefficient. We show through our evaluation that these techniques are efficiently implementable and effective by demonstrating a B+-tree implementation modified to make full use of our toolkit.

SOFORT: A Hybrid SCM-DRAM Storage Engine for Fast Data Recovery

Storage Class Memory (SCM) has the potential to significantly improve database performance. This potential has been well documented for throughput [4] and response time [25, 22]. In this paper we show that SCM has also the potential to significantly improve restart performance, a shortcoming of traditional main memory database systems. We present SOFORT, a hybrid SCM-DRAM storage engine that leverages full capabilities of SCM by doing away with a traditional log and updating the persisted data in place in small increments. We show that we can achieve restart times of a few seconds independent of instance size and transaction volume without significantly impacting transaction throughput

A Multimedia Prototype for Annotation and Illustration Using the Microsoft Foundation Class Library and C++

One of the major claims of the object-oriented programming approach is that it facilitates the development of complex programs by allowing reuse of components. Most compilers for object-oriented languages are now supplied with class libraries. In addition to those provided with the compilers, there are many others in the public domain or available from commercial suppliers. Code reuse can be maximised through the exploitation of framework class libraries for creating interactive programs. A framework library can be viewed as providing a skeleton application that can be extended and specialised through class inheritance. The evolution of application frameworks is discussed briefly in Chapter 1 with an objective to utilise one of them to develop a prototype multimedia application for annotation and illustration. This prototype is referred to as Glasgow Graphics and Sound (GGS) in this thesis. GGS deals with externally created vector or bitmap images, graphics primitives and sound objects in any sequence. GGS is designed to provide the end-users with facilities to work on external images with free-hand curves and other graphics tools, record their voice, save everything in one disk file and animate them later, if necessary. GGS has the responsibility to store different objects without knowing in advance the sequence of object types the user will create. The implementation language, C++ does not have any built-in support for object persistence. Hence, a number of techniques and strategies for adding persistence to C++ objects are reviewed in Chapter 2. The Microsoft Foundation Class (MFC) library is selected as the application framework for developing GGS and the serialization mechanism in MFC is chosen to deal with the object persistence issues. Some of the techniques for persistence, discussed in Chapter 2, are powerful but incur unacceptable overheads for lightweight applications. On the other hand, the MFC serialization is found very useful in creating transportable stream of bytes that can be stored in a file and sent away as an e-mail attachment. Chapter 3 presents the serialization internals in MFC and uncovers some undocumented details that are believed to be valuable for other MFC users. From an application programmer's viewpoint, it is straightforward to use the MFC serialization in most cases. However, the actual implementation details are complex. A sample data structure is serialized and analysed step-by-step to explain the MFC serialization mechanism. The user-friendliness of applications comes not only from an iconic user interface but also from a uniform user interface across applications. Some common user interface elements and their importance are discussed in Chapter 4 along with the document/ view architecture in MFC that separates an application's data management code from its user interface code. The multiple document interface (MDI) in GGS is based on this document/view architecture. A case study walkthrough is presented, purely from an end-user's viewpoint, to illustrate a simple use of GGS. The main classes and their hierarchy are drafted in Chapter 4 based on a high-level decomposition of GGS. Chapter 5 presents the final class hierarchy, different drawing operations and other features involving graphics primitives. Template based type-safe collection classes are used in GGS to store pointers to objects of any type. This simplifies the interaction with the document class. Basic drawing operations such as moving, deleting and highlighting graphics primitives on the screen use an efficient raster drawing mode. The implementation of view magnification together with the standard scrolling capabilities in a window is discussed that requires some special techniques. The benefits of trapping some uncommon messages from the operating system are also discussed. Chapter 5 ends with an overview of the printing process and a description of the multi-page printing features in GGS. Chapter 6 starts with a general discussion on bitmaps and metafiles. A bitmap is a complete digital representation of a picture. Each pixel in the image corresponds to one or more bits in the bitmap. A metafile, on the other hand, stores pictorial information as a series of records that correspond directly to the graphics device interface (GDI) calls. GGS can import externally created bitmaps and metafiles and treat them like any other graphic or sound objects. All commercial illustration programs do something similar. However, the motivation for developing GGS is slightly different. GGS allows the users to construct and manipulate a fairly complex picture, adding comments as they go. The process of constructing the picture is saved, not just the final picture. Sound can be an effective form of information and interface enhancement when appropriately used. It can serve purposes other than the transmission of details or factual information

Selective caching: a persistent memory approach for multi-dimensional index structures

After the introduction of Persistent Memory in the form of Intel’s Optane DC Persistent Memory on the market in 2019, it has found its way into manifold applications and systems. As Google and other cloud infrastructure providers are starting to incorporate Persistent Memory into their portfolio, it is only logical that cloud applications have to exploit its inherent properties. Persistent Memory can serve as a DRAM substitute, but guarantees persistence at the cost of compromised read/write performance compared to standard DRAM. These properties particularly affect the performance of index structures, since they are subject to frequent updates and queries. However, adapting each and every index structure to exploit the properties of Persistent Memory is tedious. Hence, we require a general technique that hides this access gap, e.g., by using DRAM caching strategies. To exploit Persistent Memory properties for analytical index structures, we propose selective caching. It is based on a mixture of dynamic and static caching of tree nodes in DRAM to reach near-DRAM access speeds for index structures. In this paper, we evaluate selective caching on the OLAP-optimized main-memory index structure Elf, because its memory layout allows for an easy caching. Our experiments show that if configured well, selective caching with a suitable replacement strategy can keep pace with pure DRAM storage of Elf while guaranteeing persistence. These results are also reflected when selective caching is used for parallel workloads