WiscSort: External Sorting For Byte-Addressable Storage

We present WiscSort, a new approach to high-performance concurrent sorting for existing and future byte-addressable storage (BAS) devices. WiscSort carefully reduces writes, exploits random reads by splitting keys and values during sorting, and performs interference-aware scheduling with thread pool sizing to avoid I/O bandwidth degradation. We introduce the BRAID model which encompasses the unique characteristics of BAS devices. Many state-of-the-art sorting systems do not comply with the BRAID model and deliver sub-optimal performance, whereas WiscSort demonstrates the effectiveness of complying with BRAID. We show that WiscSort is 2-7x faster than competing approaches on a standard sort benchmark. We evaluate the effectiveness of key-value separation on different key-value sizes and compare our concurrency optimizations with various other concurrency models. Finally, we emulate generic BAS devices and show how our techniques perform well with various combinations of hardware properties

Stop All File Systems Research

We advocate that the OS community should stop all file systems research, because file systems are fundamentally the wrong place in the storage hierarchy to innovate. Three factors combine to limit the impact of file systems research: the constraints of legacy, the diversity of implementations, and a lack of information about storage-system internals. We instead suggest that researchers focus their innovations on storage systems, namely within disk or RAID subsystems. To enable a full range of functionality within the storage system, we posit that the disk system must be "semantically smart"; that is, it must understand the on-disk structures and recognize the on-line operations of the file system above. We discuss the concepts underlying semantically-smart disk systems, present a taxonomy of the different axes of semantic knowledge, and discuss the important remaining research challenges. 1 Backgroun

Infokernel: An Evolutionary Approach to Operating System Design

Although information about the internals of the operating system is useful to applications, such information is rarely exposed. In this paper, we argue that all information about the internal state, algorithms, and costs of the OS should be exported. We term an OS that exports this information an infokernel. We show that access to information is useful so that applications can both adapt to and control the behavior of the 0S. Given that it is relatively easy to modify existing systems to expose information, we believe this is a practical way to obtain many of the benefits of more radical extensible systems

Storage-Aware Caching: Revisiting Caching for Heterogeneous Storage Systems

Modern storage environments are composed of a variety of devices with different performance characteristics. In this paper, we explore storage-aware caching algorithms, in which the file buffer replacement algorithm explicitly accounts for differences in performance across devices. We introduce a new family of storage-aware caching algorithms that partition the cache, with one partition per device. The algorithms set the partition sizes dynamically to balance work across the devices. Through simulation, we show that our storage-aware policies perform similarly to LANDLORD, a cost-aware algorithm previously shown to perform well in Web caching environments. We also demonstrate that partitions can be easily incorporated into the Clock replacement algorithm, thus increasing the likelihood of deploying cost-aware algorithms in modern operating systems