248 research outputs found
Dynamic Virtual Page-based Flash Translation Layer with Novel Hot Data Identification and Adaptive Parallelism Management
Solid-state disks (SSDs) tend to replace traditional motor-driven hard disks in high-end storage devices in past few decades. However, various inherent features, such as out-of-place update [resorting to garbage collection (GC)] and limited endurance (resorting to wear leveling), need to be reduced to a large extent before that day comes. Both the GC and wear leveling fundamentally depend on hot data identification (HDI). In this paper, we propose a hot data-aware flash translation layer architecture based on a dynamic virtual page (DVPFTL) so as to improve the performance and lifetime of NAND flash devices. First, we develop a generalized dual layer HDI (DL-HDI) framework, which is composed of a cold data pre-classifier and a hot data post-identifier. Those can efficiently follow the frequency and recency of information access. Then, we design an adaptive parallelism manager (APM) to assign the clustered data chunks to distinct resident blocks in the SSD so as to prolong its endurance. Finally, the experimental results from our realized SSD prototype indicate that the DVPFTL scheme has reliably improved the parallelizability and endurance of NAND flash devices with improved GC-costs, compared with related works.Peer reviewe
Exploiting Fine-Grained Spatial Optimization for Hybrid File System Space
Over decades, I/O optimizations implemented in legacy file systems have been concentrated on reducing HDD disk overhead, such as seek time. As SSD (Solid-State Device) is becoming the main storage medium in I/O storage subsystems, file systems integrated with SSD should take a different approach in designing I/O optimizations. This is because SSD deploys the peculiar device characteristics that do not take place in HDD, such as erasure overhead on flash blocks and absence of seek time to positioning data. In this paper, we present HP-hybrid (High Performance-hybrid) file system that provides a single hybrid file system space, by combining HDD and SSD partitions. HP-hybrid targets for optimizing I/O while considering the strength and weakness of two different partitions, to store large-scale amounts of data in a cost-effective way. Especially, HP-hybrid proposes spatial optimizations that are executed in a hierarchical, fine-grained I/O unit, to address the limited SSD storage resources. We conducted several performance experiments to verify the effectiveness of HP-hybrid while comparing to ext2, ext4 and xfs mounted on both SSD and HDD
LSM-tree based Database System Optimization using Application-Driven Flash Management
ํ์๋
ผ๋ฌธ(์์ฌ)--์์ธ๋ํ๊ต ๋ํ์ :๊ณต๊ณผ๋ํ ์ปดํจํฐ๊ณตํ๋ถ,2019. 8. ์ผํ์.Modern data centers aim to take advantage of high parallelism in storage de-
vices for I/O intensive applications such as storage servers, cache systems, and
key-value stores. Key-value stores are the most typical applications that should
provide a highly reliable service with high-performance. To increase the I/O
performance of key-value stores, many data centers have actively adopted next-
generation storage devices such as Non-Volatile Memory Express (NVMe) based
Solid State Devices (SSDs). NVMe SSDs and its protocol are characterized to
provide a high degree of parallelism. However, they may not guarantee pre-
dictable performance while providing high performance and parallelism. For
example, heavily mixed read and write requests can result in performance degra-
dation of throughput and response time due to the interference between the
requests and internal operations (e.g., Garbage Collection (GC)).
To minimize the interference and provide higher performance, this paper
presents IsoKV, an isolation scheme for key-value stores by exploiting internal
parallelism in SSDs. IsoKV manages the level of parallelism of SSD directly by
running application-driven flash management scheme. By storing data with dif-
ferent characteristics in each dedicated internal parallel units of SSD, IsoKV re-
duces interference between I/O requests. We implement IsoKV on RocksDB and
evaluate it using Open-Channel SSD. Our extensive experiments have shown
that IsoKV improves overall throughput and response time on average 1.20ร
and 43% compared with the existing scheme, respectively.์ต์ ๋ฐ์ดํฐ ์ผํฐ๋ ์คํ ๋ฆฌ์ง ์๋ฒ, ์บ์ ์์คํ
๋ฐ Key-Value stores์ ๊ฐ์ I/O
์ง์ฝ์ ์ธ ์ ํ๋ฆฌ์ผ์ด์
์ ์ํ ์คํ ๋ฆฌ์ง ์ฅ์น์ ๋์ ๋ณ๋ ฌ์ฑ์ ํ์ฉํ๋ ๊ฒ์
๋ชฉํ๋ก ํ๋ค. Key-value stores๋ ๊ณ ์ฑ๋ฅ์ ๊ณ ์ ๋ขฐ ์๋น์ค๋ฅผ ์ ๊ณตํด์ผ ํ๋ ๊ฐ์ฅ
๋ํ์ ์ธ ์์ฉํ๋ก๊ทธ๋จ์ด๋ค. Key-value stores์ I/O ์ฑ๋ฅ์ ๋์ด๊ธฐ ์ํด ๋ง์ ๋ฐ
์ดํฐ ์ผํฐ๊ฐ ๋นํ๋ฐ์ฑ ๋ฉ๋ชจ๋ฆฌ ์ต์คํ๋ ์ค(NVMe) ๊ธฐ๋ฐ SSD(Solid State Devices)
์ ๊ฐ์ ์ฐจ์ธ๋ ์คํ ๋ฆฌ์ง ์ฅ์น๋ฅผ ์ ๊ทน์ ์ผ๋ก ์ฑํํ๊ณ ์๋ค. NVMe SSD์ ๊ทธ ํ
๋กํ ์ฝ์ ๋์ ์์ค์ ๋ณ๋ ฌ์ฑ์ ์ ๊ณตํ๋ ๊ฒ์ด ํน์ง์ด๋ค. ๊ทธ๋ฌ๋ NVMe SSD๊ฐ
๋ณ๋ ฌ์ฑ์ ์ ๊ณตํ๋ฉด์๋ ์์ธก ๊ฐ๋ฅํ ์ฑ๋ฅ์ ๋ณด์ฅํ์ง๋ ๋ชปํ ์ ์๋ค. ์๋ฅผ ๋ค์ด
์ฝ๊ธฐ ๋ฐ ์ฐ๊ธฐ ์์ฒญ์ด ๋ง์ด ํผํฉ๋๋ฉด ์์ฒญ๊ณผ ๋ด๋ถ ์์
(์: GC) ์ฌ์ด์ ๊ฐ์ญ์ผ๋ก
์ธํด ์ฒ๋ฆฌ๋ ๋ฐ ์๋ต ์๊ฐ์ ์ฑ๋ฅ ์ ํ๊ฐ ๋ฐ์ํ ์ ์๋ค.
๊ฐ์ญ์ ์ต์ํํ๊ณ ์ฑ๋ฅ์ ํฅ์์ํค๊ธฐ ์ํด ๋ณธ ์ฐ๊ตฌ์์๋ Key-value stores๋ฅผ
์ํ ๊ฒฉ๋ฆฌ ๋ฐฉ์์ธ IsoKV๋ฅผ ์ ์ํ๋ค. IsoKV๋ ์ ํ๋ฆฌ์ผ์ด์
์ค์ฌ ํ๋์ ์ ์ฅ์ฅ
์น ๊ด๋ฆฌ ๋ฐฉ์์ ํตํด SSD์ ๋ณ๋ ฌํ ์์ค์ ์ง์ ๊ด๋ฆฌํ๋ค. IsoKV๋ SSD์ ๊ฐ ์ ์ฉ
๋ด๋ถ ๋ณ๋ ฌ ์ฅ์น์ ์๋ก ๋ค๋ฅธ ํน์ฑ์ ๊ฐ์ง ๋ฐ์ดํฐ๋ฅผ ์ ์ฅํจ์ผ๋ก์จ I/O ์์ฒญ ๊ฐ์
๊ฐ์ญ์ ์ค์ธ๋ค. ๋ํ IsoKV๋ SSD์ LSM ํธ๋ฆฌ ๋ก์ง๊ณผ ๋ฐ์ดํฐ ๊ด๋ฆฌ๋ฅผ ๋๊ธฐํํ
์ฌ GC๋ฅผ ์ ๊ฑฐํ๋ค. ๋ณธ ์ฐ๊ตฌ์์๋ RocksDB๋ฅผ ๊ธฐ๋ฐ์ผ๋ก IsoKV๋ฅผ ๊ตฌํํ์์ผ๋ฉฐ,
Open-Channel SSD๋ฅผ ์ฌ์ฉํ์ฌ ์ฑ๋ฅํ๊ฐํ์๋ค.. ๋ณธ ์ฐ๊ตฌ์ ์คํ ๊ฒฐ๊ณผ์ ๋ฐ๋ฅด๋ฉด
IsoKV๋ ๊ธฐ์กด์ ๋ฐ์ดํฐ ์ ์ฅ ๋ฐฉ์๊ณผ ๋น๊ตํ์ฌ ํ๊ท 1.20ร ๋น ๋ฅด๊ณ ๋ฐ 43% ๊ฐ์๋
์ฒ๋ฆฌ๋๊ณผ ์๋ต์๊ฐ ์ฑ๋ฅ ๊ฐ์ ๊ฒฐ๊ณผ๋ฅผ ์ป์๋ค. ๊ด์ ์์ 43% ๊ฐ์ํ์๋ค.Abstract
Introduction 1
Background 8
Log-Structured Merge tree based Database 8
Open-Channel SSDs 9
Preliminary Experimental Evaluation using oc bench 10
Design and Implementation 14
Overview of IsoKV 14
GC-free flash storage management synchronized with LSM-tree logic 15
I/O type Isolation through Application-Driven Flash Management 17
Dynamic Arrangement of NAND-Flash Parallelism 19
Implementation 21
Evaluation 23
Experimental Setup 23
Performance Evaluation 25
Related Work 31
Conclusion 34
Bibliography 35
์ด๋ก 40Maste
An NVM Aware MariaDB Database System and Associated IO Workload on File Systems
MariaDB is a community-developed fork of the MySQL relational database management system and originally designed and implemented in order to use the traditional spinning disk architecture. With Non-Volatile memory (NVM) technology now in the forefront and main stream for server storage (Data centers), MariaDB addresses the need by adding support for NVM devices and introduces NVM Compression method. NVM Compression is a novel hybrid technique that combines application level compression with flash awareness for optimal performance and storage efficiency. Utilizing new interface primitives exported by Flash Translation Layers (FTLs), we leverage the garbage collection available in flash devices to optimize the capacity management required by compression systems. We implement NVM Compression in the popular MariaDB database and use variants of commonly available POSIX file system interfaces to provide the extended FTL capabilities to the user space application. The experimental results show that the hybrid approach of NVM Compression can improve compression performance by 2-7x, deliver compression performance for flash devices that is within 5% of uncompressed performance, improve storage efficiency by 19% over legacy Row-Compression, reduce data writes by up to 4x when combined with other flash aware techniques such as Atomic Writes, and deliver further advantages in power efficiency and CPU utilization. Various micro benchmark measurement and findings on sparse files call for required improvement in file systems for handling of punch hole operations on files
A Survey on the Integration of NAND Flash Storage in the Design of File Systems and the Host Storage Software Stack
With the ever-increasing amount of data generate in the world, estimated to reach over 200 Zettabytes by 2025, pressure on efficient data storage systems is intensifying. The shift from HDD to flash-based SSD provides one of the most fundamental shifts in storage technology, increasing performance capabilities significantly. However, flash storage comes with different characteristics than prior HDD storage technology. Therefore, storage software was unsuitable for leveraging the capabilities of flash storage. As a result, a plethora of storage applications have been design to better integrate with flash storage and align with flash characteristics. In this literature study we evaluate the effect the introduction of flash storage has had on the design of file systems, which providing one of the most essential mechanisms for managing persistent storage. We analyze the mechanisms for effectively managing flash storage, managing overheads of introduced design requirements, and leverage the capabilities of flash storage. Numerous methods have been adopted in file systems, however prominently revolve around similar design decisions, adhering to the flash hardware constrains, and limiting software intervention. Future design of storage software remains prominent with the constant growth in flash-based storage devices and interfaces, providing an increasing possibility to enhance flash integration in the host storage software stack
- โฆ