Optimizing File Systems for High-Performance Storage Devices

학위논문 (박사)-- 서울대학교 대학원 : 공과대학 전기·컴퓨터공학부, 2018. 2. 염헌영.High-performance storage technologies such as solid-state drives (SSDs) provide low-latency, high throughput, and high I/O parallelism to legacy storage systems. SSDs access data without mechanical overhead, and they often leads to order-of-magnitude improvements in performance over legacy storage devices such as hard disk drives (HDDs). However, replacing HDDs with SSDs while keeping the software I/O stack or not exploiting SSD features does not lead to maximum performance. In this dissertation, we optimize file systems to fully exploit the SSD features (e.g., low-latency and high I/O parallelism). First, we analyze and explore I/O strategies in the existing file systems on low-latency SSDs. The file systems issue and complete several I/O requests when blocks are not contiguous, which does not take advantage of the low-latency of SSDs. To address this problem, we propose efficient I/O strategies, which transfer requests from discontiguous host memory buffers in the file systems to discontiguous storage segments in a single I/O request. Thus, they enable file systems to fully exploit the performance of low-latency SSDs. Second, we investigate the locking and I/O parallelism in the existing file systems on highly parallel SSDs. In the file systems, the coarse-grained locking to access shared data structures is used and I/O operations are serialized by a single thread. For these reasons, the file systems often face the problem of lock contention and underutilization of I/O bandwidth on multi-cores with highly parallel SSDs. To address these issues, we enable concurrent updates on data structures and parallelize I/O operations. We implement our techniques in EXT4/JBD2 and evaluate them on low-latency and highly parallel SSDs. The experimental results show that our optimized file system improves the performance compared to the existing EXT4 file system.Chapter 1 Introduction 1 1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Approach and Contributions . . . . . . . . . . . . . . . . . . . . 3 1.3 Dissertation Structure . . . . . . . . . . . . . . . . . . . . . . . . 4 Chapter 2 Background 6 2.1 High-performance Storage Devices . . . . . . . . . . . . . . . . . 6 2.2 Crash Consistency in File Systems . . . . . . . . . . . . . . . . . 7 2.3 Read and Write Operations in the Existing File Systems . . . . . 9 2.4 Journal I/O in the Journaling File Systems . . . . . . . . . . . . 10 2.5 Recovery in the Journaling File Systems . . . . . . . . . . . . . . 13 2.6 Existing Locking and I/O Parallelism in Journaling File Systems 14 Chapter 3 Design and Implementation 24 3.1 Optimizing File Systems for Low-latency Storage Devices . . . . 24 3.1.1 Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 3.1.2 Implementation . . . . . . . . . . . . . . . . . . . . . . . . 30 3.2 Optimizing File Systems for Highly Parallel Storage Devices . . . 33 3.2.1 Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 3.2.2 Implementation . . . . . . . . . . . . . . . . . . . . . . . . 39 Chapter 4 Evaluation 50 4.1 Evaluating the Optimized File System for Low-latency Storage . 50 4.1.1 Run-time Performance . . . . . . . . . . . . . . . . . . . . 52 4.1.2 Recovery Performance . . . . . . . . . . . . . . . . . . . . 57 4.1.3 Experimental Analysis . . . . . . . . . . . . . . . . . . . . 59 4.2 Evaluating the Optimized File System for Highly Parallel Storage 61 4.2.1 Run-time Performance . . . . . . . . . . . . . . . . . . . . 63 4.2.2 Recovery Performance . . . . . . . . . . . . . . . . . . . . 66 4.2.3 Experimental Analysis . . . . . . . . . . . . . . . . . . . . 67 Chapter 5 Related Work 69 5.1 Analysis and Evaluation of High-Performance storage . . . . . . 69 5.2 Study of Journaling File Systems . . . . . . . . . . . . . . . . . . 70 5.3 File and I/O System Optimizations for Low-latency Storage . . . 72 5.4 Study of Scalability in Operating Systems . . . . . . . . . . . . . 75 5.5 File and I/O System Optimizations for Highly Parallel Storage . 75 Chapter 6 Conculsion 78 6.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 6.2 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79Docto

Clinical and Cytogenetic study on 3,672 Genetic Amniocentesis: YUMC 20 years experience from 1985 to 2004 years

Objective: To systematic analyze the change of the annual distribution and indications, age distribution of the patients and chromosomal results according to patient’s age and indications in midtrimester genetic amniocentesis Methods: This study conducted between 1985 and 2004 collected 3,672 amniocenteses procedure which were done at College of Medicine, after prenatal genetic counceling for mothers who have high risk for carrying chromosomally abnormal babies. Results: 1. The incidence of amniocentesis had been in gradual increase since the 1980’’s, however, the number has increased sharply for the patiences in mid 1990’s. 2. Of the 3,672 amniocentesis cases, 32.2% was maternal age 30 to 34 which was most common age group and followed by age 35 to 39 was 29.9% and age 25 to 29 was 27.8%. 3. The indications for amniocentesis were advanced maternal age (36.1%), abnormal maternal serum markers (31.7%) and abnormal ultrasonographic findings which implies chromosomal abnormality (9.6%). In the 1980’s, amniocentesis had earlier been used primarily for those in advanced maternal age groups, at least 35 years older. Recently maternal serum markers and ultrasonography play an important role as an indicator for the amniocentesis. 4. From the 3,672 cases, 3,556 cases showed normal diploidy and 116 cases abnormal karyotype which consisted 3.16%. In autosomal disorders, 36 Down syndrome, 15 Edward syndrome, 2 Patau syndrome were diagnosed. In Sex chromosomal anomaly, 5 Turner syndrome, 6 47XYY, and 2 Klinefelter syndrome. Add to that 31 translocation including 21 Reciprocal translocation and 10 Robertsonian translocation, and 8 deletions and 4 mosaicisms were diagnosed. Of the 354 cases with abnormal ultrasonic findings, 19 (5.4%) resulted in chromosomal anomaly. Of the 1,164 casaes with positive maternal serum markers, 42 (3.6%) resulted in chromosomal anomaly. Those who had abnormal ultrasonographic findings implying chromosomal abnormality were found to have correlation with chromosomal abnomality than other indications. Conclusion: Midtrimester genetic amniocentesis is an important diagnostic tool in prenatal diagnosis, of which the annual incidence has been recently increased abruptly. Not only maternal age, but the maternal serum markers and ultrasonograms should be considered in prenatal counseling. Amniocentesis should be well informed to the general population.ope

An Optimal Resource Allocation Scheme for Increasing RocksDB Parallelism on High-Performance Computing Systems

고성능 컴퓨팅 시스템은 높아진 컴퓨팅 파워와 고성능 저장장장치의 출현으로 점점 널리 사용되고 있는 추세이다. 고성능 컴퓨팅 시스템에서 동작하는 응용의 성능은 시스템의 가용 자원을 어떻게 할당하여 사용하느냐에 따라 좌우된다. 대표적인 소셜 네트워크 서비스 운영 기업인 페이스북에서는 자사의데이터 저장 및 관리를 위해 고성능 저장장치에 최적화 된 RocksDB라는 키-밸류 스토어를 사용하고 있다. RocksDB에서는 관리자가 시스템의 가용한 자원을 최적으로 할당할 수 있도록 다양한 설정을 제공한다. 본 논문에서는 고성능 컴퓨팅 시스템의 가용 자원을 할당하는 방법이 RocksDB의 성능에 미치는 영향을 분석한다. 이를 기반으로 RocksDB의 병렬성을 높여 성능을 향상시킬 수 있는 최적의 자원 할당 기법을 제안한다. High-performance computing systems are becoming more widely used because of the increased computing power and the emergence of high-performance storage devices. The performance of applications operating on the high-performance computing systems depends on how the systems’ available resources are allocated and applied. Facebook, a leading social network service company, uses a key-value store called RocksDB, optimized for high-performance storage devices for data storage and management. RocksDB provides a variety of configurations that allow administrators to optimally allocate available resources on their systems. In this paper, we analyzed the impact of the allocation methods of available resources of the high-performance computing systems on the performance of RocksDB. Based on this, we propose an optimal resource allocation technique that can improve the performance by increasing the parallelism of RocksDB.N

An Efficient File Distribution Scheme for Multiple Storage Devices based on I/O Characteristics of the Key-Value Store

최근 많은 데이터 센터들에서는 나날이 증가하는 데이터를 관리하기 위해 고성능 저장장치를 채택하고 있다. 또한, 다수의 장치를 사용하여 대용량 데이터의 효율적으로 처리한다. 그러나, 고성능 저장 장치를 사용함에도 불구하고 읽기와 쓰기 연산이 혼재되어 발생하는 일부 응용에서는 성능이 저하될 수 있다. 이는 장치 내부에서 쓰기 연산의 처리가 완료될 때까지 읽기 연산의 처리가 지연되기 때문이다. 본 논문에서는 응용에서 생성하는 파일들의 I/O 특성을 고려하여 다수의 장치에 효율적으로 분리 저장하는 기법을 제안한다. 실험 결과를 통해 읽기와 쓰기 연산을 분리하여 처리함으로써 응용의 성능이 최대 21% 향상됨을 보인다. Recently, many data centers have adopted high-performance storage devices to manage the ever-increasing data. In addition, multiple devices are used to efficiently process large amounts of data. However, despite the use of high-performance storage devices, performance can still be degraded when read and write operations are mixed in the workload. This is because the handling read operation is delayed until the write operations are completed. Thus, in this paper, we propose an efficient file distribution scheme based on I/O characteristics. The experimental results show that the performance of the application is improved by up to 21% by separating the read and write operations.N

A Secure P2P E-coupon Service based on Blockchain

전자 쿠폰은 종이 쿠폰의 디지털 형태이며, 제품과 서비스의 구매를 유도하기 위해 고객에게 할인 또는 보상을 제공하는 것을 목적으로 한다. 전자상거래가 발전함에 따라 전자 쿠폰은 효과적인 마케팅 방법으로도 활용되며 광범위하게 사용되고 있다. 그러나 전자 쿠폰은 보안 측면에서 여러 문제점을 지니고 있다. 특히 전자 쿠폰의 부정 사용 및 쿠폰의 위변조 문제는 기업에 큰 피해를 주고 있다. 본 논문에서는 이러한 문제를 해결하기 위해 블록체인을 전자 쿠폰 서비스에 적용한 새로운 서비스를 제안한다. 제안된 전자 쿠폰 서비스는 보안 문제 해결 뿐만 아니라, 중앙의 신뢰 기관 없이 개인 간 신뢰를 기반으로 여러 상점에서 사용 가능한 장점이 있다. 쿠폰 발행 및 사용이 특정 상점에서만 가능했던 기존 서비스와 달리 상점 간 직접 대금 정산이 가능하게 함으로써 전자 쿠폰의 안정성과 효율성을 높였다. Electronic coupons are a digital form of paper coupons and are intended to provide discounts or rewards to customers in order to drive purchases of products and services. With the evolution of electronic commerce, electronic coupons have been employed as effective marketing methods and are widely used in many areas. However, electronic coupons have various problems in terms of security. In particular, the illegal use of electronic coupons and the problems of forgery and corruption are causing great damage to enterprises. In this paper, we propose a new service that applies blockchain to electronic coupon service to solve the problem of illegal use. Unlike the existing services, which issues and use coupons between individuals without a central trust institution, the proposed electronic coupon service solves security problems, prevents fraudulent use, and by enabling direct settlement between the shops, the stability, and efficiency of electronic coupons get enhanced.N

Framework-assisted Priority Boosting and Load Balancing for Improving Interactivity of Android Smartphones

Authors' final version본 논문은 KCC2012에서 안드로이드 기반 스마트폰의 사용자 응답성 향상을 위한 프레임워크 지원 우선순위 부스트 기법의 제목으로 발표된 논문을 확장한 것임최근 안드로이드 플랫폼을 탑재한 스마트폰이 널리 보급되면서 안드로이드 플랫폼에 대한 관심은 더욱 커지고 있다. 하지만 안드로이드 스마트폰은 종종 양질의 사용자 응답성을 제공하지 못하는 것으로 알려져 있다. 이는 안드로이드 상에서 대화형 태스크가 다른 태스크와 구별되지 않고 동일한 우선순위로 스케줄링 되기 때문에 사용자 입력을 처리하는 동안 여러 번의 선점을 당해 긴 응답시간을 초래할 수 있기 때문이다. 이 논문은 안드로이드 스마트폰의 사용자 응답성 향상을 위해 프레임워크 지원 우선순위 부스트 기법과 로드 밸런싱 기법을 제시한다. 프레임워크 지원 우선순위기법은 프레임워크 레벨에서 대화형 태스크를 식별하고 이를 커널에게 전달하며, 커널 레벨에서는 식별된 태스크의 우선순위를 선별적으로 부스트 시킴으로써 사용자 입력을 처리할 만큼 충분한 시간을 보장해 준다. 로드 밸런싱 기법은 부스트 된 태스크를 여전히 방해하는 태스크들을 다른 실행 큐로 이주시킴으로써, 대화형 태스크의 응답시간을 최소화 한다. 실험 결과 대화형 태스크의 응답시간이 우선순위 부스트 기법을 통해서는 기존 시스템보다 최대 22% 단축됨을 보였고 로드 밸런싱 기법을 통해서는 최대 43.31% 단축됨을 보여 제안된 기법의 효용성을 입증하였다.Smartphones on Android platform recently have been come into wide use. However, it is often reported that Android smartphones cannot provide enough interactivity because Android cannot distinguish interactive tasks and non-interactive tasks and they are scheduled with the same priority and preempted. Thus, it occurs poor response time. This paper proposes a framework assisted priority boosting and load balancing for improving interactivity of Android smartphones. The framework assisted priority boosting technique distinguishes the interactive task in the framework level and send the task ID to the kernel. The kernel ensures enough time to process user input by boosting the priority of distinguished task. The load balancing technique minimizes response time of boosted task by migrating tasks disturbing boosted task to other run-queue. The experiment results demonstrate the priority boosting technique reduces response time up to 22% and the load balancing technique along with priority boosting reduces response time up to 43.31% compared to the previous techniques.OAIID:oai:osos.snu.ac.kr:snu2012-01/102/0000004193/8SEQ:8PERF_CD:SNU2012-01EVAL_ITEM_CD:102USER_ID:0000004193ADJUST_YN:YEMP_ID:A005174DEPT_CD:4541CITE_RATE:0FILENAME:(CSTV12-07-09-15).pdfDEPT_NM:전기·컴퓨터공학부EMAIL:[email protected]_YN:NCONFIRM:

안드로이드 기반 스마트폰의 사용자 응답성 향상을 위한 프레임워크 지원 우선순위 부스트 기법

Authors' final versionOAIID:oai:osos.snu.ac.kr:snu2012-01/102/0000004193/9SEQ:9PERF_CD:SNU2012-01EVAL_ITEM_CD:102USER_ID:0000004193ADJUST_YN:NEMP_ID:A005174DEPT_CD:4541CITE_RATE:0FILENAME:12-06-27 정보과학회 - Android_Interactivity.pdfDEPT_NM:전기·컴퓨터공학부EMAIL:[email protected]: