Department of Computer Science and EngineeringThe data analytics frameworks have evolved along with the growing amount of data. There
have been numerous efforts to improve the performance of the data analytics frameworks in-
cluding MapReduce frameworks and NoSQL and NewSQL databases. These frameworks have
various target workloads and their own characteristicshowever, there is common ground as a
data analytics framework. Emerging hardware such as graphics processing units and persistent
memory is expected to open up new opportunities for such commonality. The goal of this dis-
sertation is to leverage emerging hardware to improve the performance of the data analytics
frameworks.
First, we design and implement EclipseMR, a novel MapReduce framework that efficiently
leverages an extensive amount of memory space distributed among the machines in a cluster.
EclipseMR consists of a decentralized DHT-based file system layer and an in-memory cache layer.
The in-memory cache layer is designed to store both local and remote data while balancing the
load between the servers with proposed Locality-Aware Fair (LAF) job scheduler. The design
of EclipseMR is easily extensible with emerging hardwareit can adopt persistent memory as a
primary storage layer or cache layer, or it can adopt GPU to improve the performance of map
and reduce functions. Our evaluation shows that EclipseMR outperforms Hadoop and Spark for
various applications.
Second, we propose B 3 -tree and Cache-Conscious Extendible Hashing (CCEH) for the persis-
tent memory. The fundamental challenge to design a data structure for the persistent memory is
to guarantee consistent transition with 8-bytes of fine-grained atomic write with minimum cost.
B 3 -tree is a fully persistent hybrid indexing structure of binary tree and B+-tree that benefits
from the strength of both in-memory index and block-based index, and CCEH is a variant of
extendible hashing that introduces an intermediate layer between directory and buckets to fully
benefit from a cache-sized bucket while minimizing the size of the directory. Both of the data
structures show better performance than the corresponding state-of-the-art techniques.
Third, we develop a data parallel tree traversal algorithm, Parallel Scan and Backtrack
(PSB), for k-nearest neighbor search problem on the GPU. Several studies have been proposed
to improve the performance of the query by leveraging GPU as an acceleratorhowever, most
of the works focus on the brute-force algorithms. In this work, we overcome the challenges of
traversing multi-dimensional hierarchical indexing structure on the GPU such as tiny shared
memory and runtime stack, irregular memory access pattern, and warp divergence problem.
Our evaluation shows that our data parallel PSB algorithm outperforms both the brute-force
algorithm and the traditional branch and bound algorithm.clos
