85 research outputs found
High Performance Computing using Infiniband-based clusters
L'abstract è presente nell'allegato / the abstract is in the attachmen
Chiller: Contention-centric Transaction Execution and Data Partitioning for Modern Networks
Distributed transactions on high-overhead TCP/IP-based networks were
conventionally considered to be prohibitively expensive and thus were avoided
at all costs. To that end, the primary goal of almost any existing partitioning
scheme is to minimize the number of cross-partition transactions. However, with
the new generation of fast RDMA-enabled networks, this assumption is no longer
valid. In fact, recent work has shown that distributed databases can scale even
when the majority of transactions are cross-partition. In this paper, we first
make the case that the new bottleneck which hinders truly scalable transaction
processing in modern RDMA-enabled databases is data contention, and that
optimizing for data contention leads to different partitioning layouts than
optimizing for the number of distributed transactions. We then present Chiller,
a new approach to data partitioning and transaction execution, which aims to
minimize data contention for both local and distributed transactions. Finally,
we evaluate Chiller using various workloads, and show that our partitioning and
execution strategy outperforms traditional partitioning techniques which try to
avoid distributed transactions, by up to a factor of 2
Master of Science
thesisEfficient movement of massive amounts of data over high-speed networks at high throughput is essential for a modern-day in-memory storage system. In response to the growing needs of throughput and latency demands at scale, a new class of database systems was developed in recent years. The development of these systems was guided by increased access to high throughput, low latency network fabrics, and declining cost of Dynamic Random Access Memory (DRAM). These systems were designed with On-Line Transactional Processing (OLTP) workloads in mind, and, as a result, are optimized for fast dispatch and perform well under small request-response scenarios. However, massive server responses such as those for range queries and data migration for load balancing poses challenges for this design. This thesis analyzes the effects of large transfers on scale-out systems through the lens of a modern Network Interface Card (NIC). The present-day NIC offers new and exciting opportunities and challenges for large transfers, but using them efficiently requires smart data layout and concurrency control. We evaluated the impact of modern NICs in designing data layout by measuring transmit performance and full system impact by observing the effects of Direct Memory Access (DMA), Remote Direct Memory Access (RDMA), and caching improvements such as Intel® Data Direct I/O (DDIO). We discovered that use of techniques such as Zero Copy yield around 25% savings in CPU cycles and a 50% reduction in the memory bandwidth utilization on a server by using a client-assisted design with records that are not updated in place. We also set up experiments that underlined the bottlenecks in the current approach to data migration in RAMCloud and propose guidelines for a fast and efficient migration protocol for RAMCloud
- …