16 research outputs found
Novel Architectures for Offloading and Accelerating Computations in Artificial Intelligence and Big Data
Due to the end of Moore's Law and Dennard Scaling, performance gains in general-purpose architectures have significantly slowed in recent years. While raising the number of cores has been a viable approach for further performance increases, Amdahl's Law and its implications on parallelization also limit further performance gains. Consequently, research has shifted towards different approaches, including domain-specific custom architectures tailored to specific workloads.
This has led to a new golden age for computer architecture, as noted in the Turing Award Lecture by Hennessy and Patterson, which has spawned several new architectures and architectural advances specifically targeted at highly current workloads, including Machine Learning. This thesis introduces a hierarchy of architectural improvements ranging from minor incremental changes, such as High-Bandwidth Memory, to more complex architectural extensions that offload workloads from the general-purpose CPU towards more specialized accelerators. Finally, we introduce novel architectural paradigms, namely Near-Data or In-Network Processing, as the most complex architectural improvements.
This cumulative dissertation then investigates several architectural improvements to accelerate Sum-Product Networks, a novel Machine Learning approach from the class of Probabilistic Graphical Models. Furthermore, we use these improvements as case studies to discuss the impact of novel architectures, showing that minor and major architectural changes can significantly increase performance in Machine Learning applications.
In addition, this thesis presents recent works on Near-Data Processing, which introduces Smart Storage Devices as a novel architectural paradigm that is especially interesting in the context of Big Data. We discuss how Near-Data Processing can be applied to improve performance in different database settings by offloading database operations to smart storage devices. Offloading data-reductive operations, such as selections, reduces the amount of data transferred, thus improving performance and alleviating bandwidth-related bottlenecks.
Using Near-Data Processing as a use-case, we also discuss how Machine Learning approaches, like Sum-Product Networks, can improve novel architectures. Specifically, we introduce an approach for offloading Cardinality Estimation using Sum-Product Networks that could enable more intelligent decision-making in smart storage devices. Overall, we show that Machine Learning can benefit from developing novel architectures while also showing that Machine Learning can be applied to improve the applications of novel architectures
Data-intensive Systems on Modern Hardware : Leveraging Near-Data Processing to Counter the Growth of Data
Over the last decades, a tremendous change toward using information technology in almost every daily routine of our lives can be perceived in our society, entailing an incredible growth of data collected day-by-day on Web, IoT, and AI applications.
At the same time, magneto-mechanical HDDs are being replaced by semiconductor storage such as SSDs, equipped with modern Non-Volatile Memories, like Flash, which yield significantly faster access latencies and higher levels of parallelism. Likewise, the execution speed of processing units increased considerably as nowadays server architectures comprise up to multiple hundreds of independently working CPU cores along with a variety of specialized computing co-processors such as GPUs or FPGAs.
However, the burden of moving the continuously growing data to the best fitting processing unit is inherently linked to todayâs computer architecture that is based on the data-to-code paradigm. In the light of Amdahl's Law, this leads to the conclusion that even with today's powerful processing units, the speedup of systems is limited since the fraction of parallel work is largely I/O-bound.
Therefore, throughout this cumulative dissertation, we investigate the paradigm shift toward code-to-data, formally known as Near-Data Processing (NDP), which relieves the contention on the I/O bus by offloading processing to intelligent computational storage devices, where the data is originally located.
Firstly, we identified Native Storage Management as the essential foundation for NDP due to its direct control of physical storage management within the database. Upon this, the interface is extended to propagate address mapping information and to invoke NDP functionality on the storage device. As the former can become very large, we introduce Physical Page Pointers as one novel NDP abstraction for self-contained immutable database objects.
Secondly, the on-device navigation and interpretation of data are elaborated. Therefore, we introduce cross-layer Parsers and Accessors as another NDP abstraction that can be executed on the heterogeneous processing capabilities of modern computational storage devices. Thereby, the compute placement and resource configuration per NDP request is identified as a major performance criteria. Our experimental evaluation shows an improvement in the execution durations of 1.4x to 2.7x compared to traditional systems. Moreover, we propose a framework for the automatic generation of Parsers and Accessors on FPGAs to ease their application in NDP.
Thirdly, we investigate the interplay of NDP and modern workload characteristics like HTAP. Therefore, we present different offloading models and focus on an intervention-free execution. By propagating the Shared State with the latest modifications of the database to the computational storage device, it is able to process data with transactional guarantees. Thus, we achieve to extend the design space of HTAP with NDP by providing a solution that optimizes for performance isolation, data freshness, and the reduction of data transfers. In contrast to traditional systems, we experience no significant drop in performance when an OLAP query is invoked but a steady and 30% faster throughput.
Lastly, in-situ result-set management and consumption as well as NDP pipelines are proposed to achieve flexibility in processing data on heterogeneous hardware. As those produce final and intermediary results, we continue investigating their management and identified that an on-device materialization comes at a low cost but enables novel consumption modes and reuse semantics. Thereby, we achieve significant performance improvements of up to 400x by reusing once materialized results multiple times
Towards Scalable OLTP Over Fast Networks
Online Transaction Processing (OLTP) underpins real-time data processing in many mission-critical applications, from banking to e-commerce.
These applications typically issue short-duration, latency-sensitive transactions that demand immediate processing.
High-volume applications, such as Alibaba's e-commerce platform, achieve peak transaction rates as high as 70 million transactions per second, exceeding the capacity of a single machine.
Instead, distributed OLTP database management systems (DBMS) are deployed across multiple powerful machines.
Historically, such distributed OLTP DBMSs have been primarily designed to avoid network communication, a paradigm largely unchanged since the 1980s.
However, fast networks challenge the conventional belief that network communication is the main bottleneck.
In particular, emerging network technologies, like Remote Direct Memory Access (RDMA), radically alter how data can be accessed over a network.
RDMA's primitives allow direct access to the memory of a remote machine within an order of magnitude of local memory access.
This development invalidates the notion that network communication is the primary bottleneck.
Given that traditional distributed database systems have been designed with the premise that the network is slow, they cannot efficiently exploit these fast network primitives, which requires us to reconsider how we design distributed OLTP systems.
This thesis focuses on the challenges RDMA presents and its implications on the design of distributed OLTP systems.
First, we examine distributed architectures to understand data access patterns and scalability in modern OLTP systems.
Drawing on these insights, we advocate a distributed storage engine optimized for high-speed networks.
The storage engine serves as the foundation of a database, ensuring efficient data access through three central components: indexes, synchronization primitives, and buffer management (caching).
With the introduction of RDMA, the landscape of data access has undergone a significant transformation.
This requires a comprehensive redesign of the storage engine components to exploit the potential of RDMA and similar high-speed network technologies.
Thus, as the second contribution, we design RDMA-optimized tree-based indexes â especially applicable for disaggregated databases to access remote data efficiently.
We then turn our attention to the unique challenges of RDMA.
One-sided RDMA, one of the network primitives introduced by RDMA, presents a performance advantage in enabling remote memory access while bypassing the remote CPU and the operating system.
This allows the remote CPU to process transactions uninterrupted, with no requirement to be on hand for network communication. However, that way, specialized one-sided RDMA synchronization primitives are required since traditional CPU-driven primitives are bypassed.
We found that existing RDMA one-sided synchronization schemes are unscalable or, even worse, fail to synchronize correctly, leading to hard-to-detect data corruption.
As our third contribution, we address this issue by offering guidelines to build scalable and correct one-sided RDMA synchronization primitives.
Finally, recognizing that maintaining all data in memory becomes economically unattractive, we propose a distributed buffer manager design that efficiently utilizes cost-effective NVMe flash storage.
By leveraging low-latency RDMA messages, our buffer manager provides a transparent memory abstraction, accessing the aggregated DRAM and NVMe storage across nodes.
Central to our approach is a distributed caching protocol that dynamically caches data.
With this approach, our system can outperform RDMA-enabled in-memory distributed databases while managing larger-than-memory datasets efficiently
Cooperative caching for object storage
Data is increasingly stored in data lakes, vast immutable object stores that can be accessed from anywhere in the data center. By providing low cost and scalable storage, today immutable object-storage based data lakes are used by a wide range of applications with diverse access patterns. Unfortunately, performance can suffer for applications that do not match the access patterns for which the data lake was designed. Moreover, in many of today's (non-hyperscale) data centers, limited bisectional bandwidth will limit data lake performance. Today many computer clusters integrate caches both to address the mismatch between application performance requirements and the capabilities of the shared data lake, and to reduce the demand on the data center network. However, per-cluster caching;
i) means the expensive cache resources cannot be shifted between clusters based on demand,
ii) makes sharing expensive because data accessed by multiple clusters is independently cached by each of them,
and
iii) makes it difficult for clusters to grow and shrink if their servers are being used to cache storage.
In this dissertation, we present two novel data-center wide cooperative cache architectures, Datacenter-Data-Delivery Network (D3N) and Directory-Based Datacenter-Data-Delivery Network (D4N) that are designed to be part of the data lake itself rather than part of the computer clusters that use it. D3N and D4N distribute caches across the data center to enable data sharing and elasticity of cache resources where requests are transparently directed to nearby cache nodes. They dynamically adapt to changes in access patterns and accelerate workloads while providing the same consistency, trust, availability, and resilience guarantees as the underlying data lake. We nd that exploiting the immutability of object stores significantly reduces the complexity and provides opportunities for cache management strategies that were not feasible for previous cooperative cache systems for le or block-based storage.
D3N is a multi-layer cooperative cache that targets workloads with large read-only datasets like big data analytics. It is designed to be easily integrated into existing data lakes with only limited support for write caching of intermediate data, and avoiding any global state by, for example, using consistent hashing for locating blocks and making all caching decisions based purely on local information. Our prototype is performant enough to fully exploit the (5 GB/s read) SSDs and (40, Gbit/s) NICs in our system and improve the runtime of realistic workloads by up to 3x. The simplicity of D3N has enabled us, in collaboration with industry partners, to upstream the two-layer version of D3N into the existing code base of the Ceph object store as a new experimental feature, making it available to the many data lakes around the world based on Ceph.
D4N is a directory-based cooperative cache that provides a reliable write tier and a distributed directory that maintains a global state. It explores the use of global state to implement more sophisticated cache management policies and enables application-specific tuning of caching policies to support a wider range of applications than D3N. In contrast to previous cache systems that implement their own mechanism for maintaining dirty data redundantly, D4N re-uses the existing data lake (Ceph) software for implementing a write tier and exploits the semantics of immutable objects to move aged objects to the shared data lake. This design greatly reduces the barrier to adoption and enables D4N to take advantage of sophisticated data lake features such as erasure coding. We demonstrate that D4N is performant enough to saturate the bandwidth of the SSDs, and it automatically adapts replication to the working set of the demands and outperforms the state of art cluster cache Alluxio. While it will be substantially more complicated to integrate the D4N prototype into production quality code that can be adopted by the community, these results are compelling enough that our partners are starting that effort.
D3N and D4N demonstrate that cooperative caching techniques, originally designed for file systems, can be employed to integrate caching into todayâs immutable object-based data lakes. We find that the properties of immutable object storage greatly simplify the adoption of these techniques, and enable integration of caching in a fashion that enables re-use of existing battle tested software; greatly reducing the barrier of adoption. In integrating the caching in the data lake, and not the compute cluster, this research opens the door to efficient data center wide sharing of data and resources
Redesigning Transaction Processing Systems for Non-Volatile Memory
Department of Computer Science and EngineeringTransaction Processing Systems are widely used because they make the user be able to manage
their data more efficiently. However, they suffer performance bottleneck due to the redundant
I/O for guaranteeing data consistency. In addition to the redundant I/O, slow storage device
makes the performance more degraded. Leveraging non-volatile memory is one of the promising
solutions the performance bottleneck in Transaction Processing Systems. However, since the
I/O granularity of legacy storage devices and non-volatile memory is not equal, traditional
Transaction Processing System cannot fully exploit the performance of persistent memory.
The goal of this dissertation is to fully exploit non-volatile memory for improving the performance
of Transaction Processing Systems.
Write amplification between Transaction Processing System is pointed out as a performance
bottleneck. As first approach, we redesigned Transaction Processing Systems to minimize the
redundant I/O between the Transaction Processing Systems. We present LS-MVBT that integrates
recovery information into the main database file to remove temporary files for recovery.
The LS-MVBT also employs five optimizations to reduce the write traffics in single fsync() calls.
We also exploit the persistent memory to reduce the performance bottleneck from slow storage
devices. However, since the traditional recovery method is for slow storage devices, we develop
byte-addressable differential logging, user-level heap manager, and transaction-aware persistence
to fully exploit the persistent memory. To minimize the redundant I/O for guarantee data consistency,
we present the failure-atomic slotted paging with persistent buffer cache.
Redesigning indexing structure is the second approach to exploit the non-volatile memory
fully. Since the B+-tree is originally designed for block granularity, It generates excessive I/O
traffics in persistent memory. To mitigate this traffic, we develop cache line friendly B+-tree
which aligns its node size to cache line size. It can minimize the write traffic. Moreover, with
hardware transactional memory, it can update its single node atomically without any additional
redundant I/O for guaranteeing data consistency. It can also adapt Failure-Atomic Shift and
Failure-Atomic In-place Rebalancing to eliminate unnecessary I/O.
Furthermore, We improved the persistent memory manager that exploit traditional memory
heap structure with free-list instead of segregated lists for small memory allocations to minimize
the memory allocation overhead.
Our performance evaluation shows that our improved version that consider I/O granularity
of non-volatile memory can efficiently reduce the redundant I/O traffic and improve the
performance by large of a margin.ope
Energy Measurements of High Performance Computing Systems: From Instrumentation to Analysis
Energy efficiency is a major criterion for computing in general and High Performance Computing in particular. When optimizing for energy efficiency, it is essential to measure the underlying metric: energy consumption. To fully leverage energy measurements, their quality needs to be well-understood. To that end, this thesis provides a rigorous evaluation of various energy measurement techniques. I demonstrate how the deliberate selection of instrumentation points, sensors, and analog processing schemes can enhance the temporal and spatial resolution while preserving a well-known accuracy. Further, I evaluate a scalable energy measurement solution for production HPC systems and address its shortcomings.
Such high-resolution and large-scale measurements present challenges regarding the management of large volumes of generated metric data. I address these challenges with a scalable infrastructure for collecting, storing, and analyzing metric data. With this infrastructure, I also introduce a novel persistent storage scheme for metric time series data, which allows efficient queries for aggregate timelines.
To ensure that it satisfies the demanding requirements for scalable power measurements, I conduct an extensive performance evaluation and describe a productive deployment of the infrastructure.
Finally, I describe different approaches and practical examples of analyses based on energy measurement data. In particular, I focus on the combination of energy measurements and application performance traces. However, interweaving fine-grained power recordings and application events requires accurately synchronized timestamps on both sides. To overcome this obstacle, I develop a resilient and automated technique for time synchronization, which utilizes crosscorrelation of a specifically influenced power measurement signal. Ultimately, this careful combination of sophisticated energy measurements and application performance traces yields a detailed insight into application and system energy efficiency at full-scale HPC systems and down to millisecond-range regions.:1 Introduction
2 Background and Related Work
2.1 Basic Concepts of Energy Measurements
2.1.1 Basics of Metrology
2.1.2 Measuring Voltage, Current, and Power
2.1.3 Measurement Signal Conditioning and Analog-to-Digital Conversion
2.2 Power Measurements for Computing Systems
2.2.1 Measuring Compute Nodes using External Power Meters
2.2.2 Custom Solutions for Measuring Compute Node Power
2.2.3 Measurement Solutions of System Integrators
2.2.4 CPU Energy Counters
2.2.5 Using Models to Determine Energy Consumption
2.3 Processing of Power Measurement Data
2.3.1 Time Series Databases
2.3.2 Data Center Monitoring Systems
2.4 Influences on the Energy Consumption of Computing Systems
2.4.1 Processor Power Consumption Breakdown
2.4.2 Energy-Efficient Hardware Configuration
2.5 HPC Performance and Energy Analysis
2.5.1 Performance Analysis Techniques
2.5.2 HPC Performance Analysis Tools
2.5.3 Combining Application and Power Measurements
2.6 Conclusion
3 Evaluating and Improving Energy Measurements
3.1 Description of the Systems Under Test
3.2 Instrumentation Points and Measurement Sensors
3.2.1 Analog Measurement at Voltage Regulators
3.2.2 Instrumentation with Hall Effect Transducers
3.2.3 Modular Instrumentation of DC Consumers
3.2.4 Optimal Wiring for Shunt-Based Measurements
3.2.5 Node-Level Instrumentation for HPC Systems
3.3 Analog Signal Conditioning and Analog-to-Digital Conversion
3.3.1 Signal Amplification
3.3.2 Analog Filtering and Analog-To-Digital Conversion
3.3.3 Integrated Solutions for High-Resolution Measurement
3.4 Accuracy Evaluation and Calibration
3.4.1 Synthetic Workloads for Evaluating Power Measurements
3.4.2 Improving and Evaluating the Accuracy of a Single-Node Measuring System
3.4.3 Absolute Accuracy Evaluation of a Many-Node Measuring System
3.5 Evaluating Temporal Granularity and Energy Correctness
3.5.1 Measurement Signal Bandwidth at Different Instrumentation Points
3.5.2 Retaining Energy Correctness During Digital Processing
3.6 Evaluating CPU Energy Counters
3.6.1 Energy Readouts with RAPL
3.6.2 Methodology
3.6.3 RAPL on Intel Sandy Bridge-EP
3.6.4 RAPL on Intel Haswell-EP and Skylake-SP
3.7 Conclusion
4 A Scalable Infrastructure for Processing Power Measurement Data
4.1 Requirements for Power Measurement Data Processing
4.2 Concepts and Implementation of Measurement Data Management
4.2.1 Message-Based Communication between Agents
4.2.2 Protocols
4.2.3 Application Programming Interfaces
4.2.4 Efficient Metric Time Series Storage and Retrieval
4.2.5 Hierarchical Timeline Aggregation
4.3 Performance Evaluation
4.3.1 Benchmark Hardware Specifications
4.3.2 Throughput in Symmetric Configuration with Replication
4.3.3 Throughput with Many Data Sources and Single Consumers
4.3.4 Temporary Storage in Message Queues
4.3.5 Persistent Metric Time Series Request Performance
4.3.6 Performance Comparison with Contemporary Time Series Storage Solutions
4.3.7 Practical Usage of MetricQ
4.4 Conclusion
5 Energy Efficiency Analysis
5.1 General Energy Efficiency Analysis Scenarios
5.1.1 Live Visualization of Power Measurements
5.1.2 Visualization of Long-Term Measurements
5.1.3 Integration in Application Performance Traces
5.1.4 Graphical Analysis of Application Power Traces
5.2 Correlating Power Measurements with Application Events
5.2.1 Challenges for Time Synchronization of Power Measurements
5.2.2 Reliable Automatic Time Synchronization with Correlation Sequences
5.2.3 Creating a Correlation Signal on a Power Measurement Channel
5.2.4 Processing the Correlation Signal and Measured Power Values
5.2.5 Common Oversampling of the Correlation Signals at Different Rates
5.2.6 Evaluation of Correlation and Time Synchronization
5.3 Use Cases for Application Power Traces
5.3.1 Analyzing Complex Power Anomalies
5.3.2 Quantifying C-State Transitions
5.3.3 Measuring the Dynamic Power Consumption of HPC Applications
5.4 Conclusion
6 Summary and Outloo
Characterization and optimization of network traffic in cortical simulation
Considering the great variety of obstacles the Exascale systems
have to face in the next future, a deeper attention will be given in this thesis
to the interconnect and the power consumption.
The data movement challenge involves the whole hierarchical organization
of components in HPC systems â i.e. registers, cache, memory, disks.
Running scientific applications needs to provide the most effective methods
of data transport among the levels of hierarchy. On current petaflop systems,
memory access at all the levels is the limiting factor in almost all applications.
This drives the requirement for an interconnect achieving adequate rates of
data transfer, or throughput, and reducing time delays, or latency, between
the levels.
Power consumption is identified as the largest hardware research challenge.
The annual power cost to operate the system would be above 2.5 B$
per year for an Exascale system using current technology. The research for alternative
power-efficient computing device is mandatory for the procurement
of the future HPC systems.
In this thesis, a preliminary approach will be offered to the critical process of
co-design. Co-desing is defined as the simultaneos design of both hardware
and software, to implement a desired function. This process both integrates
all components of the Exascale initiative and illuminates the trade-offs that
must be made within this complex undertaking