In the past decades, advances in the speed of commodity CPUs have far out-paced advances in memory latency. Mainmemory access is therefore increasingly a performance bottleneck for many computer applications, including HPC embedded systems. This translates into unprecedented memory performance requirements in critical systems that the commonly used DRAM memories struggle to provide. High-Bandwidth Memory (HBM) can satisfy these requirements offering high bandwidth, low power and highintegration capacity features. However, it remains unclear whether the predictability and isolation properties of HBM are compatible with the requirements of critical embedded systems. In our research, a deep characterization of the HBM for its use in MEEP applications is performed

Cervero, Teresa

Martorell Bofill, Xavier

Perdomo Hourné, Elias A.

English

UPCommons. Portal del coneixement obert de la UPC

HBM, Present and Future of HPC based on FPGAs Elias A. Perdomo Hourné*§1, Teresa Cervero*2, Xavier Martorell*§3 *Barcelona Supercomputing Center (BSC), MEEP, Til·lers c/Passeig dels Til·lers, 15. 08034 Barcelona (Spain) §Universitat Politécnica de Catalunya (UPC), Computer Architecture Dept., C6, c/Jordi Girona, 1-3. 08034 Barcelona (Spain) E-mails: 1elias.perdomo@bsc.es, 2teresa.cervero@bsc.es, 3xavier.martorell@bsc.es Keywords— FPGAs, HBM, Parallelism, Performance Optimization and Embedded systems.  EXTENDED ABSTRACT In the past decades, advances in the speed of commodity CPUs have far out-paced advances in memory latency. Main-memory access is therefore increasingly a performance bottleneck for many computer applications, including HPC embedded systems. This translates into unprecedented memory performance requirements in critical systems that the commonly used DRAM memories struggle to provide.  High-Bandwidth Memory (HBM) can satisfy these requirements offering high bandwidth, low power and high-integration capacity features. However, it remains unclear whether the predictability and isolation properties of HBM are compatible with the requirements of critical embedded systems. In our research, a deep characterization of the HBM for its use in MEEP applications is performed. A. Introduction Custom hardware – from workstations to PCs– has experienced tremendous improvements in the past decades. However, while the speed of commercial microprocessors has increased approximately 70% every year, the speed of commodity DRAM has improved by only around 50% over the whole past decade. As a result, computer systems are experiencing difficulties in achieving high processing efficiency [1]. The traditional approach for boosting performance in systems, particularly those at the edge where huge amounts of data need to be processed locally or regionally; consists on adding more computational capability into a chip and bringing more memory on-chip. But that approach no longer scales since we are now getting to the boundaries of the trifecta of von Neumann architectures, Moore’s Law and Dennard scaling. Consequently, engineers have begun focusing on solving the bottleneck between processors and memories by turning out new architectural designs at a rate no one would have anticipated before.  An alternative approach, based on recent technological advances, is moving processing elements closer to, or even into the memory. This solution looks for avoiding the penalty for replicating processing elements, which provides an acceptable tradeoff. When utilizing wide short buses (HBM being the most common example), designers avoid the penalty of going outside the die for accessing to memory and recover some of the performance tradeoff. This trend is followed by Xilinx, one of the two market giants in the area of FPGAs and the leader in adaptive computing. Xilinx is firmly committed to a transition to HBM memory as a solution to memory bottlenecks, as demonstrated in recent years. In October 2018, Xilinx launched the Alveo U200 with no HBM memory. Only 1-month after, in November 2018 the new Alveo U280 already included 8GB of HBM2 and halved DDR capacity. Their last Alveo Data center card release, the Alveo U55C, doubled HBM capacity and rescinded the DDR memory banks use [2].    B. HBM Features and Standards HBM stands for High Bandwidth Memory, a type of memory interface used in 3D-stacked DRAM (Dynamic Random Access Memory) in GPUs, servers, high-performance computing and networking and client space.  HBM technology works by vertically stacking memory chips one on top of another in order to shorten how far data has to move, while allowing for smaller form factors. Additionally, with two 128-bit channels per die, HBM memory bus is wider than the one present in other types of DRAM memory. Stacked memory chips are connected through silicon vias (TSVs) and micro-bumps and connected via the interposer, rather than on-chip, to ensure faster accesses [3]. Several versions of the HBM and future standard updates are shown in the Table 1. TABLE 1 HBM STANDARDS EVOLUTION HBM Specification HBM HBM2 / HBM2E (Current) HBM3 (Upcoming) Max Pin Transfer Rate 1 Gb/s 3.2 Gb/s ? Max Capacity 4 GB 24 GB 64GB Max Bandwidth 128 GB/s 410 GB/s 512 GB/s The current HBM2 was adopted as a JEDEC standard in December 2018 and the specifications were updated again in early 2020. It allows a bandwidth of 3.2 GB/s per pin with a max capacity of 24GB per stack (2GB per die across 12 dies per stack) and max bandwidth of 410 GB/s, delivered across a 1,024-bit memory interface separated by 8 unique channels on each stack [4]. It is therefore not surprising that HBM can overcome all the challenges of DRAM to achieve high bandwidth. Even more impressive is that it does so, while its low speed/pin consumption also improves power efficiency, as can be seen in Fig 1.  Fig. 1  Bandwidth and power consumption comparison for different memories. C. HBM in the Alveo U280 Having these precepts for the HBM, we decided to perform a first analysis of HBM from a real-time systems perspective, by exploring how it is built in the Alveo U280 (with a common topology as the Alveo U55C). We focus on the HBM device in order to capture its architectural features and functionalities, which can affect timing predictability such as device access behavior, timing properties and performance metrics. The HBM controller in the Alveo U280 has 2 stacks of 4GB each, with 8 channels and 16 pseudo-channels and 16 AXI Ports (Fig 2). These ports can access its own memory mapped 75or any other memory if the global addressing is activated but having a Lateral AXI Switch Access Throughput Loss in each micro-switch “domain change”[5].  Fig. 2 Alveo U280 HBM topology. D. AIT MatMul implementations accessing the Alveo HBM In MEEP we are enhancing the Accelerator Integration Tool (AIT), a tool in development that automatizes the bitstream generation process custom designs based on several predefined design templates using the Xilinx toolchain. Based on a previous matrix multiplication (MatMul) implementation we replaced DDR by HBM and measured performance. To accomplish that we used a MatMul implementation which has 4 matmul_block accelerators which previously accessed the 2 DDR banks in parallel and now can access 4 HBM pseudo-channels in parallel (Fig 3).  Fig. 3 MatMul implementation with 4 matmul_blocks and HBM. Table 2 shows that HBM, even with a port data width half of the DDR one, maintains a comparable performance with the DDR using the double of HBM pseudo-channels than DDR banks. Even whether MatMul is a great proof of concept and HBM can use up to 16 times the number of banks; it has its limitations because of the fact of being a CPU bounded application. Thus, we would need a memory bounded application or to stress memory accesses more thoroughly.  TABLE 2 DDR PERFORMANCE METRICS VS HBM COMPARABLE ONES Implementation Performance (GFlops) matmul_ait_u280_DDR_PW_512 287.043793 matmul_ait_u280_HBM_PW_256 255.816208 E. Custom Traffic Generators accessing the Alveo HBM With the purpose of fully characterize the HBM, a Traffic Generator set of test-bench was created. Due to hardware implementations are the only way to stress the HBM memory efficiently, we designed our own Traffic Generators to control memory access pattern generation and to simplifying the debugging task. It is highly recommended to have a common set of tests for the performance, bandwidth and latency comparison between DDR and HBM to start confirming HBM capabilities. The following table shows the results of a comparison between DDR and HBM with similar implementations (Row-Bank-Column address mapping scheme, Single row accesses and No burst). TABLE 3 DDR PERFORMANCE METRICS VS HBM COMPARABLE ONES  Implementations  DDR_TG 1 DDR DDR_TG 2 DDRs HBM_TG 32 pseudo-channels Write Latency 5 clock cycles  5 clock cycles  14 clock cycles  Read Latency 24 clock cycles  24 clock cycles  48 clock cycles  Read Throughput 0.229 GB/s  0.457 GB/s  50 GB/s  F. Conclusion and Future Enhancement Based on a similar set of test benches HBM proved to have a worse latency than the DDR. Nevertheless, as expected, the throughput performance was more than 25 times better when using all the 32 pseudo-channels in the HBM in parallel than when using the 2 memory banks present in the DDR in the Alveo U280. In order to fully characterize the HBM module, more experiments are being carried out to measure the Lateral AXI Switch Access Throughput Loss, the congestion in a channel or in its memory controller and the impact of the IDs and the reordering option. References   [1] T. C. Mowry, “Tolerating latency through software-controlled data prefetching,” PhD Thesis, Stanford University, 1994. [2] Xilinx Inc., “ALVEOTM Product Selection Guide Datasheet by Xilinx Inc.  XMP451 (v1.7),” xilinx.com, 2021. https://www.xilinx.com/support/documentation/selection-guides/alveo-product-selection-guide.pdf (accessed Mar. 15, 2022). [3] Z. Wang, H. Huang, J. Zhang, and G. Alonso, “Benchmarking high bandwidth memory on FPGAs,” arXiv preprint arXiv:2005.04324, 2020. [4] H. Jun et al., “HBM (high bandwidth memory) dram technology and architecture,” in 2017 IEEE International Memory Workshop (IMW), 2017, pp. 1–4. [5] Xilinx Inc., “AXI High Bandwidth Memory Controller v1.0 LogiCORE IP Product Guide PG276 (v1.0),” xilinx.com, Aug. 06, 2021. https://www.xilinx.com/support/documentation/ip_documentation/hbm/v1_0/pg276-axi-hbm.pdf (accessed Mar. 15, 2022). Author biography  Elias A. Perdomo Hourné was born in Havana, Cuba, in 1988. He received the B.Eng. degree in Automation Engineering in 2012, and the M.Sc. degree in Digital Systems in 2018 both from the Technological University of Havana, Havana, Cuba. In addition, he received the M.Sc. Advanced Microelectronic Systems Engineering from the University of Bristol, Bristol, United Kingdom in 2019. Since October 2020, he has been working with the MEEP group of the Barcelona Supercomputing Center (BSC) as Research Engineer as well as a PhD student at the Compuer Architecture Department of the Universitat Politecnica de Catalunya (UPC), Barcelona, Spain, under the supervision of Teresa Cervero and Xavier Martorell. His current research interests include Embedded Systems, RTL and RISCV SoC design, FPGAs, Heterogeneous Computing, automatic design generation and memory management for HPC and programming models. 76

HBM, present and future of HPC based on FPGAs

https://upcommons.upc.edu/bitstream/2117/383927/1/9BSCDS_32_HBM%20Present%20and%20Future.pdf

HBM, present and future of HPC based on FPGAs

Abstract

Similar works

Full text

Available Versions

UPCommons. Portal del coneixement obert de la UPC