Cole, Murray

Dreslinski, Ronald

Kaszyk, Kuba

O'Boyle, Michael F P

Pal, Subhankar

English

Early-stage design-space exploration (DSE) and performance/power evaluation of large-scale heterogeneous systems, such as those composed of chip multiprocessors (CMPs) coupled with fixed-function logic, have gained importance in the dark silicon era. Prior work has explored trace-based simulation techniques, that offer good trade-offs between simulation accuracy and speed, to simulate CMPs with up to 100s of threads, and similar methods are used for accelerators. However, there is lack of a unified framework for fast simulation of large-scale heterogeneous systems. In this work, we propose HETSIM, a trace-driven framework for estimating performance and power of accelerators, CMPs and heterogeneous systems with 1000s of cores. We present results on (i) a CMP and (ii) a heterogeneous accelerator, demonstrating average speed ups of 5.5× and 16.1× over detailed gem5 models with deviations in simulated time and power consumption of 4.6-28.1% and 1.7-3.3%, respectively

Edinburgh Research Explorer

     Edinburgh Research Explorer                                      HETSIM: Simulating Large-Scale Heterogeneous Systems usinga Trace-driven, Synchronization and Dependency-AwareFrameworkCitation for published version:Pal, S, Kaszyk, K, Cole, M, O'Boyle, MFP & Dreslinski, R 2020, 'HETSIM: Simulating Large-ScaleHeterogeneous Systems using a Trace-driven, Synchronization and Dependency-Aware Framework', Paperpresented at Workshop on Modeling & Simulation of Systems and Applications 2020, Virtual Workshop,12/08/20 - 12/08/20.Link:Link to publication record in Edinburgh Research ExplorerDocument Version:Peer reviewed versionGeneral rightsCopyright for the publications made accessible via the Edinburgh Research Explorer is retained by the author(s)and / or other copyright owners and it is a condition of accessing these publications that users recognise andabide by the legal requirements associated with these rights.Take down policyThe University of Edinburgh has made every reasonable effort to ensure that Edinburgh Research Explorercontent complies with UK legislation. If you believe that the public display of this file breaches copyright pleasecontact openaccess@ed.ac.uk providing details, and we will remove access to the work immediately andinvestigate your claim.Download date: 23. Jul. 2021HETSIM: Simulating Large-Scale Heterogeneous Systems using a Trace-driven, Synchronization andDependency-Aware Framework (Advances in ModSim Implementation)Subhankar Pal∗, Kuba Kaszyk†, Murray Cole†, Michael O’Boyle†, Ronald Dreslinski∗ — Universities of ∗Michigan and †EdinburghINTRODUCTION – Early-stage design-space exploration (DSE)and performance/power evaluation of large-scale heterogeneoussystems, such as those composed of chip multiprocessors (CMPs)coupled with fixed-function logic, have gained importance in thedark silicon era. Prior work has explored trace-based simulationtechniques, that offer good trade-offs between simulation accuracyand speed, to simulate CMPs with up to 100s of threads, andsimilar methods are used for accelerators. However, there islack of a unified framework for fast simulation of large-scaleheterogeneous systems. In this work, we propose HETSIM, atrace-driven framework for estimating performance and powerof accelerators, CMPs and heterogeneous systems with 1000s ofcores. We present results on (i) a CMP and (ii) a heterogeneousaccelerator, demonstrating average speedups of 5.5× and 16.1×over detailed gem5 models with deviations in simulated time andpower consumption of 4.6-28.1% and 1.7-3.3%, respectively.PROPOSED APPROACH – Figure 1 summarizes the approachused in HETSIM. The first step involves executing a multithreadedversion of the application on a native multiprocessing systemto verify correctness. Next, the application code is instrumentedwith trace-generating function calls, and run through the nativesystem to generate trace files – one per thread/core/processingelement (PE) in the target architecture. Instrumentation is a one-time overhead for DSE of the target’s memory subsystem. Tofurther reduce the burden on the end-user, we provide an LLVM-based compiler pass, which automatically identifies target-specificintrinsics and memory accesses, and re-compiles the code withinstrumentation. Lastly, the compute units in a gem5 model of thetarget are swapped with trace replay engines (TREs) that executethe “instructions” in their corresponding trace files, according tothe rates at which the cores/PEs would issue them.CAPTURED OPERATIONS – HETSIM captures broad classes ofoperations that appear in heterogeneous systems. Within a region-of-interest in the application, HETSIM captures:Memory Operations: Memory accesses are captured with highfidelity using LD/ST tokens followed by their address. In addition,a dependency list for memory operations allows HETSIM to modelflexible target architectures, such as complex in-order (InO) coresthat support prefetch instructions and multiple outstanding loads.Computation: HETSIM encodes all non-memory instructions,such as arithmetic ops, branches, etc. using the STALL token.Consecutive STALLs in a trace file are coalesced for faster tracereplay and reduced trace storage. A TRE scales the number ofSTALLs based on the level of acceleration in the target.Communication: Tokens such as PUSH and POP followed by thecore ID are used to perform buffered pushes and pops of data.These are universal primitives for fast PE-to-PE communicationin accelerators, such as systolic arrays. SIGNAL and WAIT arecommon primitives employed for handshaking between differ-ent hardware blocks. BARRIER is another useful primitive forsynchronization across a set of PEs. Finally, LOCK/UNLOCK, inaddition to the signaling and barrier synchronization primitives,are used to model Pthreads calls for CMP systems.EVALUATION – We provide a summary of our evaluation ofHETSIM for the DSE of two target systems.Target 1: In-Order Manycore CMP – We evaluate a 32-128 coreCMP system with shared 16 kB L1 (1 slice/core) and 256 kB L2 (4slices) caches, executing matrix multiplication (GeMM). Figure 2shows the execution-time deviation and speedup of HETSIM overa gem5 model that uses MinorCPU cores. We have also simulatedthis system with up to 4096 cores using HETSIM on a 64-coreThreadripper 2990WX CPU with 128 GB of RAM (not shown).Target 2: Heterogeneous Sparse Matrix Multiplication (SpMM)Accelerator – We deployed HETSIM for scalability studies on aheterogeneous SpMM accelerator prototype chip [1] that uses atiled architecture – 8 tiles with 4 custom PEs and 2 Arm Cortex-Mcores per tile. The algorithm is split into two phases – a multiplyphase that uses caches and a merge phase that uses scratchpads.Not for public release. Handle internal to DARPA/MTO and SDH performers only. Exceptions may be approved by the SDH Program Manager. 69• Implement a multithreaded version of the application using Pthreads• Run it on a nativ  multiprocessing system to verify functional correctness• Next, the application code is instrum nted wi h trace-generating function calls• The instrumentation can be done manually or using an LLVM compiler pass • Run the traces through the native system to generate trace files, one per GPE and LCP• Swap the detailed cores in Transmuter’sarchitectural model with trace replay engines that execute the “instructions” in their corresponding trace filesHetSim: A Trace-Driven Simulation ApproachTrace File 0Target Architecture Model1: Functional Verification Multithreaded Application CodeCore 0Core 1Core N…Trace Instrumentation (automatic/manual)Instrumented Application CodeCore 0Core 1Core N… …Functionally Verified CodeSMP HardwareSMP HardwareCompute SubsystemPerformance + Power EstimatesMemory Subsys-tem2: Trace Generation3: Simulation Target Architecture ModelTrace Replay Engine SubsystemMemory Subsys-temTrace FileFig. 1. Trace-based simulation approach deployed in the proposed HETSIM.05101520250%20%40%60%80%100%512 1024 2048Speedup over gem5-SEDeviation over gem5-SEMatrix Dimension (fixed density of 6.4%)Deviation Speedup02468100%20%40%60%80%100%32 64 128 32 64 128Speedup over gem5-SEDeviation over gem5-SE# CoresDeviation SpeedupMat Dim.=256 Mat Dim.=512Fig. 2. Timing accuracy and speedup of HETSIM over gem5 models of a CMPsystem executing GeMM (left) and a heterogeneous SpMM accelerator (right).Figure 2 (right) plots the accuracy and speedup of HETSIM over agem5 core od l of the accelerator that uses MinorCPUs for thePEs as well as the Arm cores. The average deviation of HETSIMfrom the measured chip performance is 34.6% for multiply and4.0% for merge in our scalability studies (see Fig. 5 in [1]).Overall, the timing and power deviations range from 4.6-28.1%to 1.7-3.3%, respectively, over detailed gem5 models, with aspeedup of 4.7-17.3× (up to 8.6× for non-DSE experiments). Notethat hand-annotated traces were used for Target 2, and automatingtrace instrumentation for accelerators is work-in-progress.TABLE ICOMPARISON OF HETSIM WITH PRIOR TRACE-DRIVEN FRAMEWORKS.Work ISA Thread-ing Exec.Sim.LimitSynchro-nizationTargetPlatformTrace Gen./ReplayElasticTraces [2] Agnostic Single OoO - - CMP gem5/gem5ElasticSim-MATE [3] Armv7/8 Multi OoO 128 OpenMP CMP gem5/gem5Synchro-Trace [4] Agnostic Multi InO 64Pthreads/OpenMP CMP Native/gem5SST/Macro [5] N/A Multi OoO 1000 MPIMulti-CMP Sys. Native/CustomHETSIM Agnostic Multi InO 4096 Pthreads/CustomCMP/accel./hetero. Native/gem5RELATED WORK – A few works have explored similar trace-driven methodologies as HETSIM, albeit only for simulatingrelatively small-scale CMP systems. We provide a qualitativecomparison over these work in Table I. One close work, Synchro-Trace [4], uses dependency and synchronization aware traces forCMP systems with simple in-order cores. In contrast, HETSIMis applicable to CMPs, accelerators and heterogeneous targets, aswell as offers flexibility to simulate complex in-order cores.DISCUSSION – HETSIM addresses the issue of simulating hetero-geneous systems with 1000s of cores within practical constraints.We have used HETSIM to evaluate the impact of bandwidth andclock speed scaling on a heterogeneous accelerator. We are in theprocess of using the same for a reconfigurable system in a multi-University project. The current effort is focused on automatic traceinstrumentation, to make it more accurate and robust and supportheterogeneous systems.REFERENCES[1] S. Pal et al., “A 7.3 M Output Non-Zeros/J Sparse Matrix-Matrix MultiplicationAccelerator using Memory Reconfiguration in 40 nm,” VLSI, 2019.[2] R. Jagtap et al., “Exploring system performance using elastic traces: Fast,accurate and portable,” SAMOS, 2017.[3] A. Nocua et al., “ElasticSimMATE: A fast and accurate gem5 trace-drivensimulator for multicore systems,” ReCoSoC, 2017.[4] K. Sangaiah et al., “SynchroTrace: Synchronization-Aware architecture-Agnostic traces for lightweight multicore simulation of CMP and HPCworkloads,” TACO, 2018.[5] C. L. Janssen et al., “A simulator for large-scale parallel computer architectures,”IJDST, 2010.

HETSIM: Simulating Large-Scale Heterogeneous Systems using a Trace-driven, Synchronization and Dependency-Aware Framework

https://www.research.ed.ac.uk/files/179706984/HETSIM_PAL_DOA30062020_AFV.pdf

HETSIM: Simulating Large-Scale Heterogeneous Systems using a Trace-driven, Synchronization and Dependency-Aware Framework

Abstract

Similar works

Full text

Available Versions

Edinburgh Research Explorer