Castellana, Vito Giovanni

Ferrandi, Fabrizio

Lattuada, Marco

Minutoli, Marco

Tumeo, Antonino

CASTELLANA, VITO GIOVANNI

TUMEO, ANTONINO

LATTUADA, MARCO

FERRANDI, FABRIZIO

Archivio istituzionale della ricerca - Politecnico di Milano

Enabling the High Level Synthesis of Data Analytics AcceleratorsMarco Minutoli, Vito Giovanni Castellana, Antonino Tumeo, Marco Lattuada, FabrizioFerrandiMarco Minutoli, Vito Giovanni Castellana, Antonino Tumeo, Marco Lattuada, and Fabrizio Ferrandi. En-abling the high level synthesis of data analytics accelerators. In Proceedings of the Eleventh IEEE/ACM/IFIPInternational Conference on Hardware/Software Codesign and System Synthesis, CODES ’16, pages 15:1–15:3, New York, NY, USA, 2016. ACMThe final pubblication is available via http://dx.doi.org/10.1145/2968456.2976764c©ACM, 2016. This is the author’s version of the work. It is posted here by permission of ACM foryour personal use. Not for redistribution. The definitive version was published in Proceedings of theEleventh IEEE/ACM/IFIP International Conference on Hardware/Software Codesign and System Syn-thesis http://doi.acm.org/10.1145/2968456.2976764Enabling the High Level Synthesis of Data AnalyticsAcceleratorsMarco Minutoli, Vito GiovanniCastellana, Antonino TumeoHigh Performance ComputingPacific Northwest National Laboratory99352 Richland, WA, USA{marco.minutoli, vitoGiovanni.castellana,antonino.tumeo}@pnnl.govMarco Lattuada, Fabrizio FerrandiDipartimento di Elettronica, Informazione eBioingegneriaPolitecnico di Milano20132 Milano, Italy{marco.lattuada,fabrizio.ferrandi}@polimi.itABSTRACTConventional High Level Synthesis (HLS) tools mainly tar-get compute intensive kernels typical of digital signal pro-cessing applications. We are developing techniques and ar-chitectural templates to enable HLS of data analytics appli-cations. These applications are memory intensive, presentfine-grained, unpredictable data accesses, and irregular, dy-namic task parallelism. We discuss an architectural tem-plate based around a distributed controller to efficiently ex-ploit thread level parallelism. We present a memory in-terface that supports parallel memory subsystems and en-ables implementing atomic memory operations. We intro-duce a dynamic task scheduling approach to efficiently ex-ecute heavily unbalanced workload. The templates are val-idated by synthesizing queries from the Lehigh UniversityBenchmark (LUBM), a well know SPARQL benchmark.1. INTRODUCTIONData Analytics applications, such as graph databases, of-ten employ pointer or linked list-based data structures that,although convenient to represents dynamically changing re-lationships among the data elements, induce an irregular be-havior [7]. These data structures, in fact, allows spawningmany concurrent activities, but present many unpredictable,fine-grained, data accesses, and require many synchroniza-tion operations. Partitioning the datasets without gener-ating load imbalance is also very difficult. Conventionalgeneral-purpose architectures are optimized for locality andreduced access latency, and do not cope well with theseworkloads, making application-specific accelerators (imple-mented, for example, on Field Programmable Gate Arrays- FPGAs) an appealing solution [6]. However, conventionalHigh Level Synthesis (HLS) flows traditionally perform wellwith compute intensive workloads (i.e., digital signal pro-cessing) that mainly expose instruction level parallelism, andcan be easily partitioned across replicated functional units.ACM acknowledges that this contribution was authored or co-authored by an em-ployee, or contractor of the national government. As such, the Government retainsa nonexclusive, royalty-free right to publish or reproduce this article, or to allow oth-ers to do so, for Government purposes only. Permission to make digital or hard copiesfor personal or classroom use is granted. Copies must bear this notice and the full ci-tation on the first page. Copyrights for components of this work owned by others thanACMmust be honored. To copy otherwise, distribute, republish, or post, requires priorspecific permission and/or a fee. Request permissions from permissions@acm.org.c© 2016 ACM. ISBN X-XXXXX-XX-X/XX/XX.DOI: http://dx.doi.org/10.1145/0000000.0000000They also usually assume a simple memory system focusingmore to reduce, rather than tolerate, access latencies.Resource Description Framework (RDF) databases havebecome one of the most prominent example of data ana-lytics application. RDF is the metadata data model typi-cally used to describe the Semantic Web. RDF databasesnaturally maps to graphs, and query languages for thesedatabases such as SPARQL basically express queries as acombination of graph methods (graph walks and graph pat-tern matching operations) and analytic functions. Amongthese, we consider GEMS, the Graph database Engine forMultithreaded Systems (GEMS) [2]. GEMS implements aRDF database on a commodity cluster by mainly employinggraph methods at all levels of his stack. To address the lim-itations of HPC systems, GEMS employs a runtime (GlobalMemory and threading - GMT) that provides: a global ad-dress space across the cluster, so that data do not need to bepartitioned, lightweight software multithreading, to toleratedata access latencies, and message aggregation, to improvenetwork utilization with fine-grained transactions. A graphapplication programming interface (API) and a set of meth-ods to ingest RDF triples and generate the related graphand dictionary (collectively named SGLib) are built withthe functions provided by the runtime. On top of the wholesystem, a translator converts query expressed in SPARQLto graph-pattern matching operations in C/C++.We have investigated acceleration of queries, as generatedfor the GEMS software stack, on FPGAs. We have devel-oped a set of architectural templates and HLS methodologiesto automatically generate specifications in hardware descrip-tion language (Verilog) of graph methods starting from Cdescriptions and have integrated them in a modified versionof an openly available High Level Synthesis tool, Bambu [1].We have then interfaced GEMS with the modified Bambu,and generated accelerators for SPARQL queries coming fromthe Lehigh University Benchmark (LUBM) [5], a referencebenchmarks for Semantic Web Repositories.2. ARCHITECTURE OVERVIEWFigure 1 shows how the GEMS stack has been integratedwith Bambu. We have changed GEMS layer so that they arenot anymore dependent on the GMT runtime. We developeda pure C version of the graph API that does not exploit anyof the functionalities of GMT. We modified the code emitteraccordingly, so that the C code generated from the SPARQLqueries only invokes the new C graph API to perform graphFigure 1: Structure of the GEMS stack and interac-tion with Bambu HLSwalks and matching and pure C functions to execute anyanalytic operation. Such a C code in practice correspondsto a set of nested loops, where each loop matches a particularedge of the graph pattern that composes the query, andbecomes the input of Bambu for the synthesis. Note thatsynthesizing full queries is not a limitation: in the manyanalytics applications, once the query is defined, it remainsstable in time, while the dataset dynamically changes. So,provided that the query execution provides a speed up, thetime required to synthesize the query on FPGA is affordable.Bambu has been modified by progressively integratingthree architectural templates, and the corresponding method-ologies to generate the instances of the templates. The threecomponents aim at providing better support for certain ofthe typical structures and behaviors that characterize paral-lel graph methods. These obviously include the graph pat-tern matching methods employed in GEMS for the queryprocessing. The components include a Parallel distributedController (PC) [3], a Hierarchical, multi-ported, MemoryInterface (HMI) [4], and a Dynamic Task Scheduler (DTS).The PC allows to generate more efficient designs that ex-ploit coarse grained (task level) parallelism than the typi-cal centralized controllers of conventional HLS flows basedaround the Finite State Machine with Datapath (FSMD)model, in terms of performance and area utilization. This isa key element in accelerating graph algorithms that basicallyare composed of a varying number of nested loops, iteratingon vertices or edges, where each iteration could identify adifferent task. By adopting the PC, it is easy to coordinateparallel execution of tasks (one, or more iterations each) onan array of replicated accelerators. The PC consists of a setof communicating modules, each one associated with one, ormore, operations. Controller modules are called ExecutionManagers (EMs). EMs start execution of the associated op-Figure 2: High Level overview of an architecturecombining the parallel controller and the hierarchi-cal memory interface.erations as soon as all their dependencies are satisfied andresource conflicts resolved. Other dedicated components,named Resource Managers (RM), arbitrate execution of op-erations on shared resources. EMs communicate through atoken-based schema: when a dependency is satisfied, a tokenis passed to the next EM. When all dependency for an EMare satisfied, it checks RMs for resource availability and, ifthe resource is free, execution starts.The HMI provides an easy way to dynamically disam-biguate fine grained memory accesses to locations of a large,multi-banked memory, while maintaining to the HLS flowand the accelerator pool the abstract view of a shared mem-ory. In cooperation with the PC, the HMI also enablesto easily support atomic memory operations. Graph algo-rithms typically access unpredictable memory locations withfine-grained transactions (i.e., the follow pointers), and theirimplementation is much easier when considering a sharedmemory abstraction. Also, they often are synchronizationintensive when parallelized, because the different tasks mayaccess the same elements concurrently. The HMI takes ininput memory access requests from N ports, which have anaddress, a data and an operation type (load/store) line. TheMIC routes requests towards one of the B output ports,corresponding to a different memory bank, by evaluatingtheir addresses. Each memory bank has non-overlapping ad-dresses. Accesses are routed towards a specific memory portat runtime, providing efficient support of the unpredictablememory access patterns typical in irregular applications.Furthermore, address can be scrambled across banks (i.e.,consecutive addresses are interleaved on different banks) toreduce hotspots. Control logic, synthesized according to thespecific scrambling function, performs the routing. Supportto atomic operations is provided through a mechanism sim-ilar to the RMs, as access to the specific memory port is notgranted until termination of the atomic operation. Figure 2provides a combined view of the PC and the HMI. RMs arehidden in the datapath (at the front of shared resources)and in the memory interface (at front of the memory ports).The DTS provides a way to execute new tasks as soon asone of the multiple accelerators is free: instead of simplyFigure 3: High level overview of an architecturetemplate using the DTSusing a fork/join model, where all currently executing taskson the set of accelerators must terminate before a new groupcould be executed, the DTS allow scheduling new tasks assoon as one of the accelerators is free. This adapt to a vari-ety of graph algorithms where certain tasks (iterations) mayexecute for a long time, while other could terminate early,such as when a graph walk is pruned early because it reachedan uninteresting part of the graph. Figure 3 shows an archi-tecture template integrating the DTS. The DTS interfacesto the set of parallel accelerators (Kernel Pool) and to theTermination Logic. The DTS itself contains a Task Queue,the Task Dispatcher, and a Status Register that holds infor-mation on the accelerators in the kernel pool. As soon asa task is terminates, the status register is updated and theDTS can schedule a new task. The Termination Logic allowsunderstanding when all the tasks have completed, i.e., whenfor example all iterations of a parallel loop have completed.3. RESULTS OVERVIEWTo validate our architectural templates and our approach,we have we have synthesized 7 queries from LUBM, and wehave tested the performance using a dataset of 5,309,056RDF ”triples”. In Table 1, we compare, in terms of exe-cution latency, a serial implementation of the architecture(Single Acc.), one that employs PC and the HMI [3] (ParallelController), and one that also includes the DTS (DynamicScheduler). The parallel architectures include 4 acceleratorsand HMIs with 4 ports. With respect to the serial imple-mentation, the architectures employing the DTS generallyshow a speed up close to the theoretical maximum. In manycases, the DTS also provides significant speed ups againstthe PC designs. This happens, in particular, with queriesthat have some iterations (tasks) that executes order of mag-nitudes longer than others. Another important effect of theDTS is that it maximizes utilization of the available memorychannels as provided by the HMI. In fact, all the architec-tures with the DTS utilize at least 3 out of 4 of the memoryports for more than 75% of the time.4. CONCLUSIONSWe have been investigating architectural templates andmethodologies to enable the HLS of data analytics appli-cations. Among all data analytics applications, we focusedour attention to RDF databases. These operate on largeamount of data that can be conveniently organized in graphdata structures and queried through languages that expressqueries as graph pattern matching operations. Modern re-Table 1: Performance comparison of the implemen-tation using the DTS+HMI against the serial andthe parallel controller implementationsSingle Acc.Parallel Dynamic SpeedupController SchedulerSingle Acc.Parallel# Cycles # Cycles # Cycles ControllerQ1 1,082,526,974 1,001,581,548 287,527,463 3.76 3.48Q2 7,359,732 2,801,694 2,672,295 2.75 1.05Q3 308,586,247 98,163,298 95,154,310 3.24 1.03Q4 63,825 42,279 19,890 3.21 2.13Q5 33,322 13,400 8,992 3.71 1.49Q6 682,949 629,671 199,749 3.42 3.15Q7 85,341,784 35,511,299 24,430,557 3.49 1.45configurable devices appear a promising target to acceler-ate this type of applications, which are massively paralleland mainly memory bound. However, conventional HLSflows typically target digital signal processing applications,which usually are compute intensive, exhibit significant in-struction level parallelism, and present regular memory ac-cess patterns. They do not cope well with the fine-grained,unpredictable memory accesses, and the highly dynamic,and sometimes very unbalanced, coarse grained (task) par-allelism, typical of graph methods. We detailed three of thearchitectural templates and the related methodologies wehave been designing to address these issues, and presentedsome initial results on a realistic benchmark. We believethat our effort can provide a solid basis to make HLS of thistype of applications more viable, thus a productive way forthe data analytic community to use custom accelerators.5. REFERENCES[1] Bambu: A Free Framework for the High-LevelSynthesis of Complex Applications.http://panda.dei.polimi.it, 2014.[2] V. Castellana, A. Morari, J. Weaver, A. Tumeo,D. Haglin, O. Villa, and J. Feo. In-memory graphdatabases for web-scale data. Computer, 48(3):24–35,Mar 2015.[3] V. G. Castellana, M. Minutoli, A. Morari, A. Tumeo,M. Lattuada, and F. Ferrandi. High Level Synthesis ofRDF Queries for Graph Analytics. In ICCAD’15:IEEE/ACM International Conference onComputer-Aided Design, pages 323–330, 2015.[4] V. G. Castellana, A. Tumeo, and F. Ferrandi. Anadaptive memory interface controller for improvingbandwidth utilization of hybrid and reconfigurablesystems. In DATE 2014: Design, Automation and Testin Europe, pages 1–4, 2014.[5] Y. Guo, Z. Pan, and J. Heflin. Lubm: A Benchmark forOWL Knowledge Base Systems. Web Semant.,3(2-3):158–182, Oct. 2005.[6] A. Putnam, A. Caulfield, E. Chung, D. Chiou,K. Constantinides, J. Demme, H. Esmaeilzadeh,J. Fowers, G. Gopal, J. Gray, M. Haselman, S. Hauck,S. Heil, A. Hormati, J.-Y. Kim, S. Lanka, J. Larus,E. Peterson, S. Pope, A. Smith, J. Thong, P. Xiao, andD. Burger. A reconfigurable fabric for acceleratinglarge-scale datacenter services. In ISCA: ACM/IEEE41st International Symposium on ComputerArchitecture (ISCA), pages 13–24, June 2014.[7] A. Tumeo and J. Feo. Irregular applications: Fromarchitectures to algorithms [guest editors’ introduction].IEEE Computer, 48(8):14–16, 2015.

Enabling the high level synthesis of data analytics accelerators

Abstract

Similar works

Full text

Available Versions

Archivio istituzionale della ricerca - Politecnico di Milano

Crossref