Improving locality of memory accesses in current and future multi-core platforms is a key to efficiently exploit those platforms. Irregular applications, which operate on pointer-based data structures, are hard to optimize in modern computer architectures due to their intrinsic unpredictable patterns of memory accesses. In this paper we explore a memory locality-driven set of data-structures in order to attenuate the memory bandwidth limitations from typical irregular algorithms. We identify the inefficiencies in the standard Java implementation of a priority-queue as one of the main memory limitations in Prim’s Minimal Spanning Tree algorithm. We also present a priority-queue using the data layout inspired in Van Emde Boas for ordering heaps. We also implement optimizations in the graph data-structure and explore ways to efficiently combine it with the memory-efficient priority-queue. In order to improve efficiency in both case studies we had to transform the data-structures in the form of array of pointer into arrays of structures or structure of arrays

Faria, Nuno Filipe Monteiro

Silva, Rui C.

Sobral, João Luís Ferreira

Universidade do Minho: RepositoriUM

Enhancing Locality in Java based IrregularApplicationsN. Faria, R. Silva and J. L. Sobral{nfaria, ruisilva, jls}@di.uminho.ptCCTC/Universidade do MinhoAbstract. Improving locality of memory accesses in current and futuremulti-core platforms is a key to e ciently exploit those platforms. Irreg-ular applications, which operate on pointer-based data structures, arehard to optimize in modern computer architectures due to their intrin-sic unpredictable patterns of memory accesses. In this paper we explorea memory locality-driven set of data-structures in order to attenuatethe memory bandwidth limitations from typical irregular algorithms.We identify the ine ciencies in the standard Java implementation ofa priority-queue as one of the main memory limitations in Prim’s Min-imal Spanning Tree algorithm. We also present a priority-queue usingthe data layout inspired in Van Emde Boas for ordering heaps. We alsoimplement optimizations in the graph data-structure and explore waysto e ciently combine it with the memory-e cient priority-queue. In or-der to improve e ciency in both case studies we had to transform thedata-structures in the form of array of pointer into arrays of structuresor structure of arrays.1 IntroductionThe gap between CPU frequency and memory bandwidth has been increasingover the last decades. Introducing multiple levels of memory hierarchy amelio-rates the impact of this gap on application performance. Although, memoryhierarchy is only e↵ective when programs provide locality in data access. Mem-ory bottleneck will also be a big hurdle in many-core platforms since severalcores usually share the available memory bandwidth, limiting the applicationscalability. On the other hand, it is expected that the e↵ective bandwidth inaccessing local caches will scale proportionally to the number of cores. Currentplatforms provide same fixed amount of L1/L2 cache per core, independentlyfrom the total number of cores. Thus, to e↵ectively use current and future many-core platforms programs should present temporal and/or spatial locality in dataaccess. Exploiting locally in the so-called regular applications (e.g., matrix op-erations) is well known and usually resorts to partitioning data into blocks thatcan fit into the cache [1]. Irregular applications that rely on pointer based datastructures, such as graphs, are harder to optimize due to their intrinsic usage ofpointers to access data and to their less-predictable pattern of data access. Onetypical case is the Prim’s algorithm to compute a Minimum Spanning Three of a*This research was supported by the Fundação para a Ciência e a Tecnologia (project Parallel Programming Refinements for Irregular Applications, UTAustin/CA/0056/2008)graph: the particular order of traversing the graph vertex depends on the weighof the edge connecting vertexes. In generic frameworks, the e↵ort to providecollections that can be used in multiple context leads to data accesses throughadditional pointer indirection. In this paper we identify this indirection as onemain source of overhead both due to pointer indirection and lack of locality ofmemory accesses and explore strategies to remove this overhead.2 Locality driven data layoutMany Java collections are implemented as Arrays of Pointers (AoP) to structures(Figure 1a). This data layout does not provide spatial locality, but it is stillpossible to exploit temporal locality. A common optimization for better temporallocality is the division of the array into blocks (e.g., by performing a domaindecomposition). If each block fits in cache and it is accessed multiple times,data will remain in cache and can be reused many times lowering the cache missrates. Optimizations to improve spatial locality need a di↵erent data layout,in which elements are contiguous and accessed in an orderly manner. In thispaper we study three di↵erent data layouts (Figure 1): (i) arrays of pointers(AoP), (ii) arrays of structures (AoS) and (iii) structure of arrays (SoA). Inthe AoS layout, fields are stored continuously in memory, as in SoA, fields arestored into a separate array. Choosing the best alternative depends on how thealgorithm accesses data. The SoA provides better locality if the algorithm doesnot require all fields of the original structure in the same time-frame. The AoS isthe alternative used for problems that require all fields of the structure at once,although this choice is di cult to implement in Java since it is not possible touse explicit pointers to data. It is also more di cult to use if the fields are not ofthe same type. We may also use an hybrid alternative, in which the data fieldsare grouped according their time-frame usage. To improve the spatial locality inJava collections it is necessary to transform an AoP implementation into an AoSor a SoA. In the latter case, the fields of the objects are converted into arrays,which normally evolves removing the encapsulation of data. This provides betterperformance, but it might enforce significant restructuring of the code. In thiswork we intend to study the impact on performance of this transformation.Several authors propose techniques to automatically improve locality in Javaapplications by relying solely on changes to the Java Virtual Machine. Hirzel et.(a) AoP (b) AoS (c) SoAFig. 1: AoP, AoS and SoA views of attributes of structures in a collectional. [2] evaluates improvements in several data layouts, by sorting objects duringgarbage copying to improve locality, which can do so by placing target objects inconsecutive memory addresses, but it still maintains the AoP layout. Wimmeret. al. [3] proposes an improvement to the JVM to automatically inline objectfields by placing the parent and children in consecutive memory places and byreplacing memory accesses by address arithmetic. It is argued that using arraysas inlining parents is complicated because the Java byte-codes for accessing arrayelements have no static type information. Thus they claim that automatic AoPto AoS transformation at JVM level is impossible without a global data flowanalysis because of the structure of the array access byte-codes. Furthermore,transforming AoP to SoA layouts also seems not feasible at JVM level.3 Case studiesTo illustrate the impact of cache-e ciency in data-structures and algorithms inJava we study a few applications that would benefit from those optimizations.We show how we optimize the cache-hit behaviour for priority-queues (PQ)and graph structures for the Prim’s Minimal Spanning Tree (MST) algorithm.For each PQ and graph implementation, were generated a random sequenceof numbers (PQ) and a random graph, both stored in file so that every PQand graph implementation uses the same memory access pattern. All resultspresented were measured with PAPI profiling tool1, integrated using Java NativeInterfaces. In order to only measure program sections we were interested in (forPQs, insertion and removal only, ignoring file computations), we start/stop PAPImeasurements at the chosen routines. The tests were ran in a cache environmentof L1 32 KB instruction caches + 32 KB write-back data caches and shared L2of 4 MB; the CPU is an Intel Core 2 Duo Mobile; the installed JVM is JavaHotSpot (TM) Server VM (version 1.6.0 24).3.1 Cache e cient priority-queuesPriority queues (PQ) are built for fast retrieval of the highest/lowest element ina set of elements. Java’s native priority queue is based on the structural layoutof a binary heap [4]. The locality driven optimizations were applied on PQsavoiding use the Comparable interface. Comparing, for instance, Integer objectswith explicit mathematical operators (<, >) and with the Comparable interfaceincurs in similar results, due to the auto-boxing/unboxing2 - comparing referencedata-types (e.g., Integer) with explicit comparators causes the auto-unboxing of1 A hardware counter-based profiling tool: http://icl.cs.utk.edu/papi/faq/index.html2 Boxing is the phenomena of converting a primitive data-type value able to be ex-plicitly stored and represented in memory into a referenced data-type value, i.e., thevalue is no longer in raw format (int, float, double, etc.) since it is now a reference toa memory location where the value is stored. Unboxing is the opposite, the conver-sion of a reference data-type value into a primitive one. Most OO frameworks, likeJava, do this automatically for primitive data-types.Fig. 2: The VEB data layout - numbers correspond to array indicesthe referenced int value. However our main reason to use explicit primitive datatypes is due to the notion of boxing - creating references to memory addressesmakes locality optimizations obsolete. After a quick analysis (see Table 1, firstthree columns) it is visible that forcing element adjacency in Java collections (BHint) substantially improves performance - less instructions were needed, cachemiss3 are also lower and the execution time is approximately three times shorter.Using primitive data-types forces heap elements to be directly accessible fromthe array (using a SoA layout), as for an array of Objects (i.e., AoP layout) wemust first resolve the pointer to the Object in order to access it; so for each arrayaccess (in int version) pointer resolving is unnecessary - BH int runs in less thanhalf the instructions of BH Integer4. Also, not resolving pointers ultimatelyresults in less cache misses, which in its turn results in less stall cycles, thusdespite a reduction in 2 times on the number of instructions we attain an overallimprovement in execution time of 3.3 times, mainly due to a bigger improvementin locality of memory addresses (6 times less misses in L1).Locality aimed heap. To improve spatial locality we implemented a variationof a binary-heap with a more cache-friendly data layout - a simplified layoutversion of Van Emde Boas (VEB) [5] (Figure 2). On Table 1 we compare BHint implementation against VEB for block sizes of 3 and 7 (i.e., block height of1 and 2, respectively). We see a slight decrease in the execution time in VEB 3int compared to BH int, we are able to achieve better cache performance. Theincrease in instructions for VEB is explained with the increase in complexity inchildren and parent index expressions.3.2 Minimal Spanning Tree algorithm problemThe MST algorithm we focus on for this case study is Prim’s MST algorithm [6]which uses a priority-queue to find the smallest weighted available edges to add3 We opted to use the number of cache misses since it is more independent fromcompiler optimizations than miss rates (e.g., poorly optimized code tends to showlower miss rates due to unessential memory loads, mainly due to register spilling).4 BH Integer is our PQ implementation (using Java’s Integer class) similar to Javanative PQ by using an AoP approach, it was developed for comparison purposes.Table 1: Benchmarks comparing binary-heap to Java’s native PQ (AoP), and toVEB-heap, storing single intsJava Integer BH Integer BH int VEB 3 VEB 7Instructions (⇥108) 132.02 153.06 73.29 74.78 96.42Cycles (⇥108) 199.81 192.87 58.16 55.94 73.26L1 accesses (⇥108) 69.22 79.68 28.14 35.62 48.82L1 misses (⇥106) 234.65 232.42 39.51 28.50 22.76L2 accesses (⇥106) 418.60 412.87 93.97 70.63 57.76L2 misses (⇥106) 279.47 272.23 50.85 39.51 30.57Time (s) 10.025 9.579 2.932 2.812 3.736to the resulting MST graph. We studied the cache-hit/miss rates in the graphrepresentation, a typically irregular structure. In Figure 3 we represent some ofthe graph representations studied. The encapsulation problem is more visible inthis case for weighted graphs, when referring to the concept of Neighbours. Ourimplementations all follow a similar interface - (i) a set of Vertices (an Object),(ii) in which each Vertex has a set of neighbours (set of objects, Neighbour),(iii) each Neighbour harbouring the attributes: weight (primitive data-type),neighbour-vertex (pointer to Vertex object) and possibly other attributes, butfor simplicity we consider only these. We distinguish the di↵erent graph imple-mentations being AoP (Array of Pointers), AoS or SoA by looking at how theVertex structure holds the Neighbour array. The process of decapsulating Neigh-bour attributes reduced redundant object instantiations between the graph andPQ structure, with the MST layer as an intermediate layer. We opened theimplementation box, by changing the program’s API in order to work directlywith the attributes, instead of encapsulating them in objects. Thus, instead ofreturning from the graph data structure and adding to the PQ a Neighbour-object with the weight and vertex attributes, we return and add these attributesseparately, to avoid redundant object instantiation, ultimately benefiting cache-hits. As stated in Table 2, the best execution time as well as, instruction, cycleand cache behaviour occur for the GVG AoS and GVG SoA implementations.The major jump in performance is mainly due to the redundant pointer resolvingoperations present in creating new object instances that has now been removedfrom the executions in AoS and SoA graph implementations. Although the cache(a) Graph-Neighbour-Vertex (GNV AoP )(b) Graph-Vertex-Graph (GVGAoS)(c) Graph-Vertex-Graph (GVGSoA)Fig. 3: Graph representationsTable 2: Benchmarks comparing all three graph implementations (GNV AoP,GVG AoS and GVG SoA), PQ implementation is VEB 3, ran with the casesAoS and SoA.VEB 3AoS VEB 3SoAGNVAoP GVGAoS GVGSoA GNVAoP GVGAoS GVGSoAInstructions (⇥108) 9.71 6.07 6.01 9.90 6.38 6.19Cycles (⇥108) 15.40 4.87 4.78 15.52 5.08 5.00L1 accesses (⇥108) 5.94 2.98 2.80 5.95 2.90 2.79L1 misses (⇥106) 9.41 1.53 1.54 9.39 1.60 1.62L2 accesses (⇥106) 12.99 3.96 3.95 12.84 3.94 3.92L2 misses (⇥106) 3.14 1.74 1.73 3.15 1.88 1.87Time (s) 0.776 0.299 0.302 0.784 0.297 0.299miss behaviour in GVG AoS is slightly better than in GVG SoA, the later showsa little less running time, because it runs in less instructions and it consumesless cycles.4 ConclusionIn this paper we presented the impact of data layout on application performance.We identify the use of array of pointers as one main source of overhead in Javacollections. This overhead is mainly due to pointer indirection and to the lackof spatial locality in data access. We explore the use of two alternative datalayouts - array of structures and structures of arrays - that showed performanceimprovements in the case studies. However, e↵ective usage of these optimizeddata layouts in modern object-oriented frameworks require component decapsu-lation in order to avoid additional object creations and data copies. In the futurewe plan to investigate techniques to perform locality-driven optimizations to datalayouts without compromising encapsulation.References1. Yotov, K., Roeder, T., Pingali, K., Gunnels, J., Gustavson, F.: An experimentalcomparison of cache-oblivious and cache-conscious programs. Proceedings of thenineteenth annual ACM symposium on Parallel algorithms and architectures SPAA07 (2007) 932. Hirzel, M.: Data Layouts for Object-Oriented Programs. In: International Confer-ence on Measurement and Modeling of Computer Systems SIGMETRICS. (2007)3. Wimmer, C., Mo¨ssenbo¨o¨k, H.: Automatic array inlining in java virtual machines.Proceedings of the sixth annual IEEEACM international symposium on Code gen-eration and optimization CGO 08 (2008) 144. Williams, J.W.J.: Algorithm 232: Heapsort. Communications of the ACM 7 (1964)347–3485. Emde Boas, P., Kaas, R., Zijlstra, E.: Design and implementation of an e cientpriority queue. Mathematical Systems Theory 10 (1976) 99–1276. Prim, R.C.: Shortest connection networks and some generalizations. Bell SystemTechnical Journal 36 (1957) 1389–1401

Enhancing locality in Java based irregular applications

http://repositorium.sdum.uminho.pt/bitstream/1822/15113/1/Inforum.pdf

Enhancing locality in Java based irregular applications

Abstract

Similar works

Full text

Available Versions

Universidade do Minho: RepositoriUM