Extending OmpSs for OpenCL kernel co-execution in heterogeneous systems

© 2017 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes,creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.Heterogeneous systems have a very high potential performance but present difficulties in their programming. OmpSs is a well known framework for task based parallel applications, which is an interesting tool to simplify the programming of these systems. However, it does not support the co-execution of a single OpenCL kernel instance on several compute devices. To overcome this limitation, this paper presents an extension of the OmpSs framework that solves two main objectives: the automatic division of datasets among several devices and the management of their memory address spaces. To adapt to different kinds of applications, the data division can be performed by the novel HGuided load balancing algorithm or by the well known Static and Dynamic. All this is accomplished with negligible impact on the programming. Experimental results reveal that there is always one load balancing algorithm that improves the performance and energy consumption of the system.This work has been supported by the University of Cantabria with grant CVE-2014-18166, the Generalitat de Catalunya under grant 2014-SGR-1051, the Spanish Ministry of Economy, Industry and Competitiveness under contracts TIN2016- 76635-C2-2-R (AEI/FEDER, UE) and TIN2015-65316-P. The Spanish Government through the Programa Severo Ochoa (SEV-2015-0493). The European Research Council under grant agreement No 321253 European Community’s Seventh Framework Programme [FP7/2007-2013] and Horizon 2020 under the Mont-Blanc Projects, grant agreement n 288777, 610402 and 671697 and the European HiPEAC Network.Peer ReviewedPostprint (published version

LEGaTO: first steps towards energy-efficient toolset for heterogeneous computing

LEGaTO is a three-year EU H2020 project which started in December 2017. The LEGaTO project will leverage task-based programming models to provide a software ecosystem for Made-in-Europe heterogeneous hardware composed of CPUs, GPUs, FPGAs and dataflow engines. The aim is to attain one order of magnitude energy savings from the edge to the converged cloud/HPC.Peer ReviewedPostprint (author's final draft

HDArray: Parallel Array Interface for Distributed Heterogeneous Devices

Heterogeneous clusters with nodes containing one or more accelerators, such as GPUs, have become common. While MPI provides a mechanism and management of interaddress space communication, and OpenCL provides a way to manage computation and communication within a process with access to heterogeneous computational resources, programmers are forced to write hybrid programs that manage the interaction of both of these systems. This paper describes an array programming interface that provides users with automatic or manual distributions of data and work. Using the distribution and information about what data is used and defined by kernels, communication among processes and among devices in a process is performed automatically. The interface provides a unified programming model to the user, thus simplifying program development

An adaptive offline implementation selector for heterogeneous parallel platforms

Heterogeneous Parallel Platforms, Comprising Multiple Processing Units And Architectures, Have Become A Cornerstone In Improving The Overall Performance And Energy Efficiency Of Scientific And Engineering Applications. Nevertheless, Taking Full Advantage Of Their Resources Comes Along With A Variety Of Difficulties: Developers Require Technical Expertise In Using Different Parallel Programming Frameworks And Previous Knowledge About The Algorithms Used Underneath By The Application. To Alleviate This Burden, We Present An Adaptive Offline Implementation Selector That Allows Users To Better Exploit Resources Provided By Heterogeneous Platforms. Specifically, This Framework Selects, At Compile Time, The Tuple Device-Implementation That Delivers The Best Performance On A Given Platform. The User Interface Of The Framework Leverages Two C&#43 &#43 Language Features: Attributes And Concepts. To Evaluate The Benefits Of This Framework, We Analyse The Global Performance And Convergence Of The Selector Using Two Different Use Cases. The Experimental Results Demonstrate That The Proposed Framework Allows Users Enhancing Performance While Minimizing Efforts To Tune Applications Targeted To Heterogeneous Platforms. Furthermore, We Also Demonstrate That Our Framework Delivers Comparable Performance Figures With Respect To Other Approaches.The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work has been partially supported by the Spanish ‘Ministerio de Economía y Competitividad’ under the project grant TIN2016-79637-P ‘Towards Unification of High Performance Computing (HPC) and Big Data Paradigms’ and the EU Projects ICT 644235 ‘RePhrase: REfactoring Parallel Heterogeneous Resource-Aware Applications’ and the FP7 609666 ‘Repara: Reengineering and Enabling Performance And poweR of Applications’

Domain Specific Languages for High Performance Computing

[ANGLÈS] High Performance Computing (HPC) relies completely on complex parallel, heterogeneus architectures and distributed systems which are hard and error-prone to exploit, even for HPC specialists. Further and further knowledge on runtime systems, dependency tracking, memory transaction optimization and many other techniques are a must-have requirement to produce high quality software capable of exploiting every single bit of power an HPC system has to other. On the other hand, domain experts like geologists or biologists are usually not technology-aware enough to produce the best software for these complex systems. Nowadays, the only way to successfully exploit an HPC system requires that computer and domain experts work closely towards producing applications to solve domain problems. Domain experts have the knowledge on the domain algorithms, while computer experts know how to ciently map these algorithms on HPC systems. This project proposes a framework that eases most of the processes related to the production of Domain Specific Languages (DSLs) that run on top of accelerator-based heretogeneus architectures. By using DSLs, domain experts can develop their applications using their own high level language, focusing only on their hard-enough issues. Meanwhile, computer experts stay only improving the implementation of these DSLs to make the most out of an HPC platform. This way, we keep each expert focused as much as possible on its natural domain of expertise.[CASTELLÀ] La computación de alto rendimiento (HPC) se basa en la utilización de sistemas distribuidos heterogeneos difíciles de explotar, incluso por expertos en el area de HPC. Para producir software HPC capaz de explotar toda la potencia de un supercomputador, cada vez se requiere más conocimiento en técnicas avanzadas de programación tales como análisis de dependencias u optimización de transacciones de memoria. Por otra parte, expertos de dominio tales como geólogos o biólogos no estan tan familiarizados con las tecnologías actuales como para producir software para arquitecturas modernas. Por este motivo, la unica manera de explotar supercomputadores actuales es hacer que los expertos de dominio trabajen muy conjuntamente con programadores expertos para producir sus aplicaciones. Este proyecto propone un sistema que agiliza muchos de los procesos asociados al desarrollo de DSLs para HPC. Mediante el uso de DSLs, los expertos de dominio pueden desarrollar sus aplicaciones utilizando su propio lenguaje de alto nivel, centrándose solamente en sus ya complicados problemas. Mientras tanto, los programadores expertos focalizan sus esfuerzos en mejorar y optimizar la implementación de estos DSLs de modo que aprovechen al máximo los recursos que ofrezca cada sistema. De este modo, cada experto se mantiene en su dominio natural, maximizando así su productividad.[CATALÀ] La computació d'alt rendiment (HPC) es basa en la utilizació de sistemes distribuïts heterogenis difícils d'explotar, inclus per experts en l'àrea de HPC. Per a produïr software HPC capaç d'explotar tota la potència d'un supercomputador, cada cop es requereix més coneixement en tècniques avançades de programació tals com anàlisi de dependències o optimizació de transaccions de memòria. D'altra banda, experts de domini tals com geòlegs o biòlegs no estan tan familiaritzats amb les tecnologies actuals com per a produïr software per a arquitectures modernes. Per aquest motiu, l'unica manera d'explotar supercomputadors actuals es fer que els experts de domini treballin molt conjuntament amb programadors experts per tal de produïr les seves aplicacions. Aquest projecte proposa un sistema que agilitza molts dels processos associats al desenvolupament de DSLs per a HPC. Mitjançant DSLs, els experts de domini poder desenvolupar les seves aplicacions fent servir el seu propi llenguatge d'alt nivell, concentrant-se en els seus prou complicats problemes. Mentres tant, els programadors experts focalitzen els seus esforços en millorar i optimitzar la implementació d'aquests DSLs de manera que aprofitin al màxim els recursos que ofereix cada sistema. D'aquesta manera, cada expert es manté al seu domini natural, maximitzant així la seva productivitat

Synchronization / communication techniques for OmpSs@FPGA

HPC machines are introducing more and more heterogeneity in their architecture on the road to exascale systems. The increasing complexity of the machines due to the variety of hardware architectures and accelerators makes efficient programming a task harder than ever. Heterogeneous parallel programming models, such as OmpSs@FPGA, help the programmer handle the most unfriendly parts of working with accelerators. This master thesis analyzes the OmpSs@FPGA communication system and proposes a set of techniques to overcome the problems related to it and potentially improve the performance of the applications. The results show that the techniques proposed speed up the applications under certain conditions and, most importantly, solves some of the limitations that had the previous communication system. In particular, the new techniques specially improve the explotation of fine-grain parallelism and open the door to explore new possibilities with regard to data communication and re-use. Moreover, a tool (autoVivado) that automatically manages the process of bitstream generation, from the synthesis of the HLS code to the generation of the device-tree, has been developed as part of this master thesis. autoVivado has been fully integrated with the OmpSs@FPGA compiler infrastructure, providing the programmers a way to transparently generate parallel heterogenous programs and bitstreams from OmpSs applications that use FPGA accelerators

Exascale machines require new programming paradigms and runtimes

Extreme scale parallel computing systems will have tens of thousands of optionally accelerator-equiped nodes with hundreds of cores each, as well as deep memory hierarchies and complex interconnect topologies. Such Exascale systems will provide hardware parallelism at multiple levels and will be energy constrained. Their extreme scale and the rapidly deteriorating reliablity of their hardware components means that Exascale systems will exhibit low mean-time-between-failure values. Furthermore, existing programming models already require heroic programming and optimisation efforts to achieve high efficiency on current supercomputers. Invariably, these efforts are platform-specific and non-portable. In this paper we will explore the shortcomings of existing programming models and runtime systems for large scale computing systems. We then propose and discuss important features of programming paradigms and runtime system to deal with large scale computing systems with a special focus on data-intensive applications and resilience. Finally, we also discuss code sustainability issues and propose several software metrics that are of paramount importance for code development for large scale computing systems

Data reuse design exploration in OmpSs@FPGA

In this thesis, the OmpSs@FPGA tool chain has been extended to try to reduce the overall communication time due to copies of data when it is possible to reuse data already in the BRAM of the accelerators

Run-time support for multi-level disjoint memory address spaces

High Performance Computing (HPC) systems have become widely used tools in many industry areas and research fields. Research to produce more powerful and efficient systems has grown in par with their popularity. As a consequence, the complexity of modern HPC architectures has increased in order to provide systems with the highest levels of performance. This increased complexity has also affected the way HPC systems are programmed. HPC users have to deal with new devices, languages and tools, and this is can be a significant access barrier to people that do not have a deep knowledge in computer science. On par with the evolution of HPC systems, programming models have also evolved to ease the task of developing applications for these machines. Two well-known examples are OpenMP and MPI. The former can be used in shared memory systems and is praised for offering an easy methodology of software development. The latter is more popular because it targets distributed environments but it is considered burdensome to use. Besides these two, many programming models have emerged to propose new methodologies or to handle new hardware devices. One of these models is OmpSs. OmpSs is a programming model for modern HPC systems that is based on OpenMP and StarSs. Developed by the Programming Models group at the Barcelona Supercomputing Center, it targets the latest generation of HPC systems while benefiting from the ease of use of OpenMP. OmpSs offers asynchronous parallelism with the concept of tasks with data dependencies. These tasks allow the specification of sections of code that can be executed in parallel while the dependencies specify the restrictions about the order in which the tasks can be executed. With this, OmpSs programs can adapt to a many different system configurations while fundamentally still being sequential programs with annotations. This thesis explores the benefits of providing OmpSs the capability to target architectures with complex memory hierarchies. An example of such systems can be the new generation of clusters that use accelerators to power their computing capabilities. The memory hierarchy of these machines is composed of a first level of distributed memory formed by the memory of each individual node, and a second level formed by the private memory of each accelerator devices. Our first contribution shows the implementation of the support of cluster of multi-cores for the OmpSs programming model. We also present two optimizations to boost the performance of applications running on top of cluster systems: a specific task scheduling policy and the addition of slave-to-slave transfers. We evaluate our implementation using a set of benchmarks coded in OmpSs and we also compare them against the same applications implemented using MPI, the most widely used programming model for these systems. We extend our initial implementation in our second contribution, which provides OmpSs with support for clusters of GPUs. We show that OmpSs programs targeting these complex systems are capable of achieving a good performance when compared against MPI+CUDA implementations. The third contribution of this thesis presents an implementation and evaluation of the performance and programmability impact of supporting non-contiguous memory regions. Offering this feature allows applications with complex data accesses to be easily annotated with OmpSs. This is important to widen the spectrum of applications that can be handled by the programming model.Els sistemes de computació d'altes prestacions (CAP) han esdevingut eines importants en diferents sectors industrials i camps de recerca. La recerca per produir sistemes més potents i eficients ha crescut proporcionalment a aquesta popularitat. Com a conseqüència, la complexitat d'aquest tipus de sistemes s'ha incrementat per tal de dotar-los d'altes prestacions. Aquest increment en la complexitat també ha afectat la manera de programar aquest tipus de sistemes. Els usuaris de sistemes CAP han de treballar amb nous dispositius, llenguatges i eines, i això pot convertir-se en una barrera d'entrada significativa per aquelles persones que no tinguin uns alts coneixements informàtics. Seguin l'evolució dels sistemes CAP, els models de programació també han evolucionat per tal de facilitar la tasca de desenvolupar aplicacions per aquests sistemes. Dos exemples ben coneguts son OpenMP i MPI. El primer es pot utilitzar en sistemes de memòria compartida i es reconegut per oferir una metodologia de desenvolupament senzilla. El segon és més popular perquè està dissenyat per sistemes distribuïts, però està considerat difícil d'utilitzar. A part d'aquests dos, altres models de programació han sorgit per proposar noves metodologies o per suportar nous components hardware. Un d'aquests nous models és OmpSs. OmpSs és un model de programació per sistemes CAP moderns que està basat en OpenMP i StarSs. Desenvolupat pel grup de Models de Programació del Barcelona Supercomputing Center, està dissenyat per suportar la darrera generació de sistemes CAP i alhora oferir la facilitat d'us d'OpenMP. OmpSs ofereix paral·lelisme asíncron mitjançant el concepte de tasques amb dependències de dades. Aquestes tasques permeten especificar regions de codi que poden ser executades en paral·lel, mentre que les dependències especifiquen les restriccions sobre l'ordre en que aquestes tasques poden ser executades. Amb això, els programes fets amb OmpSs poden adaptar-se a sistemes amb diferents configuracions tot i ser fonamentalment programes seqüencials amb anotacions. Aquesta tesi explora els beneficis de proveir a OmpSs amb la capacitat de funcionar sobre arquitectures amb jerarquies de memòria complexes. Un exemple d'un sistema així pot ser un dels clústers de nova generació que utilitzen acceleradors per tal d'oferir més capacitat de càlcul. La jerarquia de memòria en aquestes màquines està composada per un primer nivell de memòria distribuïda formada per la memòria de cada node individual, i el segon nivell està format per la memòria privada de cada accelerador. La primera contribució d'aquesta tesi mostra la implementació del suport de clústers de multi-cores pel model de programació OmpSs. També presentem dos optimitzacions per millorar el rendiment de les aplicacions quan s'executen en sistemes clúster: una política de planificació de tasques específica i la incorporació dels missatges entre nodes esclaus. Avaluem la nostra implementació usant un conjunt d'aplicacions programades en OmpSs i també les comparem amb les mateixes aplicacions implementades usant MPI, el model de programació més estès per aquest tipus de sistemes. En la segona contribució estenem la nostra implementació inicial per tal de dotar OmpSs de suport per clústers de GPUs. Mostrem que els programes OmpSs son capaços d'obtenir un bon rendiment sobre aquests tipus de sistemes, fins i tot quan els comparem amb versions implementades usant MPI+CUDA. La tercera contribució descriu la implementació i avaluació del rendiment i de l'impacte de suportar regions de memòria no contigües. Oferir aquesta funcionalitat permet implementar fàcilment amb OmpSs aplicacions amb accessos complexes a memòria, cosa que és important de cara a ampliar l'espectre d'aplicacions que poden ser tractades pel model de programació