Engsig-Karup, Allan Peter

Glimberg, Stefan Lemvig

Pawlak, Wojciech

New many-core era applications at the interface of mathematics and computer science adopt modern parallel programming paradigms and expose parallelism through proper algorithms. We present new performance results for a novel massively parallel free surface wave model suitable for advanced simulations in arbitrary size Numerical Wave Tanks.The application has already been studied in a series of works (see References) and is demonstrated to exhibit excellent performance portability and scalability using hybrid MPI-OpenCL/CUDA. Furthermore, it can be executed on arbitrary heterogeneous multi-device system sizes from desktops to large HPC systems such as superclusters and in the cloud utilizing heterogeneous devices like multi-core CPUs, GPUs, and Xeon Phi coprocessors.The numerical efficiency is evaluated on heterogeneous devices like multi-core CPUs, GPUs and Xeon Phi coprocessors to test the performance with respect to both portability and scalability. This study contributes to investigating the potential of code acceleration for reducing turn-around times of industrial CFD applications on heterogeneous hardware

Online Research Database In Technology

                       General rights Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.  • Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal   If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.     Downloaded from orbit.dtu.dk on: Nov 08, 2017Computational Hydrodynamics: How Portable and Scalable Are HeterogeneousProgramming Paradigms?Pawlak, Wojciech ; Glimberg, Stefan Lemvig; Engsig-Karup, Allan PeterPublication date:2015Document VersionPublisher's PDF, also known as Version of recordLink back to DTU OrbitCitation (APA):Pawlak, W., Glimberg, S. L., & Engsig-Karup, A. P. (2015). Computational Hydrodynamics: How Portable andScalable Are Heterogeneous Programming Paradigms?. Poster session presented at SIAM Conference onComputational Science and Engineering (SIAM CSE 2015), Salt Lake City, Utah, United States.Computational Hydrodynamics:How Portable and Scalable AreHeterogeneous Programming Paradigms?Wojciech Pawlak1, Stefan L. Glimberg2, Allan P. Engsig-Karup11 Technical University of Denmark (DTU) 2 Lloyds Register ConsultingMotivationNew many-core era applications at the interface of math-ematics and computer science adopt modern parallelprogramming paradigms and expose parallelism throughproper algorithms. We present new performance resultsfor a novel massively parallel free surface wave modelsuitable for advanced simulations in arbitrary size Nu-merical Wave Tanks.The application has already been studied in a seriesof works (see References) and is demonstrated to ex-hibit excellent performance portability and scalability us-ing hybrid MPI-OpenCL/CUDA. Furthermore, it can beexecuted on arbitrary heterogeneous multi-device sys-tem sizes from desktops to large HPC systems such assuperclusters and in the cloud utilizing heterogeneousdevices like multi-core CPUs, GPUs, and Xeon Phi co-processors.The numerical efficiency is evaluated on heterogeneousdevices like multi-core CPUs, GPUs and Xeon Phi co-processors to test the performance with respect to bothportability and scalability.This study contributes to investigating the potential ofcode acceleration for reducing turn-around times of in-dustrial CFD applications on heterogeneous hardware.ContributionsIMassively parallel implementation of an advancedfree surface wave model to enable fast engineeringanalysis and large-scale simulations of ocean waves.IAccelerated computing. Evaluation of parallelprogramming paradigms with respect to performanceand scalability across all major architectures using theGPUlab library for solving partial differential equationson heterogeneous many-core hardware.Deterministic formulationAn advanced free surface model for description of non-breaking irrotational ocean waves is this fully nonlinearand dispersive wave potential model. Dynamic and kine-matic free surface boundary conditions are∂tζ = −∇ζ · ∇φ˜+ w˜(1 +∇ζ · ∇ζ),∂tφ˜ = −gζ −12(∇φ˜ · ∇φ˜− w˜2(1 +∇ζ · ∇ζ)),where φ˜ = φ(x, ζ, t), ζ(x, t) and w˜ = ∂zφ|z=ζ arefree surface quantities and g gravitational acceleration.A Laplace problem needs to be solvedφ = φ˜, z = ζ(x, t),∇2φ+ ∂zzφ = 0, −h ≤ z < ζ(x, t),∂zφ+∇h · ∇φ = 0, z = −h.from which closure is obtained by (u, w) = (∇, ∂z)φ.020406080100051000.20.40.60.81∆σ/∆xL*max|hx/h|Spectral radious, ρ02040608010005100.20.40.60.81∆σ/∆xL*max|hx/h|Spectral radious, ρFigure 1 : Using a multigrid PDC method, theeigenvalues of the iteration matrix confirms robustnessin numerical experiments using smoothers; (a)Point-wise Gauss-Seidel, GS and (b) BlockGauss-Seidel, ZL-GS.Figure 2 : Snap shot of diffracted deterministic wavefield about a cylinder.Iterative Preconditioned DefectCorrection MethodTo enable scalable massively parallel computations, aniterative preconditioned defect correction method can beused to generate a sequence of iterations, starting froman initial guess Φ[0] by using the following short recur-renceΦ[k] = Φ[k−1] +M−1r[k−1],r[k−1] = b−AΦ[k−1], k = 1, 2, . . . ,until a given convergence criterium is satisfied.The PDC method becomes scalable by using a multigridpreconditioner, where the ideas of nested grids and de-fect corrections are combined and smoothers S are in-troduced for ensuring convergence. A two-level multigridmethod can be stated asΦk = Φk−1 +M−1rk−1,M = Sµ2(I − Ph2h(M2h)−1R2hh (Mh)−1A)Sµ1where Ph2h andR2hh are prolongation and restriction op-erations, respectively.The operatorI − Ph2h(M2h)−1R2hh (Mh)−1Ais the coarse grid correction operator responsible forglobal coupling ((M2h)−1 may be a full matrix) and dic-tates the need for global communication. Without globalcommunication the number of iterations may grow withnumber of subdomains for the outer iterative solver andthe strategy will not be scalable.Figure 3 : Performance scaling for the defect correctioniteration, on STAMPEDE, single-precision.On scalability across architecturesExperiments proved that it is possible to achieve perfor-mance portability with OpenCL programming model overvarious architectures, types and generations. OpenCLimplementation gave satisfactory results on par withor better than the algorithmically-identical CUDA imple-mentation. The goal was to avoid architecture-specificoptimization, but rather apply autotuning techniques thatadapt to the given device during simulation runtime.Scalability was achieved using domain decompositionbetween nodes through MPI and OpenCL for heteroge-neous devices on node. The final result is more clearlyobservable between GPUs and CPUs as optimal XeonPhi performance needs more multithreading and vector-ization optimizations.103 104 105 106 107 108100101102103NPerformance (single−precision) [GFlop/s]  CUDA − K40OpenCL − K40CUDA − M2070OpenCL − M2070(a) CUDA vs. OpenCL onNVIDIA hpc-oriented devices.Performance in GFlop/s.High-order residual kernel.Single precision.103 104 105 106 107 108 10910−310−210−1100101NTime (single−precision) / Iter [s]  Xeon E5−2680v2 − 1Xeon E5−2680v2 − 2Xeon E5−2680v2 − 4Xeon E5−2680v2 − 8Xeon E5−2680v2 − 16(b) OpenCL on multi Xeon E5-2680v2. Execution time in ms.One DC iteration. Single pre-cision.Figure 4 : Portability and scalability experiment results.Figure 5 : Sustained fraction of peak performance in %for all devices. Single precision.ReferencesIEngsig-Karup, A. P., Glimberg, L. S., Nielsen, A. S.and Lindberg, O. 2013. Fast hydrodynamics onheterogenous many-core hardware. Part of: Rapha¨elCouturier (Ed). Designing Scientific Applications onGPUs, 2013, CRC Press / Taylor & Francis Group.IEngsig-Karup, A. P., Madsen, M. G. and Glimberg, S.L. A massively parallel GPU-accelerated model foranalysis of fully nonlinear free surface waves.E-published in International Journal for NumericalMethods in Fluids, July, 2011.IEngsig-Karup, A.P., Bingham, H.B. and Lindberg, O.2009 An efficient flexible-order model for 3D nonlinearwater waves. Journal of Computational Physics, 288,pp. 2100–2118.Contacts: Wojciech: wojciechpawlak@gmail.com, Stefan: Stefan.Glimberg@lr.org, Allan: apek@dtu.dk

Computational Hydrodynamics: How Portable and Scalable Are Heterogeneous Programming Paradigms?

http://orbit.dtu.dk/ws/files/118349535/SIAMCSE2015_Wojciech_1.pdf

Computational Hydrodynamics: How Portable and Scalable Are Heterogeneous Programming Paradigms?

Abstract

Similar works

Full text

Available Versions

Online Research Database In Technology