Accelerating solutions of one-dimensional unsteady PDEs with GPU-based swept time-space decomposition

The expedient design of precision components in aerospace and other high-tech industries requires simulations of physical phenomena often described by partial differential equations (PDEs) without exact solutions. Modern design problems require simulations with a level of resolution difficult to achieve in reasonable amounts of time-even in effectively parallelized solvers. Though the scale of the problem relative to available computing power is the greatest impediment to accelerating these applications, significant performance gains can be achieved through careful attention to the details of memory communication and access. The swept time-space decomposition rule reduces communication between sub-domains by exhausting the domain of influence before communicating boundary values. Here we present a GPU implementation of the swept rule, which modifies the algorithm for improved performance on this processing architecture by prioritizing use of private (shared) memory, avoiding interblock communication, and overwriting unnecessary values. It shows significant improvement in the execution time of finite-difference solvers for one-dimensional unsteady PDEs, producing speedups of 2-9 x for a range of problem sizes, respectively, compared with simple GPU versions and 7-300 x compared with parallel CPU versions. However, for a more sophisticated one-dimensional system of equations discretized with a second-order finite-volume scheme, the swept rule performs 1.2-1.9 x worse than a standard implementation for all problem sizes. (C) 2017 Elsevier Inc. All rights reserved

Swept time-space domain decomposition on GPUs and heterogeneous computing systems

Modern scientific and engineering problems often require simulations with a level of resolution difficult to achieve in reasonable amounts of time—even in effectively parallelized programs. Therefore, applications that exploit high performance computing (HPC) systems have become invaluable in academia and industry over the past two decades. Addressing the questions that arise from continual scientific advancement requires solutions from hardware and software are required to supply the necessary throughput for demand across scientific disciplines. The most important development on the hardware side has been the General Purpose Graphics Processing Unit (GPGPU), a class of massively parallel device that now composes a substantial portion of the computational power of the top 500 supercomputers. As these systems grow, barriers to increased performance arise from small costs accumulated over innumerable iterations such as latency, the fixed cost of memory accesses, which becomes significantly larger when access requires communication between two distant CPU processes. This thesis implements and analyzes swept time-space domain decomposition, a communication avoiding scheme for time-stepping stencil codes, for GPGPU and heterogeneous (CPU/GPU) architectures. The GPGPU program significantly improves the execution time of finite-difference solvers for relatively simple one-dimensional time-stepping partial differential equations (PDEs). The swept decomposition code showed speedups of 2-9x compared with simple GPU domain decompositions and 7-300x compared with parallel CPU versions over a range of problem sizes: 10­­3 – 106 spatial points. However, for a more sophisticated one-dimensional system of equations discretized with a second-order finite-volume scheme, the swept rule performs 1.2-1.9x than a standard implementation for all problem sizes. The program targeting heterogeneous systems with distributed memory patterns performs significantly better on both simple problems, speedup 4-18x, and more complex equation systems, speedup 1.5-3x, over the range of problem sizes: 105-107 spatial points. This demonstrates the benefit of GPU architecture and the contingent effectiveness of swept time-space decomposition for accelerating explicit PDE solvers on current computational architectures

Depth-averaged and 3D Finite Volume numerical models for viscous fluids, with application to the simulation of lava flows

This Ph.D. project was initially born from the motivation to contribute to the depth-averaged and 3D modeling of lava flows. Still, we can frame the work done in a broader and more generalist vision. We developed two models that may be used for generic viscous fluids, and we applied efficient numerical schemes for both cases, as explained in the following. The new solvers simulate free-surface viscous fluids whose temperature changes are due to radiative, convective, and conductive heat exchanges. A temperature-dependent viscoplastic model is used for the final application to lava flows. Both the models behind the solvers were derived from mass, momentum, and energy conservation laws. Still, one was obtained by following the depth-averaged model approach and the other by the 3D model approach. The numerical schemes adopted in both our models belong to the family of finite volume methods, based on the integral form of the conservation laws. This choice of methods family is fundamental because it allows the creation and propagation of discontinuities in the solutions and enforces the conservation properties of the equations. We propose a depth-averaged model for a viscous fluid in an incompressible and laminar regime with an additional transport equation for a scalar quantity varying horizontally and a variable density that depends on such transported quantity. Viscosity and non-constant vertical profiles for the velocity and the transported quantity are assumed, overtaking the classic shallow-water formulation. The classic formulation bases on several assumptions, such as the fact that the vertical pressure distribution is hydrostatic, that the vertical component of the velocity can be neglected, and that the horizontal velocity field can be considered constant with depth because the classic formulation accounts for non-viscous fluids. When the vertical shear is essential, the last assumption is too restrictive, so it must relax, producing a modified momentum equation in which a coefficient, known as the Boussinesq factor, appears in the advective term. The spatial discretization method we employed is a modified version of the central-upwind scheme introduced by Kurganov and Petrova in 2007 for the classical shallow water equations. This method is based on a semi-discretization of the computational domain, is stable, and, being a high-order method, has a low numerical diffusion. For the temporal discretization, we used an implicit-explicit Runge-Kutta technique discussed by Russo in 2005 that permits an implicit treatment of the stiff terms. The whole scheme is proved to preserve the positivity of flow thickness and the stationary steady-states. Several numerical experiments validate the proposed method, show the incidence on the numerical solutions of shape coefficients introduced in the model and present the effects of the viscosity-related parameters on the final emplacement of a lava flow. Our 3D model describes the dynamics of two incompressible, viscous, and immiscible fluids, possibly belonging to different phases. Being interested in the final application of lava flows, we also have an equation for energy that models the thermal exchanges between the fluid and the environment. We implemented this model in OpenFOAM, which employs a segregated strategy and the Finite Volume Methods to solve the equations. The Volume of Fluid (VoF) technique introduced by Hirt and Nichols in 1981 is used to deal with the multiphase dynamics (based on the Interphase Capturing strategy), and hence a new transport equation for the volume fraction of one phase is added. The challenging effort of maintaining an accurate description of the interphase between fluids is solved by using the Multidimensional Universal Limiter for Explicit Solution (MULES) method (described by Marquez Damian in 2013) that implements the Flux-Corrected Transport (FCT) technique introduced by Boris and Book in 1973, proposing a mix of high and low order schemes. The choice of the framework to use for any new numerical code is crucial. Our contribution consists of creating a new solver called interThermalRadConvFoam in the OpenFOAM framework by modifying the already existing solver interFoam (described by Deshpande et al. in 2012). Finally, we compared the results of our simulations with some benchmarks to evaluate the performances of our model

The swept rule for breaking the latency barrier in time-advancing PDEs

Thesis: Ph. D. in Computational Science and Engineering, Massachusetts Institute of Technology, Department of Civil and Environmental Engineering, 2019Cataloged from PDF version of thesis.Includes bibliographical references (pages 103-104).This thesis describes a method to accelerate parallel, explicit time integration of unsteady PDEs. The method is motivated by our observation that network latency, not bandwidth or computing power, often limits how fast PDEs can be solved in parallel. The method is called the swept rule of space-time domain decomposition. Compared to conventional, space-only domain decomposition, it communicates similar amount of data, but in fewer messages. The swept rule achieves this by decomposing space and time among computing nodes in ways that exploit the domains of influence and the domain of dependency, making it possible to communicate once per many time steps with no redundant computation. By communicating less often, the swept rule effectively breaks the latency barrier, advancing on average more than one time step per round-trip latency of the network. The thesis describes the algorithms, presents simple theoretical analysis to the performance of the swept rule, and supports the analysis with numerical experiments.by Maitham Makki Alhubail.Ph. D. in Computational Science and EngineeringPh.D.inComputationalScienceandEngineering Massachusetts Institute of Technology, Department of Civil and Environmental Engineerin