Matrix-free GPU implementation of a preconditioned conjugate gradient solver for anisotropic elliptic PDEs

Many problems in geophysical and atmospheric modelling require the fast solution of elliptic partial differential equations (PDEs) in "flat" three dimensional geometries. In particular, an anisotropic elliptic PDE for the pressure correction has to be solved at every time step in the dynamical core of many numerical weather prediction models, and equations of a very similar structure arise in global ocean models, subsurface flow simulations and gas and oil reservoir modelling. The elliptic solve is often the bottleneck of the forecast, and an algorithmically optimal method has to be used and implemented efficiently. Graphics Processing Units have been shown to be highly efficient for a wide range of applications in scientific computing, and recently iterative solvers have been parallelised on these architectures. We describe the GPU implementation and optimisation of a Preconditioned Conjugate Gradient (PCG) algorithm for the solution of a three dimensional anisotropic elliptic PDE for the pressure correction in NWP. Our implementation exploits the strong vertical anisotropy of the elliptic operator in the construction of a suitable preconditioner. As the algorithm is memory bound, performance can be improved significantly by reducing the amount of global memory access. We achieve this by using a matrix-free implementation which does not require explicit storage of the matrix and instead recalculates the local stencil. Global memory access can also be reduced by rewriting the algorithm using loop fusion and we show that this further reduces the runtime on the GPU. We demonstrate the performance of our matrix-free GPU code by comparing it to a sequential CPU implementation and to a matrix-explicit GPU code which uses existing libraries. The absolute performance of the algorithm for different problem sizes is quantified in terms of floating point throughput and global memory bandwidth.Comment: 18 pages, 7 figure

Efficient hierarchical approximation of high-dimensional option pricing problems

A major challenge in computational finance is the pricing of options that depend on a large number of risk factors. Prominent examples are basket or index options where dozens or even hundreds of stocks constitute the underlying asset and determine the dimensionality of the corresponding degenerate parabolic equation. The objective of this article is to show how an efficient discretisation can be achieved by hierarchical approximation as well as asymptotic expansions of the underlying continuous problem. The relation to a number of state-of-the-art methods is highlighted

Hamiltonian discontinuous Galerkin FEM for linear, rotating incompressible Euler equations: inertial waves

A discontinuous Galerkin finite element method (DGFEM) has been developed and tested for linear, three-dimensional, rotating incompressible Euler equations. These equations admit complicated wave solutions. The numerical challenges concern: (i) discretisation of a divergence-free velocity field; (ii) discretisation of geostrophic boundary conditions combined with no-normal flow at solid walls; (iii) discretisation of the conserved, Hamiltonian dynamics of the inertial-waves; and, (iv) large-scale computational demands owing to the three-dimensional nature of inertial-wave dynamics and possibly its narrow zones of chaotic attraction. These issues have been resolved: (i) by employing Dirac’s method of constrained Hamiltonian dynamics to our DGFEM for linear, compressible flows, thus enforcing the incompressibility constraints; (ii) by enforcing no-normal flow at solid walls in a weak form and geostrophic tangential flow —along the wall; (iii) by applying a symplectic time discretisation; and, (iv) by combining PETSc’s linear algebra routines with our high-level software. We compared our simulations with exact solutions of three-dimensional compressible and incompressible flows, in (non)rotating periodic and partly periodic cuboids (Poincar´e waves). Additional verifications concerned semi-analytical eigenmode solutions in rotating cuboids with solid walls

Energy-based comparison between the Fourier--Galerkin method and the finite element method

The Fourier-Galerkin method (in short FFTH) has gained popularity in numerical homogenisation because it can treat problems with a huge number of degrees of freedom. Because the method incorporates the fast Fourier transform (FFT) in the linear solver, it is believed to provide an improvement in computational and memory requirements compared to the conventional finite element method (FEM). Here, we systematically compare these two methods using the energetic norm of local fields, which has the clear physical interpretation as being the error in the homogenised properties. This enables the comparison of memory and computational requirements at the same level of approximation accuracy. We show that the methods' effectiveness relies on the smoothness (regularity) of the solution and thus on the material coefficients. Thanks to its approximation properties, FEM outperforms FFTH for problems with jumps in material coefficients, while ambivalent results are observed for the case that the material coefficients vary continuously in space. FFTH profits from a good conditioning of the linear system, independent of the number of degrees of freedom, but generally needs more degrees of freedom to reach the same approximation accuracy. More studies are needed for other FFT-based schemes, non-linear problems, and dual problems (which require special treatment in FEM but not in FFTH).Comment: 24 pages, 10 figures, 2 table

Adaptive meshless centres and RBF stencils for Poisson equation

We consider adaptive meshless discretisation of the Dirichlet problem for Poisson equation based on numerical differentiation stencils obtained with the help of radial basis functions. New meshless stencil selection and adaptive refinement algorithms are proposed in 2D. Numerical experiments show that the accuracy of the solution is comparable with, and often better than that achieved by the mesh-based adaptive finite element method

Efficient Multigrid Preconditioners for Atmospheric Flow Simulations at High Aspect Ratio

Many problems in fluid modelling require the efficient solution of highly anisotropic elliptic partial differential equations (PDEs) in "flat" domains. For example, in numerical weather- and climate-prediction an elliptic PDE for the pressure correction has to be solved at every time step in a thin spherical shell representing the global atmosphere. This elliptic solve can be one of the computationally most demanding components in semi-implicit semi-Lagrangian time stepping methods which are very popular as they allow for larger model time steps and better overall performance. With increasing model resolution, algorithmically efficient and scalable algorithms are essential to run the code under tight operational time constraints. We discuss the theory and practical application of bespoke geometric multigrid preconditioners for equations of this type. The algorithms deal with the strong anisotropy in the vertical direction by using the tensor-product approach originally analysed by B\"{o}rm and Hiptmair [Numer. Algorithms, 26/3 (2001), pp. 219-234]. We extend the analysis to three dimensions under slightly weakened assumptions, and numerically demonstrate its efficiency for the solution of the elliptic PDE for the global pressure correction in atmospheric forecast models. For this we compare the performance of different multigrid preconditioners on a tensor-product grid with a semi-structured and quasi-uniform horizontal mesh and a one dimensional vertical grid. The code is implemented in the Distributed and Unified Numerics Environment (DUNE), which provides an easy-to-use and scalable environment for algorithms operating on tensor-product grids. Parallel scalability of our solvers on up to 20,480 cores is demonstrated on the HECToR supercomputer.Comment: 22 pages, 6 Figures, 2 Table