Acceleration Techniques for Industrial Large Eddy Simulation with High-Order Methods on CPU-GPU Clusters

One of the NASA's 2030 CFD Vision document key finding is that the use of CFD in the aerospace design process is severely limited by the inability to accurately and reliably predict turbulent flows with significant regions of separation. Scale-resolving simulations such as large eddy simulation (LES) are increasingly utilized with more complex problems such as flow over high lift configurations and through aircraft engines. The present work has the overall objective of reducing the computational cost of industrial LES. The high-order flux reconstruction (FR) method is used as the spatial discretization scheme. First, two acceleration techniques are investigated: the p-multigrid algorithm and Mach number preconditioning. The Weiss and Smith low Mach number preconditioner is used together with the p-multigrid method, and the third order explicit Runge-Kutta (RK3) scheme is considered as the smoother to reduce memory requirements. Mach number preconditioning significantly increased the efficiency of the p-multigrid method. For unsteady simulations, the preconditioner helped with the efficiency of the p-multigrid with larger physical time steps. In most steady cases, the preconditioned p-multigrid approach is comparable to or faster than the implicit LU-SGS algorithm and requires less memory, specially for p 2 schemes. An efficient implementation of the FR method is done for modern GPU clusters and the speedup is investigated for different polynomial orders and cell types. Approaches to improve the parallel efficiency of multi-GPU simulations are also studied. The simulation node-hour cost on the Summit supercomputer is reduced by a factor of 50 for hexahedron cells and up to 200 for tetrahedron cells. Two low memory implicit time integration methods are implemented on GPUs: the matrix-free GMRES solver and a novel local GMRES-SGS method. Parametric studies are done to evaluate their performance on LES benchmark cases. On the High-Lift Common Research Model case for the 2021 4th AIAA High-Lift Prediction Workshop, both GPU implicit time methods provide an additional speedup of 14 and 68, respectively, over the GPU explicit time simulation

GPU-Based Data Processing for 2-D Microwave Imaging on MAST

The Synthetic Aperture Microwave Imaging (SAMI) diagnostic is a Mega Amp Spherical Tokamak (MAST) diagnostic based at Culham Centre for Fusion Energy. The acceleration of the SAMI diagnostic data-processing code by a graphics processing unit is presented, demonstrating acceleration of up to 60 times compared to the original IDL (Interactive Data Language) data-processing code. SAMI will now be capable of intershot processing allowing pseudo-real-time control so that adjustments and optimizations can be made between shots. Additionally, for the first time the analysis of many shots will be possible

A Review of Element-Based Galerkin Methods for Numerical Weather Prediction: Finite Elements, Spectral Elements, and Discontinuous Galerkin

Numerical weather prediction (NWP) is in a period of transition. As resolutions increase, global models are moving towards fully nonhydrostatic dynamical cores, with the local and global models using the same governing equations; therefore we have reached a point where it will be necessary to use a single model for both applications. The new dynamical cores at the heart of these unified models are designed to scale efficiently on clusters with hundreds of thousands or even millions of CPU cores and GPUs. Operational and research NWP codes currently use a wide range of numerical methods: finite differences, spectral transform, finite volumes and, increasingly, finite/spectral elements and discontinuous Galerkin, which constitute element-based Galerkin (EBG) methods.Due to their important role in this transition, will EBGs be the dominant power behind NWP in the next 10 years, or will they just be one of many methods to choose from? One decade after the review of numerical methods for atmospheric modeling by Steppeler et al. (Meteorol Atmos Phys 82:287–301, 2003), this review discusses EBG methods as a viable numerical approach for the next-generation NWP models. One well-known weakness of EBG methods is the generation of unphysical oscillations in advection-dominated flows; special attention is hence devoted to dissipation-based stabilization methods. Since EBGs are geometrically flexible and allow both conforming and non-conforming meshes, as well as grid adaptivity, this review is concluded with a short overview of how mesh generation and dynamic mesh refinement are becoming as important for atmospheric modeling as they have been for engineering applications for many years.The authors would like to thank Prof. Eugenio Oñate (U. Politècnica de Catalunya) for his invitation to submit this review article. They are also thankful to Prof. Dale Durran (U. Washington), Dr. Tommaso Benacchio (Met Office), and Dr. Matias Avila (BSC-CNS) for their comments and corrections, as well as insightful discussion with Sam Watson, Consulting Software Engineer (Exa Corp.) Most of the contribution to this article by the first author stems from his Ph.D. thesis carried out at the Barcelona Supercomputing Center (BSCCNS) and Universitat Politècnica de Catalunya, Spain, supported by a BSC-CNS student grant, by Iberdrola Energías Renovables, and by grant N62909-09-1-4083 of the Office of Naval Research Global. At NPS, SM, AM, MK, and FXG were supported by the Office of Naval Research through program element PE-0602435N, the Air Force Office of Scientific Research through the Computational Mathematics program, and the National Science Foundation (Division of Mathematical Sciences) through program element 121670. The scalability studies of the atmospheric model NUMA that are presented in this paper used resources of the Argonne Leadership Computing Facility, which is a DOE Office of Science User Facility supported under Contract DE-AC02-06CH11357. SM, MK, and AM are grateful to the National Research Council of the National Academies.Peer ReviewedPostprint (author's final draft