Multigrid Preconditioning for a Space-Time Spectral-Element Discontinuous-Galerkin Solver

In this work we examine a multigrid preconditioning approach in the context of a high- order tensor-product discontinuous-Galerkin spectral-element solver. We couple multigrid ideas together with memory lean and efficient tensor-product preconditioned matrix-free smoothers. Block ILU(0)-preconditioned GMRES smoothers are employed on the coarsest spaces. The performance is evaluated on nonlinear problems arising from unsteady scale- resolving solutions of the Navier-Stokes equations: separated low-Mach unsteady ow over an airfoil from laminar to turbulent ow. A reduction in the number of ne space iterations is observed, which proves the efficiency of the approach in terms of preconditioning the linear systems, however this gain was not reflected in the CPU time. Finally, the preconditioner is successfully applied to problems characterized by stiff source terms such as the set of RANS equations, where the simple tensor product preconditioner fails. Theoretical justification about the findings is reported and future work is outlined

Efficient implicit spectral/hp element DG techniques for compressible flows

In the simulation of stiff problems, such as fluid flows at high Reynolds numbers, the efficiency of explicit time integration is significantly limited by the need to use very small time steps. To alleviate this limitation and to accelerate compressible flow simulations based on high-order spectral/$hp$ element methods, an implicit time integration method is developed using singly diagonally implicit Runge-Kutta temporal discretization schemes combined with a Jacobian-free Newton Krylov (JFNK) method. This thesis studies several topics influencing the efficiency, accuracy and robustness of the solver. Firstly, an efficient partially matrix-free block relaxed Jacobi (BRJ) preconditioner is proposed, in which the Jacobian matrix and preconditioning matrices are properly approximated based on studies of their influences on convergence. The preconditioner only forms and stores the diagonal part of the Jacobian matrix while the off-diagonal operators are calculated on the fly. Used together with techniques such as using single precision data, the BRJ can largely reduce the memory consumption when compared with matrix-based ones like incomplete LU factorization preconditioners (ILU). To further accelerate the solver, influences of different parts of the flux Jacobian on the preconditioning effects are studied and terms with minor influences are neglected. This reduces the computational cost of the BRJ preconditioner by about 3 times while maintaining similar preconditioning effects. Secondly, adaptive strategies for a suitable choice of some free parameters are designed to maintain temporal accuracy and relatively high efficiency. The several free parameters in the implicit solver have significant influences on the accuracy, efficiency and stability. Therefore, designing proper strategies in choosing them is essential for developing a robust general purpose solver. Based on the idea of constructing proper relations between the temporal, spatial and iterative errors, adaptive strategies are designed for determining the time step and Newton tolerance. These parameters maintain temporal accuracy of the solver in the sense that further decreasing temporal and iterative errors will not obviously improve the efficiency. Meanwhile, they maintain relatively efficient by avoiding excessively small time step and Newton tolerance. The strategies are tested in different types of cases to illustrate their performance and generality. Finally, the implicit solver is studied in high-fidelity simulations of turbulent flows based on a hierarchical implementation in the open-source spectral/$hp$ element framework Nektar++. The solver is applied to large-eddy simulations of Taylor-Green vortex flow, turbulent channel flow and flow over a circular cylinder cases. The efficiency of the solver and the prediction accuracy of these problems are studied. The results show that the solver yields good predictions in turbulence simulations whilst keeping good stability and high efficiency.Open Acces

Towards Efficient and Scalable Discontinuous Galerkin Methods for Unsteady Flows

openNegli ultimi anni, la crescente disponibilit`a di risorse computazionali ha contribuito alla diffusione della fluidodinamica computazionale per la ricerca e per la progettazione industriale. Uno degli approcci pi promettenti si basa sul metodo agli elementi finiti discontinui di Galerkin (dG). Nell’ambito di queste metodologie, il contributo della tesi e' triplice. Innanzi- tutto, il lavoro introduce un algoritmo di parallelizzazione ibrida MPI/OpenMP per l’utilizzo efficiente di risorse di super calcolo. In secondo luogo, propone strategie di soluzione efficienti, scalabili e con limitata allocazione di memoria per la soluzione di problemi complessi. Infine, confronta le strategie di soluzione introdotte con nuove tecniche di discretizzazione dette “ibridizzabili”, su problemi riguardanti la soluzione delle equazioni di Navier–Stokes non stazionarie. L’efficienza computazionale e' stata valutata su casi di crescente complessita' riguardanti la simulazione della turbolenza. In primo luogo, e' stata considerata la convezione naturale di Rayleigh-Benard e il flusso turbolento in un canale a numeri di Reynolds moderatamente alti. Le strategie di soluzione proposte sono risultate fino a cinque volte piu` veloci rispetto ai metodi standard allocando solamente il 7% della memoria. In secondo luogo, e' stato analizzato il flusso attorno ad una piastra piana con bordo arrotondato sottoposta a diversi livelli di turbolenza in ingresso. Nonostante la maggiore complessità' dovuta all’uso di elementi curvi ed anisotropi, l’algoritmo proposto e' risultato oltre tre volte piu` veloce allocando il 15% della memoria rispetto ad un metodo standard. Concludendo, viene riportata la simulazione del “Boeing Rudimentary Landing Gear” a Re = 10^6. In tutti i casi i risultati ottenuti sono in ottimo accordo con i dati sperimentali e con precedenti simulazioni numeriche pubblicate in letteratura.In recent years the increasing availability of High Performance Computing (HPC) resources strongly promoted the widespread of high fidelity simulations, such as the Large Eddy Simulation (LES), for industrial research and design. One of the most promising approaches to those kind of simulations is based on the discontinuous Galerkin (dG) discretization method. The contribution of the thesis towards this research area is three-fold. First, the work introduces an efficient hybrid MPI/OpenMP parallelisation paradigm to fruitfully exploit large HPC facilities. Second, it reports efficient, scalable and memory saving solution strategies for stiff dG discretisations. Third, it compares those solution strategies, for the first time using the same numerical framework, to hybridizable discontinuous Galerkin (HDG) methods, including a novel implementation of a p-multigrid preconditioning approach, on unsteady flow problems involving the solution of the NavierStokes equations. The improvements in computational efficiency have been evaluated on cases of growing complexity involving large eddy simulations of turbulent flows. First, the Rayleigh-Benard convection problem and the turbulent channel flow at moderately high Reynolds numbers is presented. The solution strategies proposed resulted up to five times faster than standard matrix-based methods while al- locating the 7% of the memory. A second family of test cases involve the LES simulation of a rounded leading edge flat plate under different levels of free-stream turbulence. Although the increased stiffness of the iteration matrix due to the use of curved and stretched elements, the solver resulted more than three times faster while allocating the 15% of the memory if compared to standard methods. Finally, the large eddy simulation of the Boeing Rudimentary Landing Gear at Re = 10^6 is reported. In all the cases, a remarkable agreement with experimental data as well as previous numerical simulations is documented.INGEGNERIA INDUSTRIALEopenFranciolini, Matte

Implicit time integration for high-order compressible flow solvers

The application of high-order spectral/hp element discontinuous Galerkin (DG) methods to unsteady compressible flow simulations has gained increasing popularity. However, the time step is seriously restricted when high-order methods are applied to an explicit solver. To eliminate this restriction, an implicit high-order compressible flow solver is developed using DG methods for spatial discretization, diagonally implicit Runge-Kutta methods for temporal discretization, and the Jacobian-free Newton-Krylov method as its nonlinear solver. To accelerate convergence, a block relaxed Jacobi preconditioner is partially matrix-free implementation with a hybrid calculation of analytical and numerical Jacobian.The problem of too many user-defined parameters within the implicit solver is then studied. A systematic framework of adaptive strategies is designed to relax the difficulty of parameter choices. The adaptive time-stepping strategy is based on the observation that in a fixed mesh simulation, when the total error is dominated by the spatial error, further decreasing of temporal error through decreasing the time step cannot help increase accuracy but only slow down the solver. Based on a similar error analysis, an adaptive Newton tolerance is proposed based on the idea that the iterative error should be smaller than the temporal error to guarantee temporal accuracy. An adaptive strategy to update the preconditioner based on the Krylov solver’s convergence state is also discussed. Finally, an adaptive implicit solver is developed that eliminates the need for repeated tests of tunning parameters, whose accuracy and efficiency are verified in various steady/unsteady simulations. An improved shock-capturing strategy is also proposed when the implicit solver is applied to high-speed simulations. Through comparisons among the forms of three popular artificial viscosities, we identify the importance of the density term and add density-related terms on the original bulk-stress based artificial viscosity. To stabilize the simulations involving strong shear layers, we design an extra shearstress based artificial viscosity. The new shock-capturing strategy helps dissipate oscillations at shocks but has negligible dissipation in smooth regions.Open Acces

Efficient Solvers for Space-Time Discontinuous Galerkin Spectral Element Methods

In this thesis we study efficient solvers for space-time discontinuous Galerkin spectral element methods (DG-SEM). These discretizations result in fully implicit schemes of variable order in both spatial and temporal directions. The popularity of space-time DG methods has increased in recent years and entropy stable space-time DG-SEM have been constructed for conservation laws, making them interesting for these applications. The size of the nonlinear system resulting from differential equations discretized with space-time DG-SEM is dependent on the order of the method, and the corresponding Jacobian is of block form with dense blocks. Thus, the problem arises to efficiently solve these huge nonlinear systems with regards to CPU time as well as memory consumption. The lack of good solvers for three-dimensional DG applications has been identified as one of the major obstacles before high order methods can be adapted for industrial applications.It has been proven that DG-SEM in time and Lobatto IIIC Runge-Kutta methods are equivalent, in that both methods lead to the same discrete solution. This allows to implement space-time DG-SEM in two ways: Either as a full space-time system or by decoupling the temporal elements and using implicit time-stepping with Lobatto IIIC methods. We compare theoretical properties and discuss practical aspects of the respective implementations.When considering the full space-time system, multigrid can be used as solver. We analyze this solver with the local Fourier analysis, which gives more insight into the efficiency of the space-time multigrid method. The other option is to decouple the temporal elements and use implicit Runge-Kutta time-stepping methods. We suggest to use Jacobian-free Newton-Krylov (JFNK) solvers since they are advantageous memory-wise. An efficient preconditioner for the Krylov sub-solver is needed to improve the convergence speed. However, we want to avoid constructing or storing the Jacobian, otherwise the favorable memory consumption of the JFNK approach would be obsolete. We present a preconditioner based on an auxiliary first order finite volume replacement operator. Based on the replacement operator we construct an agglomeration multigrid preconditioner with efficient smoothers using pseudo time integrators. Then only the Jacobian of the replacement operator needs to be constructed and the DG method is still Jacobian-free. Numerical experiments for hyperbolic test problems as the advection, advection-diffusion and Euler equations in several dimensions demonstrate the potential of the new approach

A matrix-free high-order discontinuous Galerkin compressible Navier-Stokes solver: A performance comparison of compressible and incompressible formulations for turbulent incompressible flows

Both compressible and incompressible Navier-Stokes solvers can be used and are used to solve incompressible turbulent flow problems. In the compressible case, the Mach number is then considered as a solver parameter that is set to a small value, $\mathrm{M}\approx 0.1$, in order to mimic incompressible flows. This strategy is widely used for high-order discontinuous Galerkin discretizations of the compressible Navier-Stokes equations. The present work raises the question regarding the computational efficiency of compressible DG solvers as compared to a genuinely incompressible formulation. Our contributions to the state-of-the-art are twofold: Firstly, we present a high-performance discontinuous Galerkin solver for the compressible Navier-Stokes equations based on a highly efficient matrix-free implementation that targets modern cache-based multicore architectures. The performance results presented in this work focus on the node-level performance and our results suggest that there is great potential for further performance improvements for current state-of-the-art discontinuous Galerkin implementations of the compressible Navier-Stokes equations. Secondly, this compressible Navier-Stokes solver is put into perspective by comparing it to an incompressible DG solver that uses the same matrix-free implementation. We discuss algorithmic differences between both solution strategies and present an in-depth numerical investigation of the performance. The considered benchmark test cases are the three-dimensional Taylor-Green vortex problem as a representative of transitional flows and the turbulent channel flow problem as a representative of wall-bounded turbulent flows

Efficiency of high-performance discontinuous Galerkin spectral element methods for under-resolved turbulent incompressible flows

The present paper addresses the numerical solution of turbulent flows with high-order discontinuous Galerkin methods for discretizing the incompressible Navier-Stokes equations. The efficiency of high-order methods when applied to under-resolved problems is an open issue in literature. This topic is carefully investigated in the present work by the example of the 3D Taylor-Green vortex problem. Our implementation is based on a generic high-performance framework for matrix-free evaluation of finite element operators with one of the best realizations currently known. We present a methodology to systematically analyze the efficiency of the incompressible Navier-Stokes solver for high polynomial degrees. Due to the absence of optimal rates of convergence in the under-resolved regime, our results reveal that demonstrating improved efficiency of high-order methods is a challenging task and that optimal computational complexity of solvers, preconditioners, and matrix-free implementations are necessary ingredients to achieve the goal of better solution quality at the same computational costs already for a geometrically simple problem such as the Taylor-Green vortex. Although the analysis is performed for a Cartesian geometry, our approach is generic and can be applied to arbitrary geometries. We present excellent performance numbers on modern, cache-based computer architectures achieving a throughput for operator evaluation of 3e8 up to 1e9 DoFs/sec on one Intel Haswell node with 28 cores. Compared to performance results published within the last 5 years for high-order DG discretizations of the compressible Navier-Stokes equations, our approach reduces computational costs by more than one order of magnitude for the same setup