Efficient distributed matrix-free multigrid methods on locally refined meshes for FEM computations

This work studies three multigrid variants for matrix-free finite-element computations on locally refined meshes: geometric local smoothing, geometric global coarsening, and polynomial global coarsening. We have integrated the algorithms into the same framework-the open-source finite-element library deal.II-, which allows us to make fair comparisons regarding their implementation complexity, computational efficiency, and parallel scalability as well as to compare the measurements with theoretically derived performance models. Serial simulations and parallel weak and strong scaling on up to 147,456 CPU cores on 3,072 compute nodes are presented. The results obtained indicate that global coarsening algorithms show a better parallel behavior for comparable smoothers due to the better load balance particularly on the expensive fine levels. In the serial case, the costs of applying hanging-node constraints might be significant, leading to advantages of local smoothing, even though the number of solver iterations needed is slightly higher.Comment: 34 pages, 17 figure

Developments in overlapping Schwarz preconditioning of high-order nodal discontinuous Galerkin discretizations

A preconditioned two-level overlapping Schwarz method for solving unstructured nodal discontinuous Galerkin discretizations of the indefinite Helmholtz problem is studied. We employ triangles in two dimensions and in a local discontinuous Galerkin (LDG) variational setting. We highlight the necessary components of the algorithm needed to achieve efficient results in the context of high-order elements and indefinite algebraic systems. More specifically, we demonstrate the importance of not only coarse-grid solution sweeps, but also for increased overlap in the subdomain solves as the order of the elements increases. In this paper, we detail the discretization strategy and offer an effective approach to solving the resulting system of equations, with numerical evidence in support

hp-FEM for Two-component Flows with Applications in Optofluidics

This thesis is concerned with the application of hp-adaptive finite element methods to a mathematical model of immiscible two-component flows. With the aim of simulating the flow processes in microfluidic optical devices based on liquid-liquid interfaces, we couple the time-dependent incompressible Navier-Stokes equations with a level set method to describe the flow of the fluids and the evolution of the interface between them

Matrix-free finite-element computations at extreme scale and for challenging applications

For numerical computations based on finite element methods (FEM), it is common practice to assemble the system matrix related to the discretized system and to pass this matrix to an iterative solver. However, the assembly step can be costly and the matrix might become locally dense, e.g., in the context of high-order, high-dimensional, or strongly coupled multicomponent FEM, leading to high costs when applying the matrix due to limited bandwidth on modern CPU- and GPU-based hardware. Matrix-free algorithms are a means of accelerating FEM computations on HPC systems, by applying the effect of the system matrix without assembling it. Despite convincing arguments for matrix-free computations as a means of improving performance, their usage still tends to be an exception at the time of writing of this thesis, not least because they have not yet proven their applicability in all areas of computational science, e.g., solid mechanics. In this thesis, we further develop a state-of-the-art matrix-free framework for high-order FEM computations with focus on the preconditioning and adopt it in novel application fields. In the context of high-order FEM, we develop means of improving cache efficiency by interleaving cell loops with vector updates, which we use to increase the throughput of preconditioned conjugate gradient methods and of block smoothers based on additive Schwarz methods; we also propose an algorithm for the fast application of hanging-node constraints in 3D for up to 137 refinement configurations. We develop efficient geometric and polynomial multigrid solvers with optimized transfer operators, whose performance is experimentally investigated in detail in the context of locally refined meshes, indicating the superiority of global-coarsening algorithms. We apply the developed solvers in the context of novel stage-parallel implicit Runge–Kutta methods and demonstrate the benefit of stage–parallel solvers in decreasing the time to solution at the scaling limit. Novel challenging application fields of matrix-free computations include high-dimensional computational plasma physics, solid-state-sintering simulations with a high and dynamically changing number of strongly coupled components, and coupled multiphysics problems with evaluation and integration at arbitrary points. In the context of these fields, we detail computational challenges, propose modified versions of the standard matrix-free algorithms for high-performance computing, and discuss preconditioning-related topics. The efficiency of the derived algorithms on the node level and at extreme scales is demonstrated experimentally on SuperMUC-NG, one of Germany’s leading supercomputers, with up to 150k processes and by solving systems of up to 5 × 1012 unknowns. Such problem sizes would not be conceivable for equivalent matrix-based algorithms. The major achievements of this thesis allow to run larger simulations faster and more efficiently, enabling progress and new possibilities for a range of application fields in computational science

Multiscale analysis of localized, nonlinear, three-dimensional thermo-structural effects

There are a wide range of computational modeling challenges associated with structures subjected to sharp, local heating effects. Problems of this nature are prevalent in diverse engineering applications such as structural analysis of hypersonic flight vehicles in extreme environments, computational modeling of weld processes, and development of semiconductor processing technology. Complex temperature gradients in the materials cause three-dimensional, localized, intense thermomechanical stress/strain variation and residual deformations, making multiphysics analysis necessary to accurately predict structural response. Localized damage or deformation may impact global structural behavior, yet bridging spatial scales between local- and structural-scale response is a nontrivial task. Because of these issues, standard finite element analysis techniques lead to cumbersome and prohibitively expensive numerical simulations for this class of problems. This study proposes a Generalized or eXtended Finite Element Method (G/XFEM) for analyzing three-dimensional solid, coupled physics problems exhibiting localized heating and thermomechanical effects. The method is based on the GFEM with global–local enrichment functions (GFEMgl), which involves the solution of interdependent coarse- (global) and fine-scale (local) problems. The global problem captures coarse-scale behavior, while local problems resolve sharp solution features in regions where fine-scale phenomena may govern the overall structural response. To address the intrinsic coupling of scales, local solution information is embedded in the global solution space via a partition of unity approach. This method extends the capabilities of traditional hp-adaptive FEM or GFEM—consisting of heavy mesh refinement (h) and local high-order polynomial approximations (p)—to one-way coupled thermo-structural problems, providing meshing flexibility while remaining accurate and efficient. Linear thermoelasticity and nonlinear thermoplasticity problems are considered, involving both steady-state and transient heating effects. The GFEMgl is further extended to capture multiscale thermal and thermomechanical effects induced by material-scale heterogeneity, which may also impact structural behavior at the coarse scale. Due to the extraordinary level of fidelity required to resolve fine-scale effects at the global scale, strategies for distributing large workloads on a parallel computer and improving the computational efficiency of the proposed method are needed. Studies have shown that the GFEMgl benefits from straightforward parallelism. However, inexact, coarse-scale boundary conditions on fine-scale may lead to large errors in global solutions. Traditional strategies aimed at improving or otherwise lessening the effect of poor local boundary conditions in the GFEMgl may be impractically expensive in the problems of interest, such as transient or nonlinear simulations involving many time or load steps. Thus, inexpensive and optimized approaches for improving boundary conditions on local problems in both linear and nonlinear problems are identified. The performance of the method is assessed on representative, large-scale, nonlinear, coupled thermo-structural problems exhibiting phenomena spanning global (structural) and local (component or even material) scales