A structure-exploiting numbering algorithm for finite elements on extruded meshes, and its performance evaluation in Firedrake

We present a generic algorithm for numbering and then efficiently iterating over the data values attached to an extruded mesh. An extruded mesh is formed by replicating an existing mesh, assumed to be unstructured, to form layers of prismatic cells. Applications of extruded meshes include, but are not limited to, the representation of three-dimensional high aspect ratio domains employed by geophysical finite element simulations. These meshes are structured in the extruded direction. The algorithm presented here exploits this structure to avoid the performance penalty traditionally associated with unstructured meshes. We evaluate the implementation of this algorithm in the Firedrake finite element system on a range of low compute intensity operations which constitute worst cases for data layout performance exploration. The experiments show that having structure along the extruded direction enables the cost of the indirect data accesses to be amortized after 10–20 layers as long as the underlying mesh is well ordered. We characterize the resulting spatial and temporal reuse in a representative set of both continuous-Galerkin and discontinuous-Galerkin discretizations. On meshes with realistic numbers of layers the performance achieved is between 70 and 90 % of a theoretical hardware-specific limit

Development of a Three-Dimensional High-Order Strand-Grids Approach

Development of a novel high-order flux correction method on strand grids is presented. The method uses a combination of flux correction in the unstructured plane and summation-by-parts operators in the strand direction to achieve high-fidelity solutions. Low-order truncation errors are cancelled with accurate flux and solution gradients in the flux correction method, thereby achieving a formal order of accuracy of 3, although higher orders are often obtained, especially for highly viscous flows. In this work, the scheme is extended to high-Reynolds number computations in both two and three dimensions. Turbulence closure is achieved with a robust version of the Spalart-Allmaras turbulence model that accommodates negative values of the turbulence working variable, and the Menter SST turbulence model, which blends the k-ε and k-ω turbulence models for better accuracy. A major advantage of this high-order formulation is the ability to implement traditional finite volume-like limiters to cleanly capture shocked and discontinuous flow. In this work, this approach is explored via a symmetric limited positive (SLIP) limiter. Extensive verification and validation is conducted in two and three dimensions to determine the accuracy and fidelity of the scheme for a number of different cases. Verification studies show that the scheme achieves better than third order accuracy for low and high-Reynolds number flow. Cost studies show that in three-dimensions, the third-order flux correction scheme requires only 30% more walltime than a traditional second-order scheme on strand grids to achieve the same level of convergence. In order to overcome meshing issues at sharp corners and other small-scale features, a unique approach to traditional geometry, coined asymptotic geometry, is explored. Asymptotic geometry is achieved by filtering out small-scale features in a level set domain through min/max flow. This approach is combined with a curvature based strand shortening strategy in order to qualitatively improve strand grid mesh quality

Stencil selection algorithms for WENO schemes on unstructured meshes

In this paper, a family of stencil selection algorithms is presented for WENO schemes on unstructured meshes. The associated freedom of stencil selection for unstructured meshes, in the context of WENO schemes present a plethora of various stencil selection algorithms. The particular focus of this paper is to assess the performance of various stencil selection algorithm, investigate the parameters that dictate their robustness, accuracy and computational efficiency. Ultimately, efficient and robust stencils are pursued that can provide significant savings in computational performance, while retaining the non-oscillatory character of WENO schemes. This is achieved when making the stencil selection algorithms adaptive, based on the quality of the cells for unstructured meshes, that can in turn reduce the computational cost of WENO schemes. For assessing the performance of the developed algorithms well established test problems are employed. These include the least square approximation of polynomial functions, linear advection equation of smooth functions and solid body rotation test problem. Euler and Navier-Stokes equations test problems are also pursued such as the Shu-Osher test problem, the Double Mach Reflection, the supersonic Forward Facing step, the Kelvin-Helmholtz instability, the Taylor-Green Vortex, and the flow past a transonic circular cylinder

The Dune framework: Basic concepts and recent developments

This paper presents the basic concepts and the module structure of the Distributed and Unified Numerics Environment and reflects on recent developments and general changes that happened since the release of the first Dune version in 2007 and the main papers describing that state Bastian etal. (2008a, 2008b). This discussion is accompanied with a description of various advanced features, such as coupling of domains and cut cells, grid modifications such as adaptation and moving domains, high order discretizations and node level performance, non-smooth multigrid methods, and multiscale methods. A brief discussion on current and future development directions of the framework concludes the paper

On the development and implementation of high-order flux reconstruction schemes for computational fluid dynamics

High-order numerical methods for unstructured grids combine the superior accuracy of high-order spectral or finite difference methods with the geometric flexibility of low-order finite volume or finite element schemes. The Flux Reconstruction (FR) approach unifies various high-order schemes for unstructured grids within a single framework. Additionally, the FR approach exhibits a significant degree of element locality, and is thus able to run efficiently on modern streaming architectures, such as graphics processing units (GPUs). The aforementioned properties of FR mean it offers a promising route to performing affordable, and hence industrially relevant, scale-resolving simulations of hitherto intractable unsteady flows within the vicinity of real-world engineering geometries. In this thesis a formulation of the FR approach that is suitable for solving non-linear advection-diffusion type problems on mixed curvilinear grids is developed. Issues around aliasing are explored in detail and techniques for mitigation outlined. A methodology for identifying symmetric quadrature rules inside of a variety of domains is also presented and used to find several rules that appear to be an improvement over those in literature. This methodology is also used to obtain improved sets of solution points inside of triangular elements. PyFR, an open-source Python based framework for solving the compressible Navier–Stokes equations using the FR approach, is also developed. It is designed to target a range of hardware platforms via use of an in-built domain specific language based on the Mako templating engine. PyFR is able to operate on mixed grids in both two and three dimensions and can target NVIDIA GPUs, AMD GPUs, and Intel CPUs. Results are presented for various benchmark flow problems, single-node performance is discussed, heterogeneous multi-node capabilities are analysed, and scalability is demonstrated on up to 2000 NVIDIA K20X GPUs for a sustained performance of 1.3 PFLOP/s.Open Acces