A tetrahedral space-filling curve for non-conforming adaptive meshes

We introduce a space-filling curve for triangular and tetrahedral red-refinement that can be computed using bitwise interleaving operations similar to the well-known Z-order or Morton curve for cubical meshes. To store sufficient information for random access, we define a low-memory encoding using 10 bytes per triangle and 14 bytes per tetrahedron. We present algorithms that compute the parent, children, and face-neighbors of a mesh element in constant time, as well as the next and previous element in the space-filling curve and whether a given element is on the boundary of the root simplex or not. Our presentation concludes with a scalability demonstration that creates and adapts selected meshes on a large distributed-memory system.Comment: 33 pages, 12 figures, 8 table

Scalable Algorithms for Parallel Tree-based Adaptive Mesh Refinement with General Element Types

In this thesis, we develop, discuss and implement algorithms for scalable parallel tree-based adaptive mesh refinement (AMR) using space-filling curves (SFCs). We create an AMR software that works independently of the used element type, such as for example lines, triangles, tetrahedra, quadrilaterals, hexahedra, and prisms. For triangular and tetrahedral elements (simplices) with red-refinement (1:4 in 2D, 1:8 in 3D), we develop a new SFC, the tetrahedral Morton space-filling curve (TM-SFC). Its construction is similar to the Morton index for quadrilaterals/hexa- hedra, as it is also based on bitwise interleaving the coordinates of a certain vertex of the simplex, the anchor node. Additionally, we interleave with a new piece of information, the so called type. For these simplices, we develop element local algorithms such as constructing the parent, children, or face-neighbors of a simplex, and show that most of them are constant-time operations independent of the refinement level. With SFC based partitioning it is possible that the mesh elements that are parti- tioned to one process do not form a face-connected domain. We prove the following upper bounds for the number of face-connected components of segments of the TM-SFC: With a maximum refine- ment level of L, the number of face-connected components is bounded by 2(L − 1) in 2D and 2L + 1 in 3D. Additionally, we perform a numerical investigation of the distribution of lengths of SFC segments. Furthermore, we develop a new approach to partition and repartition a coarse (input) mesh among the processes. Compared to previous methods it optimizes for fine mesh load-balance and reduces the parallel communication of coarse mesh data. We discuss the coarse mesh repartitioning algorithm and demonstrate that our method repartitions a coarse mesh of 371e9 trees on 917,504 processes (405,000 trees per process) on the Juqueen supercomputer in 1.2 seconds. We develop an AMR concept that works independently of the element type; achieving this independence by strictly distinguishing between functions that oper- ate on the whole mesh (high-level) and functions that locally operate on a single element or a small set of elements (low-level). We discuss a new approach to generate and manage ghost elements that fits into our element-type independent approach. We define and describe the necessary low-level algorithms. Our main idea is the computation of tree-to-tree face-neighbors of an element via the explicit construction of the element's face as a lower dimensional element. In order to optimize the runtime of this method we enhance the algorithm with a top-down search method from Isaac, Burstedde, Wilcox, and Ghattas, and demonstrate how it speeds up the computation by factors of 10 to 20 achieving runtimes comparable to state-of-the art implementations with fixed element types. With the ghost algorithm we build a straight-forward ripple version of the 2:1 balance algorithm. This is not an optimized version but it serves as a feasibility study for our element-type independent approach. We implement all algorithms that we develop in this thesis in the new AMR library t8code. Our modular approach allows us to reuse existing software, which we demonstrate by using the library p4est for quadrilateral and hexahedral elements. In a concurrent Bachelor's thesis by David Knapp (INS, Bonn) the necessary low-level algorithms for prisms were developed. With t8code we demonstrate that we can create, adapt, (re-)partition, and balance meshes, as well as create and manage a ghost layer. In various tests we show excellent strong and weak scaling behavior of our algorithms on up to 917,504 parallel processes on the Juqueen and Mira supercomputers using up to 858e9 mesh elements. We conclude this thesis by demonstrating how an application can be coupled with the AMR routines. We implement a finite volume based advection solver using t8code and show applications with triangular, quadrilateral, tetrahedral, and hexahedral elements, as well as 2D and 3D hybrid meshes, the latter consisting of tetrahedra, hexahedra, and prisms. Overall, we develop and demonstrate a new simplicial SFC and create a fast and scalable tree-based AMR software that offers a flexibility and generality that was previously not available

Scalable Algorithms for Parallel Tree-based Adaptive Mesh Refinement with General Element Types

In this thesis, we develop, discuss and implement algorithms for scalable parallel tree-based adaptive mesh refinement (AMR) using space-filling curves (SFCs). We create an AMR software that works independently of the used element type, such as for example lines, triangles, tetrahedra, quadrilaterals, hexahedra, and prisms. Along with a detailed mathematical discussion, this requires the implementation as a numerical software and its validation, as well as scalability tests on current supercomputers. For triangular and tetrahedral elements (simplices) with red-refinement (1:4 in 2D, 1:8 in 3D), we develop a new SFC index, the tetrahedral Morton index (TM-index). Its construction is similar to the Morton index for quadrilaterals/hexahedra, as it is also based on bitwise interleaving the coordinates of a certain vertex of the simplex, the anchor node. We develop and demonstrate a new simplicial SFC and create a fast and scalable tree-based AMR software that offers a flexibility and generality that was previously not available.Comment: 200 Pages, dissertation, 58 figures, Bonn (2018

Advanced git techniques

In this talk i present different advanced git techniques that i learned in my years of working with git. This talk is associated with the repository www.github.com/holke/git-advanced-workshop. I talk about git cherry-pick, interactive rebase, git reflog, hooks, filter, worktrees, git bisect. Each topic has a brief introduction and you will find more information and exercises in the github repository

The Space Filling Curve Needle

The following slides are our contribution to the Meshing Contest of the International Meshing Roundtable 2022. We show a hybrid adaptively refined mesh of the Seattle Spaceneedle

The nutritive value of black walnuts

Data contained in the paper were taken from theses submitted by Dorothy Poertner Tyrrell and Mary Holke Jenkins in partial fulfillment of the requirements for the degree of Master of Arts in the Graduate School of the University of Missouri--P. [3].Digitized 2007 AES.Includes bibliographical references (pages 10-[12])

An Optimized, Parallel Computation of the Ghost Layer for Adaptive Hybrid Forest Meshes

We discuss parallel algorithms to compute the ghost layer in computational, distributed memory, recursively adapted meshes. Its creation is a fundamental, necessary task in executing most parallel, element-based computer simulations. Common methods differ in that the ghost layer may either be inherently part of the mesh data structure that is maintained and modified, or kept separate and constructed/deleted as needed. In this work, we present a design following the latter approach, which we chose for its modularity of algorithms and data structures. We target arbitrary adaptive, nonconforming forest-of-trees meshes of mixed element shapes, such as cubes, prisms, and tetrahedra, and restrict ourselves to ghost elements across mesh faces. Our algorithm has low code complexity and redundancy since we reduce it to generic codimension-1 subalgorithms that can be flexibly combined. We recover older algorithms for cubic elements as special cases and optimize further using recursive, amortized tree searches and traversals

Dynamic Adaptive Refinement In Earth System Modelling

: Increasing the mesh resolution is one the most important tools for increasing the accuracy of numerical simulations. However, increasing the mesh resolution globally increases the amount of data and computational time substantially. In the exascale-era sub-km meshes are becoming more and more popular for atmospheric models, making it especially difficult to manage the vast amount of data efficiently. With dynamic adaptive mesh refinement (AMR) we locally control the resolution of a mesh in areas of interest, using a fine resolution only where it is explicitely needed and keeping the mesh coarse elsewhere. Thus, we concentrate the data and computing power and significantly reduce the simulation costs while keeping the same numerical accuracy. Vice versa, the resolution can be increased while keeping the same runtime. Managing adaptive meshes induces new challenges such as load-balancing, mesh management, ghost layer computation etc. Developments in the recent years have extended the scalable and efficicient tree-based AMR approach from quadrilaterals/hexahedra to various element shapes such as triangles, tetrahedra, pyramids or prisms and have been implemented in our AMR library t8code. It is a third-party library that adresses these challenges and can be integrated by solver environments in order to enable AMR. In our presentation we will give an introduction to AMR and how we use it for atmospheric simulations. We will give an overview of ongoing and past projects, such as our contributions to PilotLab Exascale Earth Sytem Modelling (Pl-ExaESM) or the lossy data compression for data coming from atmospheric simulations. Furthermore, we will demonstrate the efficiency of our methods with recent benchmark results on current supercomputers, showing that t8code scales on up to at least 1 million MPI ranks and over 1 Trillion mesh elements