Recursive Algorithms for Distributed Forests of Octrees

The forest-of-octrees approach to parallel adaptive mesh refinement and coarsening (AMR) has recently been demonstrated in the context of a number of large-scale PDE-based applications. Although linear octrees, which store only leaf octants, have an underlying tree structure by definition, it is not often exploited in previously published mesh-related algorithms. This is because the branches are not explicitly stored, and because the topological relationships in meshes, such as the adjacency between cells, introduce dependencies that do not respect the octree hierarchy. In this work we combine hierarchical and topological relationships between octree branches to design efficient recursive algorithms. We present three important algorithms with recursive implementations. The first is a parallel search for leaves matching any of a set of multiple search criteria. The second is a ghost layer construction algorithm that handles arbitrarily refined octrees that are not covered by previous algorithms, which require a 2:1 condition between neighboring leaves. The third is a universal mesh topology iterator. This iterator visits every cell in a domain partition, as well as every interface (face, edge and corner) between these cells. The iterator calculates the local topological information for every interface that it visits, taking into account the nonconforming interfaces that increase the complexity of describing the local topology. To demonstrate the utility of the topology iterator, we use it to compute the numbering and encoding of higher-order $C^0$ nodal basis functions. We analyze the complexity of the new recursive algorithms theoretically, and assess their performance, both in terms of single-processor efficiency and in terms of parallel scalability, demonstrating good weak and strong scaling up to 458k cores of the JUQUEEN supercomputer.Comment: 35 pages, 15 figures, 3 table

Discretely exact derivatives for hyperbolic PDE-constrained optimization problems discretized by the discontinuous Galerkin method

This paper discusses the computation of derivatives for optimization problems governed by linear hyperbolic systems of partial differential equations (PDEs) that are discretized by the discontinuous Galerkin (dG) method. An efficient and accurate computation of these derivatives is important, for instance, in inverse problems and optimal control problems. This computation is usually based on an adjoint PDE system, and the question addressed in this paper is how the discretization of this adjoint system should relate to the dG discretization of the hyperbolic state equation. Adjoint-based derivatives can either be computed before or after discretization; these two options are often referred to as the optimize-then-discretize and discretize-then-optimize approaches. We discuss the relation between these two options for dG discretizations in space and Runge-Kutta time integration. Discretely exact discretizations for several hyperbolic optimization problems are derived, including the advection equation, Maxwell's equations and the coupled elastic-acoustic wave equation. We find that the discrete adjoint equation inherits a natural dG discretization from the discretization of the state equation and that the expressions for the discretely exact gradient often have to take into account contributions from element faces. For the coupled elastic-acoustic wave equation, the correctness and accuracy of our derivative expressions are illustrated by comparisons with finite difference gradients. The results show that a straightforward discretization of the continuous gradient differs from the discretely exact gradient, and thus is not consistent with the discretized objective. This inconsistency may cause difficulties in the convergence of gradient based algorithms for solving optimization problems

Rapid turnover of effector-memory CD4(+) T cells in healthy humans

Memory T cells can be divided into central-memory (T(CM)) and effector-memory (T(EM)) cells, which differ in their functional properties. Although both subpopulations can persist long term, it is not known whether they are maintained by similar mechanisms. We used in vivo labeling with deuterated glucose to measure the turnover of CD4(+) T cells in healthy humans. The CD45R0(+)CCR7(-) T(EM) subpopulation was shown to have a rapid proliferation rate of 4.7% per day compared with 1.5% per day for CD45R0(+)CCR7(+) T(CM) cells; these values are equivalent to average intermitotic (doubling) times of 15 and 48 d, respectively. In contrast, the CD45RA(+)CCR7(+) naive CD4(+) T cell population was found to be much longer lived, being labeled at a rate of only 0.2% per day (corresponding to an intermitotic time of approximately 1 yr). These data indicate that human CD4(+) T(EM) cells constitute a short-lived cell population that requires continuous replenishment in vivo

Rapid turnover of T cells in acute infectious mononucleosis.

During acute infectious mononucleosis (AIM), large clones of Epstein-Barr virus-specific T lymphocytes are produced. To investigate the dynamics of clonal expansion, we measured cell proliferation during AIM using deuterated glucose to label DNA of dividing cells in vivo, analyzing cells according to CD4, CD8 and CD45 phenotype. The proportion of labeled CD8(+)CD45R0(+) T lymphocytes was dramatically increased in AIM subjects compared to controls (mean 17.5 versus 2.8%/day; p<0.005), indicating very rapid proliferation. Labeling was also increased in CD4(+)CD45R0(+) cells (7.1 versus 2.1%/day; p<0.01), but less so in CD45RA(+) cells. Mathematical modeling, accounting for death of labeled cells and changing pool sizes, gave estimated proliferation rates in CD8(+)CD45R0(+) cells of 11-130% of cells proliferating per day (mean 47%/day), equivalent to a doubling time of 1.5 days and an appearance rate in blood of about 5 x 10(9) cells/day (versus 7 x 10(7) cells/day in controls). Very rapid death rates were also observed amongst labeled cells (range 28-124, mean 57%/day),indicating very short survival times in the circulation. Thus, we have shown direct evidence for massive proliferation of CD8(+)CD45R0(+) T lymphocytes in AIM and demonstrated that rapid cell division continues concurrently with greatly accelerated rates of cell disappearance

Slab stress and strain rate as constraints on global mantle flow

Dynamically consistent global models of mantle convection with plates are developed that are consistent with detailed constraints on the state of stress and strain rate from deep focus earthquakes. Models that best fit plateness criteria and plate motion data have strong slabs that have high stresses. The regions containing the M_W 8.3 Bolivia and M_W 7.6 Tonga 1994 events are considered in detail. Modeled stress orientations match stress patterns from earthquake focal mechanisms. A yield stress of at least 100 MPa is required to fit plate motions and matches the minimum stress requirement obtained from the stress drop for the Bolivia 1994 deep focus event. The minimum strain rate determined from seismic moment release in the Tonga slab provides an upper limit of ~200 MPa on the strength in the slab

Large-scale adaptive mantle convection simulation

A new generation, parallel adaptive-mesh mantle convection code, Rhea, is described and benchmarked. Rhea targets large-scale mantle convection simulations on parallel computers, and thus has been developed with a strong focus on computational efficiency and parallel scalability of both mesh handling and numerical solvers. Rhea builds mantle convection solvers on a collection of parallel octree-based adaptive finite element libraries that support new distributed data structures and parallel algorithms for dynamic coarsening, refinement, rebalancing and repartitioning of the mesh. In this study we demonstrate scalability to 122 880 compute cores and verify correctness of the implementation. We present the numerical approximation and convergence properties using 3-D benchmark problems and other tests for variable-viscosity Stokes flow and thermal convection

Scalable Adaptive Mantle Convection Simulation on Petascale Supercomputers

Mantle convection is the principal control on the thermal and geological evolution of the Earth. Mantle convection modeling involves solution of the mass, momentum, and energy equations for a viscous, creeping, incompressible non-Newtonian fluid at high Rayleigh and Peclet numbers. Our goal is to conduct global mantle convection simulations that can resolve faulted plate boundaries, down to 1 km scales. However, uniform resolution at these scales would result in meshes with a trillion elements, which would elude even sustained petaflops supercomputers. Thus parallel adaptive mesh refinement and coarsening (AMR) is essential. We present RHEA, a new generation mantle convection code designed to scale to hundreds of thousands of cores. RHEA is built on ALPS, a parallel octree-based adaptive mesh finite element library that provides new distributed data structures and parallel algorithms for dynamic coarsening, refinement, rebalancing, and repartitioning of the mesh. ALPS currently supports low order continuous Lagrange elements, and arbitrary order discontinuous Galerkin spectral elements, on octree meshes. A forest-ofoctrees implementation permits nearly arbitrary geometries to be accommodated. Using TACC’s 579 teraflops Ranger supercomputer, we demonstrate excellent weak and strong scalability of parallel AMR on up to 62,464 cores for problems with up to 12.4 billion elements. With RHEA’s adaptive capabilities, we have been able to reduce the number of elements by over three orders of magnitude, thus enabling us to simulate large-scale mantle convection with finest local resolution of 1.5 km

Implications of Synchronous IVR Radio on Syrian Refugee Health and Community Dynamics

With 1,033,513 Syrian refugees adding a strain on the Lebanese healthcare system, innovation is key to improving access to healthcare. Our previous work identified the potential for technology to improve access to antenatal care services and increase refugee agency. Using (1) paper mock ups and a mobile based prototype, (2) process mapping, (3) focus groups and interviews and (4) key informant meetings, we explored the concept of refugee led community radio shows to deliver peer-led healthcare. We observed the influence of community radio shows on Syrian refugee health education, community dynamics and community agency in relationships between healthcare providers and refugees. Refugees were positively impacted through situating the technology within the community. We highlight issues around trust, agency, understanding, sel-forganization and privacy that resulted from running the shows through mock ups and a mobile based prototype. Our findings inform future work in community run radio shows

The hypertension cascade of care in the midst of conflict: the case of the Gaza Strip

Although hypertension constitutes a substantial burden in conflict-affected areas, little is known about its prevalence, control, and management in Gaza. This study aims to estimate the prevalence and correlates of hypertension, its diagnosis and control among adults in Gaza. We conducted a representative, cross-sectional, anonymous, household survey of 4576 persons older than 40 years in Gaza in mid-2020. Data were collected through face-to-face interviews, anthropometric, and blood pressure measurements. Hypertension was defined in anyone with an average systolic blood pressure ≥140 mmHg or average diastolic blood pressure ≥90 mmHg from two consecutive readings or a hypertension diagnosis. The mean age of participants was 56.9 ± 10.5 years, 54.0% were female and 68.5% were Palestinian refugees. The prevalence of hypertension was 56.5%, of whom 71.5% had been diagnosed. Hypertension was significantly higher among older participants, refugees, ex-smokers, those who were overweight or obese, and had other co-morbidities including mental illnesses. Two-thirds (68.3%) of those with hypertension were on treatment with one in three (35.6%) having their hypertension controlled. Having controlled hypertension was significantly higher in females, those receiving all medications for high blood pressure and those who never or rarely added salt to food. Investing in comprehensive but cost-effective initiatives that strengthen the prevention, early detection and timely treatment of hypertension in conflict settings is critical. It is essential to better understand the underlying barriers behind the lack of control and develop multi-sectoral programs to address these barriers