Parthenon -- a performance portable block-structured adaptive mesh refinement framework

On the path to exascale the landscape of computer device architectures and corresponding programming models has become much more diverse. While various low-level performance portable programming models are available, support at the application level lacks behind. To address this issue, we present the performance portable block-structured adaptive mesh refinement (AMR) framework Parthenon, derived from the well-tested and widely used Athena++ astrophysical magnetohydrodynamics code, but generalized to serve as the foundation for a variety of downstream multi-physics codes. Parthenon adopts the Kokkos programming model, and provides various levels of abstractions from multi-dimensional variables, to packages defining and separating components, to launching of parallel compute kernels. Parthenon allocates all data in device memory to reduce data movement, supports the logical packing of variables and mesh blocks to reduce kernel launch overhead, and employs one-sided, asynchronous MPI calls to reduce communication overhead in multi-node simulations. Using a hydrodynamics miniapp, we demonstrate weak and strong scaling on various architectures including AMD and NVIDIA GPUs, Intel and AMD x86 CPUs, IBM Power9 CPUs, as well as Fujitsu A64FX CPUs. At the largest scale on Frontier (the first TOP500 exascale machine), the miniapp reaches a total of $1.7\times10^{13}$ zone-cycles/s on 9,216 nodes (73,728 logical GPUs) at ~92% weak scaling parallel efficiency (starting from a single node). In combination with being an open, collaborative project, this makes Parthenon an ideal framework to target exascale simulations in which the downstream developers can focus on their specific application rather than on the complexity of handling massively-parallel, device-accelerated AMR.Comment: 17 pages, 11 figures, accepted for publication in IJHPCA, Codes available at https://github.com/parthenon-hpc-la

Thermodynamics and segregation behavior of nanoscale solids.

We present the results of an extensive atomistic study of the thermodynamics of and segregation behavior in nanoscale solids: namely, thin films and spherical particles. The primary goal of this study was to determine the effects of system size on the surface concentration and the surface free energy based upon atomic scale simulations. We also present a theoretical analysis of the effects of system size on surface segregation. This theory predicts surface composition as a function of temperature, size and the heat of segregation in a bulk system. Using this theory we can predict the surface free energy of nanoscale solids as a function of size using only values of the corresponding properties for bulk materials. The simulations show that the surface free energies of Cu-Ni thin films differ significantly from those of corresponding surfaces bounding bulk samples. Very thin films exhibit surface entropies of opposite sign relative to those for the bulk. Small particles, with low Cu concentrations also exhibit surface entropies of opposite sign to those for the bulk. Both the thin films and the particles undergo a transition from positive to negative surface entropy with increasing size, Cu concentration and/or temperature. Increasing the average Cu concentration shifts the crossover to smaller sizes. In elemental particles, the variation of the surface properties with particle size is dominated by a geometrical effect associated with the discrete spacing between atomic planes. In binary alloys, on the other hand, the limited solute content within the particle dominates the size dependence of the properties. Phase diagrams for small, spherical Au-Pt particles are very different from that of the bulk. Decreasing particle size leads to increased solubility of Au in Pt and a smaller decrease in the solubility of Pt in Au. Additionally, small particles exhibit miscibility gaps extending to higher temperatures relative to the bulk.Ph.D.Materials Science and Engineering and Scientific ComputingUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/104869/1/9610247.pdfDescription of 9610247.pdf : Restricted to UM users only