Ookami: An A64FX Computing Resource

We present a look at Ookami, a project providing community access to a testbed supercomputer with the ARM-based A64FX processors developed by a collaboration between RIKEN and Fujitsu and deployed in the Japanese supercomputer Fugaku. We provide an overview of the project and details of the hardware, and describe the user base and education/training program. We present highlights from previous performance studies of two astrophysical simulation codes and present a strong scaling study of a full 3D supernova simulation as an example of the the machine’s capability

A New Monte Carlo Method for Time-Dependent Neutrino Radiation Transport

Monte Carlo approaches to radiation transport have several attractive properties compared to deterministic methods. These include simplicity of implementation, high accuracy, and good parallel scaling. Moreover, Monte Carlo methods can handle complicated geometries and are relatively easy to extend to multiple spatial dimensions, which makes them particularly interesting in modeling complex multi-dimensional astrophysical phenomena such as core-collapse supernovae. The aim of this paper is to explore Monte Carlo methods for modeling neutrino transport in core-collapse supernovae. We generalize the implicit Monte Carlo photon transport scheme of Fleck & Cummings and gray discrete-diffusion scheme of Densmore et al. to energy-, time-, and velocity-dependent neutrino transport. Using our 1D spherically-symmetric implementation, we show that, similar to the photon transport case, the implicit scheme enables significantly larger timesteps compared with explicit time discretization, without sacrificing accuracy, while the discrete-diffusion method leads to significant speed-ups at high optical depth. Our results suggest that a combination of spectral, velocity-dependent, implicit Monte Carlo and discrete-diffusion Monte Carlo methods represents an attractive approach for use in neutrino radiation-hydrodynamics simulations of core-collapse supernovae. Our velocity-dependent scheme can easily be adapted to photon transport

A New Open-Source Code for Spherically-Symmetric Stellar Collapse to Neutron Stars and Black Holes

We present the new open-source spherically-symmetric general-relativistic (GR) hydrodynamics code GR1D. It is based on the Eulerian formulation of GR hydrodynamics (GRHD) put forth by Romero-Ibanez-Gourgoulhon and employs radial-gauge, polar-slicing coordinates in which the 3+1 equations simplify substantially. We discretize the GRHD equations with a finite-volume scheme, employing piecewise-parabolic reconstruction and an approximate Riemann solver. GR1D is intended for the simulation of stellar collapse to neutron stars and black holes and will also serve as a testbed for modeling technology to be incorporated in multi-D GR codes. Its GRHD part is coupled to various finite-temperature microphysical equations of state in tabulated form that we make available with GR1D. An approximate deleptonization scheme for the collapse phase and a neutrino-leakage/heating scheme for the postbounce epoch are included and described. We also derive the equations for effective rotation in 1D and implement them in GR1D. We present an array of standard test calculations and also show how simple analytic equations of state in combination with presupernova models from stellar evolutionary calculations can be used to study qualitative aspects of black hole formation in failing rotating core-collapse supernovae. In addition, we present a simulation with microphysical EOS and neutrino leakage/heating of a failing core-collapse supernova and black hole formation in a presupernova model of a 40 solar mass zero-age main-sequence star. We find good agreement on the time of black hole formation (within 20%) and last stable protoneutron star mass (within 10%) with predictions from simulations with full Boltzmann neutrino radiation hydrodynamics.Comment: 25 pages, 6 figures, 2 appendices. Accepted for publication to the Classical and Quantum Gravity special issue for MICRA2009. Code may be downloaded from http://www.stellarcollapse.org Update: corrected title, small modifications suggested by the referees, added source term derivation in appendix

Post-Newtonian SPH calculations of binary neutron star coalescence. II. Binary mass ratio, equation of state, and spin dependence

Using our new Post-Newtonian SPH (smoothed particle hydrodynamics) code, we study the final coalescence and merging of neutron star (NS) binaries. We vary the stiffness of the equation of state (EOS) as well as the initial binary mass ratio and stellar spins. Results are compared to those of Newtonian calculations, with and without the inclusion of the gravitational radiation reaction. We find a much steeper decrease in the gravity wave peak strain and luminosity with decreasing mass ratio than would be predicted by simple point-mass formulae. For NS with softer EOS (which we model as simple $\Gamma=2$ polytropes) we find a stronger gravity wave emission, with a different morphology than for stiffer EOS (modeled as $\Gamma=3$ polytropes as in our previous work). We also calculate the coalescence of NS binaries with an irrotational initial condition, and find that the gravity wave signal is relatively suppressed compared to the synchronized case, but shows a very significant second peak of emission. Mass shedding is also greatly reduced, and occurs via a different mechanism than in the synchronized case. We discuss the implications of our results for gravity wave astronomy with laser interferometers such as LIGO, and for theoretical models of gamma-ray bursts (GRBs) based on NS mergers.Comment: RevTeX, 38 pages, 24 figures, Minor Corrections, to appear in Phys. Rev.

Coalescing Binary Neutron Stars

Coalescing compact binaries with neutron star or black hole components provide the most promising sources of gravitational radiation for detection by the LIGO/VIRGO/GEO/TAMA laser interferometers now under construction. This fact has motivated several different theoretical studies of the inspiral and hydrodynamic merging of compact binaries. Analytic analyses of the inspiral waveforms have been performed in the Post-Newtonian approximation. Analytic and numerical treatments of the coalescence waveforms from binary neutron stars have been performed using Newtonian hydrodynamics and the quadrupole radiation approximation. Numerical simulations of coalescing black hole and neutron star binaries are also underway in full general relativity. Recent results from each of these approaches will be described and their virtues and limitations summarized.Comment: Invited Topical Review paper to appear in Classical and Quantum Gravity, 35 pages, including 5 figure

Tidally-induced thermonuclear Supernovae

We discuss the results of 3D simulations of tidal disruptions of white dwarfs by moderate-mass black holes as they may exist in the cores of globular clusters or dwarf galaxies. Our simulations follow self-consistently the hydrodynamic and nuclear evolution from the initial parabolic orbit over the disruption to the build-up of an accretion disk around the black hole. For strong enough encounters (pericentre distances smaller than about 1/3 of the tidal radius) the tidal compression is reversed by a shock and finally results in a thermonuclear explosion. These explosions are not restricted to progenitor masses close to the Chandrasekhar limit, we find exploding examples throughout the whole white dwarf mass range. There is, however, a restriction on the masses of the involved black holes: black holes more massive than $2\times 10^5$ M$_\odot$ swallow a typical 0.6 M$_\odot$ dwarf before their tidal forces can overwhelm the star's self-gravity. Therefore, this mechanism is characteristic for black holes of moderate masses. The material that remains bound to the black hole settles into an accretion disk and produces an X-ray flare close to the Eddington limit of $L_{\rm Edd} \simeq 10^{41} {\rm erg/s} M_{\rm bh}/1000 M$_\odot$), typically lasting for a few months. The combination of a peculiar thermonuclear supernova together with an X-ray flare thus whistle-blows the existence of such moderate-mass black holes. The next generation of wide field space-based instruments should be able to detect such events.Comment: 8 pages, 2 figures, EuroWD0

GenASiS: General Astrophysical Simulation System. I. Refinable Mesh and Nonrelativistic Hydrodynamics

GenASiS (General Astrophysical Simulation System) is a new code being developed initially and primarily, though by no means exclusively, for the simulation of core-collapse supernovae on the world's leading capability supercomputers. This paper---the first in a series---demonstrates a centrally refined coordinate patch suitable for gravitational collapse and documents methods for compressible nonrelativistic hydrodynamics. We benchmark the hydrodynamics capabilities of GenASiS against many standard test problems; the results illustrate the basic competence of our implementation, demonstrate the strengths and limitations of the HLLC relative to the HLL Riemann solver in a number of interesting cases, and provide preliminary indications of the code's ability to scale and to function with cell-by-cell fixed-mesh refinement.Comment: Belated update to version accepted ApJ

Gravitational Waves from Gravitational Collapse

Gravitational wave emission from the gravitational collapse of massive stars has been studied for more than three decades. Current state of the art numerical investigations of collapse include those that use progenitors with realistic angular momentum profiles, properly treat microphysics issues, account for general relativity, and examine non--axisymmetric effects in three dimensions. Such simulations predict that gravitational waves from various phenomena associated with gravitational collapse could be detectable with advanced ground--based and future space--based interferometric observatories.Comment: 68 pages including 13 figures; revised version accepted for publication in Living Reviews in Relativity (http://www.livingreviews.org

Computational Models of Stellar Collapse and Core-Collapse Supernovae

Core-collapse supernovae are among Nature's most energetic events. They mark the end of massive star evolution and pollute the interstellar medium with the life-enabling ashes of thermonuclear burning. Despite their importance for the evolution of galaxies and life in the universe, the details of the core-collapse supernova explosion mechanism remain in the dark and pose a daunting computational challenge. We outline the multi-dimensional, multi-scale, and multi-physics nature of the core-collapse supernova problem and discuss computational strategies and requirements for its solution. Specifically, we highlight the axisymmetric (2D) radiation-MHD code VULCAN/2D and present results obtained from the first full-2D angle-dependent neutrino radiation-hydrodynamics simulations of the post-core-bounce supernova evolution. We then go on to discuss the new code Zelmani which is based on the open-source HPC Cactus framework and provides a scalable AMR approach for 3D fully general-relativistic modeling of stellar collapse, core-collapse supernovae and black hole formation on current and future massively-parallel HPC systems. We show Zelmani's scaling properties to more than 16,000 compute cores and discuss first 3D general-relativistic core-collapse results.Comment: 16 pages, 5 figures, to appear in the proceedings of the DOE/SciDAC 2009 conference. A version with high-resolution figures is available from http://stellarcollapse.org/papers/Ott_SciDAC2009.pd

Numerical hydrodynamics in general relativity

The current status of numerical solutions for the equations of ideal general relativistic hydrodynamics is reviewed. With respect to an earlier version of the article the present update provides additional information on numerical schemes and extends the discussion of astrophysical simulations in general relativistic hydrodynamics. Different formulations of the equations are presented, with special mention of conservative and hyperbolic formulations well-adapted to advanced numerical methods. A large sample of available numerical schemes is discussed, paying particular attention to solution procedures based on schemes exploiting the characteristic structure of the equations through linearized Riemann solvers. A comprehensive summary of astrophysical simulations in strong gravitational fields is presented. These include gravitational collapse, accretion onto black holes and hydrodynamical evolutions of neutron stars. The material contained in these sections highlights the numerical challenges of various representative simulations. It also follows, to some extent, the chronological development of the field, concerning advances on the formulation of the gravitational field and hydrodynamic equations and the numerical methodology designed to solve them.Comment: 105 pages, 12 figures. The full online-readable version of this article, including several animations, will be published in Living Reviews in Relativity at http://www.livingreviews.or