Ray Tracing Structured AMR Data Using ExaBricks

Structured Adaptive Mesh Refinement (Structured AMR) enables simulations to adapt the domain resolution to save computation and storage, and has become one of the dominant data representations used by scientific simulations; however, efficiently rendering such data remains a challenge. We present an efficient approach for volume- and iso-surface ray tracing of Structured AMR data on GPU-equipped workstations, using a combination of two different data structures. Together, these data structures allow a ray tracing based renderer to quickly determine which segments along the ray need to be integrated and at what frequency, while also providing quick access to all data values required for a smooth sample reconstruction kernel. Our method makes use of the RTX ray tracing hardware for surface rendering, ray marching, space skipping, and adaptive sampling; and allows for interactive changes to the transfer function and implicit iso-surfacing thresholds. We demonstrate that our method achieves high performance with little memory overhead, enabling interactive high quality rendering of complex AMR data sets on individual GPU workstations

Formation of Precessing Jets by Tilted Black-hole Discs in 3D General Relativistic MHD Simulations

Gas falling into a black hole (BH) from large distances is unaware of BH spin direction, and misalignment between the accretion disc and BH spin is expected to be common. However, the physics of tilted discs (e.g., angular momentum transport and jet formation) is poorly understood. Using our new GPU-accelerated code H-AMR, we performed 3D general relativistic magnetohydrodynamic simulations of tilted thick accretion discs around rapidly spinning BHs, at the highest resolution to date. We explored the limit where disc thermal pressure dominates magnetic pressure, and showed for the first time that, for different magnetic field strengths on the BH, these flows launch magnetized relativistic jets propagating along the rotation axis of the tilted disc (rather than of the BH). If strong large-scale magnetic flux reaches the BH, it bends the inner few gravitational radii of the disc and jets into partial alignment with the BH spin. On longer time scales, the simulated disc-jet system as a whole undergoes Lense-Thirring precession and approaches alignment, demonstrating for the first time that jets can be used as probes of disc precession. When the disc turbulence is well-resolved, our isolated discs spread out, causing both the alignment and precession to slow down.Comment: MNRAS Letters, accepted. Animations available at https://www.youtube.com/playlist?list=PL39mDr1uU6a5RYZdXLAjKE1C_GAJkQJN

Simulating streamer discharges in 3D with the parallel adaptive Afivo framework

We present an open-source plasma fluid code for 2D, cylindrical and 3D simulations of streamer discharges, based on the Afivo framework that features adaptive mesh refinement, geometric multigrid methods for Poisson's equation, and OpenMP parallelism. We describe the numerical implementation of a fluid model of the drift-diffusion-reaction type, combined with the local field approximation. Then we demonstrate its functionality with 3D simulations of long positive streamers in nitrogen in undervolted gaps, using three examples. The first example shows how a stochastic background density affects streamer propagation and branching. The second one focuses on the interaction of a streamer with preionized regions, and the third one investigates the interaction between two streamers. The simulations run on up to $10^8$ grid cells within less than a day. Without mesh refinement, they would require $4\cdot 10^{12}$ grid cells

Numerical simulations of possible finite time singularities in the incompressible Euler equations: comparison of numerical methods

The numerical simulation of the 3D incompressible Euler equation is analyzed with respect to different integration methods. The numerical schemes we considered include spectral methods with different strategies for dealiasing and two variants of finite difference methods. Based on this comparison, a Kida-Pelz like initial condition is integrated using adaptive mesh refinement and estimates on the necessary numerical resolution are given. This estimate is based on analyzing the scaling behavior similar to the procedure in critical phenomena and present simulations are put into perspective.Comment: Euler equations: 250 years o

Effect of dust on Kelvin-Helmholtz instabilities

Dust is present in a large variety of astrophysical fluids, from tori around supermassive black holes to molecular clouds, protoplanetary discs, and cometary outflows. In many such fluids, shearing flows are present, leading to the formation of Kelvin-Helmholtz instabilities (KHI) and changing the properties and structures of the fluid through processes such as mixing and clumping of dust. We investigate how dust changes the growth rates of the KHI in 2D and 3D and how the it redistributes and clumps dust. We investigate if similarities can be found between the structures in 3D KHI and those seen in observations of molecular clouds. We do this by performing numerical hydrodynamical dust+gas simulations with in addition to gas a number of dust fluids. Each dust fluid represents a portion of the particle size-distribution. We study how dust-to-gas mass density ratios between 0.01 and 1 alter the growth rate in the linear phase of the KHI. We do this for a wide range of perturbation wavelengths, and compare these values to the analytical gas-only growth rates. As the formation of high-density dust structures is of interest in many astrophysical environments, we scale our simulations with physical quantities similar to values in molecular clouds. Large differences in dynamics are seen for different grain sizes. We demonstrate that high dust-to-gas ratios significantly reduce the growth rate of the KHI, especially for short wavelengths. We compare the dynamics in 2D and 3D simulations, where the latter demonstrates additional full 3D instabilities during the non-linear phase, leading to increased dust densities. We compare the structures formed by the KHI in 3D simulations with those in molecular clouds and see how the column density distribution of the simulation shares similarities with log-normal distributions with power-law tails sometimes seen in observations of molecular clouds.Comment: 14 pages, 20 figure

Adaptive Mesh Refinement for Supersonic Molecular Cloud Turbulence

We performed a series of three-dimensional numerical simulations of supersonic homogeneous Euler turbulence with adaptive mesh refinement (AMR) and effective grid resolution up to 1024^3 zones. Our experiments describe non-magnetized driven supersonic turbulent flows with an isothermal equation of state. Mesh refinement on shocks and shear is implemented to cover dynamically important structures with the highest resolution subgrids and calibrated to match the turbulence statistics obtained from the equivalent uniform grid simulations. We found that at a level of resolution slightly below 512^3, when a sufficient integral/dissipation scale separation is first achieved, the fraction of the box volume covered by the AMR subgrids first becomes smaller than unity. At the higher AMR levels subgrids start covering smaller and smaller fractions of the whole volume, which scale with the Reynolds number as Re^{-1/4}. We demonstrate the consistency of this scaling with a hypothesis that the most dynamically important structures in intermittent supersonic turbulence are strong shocks with a fractal dimension of two. We show that turbulence statistics derived from AMR simulations and simulations performed on uniform grids agree surprisingly well, even though only a fraction of the volume is covered by AMR subgrids. Based on these results, we discuss the signature of dissipative structures in the statistical properties of supersonic turbulence and their role in overall flow dynamics.Comment: 5 pages, 5 figures, revised versio

A Multi-Code Analysis Toolkit for Astrophysical Simulation Data

The analysis of complex multiphysics astrophysical simulations presents a unique and rapidly growing set of challenges: reproducibility, parallelization, and vast increases in data size and complexity chief among them. In order to meet these challenges, and in order to open up new avenues for collaboration between users of multiple simulation platforms, we present yt (available at http://yt.enzotools.org/), an open source, community-developed astrophysical analysis and visualization toolkit. Analysis and visualization with yt are oriented around physically relevant quantities rather than quantities native to astrophysical simulation codes. While originally designed for handling Enzo's structure adaptive mesh refinement (AMR) data, yt has been extended to work with several different simulation methods and simulation codes including Orion, RAMSES, and FLASH. We report on its methods for reading, handling, and visualizing data, including projections, multivariate volume rendering, multi-dimensional histograms, halo finding, light cone generation and topologically-connected isocontour identification. Furthermore, we discuss the underlying algorithms yt uses for processing and visualizing data, and its mechanisms for parallelization of analysis tasks.Comment: 18 pages, 6 figures, emulateapj format. Resubmitted to Astrophysical Journal Supplement Series with revisions from referee. yt can be found at http://yt.enzotools.org