Magnetic flux plays an important role during a BHXRB outburst in radiative 2T GRMHD simulations

Black hole (BH) X-ray binaries cycle through different spectral states of accretion over the course of months to years. Although fluctuations in the BH mass accretion rate are generally recognized as the most important component of state transitions, it is becoming increasingly evident that magnetic fields play a similarly important role. In this article, we present the first radiative two-temperature (2T) general relativistic magnetohydrodynamics (GRMHD) simulations in which an accretion disk transitions from a quiescent state at an accretion rate of $\dot{M} \sim 10^{-10} \dot{M}_{\rm Edd}$ to a hard-intermediate state at an accretion rate of $\dot{M} \sim 10^{-2} \dot{M}_{\rm Edd}$. This huge parameter space in mass accretion rate is bridged by artificially rescaling the gas density scale of the simulations. We present two jetted BH models with varying degrees of magnetic flux saturation. We demonstrate that in `Standard and Normal Evolution' models, which are unsaturated with magnetic flux, the hot torus collapses into a thin and cold accretion disk when $\dot{M} \gtrsim 5\times 10^{-3} \dot{M}_{\rm Edd}$. On the other hand, in `Magnetically Arrested Disk' models, which are fully saturated with vertical magnetic flux, the plasma remains mostly hot with substructures that condense into cold clumps of gas when $\dot{M} \gtrsim 1 \times 10^{-2} \dot{M}_{\rm Edd}$. This suggests that the spectral signatures observed during state transitions are closely tied to the level of magnetic flux saturation.Comment: 8 pages, 5 figures, accompanying animations included in YouTube playlist: https://www.youtube.com/playlist?list=PLDO1oeU33Gwm1Thyw0iHC14BbvBWaG5c

H-AMR: A New GPU-accelerated GRMHD Code for Exascale Computing With 3D Adaptive Mesh Refinement and Local Adaptive Time-stepping

General-relativistic magnetohydrodynamic (GRMHD) simulations have revolutionized our understanding of black-hole accretion. Here, we present a GPU-accelerated GRMHD code H-AMR with multi-faceted optimizations that, collectively, accelerate computation by 2-5 orders of magnitude for a wide range of applications. Firstly, it involves a novel implementation of a spherical-polar grid with 3D adaptive mesh refinement that operates in each of the 3 dimensions independently. This allows us to circumvent the Courant condition near the polar singularity, which otherwise cripples high-res computational performance. Secondly, we demonstrate that local adaptive time-stepping (LAT) on a logarithmic spherical-polar grid accelerates computation by a factor of $\lesssim10$ compared to traditional hierarchical time-stepping approaches. Jointly, these unique features lead to an effective speed of $\sim10^9$ zone-cycles-per-second-per-node on 5,400 NVIDIA V100 GPUs (i.e., 900 nodes of the OLCF Summit supercomputer). We demonstrate its computational performance by presenting the first GRMHD simulation of a tilted thin accretion disk threaded by a toroidal magnetic field around a rapidly spinning black hole. With an effective resolution of $13$,$440\times4$,$608\times8$,$092$ cells, and a total of $\lesssim22$ billion cells and $\sim0.65\times10^8$ timesteps, it is among the largest astrophysical simulations ever performed. We find that frame-dragging by the black hole tears up the disk into two independently precessing sub-disks. The innermost sub-disk rotation axis intermittently aligns with the black hole spin, demonstrating for the first time that such long-sought alignment is possible in the absence of large-scale poloidal magnetic fields.Comment: 10 pages, 5 figures, submitted to MNRAS, for the YouTube playlist, see https://youtu.be/rIOjKUfzcv

Radiation Transport Two-temperature GRMHD Simulations of Warped Accretion Disks

In many black hole (BH) systems, the accretion disk is expected to be misaligned with respect to the BH spin axis. If the scale height of the disk is much smaller than the misalignment angle, the spin of the BH can tear the disk into multiple, independently precessing “sub-disks.” This is most likely to happen during outbursts in black hole X-Ray binaries (BHXRBs) and in active galactic nuclei (AGNs) accreting above a few percent of the Eddington limit, because the disk becomes razor-thin. Disk tearing has the potential to explain variability phenomena including quasi-periodic oscillations in BHXRBs and changing-look phenomena in AGNs. Here, we present the first radiative two-temperature general relativistic magnetohydrodynamic (GRMHD) simulation of a strongly tilted (65°) accretion disk around an M _BH = 10 M _⊙ BH, which tears and precesses. This leads to luminosity swings between a few percent and 50% of the Eddington limit on sub-viscous timescales. Surprisingly, even where the disk is radiation-pressure-dominated, the accretion disk is thermally stable over t ≳ 14,000 r _g / c . This suggests warps play an important role in stabilizing the disk against thermal collapse. The disk forms two nozzle shocks perpendicular to the line of nodes where the scale height of the disk decreases tenfold and the electron temperature reaches T _e ∼ 10 ^8 –10 ^9 K. In addition, optically thin gas crossing the tear between the inner and outer disk gets heated to T _e ∼ 10 ^8 K. This suggests that warped disks may emit a Comptonized spectrum that deviates substantially from idealized models

H-AMR: A New GPU-accelerated GRMHD Code for Exascale Computing with 3D Adaptive Mesh Refinement and Local Adaptive Time Stepping

General relativistic magnetohydrodynamic (GRMHD) simulations have revolutionized our understanding of black hole accretion. Here, we present a GPU-accelerated GRMHD code H-AMR with multifaceted optimizations that, collectively, accelerate computation by 2-5 orders of magnitude for a wide range of applications. First, it introduces a spherical grid with 3D adaptive mesh refinement that operates in each of the three dimensions independently. This allows us to circumvent the Courant condition near the polar singularity, which otherwise cripples high-resolution computational performance. Second, we demonstrate that local adaptive time stepping on a logarithmic spherical-polar grid accelerates computation by a factor of ≲10 compared to traditional hierarchical time-stepping approaches. Jointly, these unique features lead to an effective speed of ∼109 zone cycles per second per node on 5400 NVIDIA V100 GPUs (i.e., 900 nodes of the OLCF Summit supercomputer). We illustrate H-AMR's computational performance by presenting the first GRMHD simulation of a tilted thin accretion disk threaded by a toroidal magnetic field around a rapidly spinning black hole. With an effective resolution of 13,440 × 4608 × 8092 cells and a total of ≲22 billion cells and ∼0.65 × 108 time steps, it is among the largest astrophysical simulations ever performed. We find that frame dragging by the black hole tears up the disk into two independently precessing subdisks. The innermost subdisk rotation axis intermittently aligns with the black hole spin, demonstrating for the first time that such long-sought alignment is possible in the absence of large-scale poloidal magnetic fields

H-AMR: A New GPU-accelerated GRMHD Code for Exascale Computing with 3D Adaptive Mesh Refinement and Local Adaptive Time Stepping

General relativistic magnetohydrodynamic (GRMHD) simulations have revolutionized our understanding of black hole accretion. Here, we present a GPU-accelerated GRMHD code H-AMR with multifaceted optimizations that, collectively, accelerate computation by 2-5 orders of magnitude for a wide range of applications. First, it introduces a spherical grid with 3D adaptive mesh refinement that operates in each of the three dimensions independently. This allows us to circumvent the Courant condition near the polar singularity, which otherwise cripples high-resolution computational performance. Second, we demonstrate that local adaptive time stepping on a logarithmic spherical-polar grid accelerates computation by a factor of ≲10 compared to traditional hierarchical time-stepping approaches. Jointly, these unique features lead to an effective speed of ∼109 zone cycles per second per node on 5400 NVIDIA V100 GPUs (i.e., 900 nodes of the OLCF Summit supercomputer). We illustrate H-AMR's computational performance by presenting the first GRMHD simulation of a tilted thin accretion disk threaded by a toroidal magnetic field around a rapidly spinning black hole. With an effective resolution of 13,440 × 4608 × 8092 cells and a total of ≲22 billion cells and ∼0.65 × 108 time steps, it is among the largest astrophysical simulations ever performed. We find that frame dragging by the black hole tears up the disk into two independently precessing subdisks. The innermost subdisk rotation axis intermittently aligns with the black hole spin, demonstrating for the first time that such long-sought alignment is possible in the absence of large-scale poloidal magnetic fields