Parallel HOP: A Scalable Halo Finder for Massive Cosmological Data Sets

Modern N-body cosmological simulations contain billions ($10^9$) of dark matter particles. These simulations require hundreds to thousands of gigabytes of memory, and employ hundreds to tens of thousands of processing cores on many compute nodes. In order to study the distribution of dark matter in a cosmological simulation, the dark matter halos must be identified using a halo finder, which establishes the halo membership of every particle in the simulation. The resources required for halo finding are similar to the requirements for the simulation itself. In particular, simulations have become too extensive to use commonly-employed halo finders, such that the computational requirements to identify halos must now be spread across multiple nodes and cores. Here we present a scalable-parallel halo finding method called Parallel HOP for large-scale cosmological simulation data. Based on the halo finder HOP, it utilizes MPI and domain decomposition to distribute the halo finding workload across multiple compute nodes, enabling analysis of much larger datasets than is possible with the strictly serial or previous parallel implementations of HOP. We provide a reference implementation of this method as a part of the toolkit yt, an analysis toolkit for Adaptive Mesh Refinement (AMR) data that includes complementary analysis modules. Additionally, we discuss a suite of benchmarks that demonstrate that this method scales well up to several hundred tasks and datasets in excess of $2000^3$ particles. The Parallel HOP method and our implementation can be readily applied to any kind of N-body simulation data and is therefore widely applicable.Comment: 29 pages, 11 figures, 2 table

The Two-Point Correlation of 2QZ Quasars and 2SLAQ LRGs: From a Quasar Fueling Perspective

Public data from the 2dF quasar survey (2QZ) and 2dF/SDSS LRG & QSO (2SLAQ), with their vast reservoirs of spectroscopically located and identified sources, afford us the chance to more accurately study their real space correlations in the hopes of identifying the physical processes that trigger quasar activity. We have used these two public databases to measure the projected cross correlation, $\omega_p$, between quasars and luminous red galaxies. We find the projected two-point correlation to have a fitted clustering radius of $r_0, = 5.3 \pm 0.6$ and a slope, $\gamma =1.83 \pm 0.42$ on scales from 0.7-27$h^{-1}$Mpc. We attempt to understand this strong correlation by separating the LRG sample into 2 populations of blue and red galaxies. We measure at the cross correlation with each population. We find that these quasars have a stronger correlation amplitude with the bluer, more recently starforming population in our sample than the redder passively evolving population, which has a correlation that is much more noisy and seems to flatten on scales $< 5h^{-1}$Mpc. We compare this result to published work on hierarchical models. The stronger correlation of bright quasars with LRGs that have undergone a recent burst of starformation suggests that the physical mechanisms that produce both activities are related and that minor mergers or tidal effects may be important triggers of bright quasar activity and/or that bright quasars are less highly biased than faint quasars.Comment: Accepted for publication in Ap

Particle mesh simulations of the Lyman-alpha forest and the signature of Baryon Acoustic Oscillations in the intergalactic medium

We present a set of ultra-large particle-mesh simulations of the LyA forest targeted at understanding the imprint of baryon acoustic oscillations (BAO) in the inter-galactic medium. We use 9 dark matter only simulations which can, for the first time, simultaneously resolve the Jeans scale of the intergalactic gas while covering the large volumes required to adequately sample the acoustic feature. Mock absorption spectra are generated using the fluctuating Gunn-Peterson approximation which have approximately correct flux probability density functions (PDFs) and small-scale power spectra. On larger scales there is clear evidence in the redshift space correlation function for an acoustic feature, which matches a linear theory template with constant bias. These spectra, which we make publicly available, can be used to test pipelines, plan future experiments and model various physical effects. As an illustration we discuss the basic properties of the acoustic signal in the forest, the scaling of errors with noise and source number density, modified statistics to treat mean flux evolution and misestimation, and non-gravitational sources such as fluctuations in the photo-ionizing background and temperature fluctuations due to HeII reionization.Comment: 11 pages, 10 figures, minor changes to address referee repor

A New Approach for Simulating Galaxy Cluster Properties

We describe a subgrid model for including galaxies into hydrodynamical cosmological simulations of galaxy cluster evolution. Each galaxy construct- or galcon- is modeled as a physically extended object within which star formation, galactic winds, and ram pressure stripping of gas are modeled analytically. Galcons are initialized at high redshift (z~3) after galaxy dark matter halos have formed but before the cluster has virialized. Each galcon moves self-consistently within the evolving cluster potential and injects mass, metals, and energy into intracluster (IC) gas through a well-resolved spherical interface layer. We have implemented galcons into the Enzo adaptive mesh refinement code and carried out a simulation of cluster formation in a LambdaCDM universe. With our approach, we are able to economically follow the impact of a large number of galaxies on IC gas. We compare the results of the galcon simulation with a second, more standard simulation where star formation and feedback are treated using a popular heuristic prescription. One advantage of the galcon approach is explicit control over the star formation history of cluster galaxies. Using a galactic SFR derived from the cosmic star formation density, we find the galcon simulation produces a lower stellar fraction, a larger gas core radius, a more isothermal temperature profile, and a flatter metallicity gradient than the standard simulation, in better agreement with observations.Comment: 4 pages, 2 figures, submitted for publication in ApJ

The Biermann Battery in Cosmological MHD Simulations of Population III Star Formation

We report the results of the first self-consistent three-dimensional adaptive mesh refinement magnetohydrodynamical simulations of Population III star formation including the Biermann Battery effect. We find that the Population III stars formed including this effect are both qualitatively and quantitatively similar to those from hydrodynamics-only (non-MHD) cosmological simulations. We observe peak magnetic fields of ~10^-9 G in the center of our star-forming halo at z ~ 17.55. The magnetic fields created by the Biermann Battery effect are predominantly formed early in the evolution of the primordial halo at low density and large spatial scales, and then grow through compression and by shear flows. The fields seen in this calculation are never large enough to be dynamically important (with beta >= 10^{15} at all times), and should be considered the minimum possible fields in existence during Population III star formation, and may be seed fields for the stellar dynamo or the magnetorotational instability at higher densities and smaller spatial scales.Comment: 4 pages, 3 figures; Submitted to the Astrophysical Journal Letters. Comments welcome. Typo found (and fixed) in equation

The Properties of X-ray Cold Fronts in a Statistical Sample of Simulated Galaxy Clusters

We examine the incidence of cold fronts in a large sample of galaxy clusters extracted from a (512h^-1 Mpc) hydrodynamic/N-body cosmological simulation with adiabatic gas physics computed with the Enzo adaptive mesh refinement code. This simulation contains a sample of roughly 4000 galaxy clusters with M > 10^14 M_sun at z=0. For each simulated galaxy cluster, we have created mock 0.3-8.0 keV X-ray observations and spectroscopic-like temperature maps. We have searched these maps with a new automated algorithm to identify the presence of cold fronts in projection. Using a threshold of a minimum of 10 cold front pixels in our images, corresponding to a total comoving length L_cf > 156h^-1 kpc, we find that roughly 10-12% of all projections in a mass-limited sample would be classified as cold front clusters. Interestingly, the fraction of clusters with extended cold front features in our synthetic maps of a mass-limited sample trends only weakly with redshift out to z=1.0. However, when using different selection functions, including a simulated flux limit, the trending with redshift changes significantly. The likelihood of finding cold fronts in the simulated clusters in our sample is a strong function of cluster mass. In clusters with M>7.5x10^14 M_sun the cold front fraction is 40-50%. We also show that the presence of cold fronts is strongly correlated with disturbed morphology as measured by quantitative structure measures. Finally, we find that the incidence of cold fronts in the simulated cluster images is strongly dependent on baryonic physics.Comment: 16 pages, 21 figures, Accepted to Ap

Fast, large volume, GPU enabled simulations for the Ly-alpha forest: power spectrum forecasts for baryon acoustic oscillation experiments

High redshift measurements of the baryonic acoustic oscillation scale (BAO) from large Ly-alpha forest surveys represent the next frontier of dark energy studies. As part of this effort, efficient simulations of the BAO signature from the Ly-alpha forest will be required. We construct a model for producing fast, large volume simulations of the Ly-alpha forest for this purpose. Utilising a calibrated semi-analytic approach, we are able to run very large simulations in 1 Gpc^3 volumes which fully resolve the Jeans scale in less than a day on a desktop PC using a GPU enabled version of our code. The Ly-alpha forest spectra extracted from our semi-analytical simulations are in excellent agreement with those obtained from a fully hydrodynamical reference simulation. Furthermore, we find our simulated data are in broad agreement with observational measurements of the flux probability distribution and 1D flux power spectrum. We are able to correctly recover the input BAO scale from the 3D Ly-alpha flux power spectrum measured from our simulated data, and estimate that a BOSS-like 10^4 deg^2 survey with ~15 background sources per square degree and a signal-to-noise of ~5 per pixel should achieve a measurement of the BAO scale to within ~1.4 per cent. We also use our simulations to provide simple power-law expressions for estimating the fractional error on the BAO scale on varying the signal-to-noise and the number density of background sources. The speed and flexibility of our approach is well suited for exploring parameter space and the impact of observational and astrophysical systematics on the recovery of the BAO signature from forthcoming large scale spectroscopic surveys.Comment: 16 pages, 11 figures, accepted to MNRA

Structure and Turbulence in Simulated Galaxy Clusters and the Implications for the Formation of Radio Halos

We track the histories of massive clusters of galaxies formed within a cosmological hydrodynamic simulation. Specifically, we track the time evolution of the energy in random bulk motions of the intracluster medium and X-ray measures of cluster structure and their relationship to cluster mergers. We aim to assess the viability of the turbulent re-acceleration model for the generation of giant radio halos by comparing the level of turbulent kinetic energy in simulated clusters with the observed properties of radio halo clusters, giving particular attention to the association of radio halos to clusters with disturbedX-ray structures. The evolution of X-ray cluster structure and turbulence kinetic energy, k, in simulations can then inform us about the expected lifetime of radio halos and the fraction of clusters as a function of redshift expected to host them. We find strong statistical correlation of disturbed structure measures and the presence of enhancements in k. Specifically, quantitatively "disturbed", radio halo-like X-ray morphology in our sample indicates a 92% chance of the cluster in question having k elevated to more than twice its minimum value over the cluster's life. The typical lifetime of episodes of elevated turbulence is on the order of 1 Gyr, though these periods can last 5 Gyrs or more. This variation reflects the wide range of cluster histories; while some clusters undergo complex and repeated mergers spending a majority of their time in elevated k states, other clusters are relaxed over nearly their entire history. We do not find a bimodal relationship between cluster X-ray luminosity and the total energy in turbulence that might account directly for a bimodal L_X-P_{1.4 GHz} relation. However, our result may be consistent with the observed bimodality, as here we are not including a full treatment of cosmic rays sources and magnetic fields.Comment: 15 pages, 12 figures, MNRAS Submitte

The Santa Fe Light Cone Simulation Project: II. The Prospects for Direct Detection of the WHIM with SZE Surveys

Detection of the Warm-Hot Intergalactic Medium (WHIM) using Sunyaev-Zeldovich effect (SZE) surveys is an intriguing possibility, and one that may allow observers to quantify the amount of "missing baryons" in the WHIM phase. We estimate the necessary sensitivity for detecting low density WHIM gas with the South Pole Telescope (SPT) and Planck Surveyor for a synthetic 100 square degree sky survey. This survey is generated from a very large, high dynamic range adaptive mesh refinement cosmological simulation performed with the Enzo code. We find that for a modest increase in the SPT survey sensitivity (a factor of 2-4), the WHIM gas makes a detectable contribution to the integrated sky signal. For a Planck-like satellite, similar detections are possible with a more significant increase in sensitivity (a factor of 8-10). We point out that for the WHIM gas, the kinematic SZE signal can sometimes dominate the thermal SZE where the thermal SZE decrement is maximal (150 GHz), and that using the combination of the two increases the chance of WHIM detection using SZE surveys. However, we find no evidence of unique features in the thermal SZE angular power spectrum that may aid in its detection. Interestingly, there are differences in the power spectrum of the kinematic SZE, which may not allow us to detect the WHIM directly, but could be an important contaminant in cosmological analyses of the kSZE-derived velocity field. Corrections derived from numerical simulations may be necessary to account for this contamination.Comment: 9 pages, submitted to Astrophysical Journa

The Lyman-alpha forest at redshifts 0.1 -- 1.6: good agreement between a large hydrodynamic simulation and HST spectra

We give a comprehensive statistical description of the Lyman-alpha absorption from the intergalactic medium in a hydrodynamic simulation at redshifts 0.1-1.6, the range of redshifts covered by HST spectra of QSOs. We use the ENZO code to make a 76 comoving Mpc cube simulation using 75 kpc cells, for a Hubble constant of 71 km/s/Mpc. The best prior work, by \citet{dave99},used an SPH simulation in a 15.6 Mpc box with an effective resolution of 245 kpc and slightly different cosmological parameters. At redshifts z=2 this simulation is different from data. \citet{tytler07b} found that the simulated spectra at z=2 have too little power on large scales, Lyman-alpha lines are too wide, there is a lack high column density lines, and there is a lack of pixels with low flux. Here we present statistics at z<1.6, including the flux distribution, the mean flux, the effective opacity, and the power and correlation of the flux. We also give statistics of the lyman alpha lines including the line width distribution, the column density distribution, the number of lines per unit equivalent width and redshift, and the correlation between the line width and column density. We find that the mean amount of absorption in the simulated spectra changes smoothly with redshift with DA(z)=0.01(1+z)^{2.25}. Both the trend and absolute values are close to measurements of HST spectra by \citet{kirkman07a}. The column density and line width distributions are also close to those measured from HST spectra by \citet{janknecht06a}, except for the mode of the line width distribution which is smaller in the HST spectra. Although some differences that we saw at z=2 are too subtle to be seen in existing HST spectra, overall, the simulation gives an good description of HST spectra at 0.1<z<1.6