154 research outputs found
ASCR/HEP Exascale Requirements Review Report
This draft report summarizes and details the findings, results, and
recommendations derived from the ASCR/HEP Exascale Requirements Review meeting
held in June, 2015. The main conclusions are as follows. 1) Larger, more
capable computing and data facilities are needed to support HEP science goals
in all three frontiers: Energy, Intensity, and Cosmic. The expected scale of
the demand at the 2025 timescale is at least two orders of magnitude -- and in
some cases greater -- than that available currently. 2) The growth rate of data
produced by simulations is overwhelming the current ability, of both facilities
and researchers, to store and analyze it. Additional resources and new
techniques for data analysis are urgently needed. 3) Data rates and volumes
from HEP experimental facilities are also straining the ability to store and
analyze large and complex data volumes. Appropriately configured
leadership-class facilities can play a transformational role in enabling
scientific discovery from these datasets. 4) A close integration of HPC
simulation and data analysis will aid greatly in interpreting results from HEP
experiments. Such an integration will minimize data movement and facilitate
interdependent workflows. 5) Long-range planning between HEP and ASCR will be
required to meet HEP's research needs. To best use ASCR HPC resources the
experimental HEP program needs a) an established long-term plan for access to
ASCR computational and data resources, b) an ability to map workflows onto HPC
resources, c) the ability for ASCR facilities to accommodate workflows run by
collaborations that can have thousands of individual members, d) to transition
codes to the next-generation HPC platforms that will be available at ASCR
facilities, e) to build up and train a workforce capable of developing and
using simulations and analysis to support HEP scientific research on
next-generation systems.Comment: 77 pages, 13 Figures; draft report, subject to further revisio
2HOT: An Improved Parallel Hashed Oct-Tree N-Body Algorithm for Cosmological Simulation
We report on improvements made over the past two decades to our adaptive
treecode N-body method (HOT). A mathematical and computational approach to the
cosmological N-body problem is described, with performance and scalability
measured up to 256k () processors. We present error analysis and
scientific application results from a series of more than ten 69 billion
() particle cosmological simulations, accounting for
floating point operations. These results include the first simulations using
the new constraints on the standard model of cosmology from the Planck
satellite. Our simulations set a new standard for accuracy and scientific
throughput, while meeting or exceeding the computational efficiency of the
latest generation of hybrid TreePM N-body methods.Comment: 12 pages, 8 figures, 77 references; To appear in Proceedings of SC
'1
A survey of high level frameworks in block-structured adaptive mesh refinement packages
pre-printOver the last decade block-structured adaptive mesh refinement (SAMR) has found increasing use in large, publicly available codes and frameworks. SAMR frameworks have evolved along different paths. Some have stayed focused on specific domain areas, others have pursued a more general functionality, providing the building blocks for a larger variety of applications. In this survey paper we examine a representative set of SAMR packages and SAMR-based codes that have been in existence for half a decade or more, have a reasonably sized and active user base outside of their home institutions, and are publicly available. The set consists of a mix of SAMR packages and application codes that cover a broad range of scientific domains. We look at their high-level frameworks, their design trade-offs and their approach to dealing with the advent of radical changes in hardware architecture. The codes included in this survey are BoxLib, Cactus, Chombo, Enzo, FLASH, and Uintah
Gasoline: An adaptable implementation of TreeSPH
The key algorithms and features of the Gasoline code for parallel
hydrodynamics with self-gravity are described. Gasoline is an extension of the
efficient Pkdgrav parallel N-body code using smoothed particle hydrodynamics.
Accuracy measurements, performance analysis and tests of the code are
presented. Recent successful Gasoline applications are summarized. These cover
a diverse set of areas in astrophysics including galaxy clusters, galaxy
formation and gas-giant planets. Future directions for gasdynamical simulations
in astrophysics and code development strategies for tackling cutting edge
problems are discussed.Comment: 30 pages, 11 figures, submitted to New Astronomy. Movies and high
resolution images are available at http://imp.mcmaster.ca/image
Numerical Simulations of the Dark Universe: State of the Art and the Next Decade
We present a review of the current state of the art of cosmological dark
matter simulations, with particular emphasis on the implications for dark
matter detection efforts and studies of dark energy. This review is intended
both for particle physicists, who may find the cosmological simulation
literature opaque or confusing, and for astro-physicists, who may not be
familiar with the role of simulations for observational and experimental probes
of dark matter and dark energy. Our work is complementary to the contribution
by M. Baldi in this issue, which focuses on the treatment of dark energy and
cosmic acceleration in dedicated N-body simulations. Truly massive dark
matter-only simulations are being conducted on national supercomputing centers,
employing from several billion to over half a trillion particles to simulate
the formation and evolution of cosmologically representative volumes (cosmic
scale) or to zoom in on individual halos (cluster and galactic scale). These
simulations cost millions of core-hours, require tens to hundreds of terabytes
of memory, and use up to petabytes of disk storage. The field is quite
internationally diverse, with top simulations having been run in China, France,
Germany, Korea, Spain, and the USA. Predictions from such simulations touch on
almost every aspect of dark matter and dark energy studies, and we give a
comprehensive overview of this connection. We also discuss the limitations of
the cold and collisionless DM-only approach, and describe in some detail
efforts to include different particle physics as well as baryonic physics in
cosmological galaxy formation simulations, including a discussion of recent
results highlighting how the distribution of dark matter in halos may be
altered. We end with an outlook for the next decade, presenting our view of how
the field can be expected to progress. (abridged)Comment: 54 pages, 4 figures, 3 tables; invited contribution to the special
issue "The next decade in Dark Matter and Dark Energy" of the new Open Access
journal "Physics of the Dark Universe". Replaced with accepted versio
Ray Tracing Structured AMR Data Using ExaBricks
Structured Adaptive Mesh Refinement (Structured AMR) enables simulations to
adapt the domain resolution to save computation and storage, and has become one
of the dominant data representations used by scientific simulations; however,
efficiently rendering such data remains a challenge. We present an efficient
approach for volume- and iso-surface ray tracing of Structured AMR data on
GPU-equipped workstations, using a combination of two different data
structures. Together, these data structures allow a ray tracing based renderer
to quickly determine which segments along the ray need to be integrated and at
what frequency, while also providing quick access to all data values required
for a smooth sample reconstruction kernel. Our method makes use of the RTX ray
tracing hardware for surface rendering, ray marching, space skipping, and
adaptive sampling; and allows for interactive changes to the transfer function
and implicit iso-surfacing thresholds. We demonstrate that our method achieves
high performance with little memory overhead, enabling interactive high quality
rendering of complex AMR data sets on individual GPU workstations
Fully-Coupled Simulation of Cosmic Reionization. I: Numerical Methods and Tests
We describe an extension of the Enzo code to enable fully-coupled radiation
hydrodynamical simulation of inhomogeneous reionization in large cosmological volumes with thousands to millions of point sources. We
solve all dynamical, radiative transfer, thermal, and ionization processes
self-consistently on the same mesh, as opposed to a postprocessing approach
which coarse-grains the radiative transfer. We do, however, employ a simple
subgrid model for star formation which we calibrate to observations. Radiation
transport is done in the grey flux-limited diffusion (FLD) approximation, which
is solved by implicit time integration split off from the gas energy and
ionization equations, which are solved separately. This results in a faster and
more robust scheme for cosmological applications compared to the earlier
method. The FLD equation is solved using the hypre optimally scalable geometric
multigrid solver from LLNL. By treating the ionizing radiation as a grid field
as opposed to rays, our method is scalable with respect to the number of
ionizing sources, limited only by the parallel scaling properties of the
radiation solver. We test the speed and accuracy of our approach on a number of
standard verification and validation tests. We show by direct comparison with
Enzo's adaptive ray tracing method Moray that the well-known inability of FLD
to cast a shadow behind opaque clouds has a minor effect on the evolution of
ionized volume and mass fractions in a reionization simulation validation test.
We illustrate an application of our method to the problem of inhomogeneous
reionization in a 80 Mpc comoving box resolved with Eulerian grid
cells and dark matter particles.Comment: 32 pages, 23 figures. ApJ Supp accepted. New title and substantial
revisions re. v
Doctor of Philosophy
dissertationSolutions to Partial Di erential Equations (PDEs) are often computed by discretizing the domain into a collection of computational elements referred to as a mesh. This solution is an approximation with an error that decreases as the mesh spacing decreases. However, decreasing the mesh spacing also increases the computational requirements. Adaptive mesh re nement (AMR) attempts to reduce the error while limiting the increase in computational requirements by re ning the mesh locally in regions of the domain that have large error while maintaining a coarse mesh in other portions of the domain. This approach often provides a solution that is as accurate as that obtained from a much larger xed mesh simulation, thus saving on both computational time and memory. However, historically, these AMR operations often limit the overall scalability of the application. Adapting the mesh at runtime necessitates scalable regridding and load balancing algorithms. This dissertation analyzes the performance bottlenecks for a widely used regridding algorithm and presents two new algorithms which exhibit ideal scalability. In addition, a scalable space- lling curve generation algorithm for dynamic load balancing is also presented. The performance of these algorithms is analyzed by determining their theoretical complexity, deriving performance models, and comparing the observed performance to those performance models. The models are then used to predict performance on larger numbers of processors. This analysis demonstrates the necessity of these algorithms at larger numbers of processors. This dissertation also investigates methods to more accurately predict workloads based on measurements taken at runtime. While the methods used are not new, the application of these methods to the load balancing process is. These methods are shown to be highly accurate and able to predict the workload within 3% error. By improving the accuracy of these estimations, the load imbalance of the simulation can be reduced, thereby increasing the overall performance
- …