Particle tracking in large-scale numerical simulations of turbulent flows
presents one of the major bottlenecks in parallel performance and scaling
efficiency. Here, we describe a particle tracking algorithm for large-scale
parallel pseudo-spectral simulations of turbulence which scales well up to
billions of tracer particles on modern high-performance computing
architectures. We summarize the standard parallel methods used to solve the
fluid equations in our hybrid MPI/OpenMP implementation. As the main focus, we
describe the implementation of the particle tracking algorithm and document its
computational performance. To address the extensive inter-process communication
required by particle tracking, we introduce a task-based approach to overlap
point-to-point communications with computations, thereby enabling improved
resource utilization. We characterize the computational cost as a function of
the number of particles tracked and compare it with the flow field computation,
showing that the cost of particle tracking is very small for typical
applications
