Impact of the floating-point precision and interpolation scheme on the results of DNS of turbulence by pseudo-spectral codes

In this paper we investigate the impact of the floating-point precision and interpolation scheme on the results of direct numerical simulations (DNS) of turbulence by pseudo-spectral codes. Three different types of floating-point precision configurations show no differences in the statistical results. This implies that single precision computations allow for increased Reynolds numbers due to the reduced amount of memory needed. The interpolation scheme for obtaining velocity values at particle positions has a noticeable impact on the Lagrangian acceleration statistics. A tri-cubic scheme results in a slightly broader acceleration probability density function than a tri-linear scheme. Furthermore the scaling behavior obtained by the cubic interpolation scheme exhibits a tendency towards a slightly increased degree of intermittency compared to the linear one.Comment: to appear in Comp. Phys. Com

Petascale turbulence simulation using a highly parallel fast multipole method on GPUs

This paper reports large-scale direct numerical simulations of homogeneous-isotropic fluid turbulence, achieving sustained performance of 1.08 petaflop/s on gpu hardware using single precision. The simulations use a vortex particle method to solve the Navier-Stokes equations, with a highly parallel fast multipole method (FMM) as numerical engine, and match the current record in mesh size for this application, a cube of 4096^3 computational points solved with a spectral method. The standard numerical approach used in this field is the pseudo-spectral method, relying on the FFT algorithm as numerical engine. The particle-based simulations presented in this paper quantitatively match the kinetic energy spectrum obtained with a pseudo-spectral method, using a trusted code. In terms of parallel performance, weak scaling results show the fmm-based vortex method achieving 74% parallel efficiency on 4096 processes (one gpu per mpi process, 3 gpus per node of the TSUBAME-2.0 system). The FFT-based spectral method is able to achieve just 14% parallel efficiency on the same number of mpi processes (using only cpu cores), due to the all-to-all communication pattern of the FFT algorithm. The calculation time for one time step was 108 seconds for the vortex method and 154 seconds for the spectral method, under these conditions. Computing with 69 billion particles, this work exceeds by an order of magnitude the largest vortex method calculations to date

More efficient time integration for Fourier pseudo-spectral DNS of incompressible turbulence

Time integration of Fourier pseudo-spectral DNS is usually performed using the classical fourth-order accurate Runge--Kutta method, or other methods of second or third order, with a fixed step size. We investigate the use of higher-order Runge-Kutta pairs and automatic step size control based on local error estimation. We find that the fifth-order accurate Runge--Kutta pair of Bogacki \& Shampine gives much greater accuracy at a significantly reduced computational cost. Specifically, we demonstrate speedups of 2x-10x for the same accuracy. Numerical tests (including the Taylor-Green vortex, Rayleigh-Taylor instability, and homogeneous isotropic turbulence) confirm the reliability and efficiency of the method. We also show that adaptive time stepping provides a significant computational advantage for some problems (like the development of a Rayleigh-Taylor instability) without compromising accuracy

FMM-based vortex method for simulation of isotropic turbulence on GPUs, compared with a spectral method

The Lagrangian vortex method offers an alternative numerical approach for direct numerical simulation of turbulence. The fact that it uses the fast multipole method (FMM)--a hierarchical algorithm for N-body problems with highly scalable parallel implementations--as numerical engine makes it a potentially good candidate for exascale systems. However, there have been few validation studies of Lagrangian vortex simulations and the insufficient comparisons against standard DNS codes has left ample room for skepticism. This paper presents a comparison between a Lagrangian vortex method and a pseudo-spectral method for the simulation of decaying homogeneous isotropic turbulence. This flow field is chosen despite the fact that it is not the most favorable flow problem for particle methods (which shine in wake flows or where vorticity is compact), due to the fact that it is ideal for the quantitative validation of DNS codes. We use a 256^3 grid with Re_lambda=50 and 100 and look at the turbulence statistics, including high-order moments. The focus is on the effect of the various parameters in the vortex method, e.g., order of FMM series expansion, frequency of reinitialization, overlap ratio and time step. The vortex method uses an FMM code (exaFMM) that runs on GPU hardware using CUDA, while the spectral code (hit3d) runs on CPU only. Results indicate that, for this application (and with the current code implementations), the spectral method is an order of magnitude faster than the vortex method when using a single GPU for the FMM and six CPU cores for the FFT

Estimating intermittency in three-dimensional Navier-Stokes turbulence

The issue of why computational resolution in Navier-Stokes turbulence is so hard to achieve is addressed. It is shown that Navier-Stokes solutions can potentially behave differently in two distinct regions of space-time $\mathbb{R}^{\pm}$ where $\mathbb{R}^{-}$ is comprised of a union of disjoint space-time `anomalies'. Large values of |\nabla\bom| dominate $\mathbb{R}^{-}$, which is consistent with the formation of vortex sheets or tightly-coiled filaments. The local number of degrees of freedom $\mathcal{N}^{\pm}$ needed to resolve the regions in $\mathbb{R}^{\pm}$ satisfies \mathcal{N}^{\pm}(\bx, t)\lessgtr c_{\pm}\mathcal{R}_{u}^{3} where $\mathcal{R}_{u} = uL/\nu$ is a Reynolds number dependent on the local velocity field u(\bx, t)

Kinematic simulation for stably stratified and rotating turbulence

The properties of one-particle and particle-pair diffusion in rotating and stratified turbulence are studied by applying the rapid distortion theory (RDT) to a kinematic simulation (KS) of the Boussinesq equation with a Coriolis term. Scalings for one- and two-particle horizontal and vertical diffusions in purely rotating turbulence are proposed for small Rossby numbers. Particular attention is given to the locality-in-scale hypothesis for two-particle diffusion in purely rotating turbulence both in the horizontal and the vertical directions. It is observed that both rotation and stratification decrease the pair diffusivity and improve the validity of the locality-in-scale hypothesis. In the case of stratification the range of scales over which the locality-in-scale hypothesis is observed is increased. It is found that rotation decreases the diffusion in the horizontal direction as well as, though to a much lesser extent, in the vertical direction