A low-cost parallel implementation of direct numerical simulation of wall turbulence

A numerical method for the direct numerical simulation of incompressible wall turbulence in rectangular and cylindrical geometries is presented. The distinctive feature resides in its design being targeted towards an efficient distributed-memory parallel computing on commodity hardware. The adopted discretization is spectral in the two homogeneous directions; fourth-order accurate, compact finite-difference schemes over a variable-spacing mesh in the wall-normal direction are key to our parallel implementation. The parallel algorithm is designed in such a way as to minimize data exchange among the computing machines, and in particular to avoid taking a global transpose of the data during the pseudo-spectral evaluation of the non-linear terms. The computing machines can then be connected to each other through low-cost network devices. The code is optimized for memory requirements, which can moreover be subdivided among the computing nodes. The layout of a simple, dedicated and optimized computing system based on commodity hardware is described. The performance of the numerical method on this computing system is evaluated and compared with that of other codes described in the literature, as well as with that of the same code implementing a commonly employed strategy for the pseudo-spectral calculation.Comment: To be published in J. Comp. Physic

High performance Python for direct numerical simulations of turbulent flows

Direct Numerical Simulations (DNS) of the Navier Stokes equations is an invaluable research tool in fluid dynamics. Still, there are few publicly available research codes and, due to the heavy number crunching implied, available codes are usually written in low-level languages such as C/C++ or Fortran. In this paper we describe a pure scientific Python pseudo-spectral DNS code that nearly matches the performance of C++ for thousands of processors and billions of unknowns. We also describe a version optimized through Cython, that is found to match the speed of C++. The solvers are written from scratch in Python, both the mesh, the MPI domain decomposition, and the temporal integrators. The solvers have been verified and benchmarked on the Shaheen supercomputer at the KAUST supercomputing laboratory, and we are able to show very good scaling up to several thousand cores. A very important part of the implementation is the mesh decomposition (we implement both slab and pencil decompositions) and 3D parallel Fast Fourier Transforms (FFT). The mesh decomposition and FFT routines have been implemented in Python using serial FFT routines (either NumPy, pyFFTW or any other serial FFT module), NumPy array manipulations and with MPI communications handled by MPI for Python (mpi4py). We show how we are able to execute a 3D parallel FFT in Python for a slab mesh decomposition using 4 lines of compact Python code, for which the parallel performance on Shaheen is found to be slightly better than similar routines provided through the FFTW library. For a pencil mesh decomposition 7 lines of code is required to execute a transform

More efficient time integration for Fourier pseudo-spectral DNS of incompressible turbulence

Time integration of Fourier pseudo-spectral DNS is usually performed using the classical fourth-order accurate Runge--Kutta method, or other methods of second or third order, with a fixed step size. We investigate the use of higher-order Runge-Kutta pairs and automatic step size control based on local error estimation. We find that the fifth-order accurate Runge--Kutta pair of Bogacki \& Shampine gives much greater accuracy at a significantly reduced computational cost. Specifically, we demonstrate speedups of 2x-10x for the same accuracy. Numerical tests (including the Taylor-Green vortex, Rayleigh-Taylor instability, and homogeneous isotropic turbulence) confirm the reliability and efficiency of the method. We also show that adaptive time stepping provides a significant computational advantage for some problems (like the development of a Rayleigh-Taylor instability) without compromising accuracy

Convective instability and transient growth in flow over a backward-facing step

Transient energy growths of two- and three-dimensional optimal linear perturbations to two-dimensional flow in a rectangular backward-facing-step geometry with expansion ratio two are presented. Reynolds numbers based on the step height and peak inflow speed are considered in the range 0–500, which is below the value for the onset of three-dimensional asymptotic instability. As is well known, the flow has a strong local convective instability, and the maximum linear transient energy growth values computed here are of order 80×103 at Re = 500. The critical Reynolds number below which there is no growth over any time interval is determined to be Re = 57.7 in the two-dimensional case. The centroidal location of the energy distribution for maximum transient growth is typically downstream of all the stagnation/reattachment points of the steady base flow. Sub-optimal transient modes are also computed and discussed. A direct study of weakly nonlinear effects demonstrates that nonlinearity is stablizing at Re = 500. The optimal three-dimensional disturbances have spanwise wavelength of order ten step heights. Though they have slightly larger growths than two-dimensional cases, they are broadly similar in character. When the inflow of the full nonlinear system is perturbed with white noise, narrowband random velocity perturbations are observed in the downstream channel at locations corresponding to maximum linear transient growth. The centre frequency of this response matches that computed from the streamwise wavelength and mean advection speed of the predicted optimal disturbance. Linkage between the response of the driven flow and the optimal disturbance is further demonstrated by a partition of response energy into velocity components

Phase-field simulation of core-annular pipe flow

Phase-field methods have long been used to model the flow of immiscible fluids. Their ability to naturally capture interface topological changes is widely recognized, but their accuracy in simulating flows of real fluids in practical geometries is not established. We here quantitatively investigate the convergence of the phase-field method to the sharp-interface limit with simulations of two-phase pipe flow. We focus on core-annular flows, in which a highly viscous fluid is lubricated by a less viscous fluid, and validate our simulations with an analytic laminar solution, a formal linear stability analysis and also in the fully nonlinear regime. We demonstrate the ability of the phase-field method to accurately deal with non-rectangular geometry, strong advection, unsteady fluctuations and large viscosity contrast. We argue that phase-field methods are very promising for quantitatively studying moderately turbulent flows, especially at high concentrations of the disperse phase.Comment: Paper accepted for publication in International Journal of Multiphase Flo

The effect of Mach number on unstable disturbances in shock/boundary-layer interactions

The effect of Mach number on the growth of unstable disturbances in a boundary layer undergoing a strong interaction with an impinging oblique shock wave is studied by direct numerical simulation and linear stability theory (LST). To reduce the number of independent parameters, test cases are arranged so that both the interaction location Reynolds number (based on the distance from the plate leading edge to the shock impingement location for a corresponding inviscid flow) and the separation bubble length Reynolds number are held fixed. Small-amplitude disturbances are introduced via both white-noise and harmonic forcing and, after verification that the disturbances are convective in nature, linear growth rates are extracted from the simulations for comparison with parallel flow LST and solutions of the parabolized stability equations (PSE). At Mach 2.0, the oblique modes are dominant and consistent results are obtained from simulation and theory. At Mach 4.5 and Mach 6.85, the linear Navier-Stokes results show large reductions in disturbance energy at the point where the shock impinges on the top of the separated shear layer. The most unstable second mode has only weak growth over the bubble region, which instead shows significant growth of streamwise structures. The two higher Mach number cases are not well predicted by parallel flow LST, which gives frequencies and spanwise wave numbers that are significantly different from the simulations. The PSE approach leads to good qualitative predictions of the dominant frequency and wavenumber at Mach 2.0 and 4.5, but suffers from reduced accuracy in the region immediately after the shock impingement. Three-dimensional Navier-Stokes simulations are used to demonstrate that at finite amplitudes the flow structures undergo a nonlinear breakdown to turbulence. This breakdown is enhanced when the oblique-mode disturbances are supplemented with unstable Mack modes

Poloidal-toroidal decomposition in a finite cylinder. II. Discretization, regularization and validation

The Navier-Stokes equations in a finite cylinder are written in terms of poloidal and toroidal potentials in order to impose incompressibility. Regularity of the solutions is ensured in several ways: First, the potentials are represented using a spectral basis which is analytic at the cylindrical axis. Second, the non-physical discontinuous boundary conditions at the cylindrical corners are smoothed using a polynomial approximation to a steep exponential profile. Third, the nonlinear term is evaluated in such a way as to eliminate singularities. The resulting pseudo-spectral code is tested using exact polynomial solutions and the spectral convergence of the coefficients is demonstrated. Our solutions are shown to agree with exact polynomial solutions and with previous axisymmetric calculations of vortex breakdown and of nonaxisymmetric calculations of onset of helical spirals. Parallelization by azimuthal wavenumber is shown to be highly effective