STREAmS: a high-fidelity accelerated solver for direct numerical simulation of compressible turbulent flow

We present STREAmS, an in-house high-fidelity solver for large-scale, massively parallel direct numerical simulations (DNS) of compressible turbulent flows on graphical processing units (GPUs). STREAmS is written in the Fortran 90 language and it is tailored to carry out DNS of canonical compressible wall-bounded flows, namely turbulent plane channel, zero-pressure gradient turbulent boundary layer and supersonic oblique shock-wave/boundary layer interactions. The solver incorporates state-of-the-art numerical algorithms, specifically designed to cope with the challenging problems associated with the solution of high-speed turbulent flows and can be used across a wide range of Mach numbers, extending from the low subsonic up to the hypersonic regime. The use of cuf automatic kernels allowed an easy and efficient porting on the GPU architecture minimizing the changes to the original CPU code, which is also maintained. We discuss a memory allocation strategy based on duplicated arrays for host and device which carefully minimizes the memory usage making the solver suitable for large scale computations on the latest GPU cards. Comparison between different CPUs and GPUs architectures strongly favor the latter, and executing the solver on a single NVIDIA Tesla P100 corresponds to using approximately 330 Intel Knights Landing CPU cores. STREAmS shows very good strong scalability and essentially ideal weak scalability up to 2048 GPUs, paving the way to simulations in the genuine high-Reynolds number regime, possibly at friction Reynolds number $Re_{\tau} > 10^4$. The solver is released open source under GPLv3 license and is available at https://github.com/matteobernardini/STREAmS.Comment: 11 pages, 11 figure

STREAmS: A high-fidelity accelerated solver for direct numerical simulation of compressible turbulent flows

We present STREAmS, an in-house high-fidelity solver for direct numerical simulations (DNS) of canonical compressible wall-bounded flows, namely turbulent plane channel, zero-pressure gradient turbulent boundary layer and supersonic oblique shock-wave/boundary layer interaction. The solver incorporates state-of-the-art numerical algorithms, specifically designed to cope with the challenging problems associated with the solution of high-speed turbulent flows and can be used across a wide range of Mach numbers, extending from the low subsonic up to the hypersonic regime. From the computational viewpoint, STREAmS is oriented to modern HPC platforms thanks to MPI parallelization and the ability to run on multi-GPU architectures. This paper discusses the main implementation strategies, with particular reference to the CUDA paradigm, the management of a single code for traditional and multi-GPU architectures, and the optimization process to take advantage of the latest generation of NVIDIA GPUs. Performance measurements show that single-GPU optimization more than halves the computing time as compared to the baseline version. At the same time, the asynchronous patterns implemented in STREAmS for MPI communications guarantee very good parallel performance especially in the weak scaling spirit, with efficiency exceeding 97% on 1024 GPUs. For overall evaluation of STREAmS with respect to other compressible solvers, comparison with a recent GPU-enabled community solver is presented. It turns out that, although STREAmS is much more limited in terms of flow configurations that can be addressed, the advantage in terms of accuracy, computing time and memory occupation is substantial, which makes it an ideal candidate for large-scale simulations of high-Reynolds number, compressible wall-bounded turbulent flows. The solver is released open source under GPLv3 license. Program summary: Program Title: STREAmS CPC Library link to program files: https://doi.org/10.17632/hdcgjpzr3y.1 Developer's repository link: https://github.com/matteobernardini/STREAmS Code Ocean capsule: https://codeocean.com/capsule/8931507/tree/v2 Licensing provisions: GPLv3 Programming language: Fortran 90, CUDA Fortran, MPI Nature of problem: Solving the three-dimensional compressible Navier–Stokes equations for low and high Mach regimes in a Cartesian domain configured for channel, boundary layer or shock-boundary layer interaction flows. Solution method: The convective terms are discretized using a hybrid energy-conservative shock-capturing scheme in locally conservative form. Shock-capturing capabilities rely on the use of Lax–Friedrichs flux vector splitting and weighted essentially non-oscillatory (WENO) reconstruction. The system is advanced in time using a three-stage, third-order RK scheme. Two-dimensional pencil distributed MPI parallelization is implemented alongside different patterns of GPU (CUDA Fortran) accelerated routines

FluTAS: A GPU-accelerated finite difference code for multiphase flows

We present the Fluid Transport Accelerated Solver, FluTAS, a scalable GPU code for multiphase flows with thermal effects. The code solves the incompressible Navier-Stokes equation for two-fluid systems, with a direct FFT-based Poisson solver for the pressure equation. The interface between the two fluids is represented with the Volume of Fluid (VoF) method, which is mass conserving and well suited for complex flows thanks to its capacity of handling topological changes. The energy equation is explicitly solved and coupled with the momentum equation through the Boussinesq approximation. The code is conceived in a modular fashion so that different numerical methods can be used independently, the existing routines can be modified, and new ones can be included in a straightforward and sustainable manner. FluTAS is written in modern Fortran and parallelized using hybrid MPI/OpenMP in the CPU-only version and accelerated with OpenACC directives in the GPU implementation. We present different benchmarks to validate the code, and two large-scale simulations of fundamental interest in turbulent multiphase flows: isothermal emulsions in HIT and two-layer Rayleigh-B\'enard convection. FluTAS is distributed through a MIT license and arises from a collaborative effort of several scientists, aiming to become a flexible tool to study complex multiphase flows

An interface capturing method for liquid-gas flows at low-Mach number

Multiphase, compressible and viscous flows are of crucial importance in a wide range of scientific and engineering problems. Despite the large effort paid in the last decades to develop accurate and efficient numerical techniques to address this kind of problems, current models need to be further improved to address realistic applications. In this context, we propose a numerical approach to the simulation of multiphase, viscous flows where a compressible and an incompressible phase interact in the low-Mach number regime. In this frame, acoustics is neglected but large density variations of the compressible phase can be accounted for as well as heat transfer, convection and diffusion processes. The problem is addressed in a fully Eulerian framework exploiting a low-Mach number asymptotic expansion of the Navier-Stokes equations. A Volume of Fluid approach (VOF) is used to capture the liquid-gas interface, built on top of a massive parallel solver, second order accurate both in time and space. The second-order-pressure term is treated implicitly and the resulting pressure equation is solved with the eigenexpansion method employing a robust and novel formulation. We provide a detailed and complete description of the theoretical approach together with information about the numerical technique and implementation details. Results of benchmarking tests are provided for five different test cases

Simulazioni alle grandi scale del trasporto di uno scalare passivo in flussi turbolenti di canale

Il trasporto di uno scalare passivo ad opera di un flusso turbolento risulta di notevole importanza in molti ambiti delle scienze e dell’ingegneria. Un tipico esempio riguarda il trasporto di inquinanti ad opera del vento o, più in generale, delle correnti d’aria. In questo contesto, la presente tesi riguarda la simulazione alle grandi scale, Large Eddy Simulation (LES), del trasporto di una grandezza scalare passiva in un flusso turbolento di canale. Le equazioni che governano il trasporto della grandezza scalare sono state implementate nel software open-source CaNS. Come modello di turbolenza per le scale non risolte è stato impiegato un modello Wall-Adapting Local Eddy-viscosity (WALE). L’algoritmo risolutivo per la LES è basato sulle equazioni di Navier − Stokes incomprimibili. Si è fatto inoltre uso del numero di Schmidt turbolento per calcolare la viscosità turbolenta. E’ stata infine eseguita una simulazione numerica diretta, Direct Numerical Simulation (DNS), sempre basata sulle equazioni di Navier−Stokes a densità costante. Tutte le simulazioni considerate riproducono un canale periodico in direzione x e y, mentre, per quanto riguarda la direzione verticale, esso presenta due pareti a temperatura diversa, quella inferiore più fredda rispetto a quella superiore. La simulazione è stata eseguita in parallelo grazie a OpenMPI, utilizzando un approccio many − CPUs. Tramite la LES sono stati ricavati i risultati relativi al trasporto della grandezza scalare, alla velocità nelle tre componenti e alla viscosità turbolenta. I risultati della LES sono stati comparati con quelli della DNS e successivamente sono stati confrontati con i dati di riferimento, relativi allo studio effettuato da Wang e Pletcher nel 1996 al fine di validare l’implementazione delle equazioni

Investigation of protein-protein interactions: multibody docking, association/dissociation kinetics and macromolecular crowding

Protein-protein interactions are central to understanding how cells carry out their wide array of functions and metabolic procedures. Conventional studies on specific protein interactions focus either on details of one-to-one binding interfaces, or on large networks that require a priori knowledge of binding strengths. Moreover, specific protein interactions, occurring within a crowded macromolecular environment, which is precisely the case for interactions in a real cell, are often under-investigated. A macromolecular simulation package, called BioSimz, has been developed to perform Langevin dynamics simulations on multiple protein-protein interactions at atomic resolution, aimed at bridging the gaps between structural, kinetic and crowding studies on protein-protein interactions. Simulations on twenty-seven experimentally determined protein-protein interactions, indicated that the use of contact frequency information of proteins forming specific encounters can guide docking algorithms towards the most likely binding regions. Further evidence from eleven benchmarked protein interactions showed that the association rate constant of a complex, kon, can be estimated, with good agreement to experimental values, based on the retention time of its specific encounter. Performing these simulations with ten types of environmental protein crowders, it suggests, from the change of kon, that macromolecular crowding improves the association kinetics of slower-binding proteins, while it damps the association kinetics of fast, electrostatics-driven protein-protein interactions. It is hypothesised, based on evidence from docking, kinetics and crowding, that the dynamics of specific protein-protein encounters is vitally important in determining their association affinity. There are multiple factors by which encounter dynamics, and subsequently the kon, can be influenced, such as anchor residues, long-range forces, and environmental steering via crowders’ electrostatics and/or volume exclusion. The capacity of emulating these conditions on a common platform not only provides a holistic view of interacting dynamics, but also offers the possibility of evaluating and engineering protein-protein interactions from aspects that have never been opened before