Parallel load balancing strategy for Volume-of-Fluid methods on 3-D unstructured meshes

© 2016. This version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/l Volume-of-Fluid (VOF) is one of the methods of choice to reproduce the interface motion in the simulation of multi-fluid flows. One of its main strengths is its accuracy in capturing sharp interface geometries, although requiring for it a number of geometric calculations. Under these circumstances, achieving parallel performance on current supercomputers is a must. The main obstacle for the parallelization is that the computing costs are concentrated only in the discrete elements that lie on the interface between fluids. Consequently, if the interface is not homogeneously distributed throughout the domain, standard domain decomposition (DD) strategies lead to imbalanced workload distributions. In this paper, we present a new parallelization strategy for general unstructured VOF solvers, based on a dynamic load balancing process complementary to the underlying DD. Its parallel efficiency has been analyzed and compared to the DD one using up to 1024 CPU-cores on an Intel SandyBridge based supercomputer. The results obtained on the solution of several artificially generated test cases show a speedup of up to similar to 12x with respect to the standard DD, depending on the interface size, the initial distribution and the number of parallel processes engaged. Moreover, the new parallelization strategy presented is of general purpose, therefore, it could be used to parallelize any VOF solver without requiring changes on the coupled flow solver. Finally, note that although designed for the VOF method, our approach could be easily adapted to other interface-capturing methods, such as the Level-Set, which may present similar workload imbalances. (C) 2014 Elsevier Inc. Allrights reserved.Peer ReviewedPostprint (author's final draft

HPC compact quasi-Newton algorithm for interface problems

In this work we present a robust interface coupling algorithm called Compact Interface quasi-Newton (CIQN). It is designed for computationally intensive applications using an MPI multi-code partitioned scheme. The algorithm allows to reuse information from previous time steps, feature that has been previously proposed to accelerate convergence. Through algebraic manipulation, an efficient usage of the computational resources is achieved by: avoiding construction of dense matrices and reduce every multiplication to a matrix-vector product and reusing the computationally expensive loops. This leads to a compact version of the original quasi-Newton algorithm. Altogether with an efficient communication, in this paper we show an efficient scalability up to 4800 cores. Three examples with qualitatively different dynamics are shown to prove that the algorithm can efficiently deal with added mass instability and two-field coupled problems. We also show how reusing histories and filtering does not necessarily makes a more robust scheme and, finally, we prove the necessity of this HPC version of the algorithm. The novelty of this article lies in the HPC focused implementation of the algorithm, detailing how to fuse and combine the composing blocks to obtain an scalable MPI implementation. Such an implementation is mandatory in large scale cases, for which the contact surface cannot be stored in a single computational node, or the number of contact nodes is not negligible compared with the size of the domain. \c{opyright} Elsevier. This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/Comment: 33 pages: 23 manuscript, 10 appendix. 16 figures: 4 manuscript, 12 appendix. 10 Tables: 3 manuscript, 7 appendi

Direct numerical simulation of the turbulent natural convection flow in an open cavity of aspect ratio 4

In this paper, three-dimensional turbulent natural convection heat transfer in an open cavity with an isothermal wall facing the overture has been studied. The aspect ratio chosen for the cavity has been 4 to complement the studies by Trias et al. [1, 2] of closed cavities with the same aspect ratio. Direct numerical simulations (DNS) of the cavity are presented and analyzed. Rayleigh numbers up to Ra = 1012 has been considered.Peer ReviewedPostprint (published version

Direct numerical simulation of a NACA0012 in full stall

This work aims at investigating the mechanisms of separation and the transition to turbulence in the separated shear-layer of aerodynamic profiles, while at the same time to gain insight into coherent structures formed in the separated zone at low-to-moderate Reynolds numbers. To do this, direct numerical simulations of the flow past a NACA0012 airfoil at Reynolds numbers Re = 50,000 (based on the free-stream velocity and the airfoil chord) and angles of attack AOA = 9.25 and AOA = 12 have been carried out. At low-to-moderate Reynolds numbers, NACA0012 exhibits a combination of leading-edge/trailing-edge stall which causes the massive separation of the flow on the suction side of the airfoil. The initially laminar shear layer undergoes transition to turbulence and vortices formed are shed forming a von Kármán like vortex street in the airfoil wake. The main characteristics of this flow together with its main features, including power spectra of a set of selected monitoring probes at different positions on the suction side and in the wake of the airfoil are provided and discussed in detail.Peer ReviewedPreprin

Efficient CFD code implementation for the ARM-based Mont-Blanc architecture

Since 2011, the European project Mont-Blanc has been focused on enabling ARM-based technology for HPC, developing both hardware platforms and system software. The latest Mont-Blanc prototypes use system-on-chip (SoC) devices that combine a CPU and a GPU sharing a common main memory. Specific developments of parallel computing software and well-suited implementation approaches are of crucial importance for such a heterogeneous architecture in order to efficiently exploit its potential. This paper is devoted to the optimizations carried out in the TermoFluids CFD code to efficiently run it on the Mont-Blanc system. The underlying numerical method is based on an unstructured finite-volume discretization of the Navier–Stokes equations for the numerical simulation of incompressible turbulent flows. It is implemented using a portable and modular operational approach based on a minimal set of linear algebra operations. An architecture-specific heterogeneous multilevel MPI+OpenMP+OpenCL implementation of such kernels is proposed. It includes optimizations of the storage formats, dynamic load balancing between the CPU and GPU devices and hiding of communication overheads by overlapping computations and data transfers. A detailed performance study shows time reductions of up to on the kernels’ execution with the new heterogeneous implementation, its scalability on up to 128 Mont-Blanc nodes and the energy savings (around ) achieved with the Mont-Blanc system versus the high-end hybrid supercomputer MinoTauro.The research leading to these results has received funding from the European Community’s Seventh Framework Programme [FP7/2007–2013] and Horizon 2020 under the Mont-Blanc Project (www.montblanc-project.eu), grant agreement n 288777, 610402 and 671697. The work has been financially supported by the Ministerio de Ciencia e Innovación, Spain (ENE- 2014-60577-R), the Russian Science Foundation, project 15-11-30039, CONICYT Becas Chile Doctorado 2012, the Juan de la Cierva posdoctoral grant (IJCI-2014-21034), and the Initial Training Network SEDITRANS (GA number: 607394), implemented within the 7th Framework Programme of the European Commission under call FP7-PEOPLE- 2013-ITN. Our calculations have been performed on the resources of the Barcelona Supercomputing Center. The authors thankfully acknowledge these institutions.Peer ReviewedPostprint (published version

Low-frequency unsteadiness in the vortex formation region of a circular cylinder

Electronic version of an article published as "Physics of fluids", vol. 25, no 8, 2013. DOI: http://dx.doi.org/10.1063/1.4818641.The presence of low-frequency fluctuations in the wake of bluff bodies have been observed in several investigations. Even though the flow past a circular cylinder at Re = 3900 (Re = U ref D/ν) has been the object of several experimental and numerical investigations, there is a large scattering in the average statistics in the near wake. In the present work, the flow dynamics of the near wake region behind a circular cylinder has been investigated by means of direct numerical simulations and statistics have been computed for more than 858 shedding cycles. The analysis of instantaneous velocity signals of several probes located in the vortex formation region, point out the existence of a low-frequency fluctuation at the non-dimensional frequency of f m = 0.0064. This large-scale almost periodic motion seems to be related with the modulation of the recirculation bubble which causes its shrinking and enlargement over the time. Two different configurations have been identified: (i) a high-energy mode with larger fluctuations in the shear-layer and in the vortex formation region (Mode H) and (ii) a low-energy mode with weaker fluctuations in the shear layer (Mode L). The influence of such a low-frequency in the wake topology has been studied not only by means of the phase-average flow field for each mode, but also by the analysis of the time-average first- and second-order statistics of each wake mode. The results are compared with the long-term averaged solution and with results in the existing literature.Peer ReviewedPostprint (published version

On the flow past a circular cylinder from critical to super-critical Reynolds numbers: Wake topology and vortex shedding

Large-eddy simulations (LES) of the flow past a circular cylinder are used to investigate the flow topology and the vortex shedding process at Reynolds numbers Re=2.5×105-8.5×105Re=2.5×105-8.5×105. This range encompasses both the critical and super-critical regimes. As the flow enters the critical regime, major changes occur which affect the flow configuration. Asymmetries in the flow are found in the critical regime, whereas the wake recovers its symmetry and stabilizes in the super-critical regime. Wake characteristic lengths are measured and compared between the different Reynolds numbers. It is shown that the super-critical regime is characterised by a plateau in the drag coefficient at about CD˜0.22CD˜0.22, and a quasi-stable wake which has a non-dimensional width of dw/D˜0.4dw/D˜0.4. The periodic nature of the flow is analysed by means of measurements of the unsteady drag and lift coefficients. Power spectra of the lift fluctuations are computed. Wake vortex shedding is found to occur for both regimes investigated, although a jump in frequencies is observed when the flow enters the super-critical regime. In this regime, non-dimensional vortex-shedding frequency is almost constant and equal to St=fvsD/Uref˜0.44St=fvsD/Uref˜0.44. The analysis also shows a steep decrease in the fluctuating lift when entering the super-critical regime. The combined analysis of both wake topology and vortex shedding complements the physical picture of a stable and highly coherent flow in the super-critical regime.Peer ReviewedPostprint (author's final draft

Low-frequency variations in the wake of a circular cylinder at Re = 3900

Flow around cylindrical structures is of relevance for many practical applications. Knowledge of flow-related unsteady loading of such structures is crucial for hydro- and aerodynamic control and design. In order to obtain a deeper knowledge of this kind of flow, a DNS have been performed at ReD = 3900 (ReD = UrefD/v). The instantaneous velocity signals of probes located in the separated shear-layer and in the vortex formation region exhibit the presence of low-frequency variations. The statistical analysis of these signals suggest that low-frequency variations in the vortex formation length, suction base pressure and intermittencies in the shear layer are closely related. It is shown that these variations are the responsible of the large scattering of data obtained in different experimental and numerical results, as well as the U-shape and V-shape stream-wise velocity profiles observed in the very near wake of the cylinder.Peer ReviewedPreprin