High performance computing for multiphase fluid flows

Multiphase fluid flows are very common in engineering and science applications. Examples include air ow on water surface, metallurgical flow and blood flow in the body. In these flows, fluids are separated by a sharp interface and form different phases. The flow is characterized by the movement of this interface. Accurate modelling of the interface movement is a fundamental problem in the numerical simulation of these flows. Velocities for the movement are provided by the numerical solution of the Navier-Stokes (N-S) equations. These equations are discretized and converted into linear systems of equations. Research in the direction towards solving these systems efficiently has been the main focus of many researchers in the field of Computational Fluid Dynamics (CFD). A modified Volume of Fluid (VOF) method for modelling two phase flows is implemented using an analytic relation for its reconstruction step. The Finite Volume Method (FVM) is utilized, by incorporating a staggered grid, to discretize the two-dimensional (2-D) N-S equations. A preconditioned Krylov-Subspace iterative method, namely, the Bi-Conjugate Gradient Stabilized (Bi-CGSTAB) method is employed to solve the linear systems of equations. Solving the linear system usually consumes most of the simulation time for multiphase flow problems. Novel algorithms for the Incomplete LU Threshold (ILUT) preconditioner, forward and backward substitution and other matrix operations for penta-diagonal matrices are proposed here by adopting a diagonal sparse matrices format. The novel algorithm for ILUT reduces the computational complexity from O(n3 − n2) to O(n) in comparison to dense format. Further, it brings down the communication overhead, consequently facilitating parallelization. Parallel versions of these algorithms are developed using a new load balancing scheme. The MPI C++ communication library is utilized to develop the parallel version. The 2-D VOF code is applied to shape advection problems and results are found to be in good agreement with those available in literature. In the case of translation of a square box, it provides more accurate results than other VOF methods. The code for the VOF method and the parallel iterative solvers are integrated with 2-D N-S code in C++. The whole code is then implemented to simulate several two phase flow problems: dam breaking with and without an obstacle, rising of an air bubble and lid driven cavity flows. Speedup data from parallel programs implemented on these problems are generated

Accelerating the Lawson-Hanson NNLS solver for large-scale Tchakaloff regression designs

We deal with the problem of computing near G-optimal compressed designs for high-degree polynomial regression on fine discretizations of 2d and 3d regions of arbitrary shape. The key tool is Tchakaloff-like compression of discrete probability measures, via an improved version of the Lawson-Hanson NNLS solver for the corresponding full and large-scale underdetermined moment system, that can have for example a size order of 10\u2c63 (basis polynomials) x 10\u2c64 (nodes)

SOLID-SHELL FINITE ELEMENT MODELS FOR EXPLICIT SIMULATIONS OF CRACK PROPAGATION IN THIN STRUCTURES

Crack propagation in thin shell structures due to cutting is conveniently simulated using explicit finite element approaches, in view of the high nonlinearity of the problem. Solidshell elements are usually preferred for the discretization in the presence of complex material behavior and degradation phenomena such as delamination, since they allow for a correct representation of the thickness geometry. However, in solid-shell elements the small thickness leads to a very high maximum eigenfrequency, which imply very small stable time-steps. A new selective mass scaling technique is proposed to increase the time-step size without affecting accuracy. New ”directional” cohesive interface elements are used in conjunction with selective mass scaling to account for the interaction with a sharp blade in cutting processes of thin ductile shells

Numerical evaluation of aerodynamic roughness of the built environment and complex terrain

Aerodynamic drag in the atmospheric boundary layer (ABL) is affected by the structure and density of obstacles (surface roughness) and nature of the terrain (topography). In building codes and standards, average roughness is usually determined somewhat subjectively by examination of aerial photographs. For detailed wind mapping, boundary layer wind tunnel (BLWT) testing is usually recommended. This may not be cost effective for many projects, in which case numerical studies become good alternatives. This thesis examines Computational Fluid Dynamics (CFD) for evaluation of aerodynamic roughness of the built environment and complex terrain. The present study started from development of an in-house CFD software tailored for ABL simulations. A three-dimensional finite-volume code was developed using flexible polyhedral elements as building blocks. The program is parallelized using MPI to run on clusters of processors so that micro-scale simulations can be conducted quickly. The program can also utilize the power of latest technology in high performance computing, namely GPUs. Various turbulence models including mixing-length, RANS, and LES models are implemented, and their suitability for ABL simulations assessed. Then the effect of surface roughness alone on wind profiles is assessed using CFD. Cases with various levels of complexity are considered including simplified models with roughness blocks of different arrangement, multiple roughness patches, semi-idealized urban model, and real built environment. Comparison with BLWT data for the first three cases showed good agreement thereby justifying explicit three-dimensional numerical approach. Due to lack of validation data, the real built environment case served only to demonstrate use of CFD for such purposes. Finally, the effect of topographic features on wind profiles was investigated using CFD. This work extends prior work done by the research team on multiple idealized two-dimensional topographic features to more elaborate three-dimensional simulations. It is found that two-dimensional simulations overestimate speed up over crests of hills and also show larger recirculation zones. The current study also emphasized turbulence characterization behind hills. Finally a real complex terrain case of the well-known Askervein hill was simulated and the results validated against published field observations. In general the results obtained from the current simulations compared well with those reported in literature

Parallel algorithms for computational fluid dynamics on unstructured meshes

La simulació numèrica directa (DNS) de fluxos complexes és actualment una utopia per la majoria d'aplicacions industrials ja que els requeriments computacionals son massa elevats. Donat un flux, la diferència entre els recursos computacionals necessaris i els disponibles és cobreix mitjançant la modelització/simplificació d'alguns termes de les equacions originals que regeixen el seu comportament. El creixement continuat dels recursos computacionals disponibles, principalment en forma de super-ordinadors, contribueix a reduir la part del flux que és necessari aproximar. De totes maneres, obtenir la eficiència esperada dels nous super-ordinadors no és una tasca senzilla i, per aquest motiu, part de la recerca en el camp de la Mecànica de Fluids Computacional es centra en aquest objectiu. En aquest sentit, algunes contribucions s'han presentat en el marc d'aquesta tesis. El primer objectiu va ser el desenvolupament d'un codi de CFD de propòsit general i paral·lel, basat en la metodologia de volums finits en malles no estructurades, per resoldre problemes de multi-física. Aquest codi, anomenat TermoFluids (TF), té un disseny orientat a objectes i pensat per ser usat de forma altament eficient en els super-ordinadors actuals. Amb el temps, ha esdevingut pel grup una eina fonamental en projectes tant de recerca bàsica com d'interès industrial. En el context d'aquesta tesis, el treball s'ha focalitzat en el desenvolupament de dos de les llibreries més bàsiques de TermoFluids: i) La Basics Objects Library (BOL), que es una plataforma de software sobre la qual estan programades la resta de llibreries del codi, i que conté els mètodes algebraics i geomètrics fonamentals per la implementació paral·lela dels algoritmes de discretització, ii) la Linear Solvers Library (LSL), que conté un gran nombre de mètodes per resoldre els sistemes d'equacions lineals derivats de les discretitzacions. El primer capítol d'aquesta tesi conté les principals idees subjacents al disseny i la implementació de la BOL i la LSL, juntament amb alguns exemples i algunes aplicacions industrials. En els capítols posteriors hi ha una explicació detallada de solvers específics per algunes aplicacions concretes. En el segon capítol, es presenta un solver paral·lel i directe per la resolució de l'equació de Poisson per casos en els quals una de les direccions del domini té condicions d'homogeneïtat. En la simulació de fluxos incompressibles, l'equació de Poisson es resol almenys una vegada en cada pas de temps, convertint-se en una de les parts més costoses i difícils de paral·lelitzar del codi. El mètode que proposem és una combinació d'una descomposició directa de Schur (DDS) i una diagonalització de Fourier. La darrera descompon el sistema original en un conjunt de sub-sistemes 2D independents que es resolen mitjançant l'algorisme DDS. Atès que no s'imposen restriccions a les direccions no periòdiques del domini, aquest mètode és aplicable a la resolució de problemes discretitzats mitjançat l'extrusió de malles 2D no estructurades. L'escalabilitat d'aquest mètode ha estat provada amb èxit amb un màxim de 8192 CPU per malles de fins a ~10⁹ volums de control. En el darrer capitol capítol, es presenta un mètode de resolució per l'equació de Transport de Boltzmann (BTE). La estratègia emprada es basa en el mètode d'Ordenades Discretes i pot ser aplicat en discretitzacions no estructurades. El flux per a cada ordenada angular es resol amb un mètode de substitució equivalent a la resolució d'un sistema lineal triangular. La naturalesa seqüencial d'aquest procés fa de la paral·lelització de l'algoritme el principal repte. Diversos algorismes de substitució han estat analitzats, esdevenint una de les heurístiques proposades la millor opció en totes les situacions analitzades, amb excel·lents resultats. Els testos d'eficiència paral·lela s'han realitzat usant fins a 2560 CPU.Direct Numerical Simulation (DNS) of complex flows is currently an utopia for most of industrial applications because computational requirements are too high. For a given flow, the gap between the required and the available computing resources is covered by modeling/simplifying of some terms of the original equations. On the other hand, the continuous growth of the computing power of modern supercomputers contributes to reduce this gap, reducing hence the unresolved physics that need to be attempted with approximated models. This growth, widely relies on parallel computing technologies. However, getting the expected performance from new complex computing systems is becoming more and more difficult, and therefore part of the CFD research is focused on this goal. Regarding to it, some contributions are presented in this thesis. The first objective was to contribute to the development of a general purpose multi-physics CFD code. referred to as TermoFluids (TF). TF is programmed following the object oriented paradigm and designed to run in modern parallel computing systems. It is also intensively involved in many different projects ranging from basic research to industry applications. Besides, one of the strengths of TF is its good parallel performance demonstrated in several supercomputers. In the context of this thesis, the work was focused on the development of two of the most basic libraries that compose TF: I) the Basic Objects Library (BOL), which is a parallel unstructured CFD application programming interface, on the top of which the rest of libraries that compose TF are written, ii) the Linear Solvers Library (LSL) containing many different algorithms to solve the linear systems arising from the discretization of the equations. The first chapter of this thesis contains the main ideas underlying the design and the implementation of the BOL and LSL libraries, together with some examples and some industrial applications. A detailed description of some application-specific linear solvers included in the LSL is carried out in the following chapters. In the second chapter, a parallel direct Poisson solver restricted to problems with one uniform periodic direction is presented. The Poisson equation is solved, at least, once per time-step when modeling incompressible flows, becoming one of the most time consuming and difficult to parallelize parts of the code. The solver here proposed is a combination of a direct Schur-complement based decomposition (DSD) and a Fourier diagonalization. The latter decomposes the original system into a set of mutually independent 2D sub-systems which are solved by means of the DSD algorithm. Since no restrictions are imposed in the non-periodic directions, the overall algorithm is well-suited for solving problems discretized on extruded 2D unstructured meshes. The scalability of the solver has been successfully tested using up to 8192 CPU cores for meshes with up to 10 9 grid points. In the last chapter, a solver for the Boltzmann Transport Equation (BTE) is presented. It can be used to solve radiation phenomena interacting with flows. The solver is based on the Discrete Ordinates Method and can be applied to unstructured discretizations. The flux for each angular ordinate is swept across the computational grid, within a source iteration loop that accounts for the coupling between the different ordinates. The sequential nature of the sweep process makes the parallelization of the overall algorithm the most challenging aspect. Several parallel sweep algorithms, which represent different options of interleaving communications and calculations, are analyzed. One of the heuristics proposed consistently stands out as the best option in all the situations analyzed. With this algorithm, good scalability results have been achieved regarding both weak and strong speedup tests with up to 2560 CPUs

Fully iterative ILU preconditioning of the unsteady Navier-Stokes equations for GPGPU

none2In this work we investigate the numerical difficulties that arise in optimizing the efficiency of Newtonian fluids simulations on a massively parallel computing hardware like a GPU. In particular, we will concentrate on the resulting algebraic problem. We will present an approximate, fully-iterative, ILU preconditioner and we will show that this solution is very efficient on a GPU if compared with an intrinsic massively parallel preconditioner like the diagonal preconditioner, which indeed goes faster than more robust techniques, like ILU, despite their strong decrease in the number of iterations. We refer to GMRES as the iterative scheme used to solve the linear system. In particular, we will deal with the solution of incompressible flows with variable density and we will investigate the interplay between Reynolds and Atwood numbers. We will show that the numerical simulation at medium-high Reynolds numbers produces linear systems whose matrices can be reasonably preconditioned with the diagonal preconditioner, while at low Reynolds numbers the higher viscosity of the fluid makes the diagonal preconditioner ineffective in the solution time requested from GMRES and, decreasing the Reynolds number, unable to let GMRES converge at all. In this situation, we will show how an adequate iterative approach to the parallel solution of the triangular systems that result from the ILU preconditioning, turns out to be robust and efficient. We will show numerical results for variable-density fluids, discretized with the scheme described in Calgaro et al. [8], in classical benchmarks and, in particular, in the well-known Rayleigh-Taylor instability.mixedMonica Dessole; Fabio MarcuzziDessole, Monica; Marcuzzi, Fabi