Parallel Adaptive High-Order CFD Simulations Characterizing SOFIA Cavitiy Acoustics

This paper presents large-scale MPI-parallel computational uid dynamics simulations for the Stratospheric Observatory for Infrared Astronomy (SOFIA). SOFIA is an airborne, 2.5-meter infrared telescope mounted in an open cavity in the aft fuselage of a Boeing 747SP. These simulations focus on how the unsteady ow eld inside and over the cavity interferes with the optical path and mounting structure of the telescope. A tempo- rally fourth-order accurate Runge-Kutta, and a spatially fth-order accurate WENO-5Z scheme were used to perform implicit large eddy simulations. An immersed boundary method provides automated gridding for complex geometries and natural coupling to a block-structured Cartesian adaptive mesh re nement framework. Strong scaling studies using NASA's Pleiades supercomputer with up to 32k CPU cores and 4 billion compu- tational cells shows excellent scaling. Dynamic load balancing based on execution time on individual AMR blocks addresses irregular numerical cost associated with blocks con- taining boundaries. Limits to scaling beyond 32k cores are identi ed, and targeted code optimizations are discussed

Parallel Adjective High-Order CFD Simulations Characterizing SOFIA Cavity Acoustics

This paper presents large-scale MPI-parallel computational uid dynamics simulations for the Stratospheric Observatory for Infrared Astronomy (SOFIA). SOFIA is an airborne, 2.5-meter infrared telescope mounted in an open cavity in the aft fuselage of a Boeing 747SP. These simulations focus on how the unsteady ow eld inside and over the cavity interferes with the optical path and mounting structure of the telescope. A temporally fourth-order accurate Runge-Kutta, and spatially fth-order accurate WENO- 5Z scheme was used to perform implicit large eddy simulations. An immersed boundary method provides automated gridding for complex geometries and natural coupling to a block-structured Cartesian adaptive mesh re nement framework. Strong scaling studies using NASA's Pleiades supercomputer with up to 32k CPU cores and 4 billion compu- tational cells shows excellent scaling. Dynamic load balancing based on execution time on individual AMR blocks addresses irregular numerical cost associated with blocks con- taining boundaries. Limits to scaling beyond 32k cores are identi ed, and targeted code optimizations are discussed

Heat Transfer Mechanism In Particle-Laden Turbulent Shearless Flows

Particle-laden turbulent flows are one of the complex flow regimes involved in a wide range of environmental, industrial, biomedical and aeronautical applications. Recently the interest has included also the interaction between scalars and particles, and the complex scenario which arises from the interaction of particle finite inertia, temperature transport, and momentum and heat feedback of particles on the flow leads to a multi-scale and multi-physics phenomenon which is not yet fully understood. The present work aims to investigate the fluid-particle thermal interaction in turbulent mixing under one-way and two-way coupling regimes. A recent novel numerical framework has been used to investigate the impact of suspended sub-Kolmogorov inertial particles on heat transfer within the mixing layer which develops at the interface of two regions with different temperature in an isotropic turbulent flow. Temperature has been considered a passive scalar, advected by the solenoidal velocity field, and subject to the particle thermal feedback in the two-way regime. A self-similar stage always develops where all single-point statistics of the carrier fluid and the suspended particles collapse when properly re-scaled. We quantify the effect of particle inertial, parametrized through the Stokes and thermal Stokes numbers, on the heat transfer through the Nusselt number, defined as the ratio of the heat transfer to the thermal diffusion. A scale analysis will be presented. We show how the modulation of fluid temperature gradients due to the statistical alignments of the particle velocity and the local carrier flow temperature gradient field, impacts the overall heat transfer in the two-way coupling regime

A dynamic load balancing method for the evaluation of chemical reaction rates in parallel combustion simulations

The development and assessment of an efficient parallelization method for the evaluation of reaction rates in combustion simulations is presented. Combustion simulations where the finite-rate chemistry model is employed are computationally expensive. In such simulations, a transport equation for each species in the chemical reaction mechanism has to be solved, and the resulting system of equations is typically stiff. As a result, advanced implicit methods must be applied to obtain accurate solutions using reasonable time-steps at expenses of higher computational resources than explicit or classical implicit methods. In the present work, a new algorithm aimed to enhance the numerical performance of the time integration of stiffsystems of equations in parallel combustion simulations is presented. The algorithm is based on a runtime load balancing mechanism, increasing noteworthy the computational performance of the simulations, and consequently, reducing significantly the computer time required to perform the numerical combustion studies.Peer ReviewedPostprint (published version

Automatic parallel octree grid generation software with an extensible solver framework and a focus on urban simulation

The development of an automatic, dynamic, parallel, Cartesian, linear forest-of-octree grid generator and partial differential equation (PDE) solver framework is presented. This research is bundled into an application programmed with C++ which uses MPI for distributed parallelism. The application is named paros which stands for PARallel Octree Solver. In its current implementation, the application provides a \u27zeroth\u27 order representation of the target geometry, and as such, no cut-cell algorithm, projection method, or immersed boundary condition are implemented. In this case, \u27zeroth\u27 order means that no geometry is ever exactly represented in the final computational mesh: an octree element is either completely in the domain or entirely outside of it. Any element that contains or is intersected by a geometry facet is removed from the final mesh which results in a \u27blocky\u27 or \u27stepped\u27 geometry representation and simplifies boundary computations. The computational octree mesh creation is completely parallel and automated. The algorithm is dynamic in the sense that it is repartitioned dynamically throughout the grid generation process to maintain optimal load balancing during all phases of the mesh genesis. A linear octree data structure is used to store the octree mesh elements and is leveraged for optimal load balancing. An additional hierarchical octree is used to significantly improve algorithms that suffer from this linear storage paradigm. This work focuses on, but is not limited to, applications related to urban simulations and may be applied to plume/contaminant propagation. Within the PDE solution framework a cell-centered, incompressible, unsteady, Navier Stokes solver with an energy term to account for thermal buoyancy is implemented and validated using canonical test cases. Turbulence closure is implemented in the form of the Smagorinski large eddy simulation (LES) model. The parallel grid generation and solution process is tested on a large scale cityscape geometry and shown to be robust and efficient. Additionally, an implementation of the compressible Navier-Stokes equations is coded within the framework. The framework is extensible such that adding other types of numerical PDE solvers should not be difficult. Other features including adaptive mesh refinement (AMR) and contaminant transport functionality are included

Parallel simulation of multiphase flows using octree adaptivity and the volume-of-fluid method

International audienceWe describe computations performed using the Gerris code, an open-source software implementing finite volumesolvers on an octree adaptive grid together with a piecewise linear volume of fluid interface tracking method. Theparallelisation of Gerris is achieved by domain decomposition. We show examples of the capabilities of Gerris onseveral types of problems. The impact of a droplet on a layer of the same liquid results in the formation of a thinair layer trapped between the droplet and the liquid layer that the adaptive refinement allows to capture. It isfollowed by the jetting of a thin corolla emerging from below the impacting droplet. The jet atomization problemis another extremely challenging computational problem, in which a large number of small scales are generated.Finally we show an example of a turbulent jet computation in an equivalent resolution of 6 × 10243 cells. Thejet simulation is based on the configuration of the Deepwater Horizon oil lea

Modelling and numerical simulation of combustion and multi-phase flows using finite volume methods on unstructured meshes

The present thesis is devoted to the development and implementation of mathematical models and numerical methods in order to carry out computational simulations of complex heat and mass transfer phenomena. Several areas and topics in the field of Computational Fluid Dynamics (CFD) have been treated and covered during the development of the current thesis, specially combustion and dispersed multi-phase flows. This type of simulations requires the implementation and coupling of different physics. The numerical simulation of multiphysics phenomena is challenging due to the wide range of spatial and temporal scales which can characterize each one of the physics involved in the problem. Moreover, when solving turbulent flows, turbulence itself is a very complex physical phenomenon that can demand a huge computational effort. Hence, in order to make turbulent flow simulations computationally affordable, the turbulence should be modelled. Therefore, throughout this thesis different numerical methods and algorithms have been developed and implemented aiming to perform multiphysics simulations in turbulent flows. The first topic addressed is turbulent combustion. Chapter 2 presents a combustion model able to notably reduce the computational cost of the simulation. The model, namely the Progress-Variable (PV) model, relies on a separation of the spatio-temporal scales between the flow and the chemistry. Moreover, in order to account for the influence of the sub-grid species concentrations and energy fluctuations, the PV model is coupled to the Presumed Conditional Moment (PCM) model. Chapter 2 also shows the development of a smart load-balancing method for the evaluation of chemical reaction rates in parallel combustion simulations. Chapter 3 is devoted to dispersed multiphase flows. This type of flows are composed of a continuous phase and a dispersed phase in the form of unconnected particles or droplets. In this thesis, the Eulergian-Lagrangian approach has been selected. This type of model is the best-suited for dispersed multiphase flows with thousands or millions of particles, and with a flow regime ranging from the very dilute up to relatively dense. In Chapter 4, a new method capable of performing parallel numerical simulations using non-overlapping disconnected mesh domains with adjacent boundaries is presented. The presented algorithm stitches at each iteration independent meshes and solves them as a unique domain. Finally, Chapter 5 addresses a transversal aspect to the previously covered topics throughout the thesis. In this chapter, a self-adaptive strategy for the maximisation of the time-step for the numerical solution of convection-diffusion equations is discussed. The method is capable of determining dynamically at each iteration which is the maximum allowable time-step which assures a stable time integration. Moreover, the method also smartly modifies the temporal integration scheme in order to maximize its stability region depending on the properties of the system matrix.La present tesis està dedicada al desenvolupament e implementació de models matemàtics i mètodes numèrics amb l’objectiu de realitzar simulacions computacionals de fenòmens complexos de transferència de calor i massa. Diverses àrees i temes en el camp de la Dinàmica de Fluids Computacional (CFD) han sigut tractats i coberts durant el desenvolupament de la present tesi, en especial, la combustió i els fluxos multi-fase dispersos. Aquest tipus de simulacions de fenòmens multi-físics es desafiant degut al gran rang d’escales espaio-temporals que poden caracteritzar cada una de les físiques involucrades en el problema. D’altra banda, quan es resolen fluxos turbulents, la pròpia turbulència ja és un fenomen físic molt complex que pot requerir un gran esforç computacional. Per tant, amb l’objectiu de fer les simulacions computacionals de fluxos turbulents computacionalment assequibles, la turbulència ha de ser modelada. Per tant, durant aquesta tesis diferents mètodes i algoritmes han sigut desenvolupats e implementats amb l’objectiu de realitzar simulacions multi-físiques en fluxos turbulents. El primer tema abordat és la combustió turbulenta. El Capítol 2 presenta un model de combustió capaç de reduir notablement el cost computacional de la simulació. El model, anomenat el model Progress-Variable (PV), està basat en la separació d’escales espaio-temporals entre el fluid i la química. A més, amb l’objectiu de tenir en compte l’influencia de les fluctuacions a nivell sub-grid d’energia i concentracions d'espècies, el model PV s’acobla amb el model Presumed Conditional Moment (PCM). El Capítol 2 també mostra el desenvolupament d’un mètode intel·ligent de balanceig de càrrega per l'avaluació de el rati de reacció químic en simulacions de combustió paral·leles. El Capítol 3 està dedicat als fluxos multi-fase dispersos. Aquest tipus de fluids estan formats per una fase continua i una fase dispersa en forma de partícules o gotes inconnexes. En aquesta tesis, l’aproximació Euleriana-Lagrangiana ha sigut la seleccionada. Aquest tipus de model és el més adequat per fluxos multi-fase dispersos amb milers o milions de partícules, i amb règims que van des del molt diluït fins al relativament dens. Al Capítol 4, es presenta un nou mètode capaç de realitzar simulacions numèriques paral·leles utilitzant malles inconnexes no solapades que tenen fronteres adjacents. L’algoritme presentat cus a cada iteració les malles independents i les resol com un únic domini. Finalment, el Capítol 5 tracta un aspecte transversal a tots els temes coberts al llarg de la tesi. En aquest capítol es discuteix una estratègia auto-adaptativa destinada a la maximització del pas de temps per a la solució numèrica d’equacions de convecció-difusió. El mètode es capaç de determinar dinàmicament a cada iteració quin és el màxim pas de temps possible que assegura una integració temporal estable. A més, el mètode també modifica de forma intel·ligent la regió d’estabilitat en funció de les propietats de la matriu del sistema.Postprint (published version

Das unstetige Galerkinverfahren für Strömungen mit freier Oberfläche und im Grundwasserbereich in geophysikalischen Anwendungen

Free surface flows and subsurface flows appear in a broad range of geophysical applications and in many environmental settings situations arise which even require the coupling of free surface and subsurface flows. Many of these application scenarios are characterized by large domain sizes and long simulation times. Hence, they need considerable amounts of computational work to achieve accurate solutions and the use of efficient algorithms and high performance computing resources to obtain results within a reasonable time frame is mandatory. Discontinuous Galerkin methods are a class of numerical methods for solving differential equations that share characteristics with methods from the finite volume and finite element frameworks. They feature high approximation orders, offer a large degree of flexibility, and are well-suited for parallel computing. This thesis consists of eight articles and an extended summary that describe the application of discontinuous Galerkin methods to mathematical models including free surface and subsurface flow scenarios with a strong focus on computational aspects. It covers discretization and implementation aspects, the parallelization of the method, and discrete stability analysis of the coupled model.Für viele geophysikalische Anwendungen spielen Strömungen mit freier Oberfläche und im Grundwasserbereich oder sogar die Kopplung dieser beiden eine zentrale Rolle. Oftmals charakteristisch für diese Anwendungsszenarien sind große Rechengebiete und lange Simulationszeiten. Folglich ist das Berechnen akkurater Lösungen mit beträchtlichem Rechenaufwand verbunden und der Einsatz effizienter Lösungsverfahren sowie von Techniken des Hochleistungsrechnens obligatorisch, um Ergebnisse innerhalb eines annehmbaren Zeitrahmens zu erhalten. Unstetige Galerkinverfahren stellen eine Gruppe numerischer Verfahren zum Lösen von Differentialgleichungen dar, und kombinieren Eigenschaften von Methoden der Finiten Volumen- und Finiten Elementeverfahren. Sie ermöglichen hohe Approximationsordnungen, bieten einen hohen Grad an Flexibilität und sind für paralleles Rechnen gut geeignet. Diese Dissertation besteht aus acht Artikeln und einer erweiterten Zusammenfassung, in diesen die Anwendung unstetiger Galerkinverfahren auf mathematische Modelle inklusive solcher für Strömungen mit freier Oberfläche und im Grundwasserbereich beschrieben wird. Die behandelten Themen umfassen Diskretisierungs- und Implementierungsaspekte, die Parallelisierung der Methode sowie eine diskrete Stabilitätsanalyse des gekoppelten Modells