Design and Analysis of a Task-based Parallelization over a Runtime System of an Explicit Finite-Volume CFD Code with Adaptive Time Stepping

FLUSEPA (Registered trademark in France No. 134009261) is an advanced simulation tool which performs a large panel of aerodynamic studies. It is the unstructured finite-volume solver developed by Airbus Safran Launchers company to calculate compressible, multidimensional, unsteady, viscous and reactive flows around bodies in relative motion. The time integration in FLUSEPA is done using an explicit temporal adaptive method. The current production version of the code is based on MPI and OpenMP. This implementation leads to important synchronizations that must be reduced. To tackle this problem, we present the study of a task-based parallelization of the aerodynamic solver of FLUSEPA using the runtime system StarPU and combining up to three levels of parallelism. We validate our solution by the simulation (using a finite-volume mesh with 80 million cells) of a take-off blast wave propagation for Ariane 5 launcher.Comment: Accepted manuscript of a paper in Journal of Computational Scienc

MPMD parallelization of an aerodynamic code with bodies in relative motion

International audienceFLUSEPA is an advanced simulation tool which performs a large panel of aerodynamic studies. It is the unstructured finite-volume solver developed by Airbus Defence & Space company to calculate compressible, multidimensional, unsteady, viscous and reactive flows around bodies in relative motion. The numerical strategy in FLUSEPA is designed for highly compressible flows and keeps its accuracy in non-Cartesian grids. According the desired accuracy, a second-order accurate shock-capturing scheme is generally used for RANS and URANS simulations and a fourth order accurate vortex-centered scheme is used for hybrid RANS/LES simulations. The meshing strategy is based on multi-overlapping grid intersection which is conservative and allows to quickly and properly mesh 3D complex geometries. It can be seen as a CHIMERA strategy without interpolation. Each body is meshed independently and immersed in background grids.MPMD parallelisation of this code is presented

Task-based parallelization of a CFD code over a Runtime System

International audienceFLUSEPA is an advanced simulation tool which performs a large panel of aerodynamic studies. It is the unstructured nite-volume solver developed by Airbus Defence & Space company to calculate compressible, multidimensional, unsteady, viscous and reactive flows around bodies in relative motion [2](Figure 1). The numerical strategy in FLUSEPA is designed for highly compressible tlows and to remain accurate when using non-Cartesian grids. The meshing strategy is based on multi-overlapping grid intersection which is conservative and allows to quickly mesh 3D complex geometries. The time integration in FLUSEPA is done using an explicit temporal adaptive method [3]. Instead of using a time step imposed by the slowest cell, cells are grouped inside temporal classes according to their maximum allowed time step. This method allows to perform a fewer number of operations at the expense of some complexity. An iteration of the aerodynamic solver is separated into several sub-iterations. Every cell is not integrated during each sub-iteration but cells which share a face with a slower one are interpolated. At the end of the iteration, each cell has reached the same time. Between iterations, temporal classes can evolve.The current version of FLUSEPA is parallelized using a classical MPI-OpenMP approach and domain decomposition: border faces are duplicated and ghost cells are used. However, the way the time is integrated leads to important time wasted in synchronization despite computation-communication overlapping. The time integration implies an order for the cells to be processed depending on their temporal class: neighbor cells must be at the same time during processing. This locality information is partially lost with the current parallelization. In this paper, we present a way to overcome this issue by working on sub-domains inside each process. We capture the dependencies more precisely through the use of a task-based parallel expression with the help of a well suited runtime system

Towards an efficient Task-based Parallelization over a Runtime System of an Explicit Finite-Volume CFD Code with Adaptive Time Stepping

International audienceFLUSEPA is an advanced simulation tool which performs a large panel of aerodynamic studies. It is the unstruc-tured finite-volume solver developed by Airbus Defence & Space company to calculate compressible, multidimensional, unsteady, viscous and reactive flows around bodies in relative motion. The time integration in FLUSEPA is done using an explicit temporal adaptive method. The current production version of the code is based on MPI and OpenMP. This implementation leads to important synchronizations that must be reduced. To tackle this problem, we present a first study of a task-based parallelization of the solver part of FLUSEPA using the runtime system StarPU and combining up to three levels of parallelism. We validate our solution on the simulation of a takeoff blast wave propagation for Ariane 5 launcher

Contribution à la parallélisation et au passage à l'échelle du code FLUSEPA

There are different kinds of satellites that offer different services like communication, navigationor observation. They are put into orbit through the use of launchers whose design is oneof the main activities of Airbus Defence and Space. Relying on experiments is not easy : windtunnel cannot be used to evaluate every critical situation that a launcher will face during itsmission. Numerical simulation is therefore mandatory for spatial industry.In order to have more reliable simulations, more computational power is needed and supercomputersare used. Those supercomputers become more and more complex and this impliesto adapt existing codes to make them run efficiently. Nowadays, it seems important to rely onabstractions in order to ensure a good portability of performance. Airbus Defence and Spacedeveloped for more than 20 years the FLUSEPA code which is used to compute unsteady phenomenalike take-off blast wave or stage separation. The aerodynamic solver relies on a finitevolume formulation and an explicit temporal adaptive solver. Bodies in relative motion are takeninto account through the use of multiple meshes that are overlapped.This thesis is about the parallelization of the FLUSEPA code. At the start of the thesis,the only parallel version available was in shared memory through OpenMP. A first distributedmemory version was realized and relies on MPI and OpenMP. The performance improvementof this version was evaluated on two industrial test cases. A task-based demonstrator of theaerodynamic solver was also realized over a runtime system.Les satellites sont mis en orbite en utilisant des lanceurs dont la conception est une des activités principales d’Airbus Defence and Space. Pour ce faire, se baser sur des expériences n’est pas facile : les souffleries ne permettent pas d’évaluer toutes les situations auxquelles un lanceur est confronté au cours de sa mission. La simulation numérique est donc essentielle pour l’industrie spatiale. Afin de disposer de simulations toujours plus fidèles, il est nécessaire d’utiliser des supercalculateurs de plus en plus puissants. Cependant, ces machines voient leur complexité augmenter et pour pouvoir exploiter leur plein potentiel, il est nécessaire d’adapter les codes existants. Désormais, il semble essentiel de passer par des couches d’abstraction afin d’assurer une bonne portabilité des performances. ADS a développé depuis plus de 20 ans le code FLUSEPA qui est utilisé pour le calcul de phénomènes instationnaires comme les calculs d’onde de souffle au décollage ou les séparations d’étages. Le solveur aérodynamique est basé sur une formulation volume fini et une technique d’intégration temporelle adaptative. Les corps en mouvement sont pris en compte via l’utilisation de plusieurs maillages qui sont combinés par intersections.Cette thèse porte sur la parallélisation du code FLUSEPA. Au début de la thèse, la seule version parallèle disponible était en mémoire partagée. Une première version parallèle en mémoire distribuée a d’abord été réalisée. Les gains en performance de cette version ont été évalués via l’utilisation de deux cas tests industriels. Un démonstrateur du solveur aérodynamique utilisant la programmation par tâche au dessus d’un runtime a aussi été réalisé

Contributions to the parallelization and the scalability of the FLUSEPA code

Les satellites sont mis en orbite en utilisant des lanceurs dont la conception est une des activités principales d’Airbus Defence and Space. Pour ce faire, se baser sur des expériences n’est pas facile : les souffleries ne permettent pas d’évaluer toutes les situations auxquelles un lanceur est confronté au cours de sa mission. La simulation numérique est donc essentielle pour l’industrie spatiale. Afin de disposer de simulations toujours plus fidèles, il est nécessaire d’utiliser des supercalculateurs de plus en plus puissants. Cependant, ces machines voient leur complexité augmenter et pour pouvoir exploiter leur plein potentiel, il est nécessaire d’adapter les codes existants. Désormais, il semble essentiel de passer par des couches d’abstraction afin d’assurer une bonne portabilité des performances. ADS a développé depuis plus de 20 ans le code FLUSEPA qui est utilisé pour le calcul de phénomènes instationnaires comme les calculs d’onde de souffle au décollage ou les séparations d’étages. Le solveur aérodynamique est basé sur une formulation volume fini et une technique d’intégration temporelle adaptative. Les corps en mouvement sont pris en compte via l’utilisation de plusieurs maillages qui sont combinés par intersections.Cette thèse porte sur la parallélisation du code FLUSEPA. Au début de la thèse, la seule version parallèle disponible était en mémoire partagée. Une première version parallèle en mémoire distribuée a d’abord été réalisée. Les gains en performance de cette version ont été évalués via l’utilisation de deux cas tests industriels. Un démonstrateur du solveur aérodynamique utilisant la programmation par tâche au dessus d’un runtime a aussi été réalisé.There are different kinds of satellites that offer different services like communication, navigationor observation. They are put into orbit through the use of launchers whose design is oneof the main activities of Airbus Defence and Space. Relying on experiments is not easy : windtunnel cannot be used to evaluate every critical situation that a launcher will face during itsmission. Numerical simulation is therefore mandatory for spatial industry.In order to have more reliable simulations, more computational power is needed and supercomputersare used. Those supercomputers become more and more complex and this impliesto adapt existing codes to make them run efficiently. Nowadays, it seems important to rely onabstractions in order to ensure a good portability of performance. Airbus Defence and Spacedeveloped for more than 20 years the FLUSEPA code which is used to compute unsteady phenomenalike take-off blast wave or stage separation. The aerodynamic solver relies on a finitevolume formulation and an explicit temporal adaptive solver. Bodies in relative motion are takeninto account through the use of multiple meshes that are overlapped.This thesis is about the parallelization of the FLUSEPA code. At the start of the thesis,the only parallel version available was in shared memory through OpenMP. A first distributedmemory version was realized and relies on MPI and OpenMP. The performance improvementof this version was evaluated on two industrial test cases. A task-based demonstrator of theaerodynamic solver was also realized over a runtime system

Contributions to the parallelization and the scalability of the FLUSEPA code

Les satellites sont mis en orbite en utilisant des lanceurs dont la conception est une des activités principales d’Airbus Defence and Space. Pour ce faire, se baser sur des expériences n’est pas facile : les souffleries ne permettent pas d’évaluer toutes les situations auxquelles un lanceur est confronté au cours de sa mission. La simulation numérique est donc essentielle pour l’industrie spatiale. Afin de disposer de simulations toujours plus fidèles, il est nécessaire d’utiliser des supercalculateurs de plus en plus puissants. Cependant, ces machines voient leur complexité augmenter et pour pouvoir exploiter leur plein potentiel, il est nécessaire d’adapter les codes existants. Désormais, il semble essentiel de passer par des couches d’abstraction afin d’assurer une bonne portabilité des performances. ADS a développé depuis plus de 20 ans le code FLUSEPA qui est utilisé pour le calcul de phénomènes instationnaires comme les calculs d’onde de souffle au décollage ou les séparations d’étages. Le solveur aérodynamique est basé sur une formulation volume fini et une technique d’intégration temporelle adaptative. Les corps en mouvement sont pris en compte via l’utilisation de plusieurs maillages qui sont combinés par intersections.Cette thèse porte sur la parallélisation du code FLUSEPA. Au début de la thèse, la seule version parallèle disponible était en mémoire partagée. Une première version parallèle en mémoire distribuée a d’abord été réalisée. Les gains en performance de cette version ont été évalués via l’utilisation de deux cas tests industriels. Un démonstrateur du solveur aérodynamique utilisant la programmation par tâche au dessus d’un runtime a aussi été réalisé.There are different kinds of satellites that offer different services like communication, navigationor observation. They are put into orbit through the use of launchers whose design is oneof the main activities of Airbus Defence and Space. Relying on experiments is not easy : windtunnel cannot be used to evaluate every critical situation that a launcher will face during itsmission. Numerical simulation is therefore mandatory for spatial industry.In order to have more reliable simulations, more computational power is needed and supercomputersare used. Those supercomputers become more and more complex and this impliesto adapt existing codes to make them run efficiently. Nowadays, it seems important to rely onabstractions in order to ensure a good portability of performance. Airbus Defence and Spacedeveloped for more than 20 years the FLUSEPA code which is used to compute unsteady phenomenalike take-off blast wave or stage separation. The aerodynamic solver relies on a finitevolume formulation and an explicit temporal adaptive solver. Bodies in relative motion are takeninto account through the use of multiple meshes that are overlapped.This thesis is about the parallelization of the FLUSEPA code. At the start of the thesis,the only parallel version available was in shared memory through OpenMP. A first distributedmemory version was realized and relies on MPI and OpenMP. The performance improvementof this version was evaluated on two industrial test cases. A task-based demonstrator of theaerodynamic solver was also realized over a runtime system

Contribution à la parallélisation et au passage à l'échelle du code FLUSEPA

There are different kinds of satellites that offer different services like communication, navigationor observation. They are put into orbit through the use of launchers whose design is oneof the main activities of Airbus Defence and Space. Relying on experiments is not easy : windtunnel cannot be used to evaluate every critical situation that a launcher will face during itsmission. Numerical simulation is therefore mandatory for spatial industry.In order to have more reliable simulations, more computational power is needed and supercomputersare used. Those supercomputers become more and more complex and this impliesto adapt existing codes to make them run efficiently. Nowadays, it seems important to rely onabstractions in order to ensure a good portability of performance. Airbus Defence and Spacedeveloped for more than 20 years the FLUSEPA code which is used to compute unsteady phenomenalike take-off blast wave or stage separation. The aerodynamic solver relies on a finitevolume formulation and an explicit temporal adaptive solver. Bodies in relative motion are takeninto account through the use of multiple meshes that are overlapped.This thesis is about the parallelization of the FLUSEPA code. At the start of the thesis,the only parallel version available was in shared memory through OpenMP. A first distributedmemory version was realized and relies on MPI and OpenMP. The performance improvementof this version was evaluated on two industrial test cases. A task-based demonstrator of theaerodynamic solver was also realized over a runtime system.Les satellites sont mis en orbite en utilisant des lanceurs dont la conception est une des activités principales d’Airbus Defence and Space. Pour ce faire, se baser sur des expériences n’est pas facile : les souffleries ne permettent pas d’évaluer toutes les situations auxquelles un lanceur est confronté au cours de sa mission. La simulation numérique est donc essentielle pour l’industrie spatiale. Afin de disposer de simulations toujours plus fidèles, il est nécessaire d’utiliser des supercalculateurs de plus en plus puissants. Cependant, ces machines voient leur complexité augmenter et pour pouvoir exploiter leur plein potentiel, il est nécessaire d’adapter les codes existants. Désormais, il semble essentiel de passer par des couches d’abstraction afin d’assurer une bonne portabilité des performances. ADS a développé depuis plus de 20 ans le code FLUSEPA qui est utilisé pour le calcul de phénomènes instationnaires comme les calculs d’onde de souffle au décollage ou les séparations d’étages. Le solveur aérodynamique est basé sur une formulation volume fini et une technique d’intégration temporelle adaptative. Les corps en mouvement sont pris en compte via l’utilisation de plusieurs maillages qui sont combinés par intersections.Cette thèse porte sur la parallélisation du code FLUSEPA. Au début de la thèse, la seule version parallèle disponible était en mémoire partagée. Une première version parallèle en mémoire distribuée a d’abord été réalisée. Les gains en performance de cette version ont été évalués via l’utilisation de deux cas tests industriels. Un démonstrateur du solveur aérodynamique utilisant la programmation par tâche au dessus d’un runtime a aussi été réalisé

StarPU

StarPU is a runtime system that offers support for heterogeneous multicore machines