Massively Parallel Algorithm for Solving the Eikonal Equation on Multiple Accelerator Platforms

The research presented in this thesis investigates parallel implementations of the Fast Sweeping Method (FSM) for Graphics Processing Unit (GPU)-based computational plat forms and proposes a new parallel algorithm for distributed computing platforms with accelerators. Hardware accelerators such as GPUs and co-processors have emerged as general- purpose processors in today’s high performance computing (HPC) platforms, thereby increasing platforms’ performance capabilities. This trend has allowed greater parallelism and substantial acceleration of scientific simulation software. In order to leverage the power of new HPC platforms, scientific applications must be written in specific lower-level programming languages, which used to be platform specific. Newer programming models such as OpenACC simplifies implementation and assures portability of applications to run across GPUs from different vendors and multi-core processors. The distance field is a representation of a surface geometry or shape required by many algorithms within the areas of computer graphics, visualization, computational fluid dynamics and more. It can be calculated by solving the eikonal equation using the FSM. The parallel FSMs explored in this thesis have not been implemented on GPU platforms and do not scale to a large problem size. This thesis addresses this problem by designing a parallel algorithm that utilizes a domain decomposition strategy for multi-accelerated distributed platforms. The proposed algorithm applies first coarse grain parallelism using MPI to distribute subdomains across multiple nodes and then fine grain parallelism to optimize performance by utilizing accelerators. The results of the parallel implementations of FSM for GPU-based platforms showed speedup greater than 20× compared to the serial version for some problems and the newly developed parallel algorithm eliminates the limitation of current algorithms to solve large memory problems with comparable runtime efficiency

A FAST ITERATIVE METHOD FOR EIKONAL EQUATIONS

In this paper we propose a novel computational technique to solve the Eikonal equation efficiently on parallel architectures. The proposed method manages the list of active nodes and iteratively updates the solutions on those nodes until they converge. Nodes are added to or removed from the list based on a convergence measure, but the management of this list does not entail an extra burden of expensive ordered data structures or special updating sequences. The proposed method has suboptimal worst-case performance but, in practice, on real and synthetic datasets, runs faster than guaranteed-optimal alternatives. Furthermore, the proposed method uses only local, synchronous updates and therefore has better cache coherency, is simple to implement, and scales efficiently on parallel architectures. This paper describes the method, proves its consistency, gives a performance analysis that compares the proposed method against the state-of-the-art Eikonal solvers, and describes the implementation on a single instruction multiple datastream (SIMD) parallel architecture.open393

Doctor of Philosophy

dissertationPartial differential equations (PDEs) are widely used in science and engineering to model phenomena such as sound, heat, and electrostatics. In many practical science and engineering applications, the solutions of PDEs require the tessellation of computational domains into unstructured meshes and entail computationally expensive and time-consuming processes. Therefore, efficient and fast PDE solving techniques on unstructured meshes are important in these applications. Relative to CPUs, the faster growth curves in the speed and greater power efficiency of the SIMD streaming processors, such as GPUs, have gained them an increasingly important role in the high-performance computing area. Combining suitable parallel algorithms and these streaming processors, we can develop very efficient numerical solvers of PDEs. The contributions of this dissertation are twofold: proposal of two general strategies to design efficient PDE solvers on GPUs and the specific applications of these strategies to solve different types of PDEs. Specifically, this dissertation consists of four parts. First, we describe the general strategies, the domain decomposition strategy and the hybrid gathering strategy. Next, we introduce a parallel algorithm for solving the eikonal equation on fully unstructured meshes efficiently. Third, we present the algorithms and data structures necessary to move the entire FEM pipeline to the GPU. Fourth, we propose a parallel algorithm for solving the levelset equation on fully unstructured 2D or 3D meshes or manifolds. This algorithm combines a narrowband scheme with domain decomposition for efficient levelset equation solving

Wall Distance Evaluation Via Eikonal Solver for RANS Applications

RÉSUMÉ Les logiciels de mécanique des fluides assistée par ordinateur (CFD) sont de plus en plus utilisés pour la conception d’aéronefs. L’utilisation de grappes informatiques haute performance permet d’augmenter la puissance de calcul, aux prix de modifier la structure du code. Dans les codes CFD, les équations de Navier-Stokes moyennées (plus connues sous le nom des équations RANS) sont souvent résolues. Par conséquent, les modèles de turbulence sont utilisés pour approximer les effets de la turbulence. Dans l’industrie aéronautique, le modèle Spalart-Allmaras est bien accepté. La distance à la paroi dans ce modèle, par exemple, joue un rôle clé dans l’évaluation des forces aérodynamiques. L’évaluation de ce paramètre géométrique doit alors être précis et son calcul efficace. Avec les nouveaux développement des hardwares, un besoin se crée dans la communauté afin d’adapter les codes CFD à ceux-ci. Les algorithmes de recherche comme les distances euclidienne et projetée sont des méthodes souvent utilisées pour le calcul de la distance à la paroi et ont tendance à présenter une mauvaise scalabilité. Pour cette raison, un nouveau solveur pour la distance à la paroi doit être développé. Pour utiliser les solveurs et techniques d’accélération déjà existantes au sein du code CFD, l’équation Eikonal, une équation aux différentielles partielles non-linéaires, a été choisie. Dans la première partie du projet, le solveur d’équation Eikonal est développé en 2D et est résolue dans sa forme advective au centre de cellule. Les méthodes des différences finies et des volumes finis sont testées. L’équation est résolue à l’aide d’une discrétisation spatiale de premier ordre en amont. Les solveurs ont été vérifiés sur des cas canoniques, tels une plaque plane et un cylindre. Les deux méthodes de discrétisation réussissent à corriger les effets de maillages obliques et courbes. La méthode des différences finies possède un taux de convergence en maillage de deuxième ordre tandis que la méthode des volumes finis a un taux de convergence de premier ordre. L’addition d’une reconstruction linéaire de la solution à la face permet d’étendre la méthode des volumes finis à une méthode de deuxième ordre. De plus, les méthodes de différence finie et de volume fini de deuxième ordre permettent de bien représenter la distance à la paroi dans les zones de fort élargissement des cellules. L’équation Eikonal est ensuite vérifié sur plusieurs cas dont un profil NACA0012 en utilisant trois modèles de turbulence : Spalart-Allmaras, Menter SST et Mener-Langtry SST transitionnel.----------ABSTRACT Computational fluid dynamics (CFD) software is being used more often nowadays in aircraft design. The use of high performance computing clusters can increase computing power, but requires change in the structure of the software. In the aeronautical industry, CFD codes are often used to solve the Reynolds-Averaged Navier-Stokes (RANS) equations, and turbulence models are frequently used to approximate turbulent effects on flow. The Spalart-Allmaras turbulence model is widely accepted in the industry. In this model, wall distance plays a key role in the evaluation of aerodynamic forces. Therefore calculation of this geometric parameter needs to be accurate and efficient. With new developments in computing hardware, there is a need to adapt CFD codes. Search algorithms such as Euclidean and projected distance are often the methods used for computation of wall distance but tend to exhibit poor scalability. For this reason, a new wall distance solver is developed here using the Eikonal equation, a non-linear partial differential equation, chosen to make use of existing solvers and acceleration techniques in RANS solvers. In the first part of the project, the Eikonal equation solver was developed in 2D and solved in its advective form at the cell center. Both finite difference and finite volume methods were tested. The Eikonal equation was also solved using a first-order upwind spatial discretization. The solvers were verified through canonical cases like a flat plate and a cylinder. Both methods were able to correct the effects of skewed and curved meshes. The finite difference method converged at a second-order rate in space while the finite volume method converged at a first-order rate. The addition of a linear reconstruction of the solution at the face extended the finite volume method to a second-order method. Moreover, both finite difference and second-order finite volume methods were well represented by wall distance in zones of strong cell growth. The finite difference method was chosen, as it required less computing time. The Eikonal equation was then verified for several cases including a NACA0012 using three turbulence models: Spalart-Allmaras, Menter’s SST and Menter-Langtry transitional SST. For the first model, the Eikonal equation was able to correct grid skewness on the turbulent viscosity as well as on the aerodynamic coefficients, while for the other two yielded results similar to Euclidean and projected distance. To verify the implementation and convergence of the multi-grid scheme, the new wall distance solver was tested on an ice-accreted airfoil. In addition, the overset grid capabilities of the wall distance solver were verified on the McDonnell Douglas airfoil. Finally, the DLR-F6, a 3D case, was solved to show that the Eikonal equation can be extended to 3D meshes

Wall Distance Evaluation Via Eikonal Solver for RANS Applications

RÉSUMÉ Les logiciels de mécanique des fluides assistée par ordinateur (CFD) sont de plus en plus utilisés pour la conception d’aéronefs. L’utilisation de grappes informatiques haute performance permet d’augmenter la puissance de calcul, aux prix de modifier la structure du code. Dans les codes CFD, les équations de Navier-Stokes moyennées (plus connues sous le nom des équations RANS) sont souvent résolues. Par conséquent, les modèles de turbulence sont utilisés pour approximer les effets de la turbulence. Dans l’industrie aéronautique, le modèle Spalart-Allmaras est bien accepté. La distance à la paroi dans ce modèle, par exemple, joue un rôle clé dans l’évaluation des forces aérodynamiques. L’évaluation de ce paramètre géométrique doit alors être précis et son calcul efficace. Avec les nouveaux développement des hardwares, un besoin se crée dans la communauté afin d’adapter les codes CFD à ceux-ci. Les algorithmes de recherche comme les distances euclidienne et projetée sont des méthodes souvent utilisées pour le calcul de la distance à la paroi et ont tendance à présenter une mauvaise scalabilité. Pour cette raison, un nouveau solveur pour la distance à la paroi doit être développé. Pour utiliser les solveurs et techniques d’accélération déjà existantes au sein du code CFD, l’équation Eikonal, une équation aux différentielles partielles non-linéaires, a été choisie. Dans la première partie du projet, le solveur d’équation Eikonal est développé en 2D et est résolue dans sa forme advective au centre de cellule. Les méthodes des différences finies et des volumes finis sont testées. L’équation est résolue à l’aide d’une discrétisation spatiale de premier ordre en amont. Les solveurs ont été vérifiés sur des cas canoniques, tels une plaque plane et un cylindre. Les deux méthodes de discrétisation réussissent à corriger les effets de maillages obliques et courbes. La méthode des différences finies possède un taux de convergence en maillage de deuxième ordre tandis que la méthode des volumes finis a un taux de convergence de premier ordre. L’addition d’une reconstruction linéaire de la solution à la face permet d’étendre la méthode des volumes finis à une méthode de deuxième ordre. De plus, les méthodes de différence finie et de volume fini de deuxième ordre permettent de bien représenter la distance à la paroi dans les zones de fort élargissement des cellules. L’équation Eikonal est ensuite vérifié sur plusieurs cas dont un profil NACA0012 en utilisant trois modèles de turbulence : Spalart-Allmaras, Menter SST et Mener-Langtry SST transitionnel.----------ABSTRACT Computational fluid dynamics (CFD) software is being used more often nowadays in aircraft design. The use of high performance computing clusters can increase computing power, but requires change in the structure of the software. In the aeronautical industry, CFD codes are often used to solve the Reynolds-Averaged Navier-Stokes (RANS) equations, and turbulence models are frequently used to approximate turbulent effects on flow. The Spalart-Allmaras turbulence model is widely accepted in the industry. In this model, wall distance plays a key role in the evaluation of aerodynamic forces. Therefore calculation of this geometric parameter needs to be accurate and efficient. With new developments in computing hardware, there is a need to adapt CFD codes. Search algorithms such as Euclidean and projected distance are often the methods used for computation of wall distance but tend to exhibit poor scalability. For this reason, a new wall distance solver is developed here using the Eikonal equation, a non-linear partial differential equation, chosen to make use of existing solvers and acceleration techniques in RANS solvers. In the first part of the project, the Eikonal equation solver was developed in 2D and solved in its advective form at the cell center. Both finite difference and finite volume methods were tested. The Eikonal equation was also solved using a first-order upwind spatial discretization. The solvers were verified through canonical cases like a flat plate and a cylinder. Both methods were able to correct the effects of skewed and curved meshes. The finite difference method converged at a second-order rate in space while the finite volume method converged at a first-order rate. The addition of a linear reconstruction of the solution at the face extended the finite volume method to a second-order method. Moreover, both finite difference and second-order finite volume methods were well represented by wall distance in zones of strong cell growth. The finite difference method was chosen, as it required less computing time. The Eikonal equation was then verified for several cases including a NACA0012 using three turbulence models: Spalart-Allmaras, Menter’s SST and Menter-Langtry transitional SST. For the first model, the Eikonal equation was able to correct grid skewness on the turbulent viscosity as well as on the aerodynamic coefficients, while for the other two yielded results similar to Euclidean and projected distance. To verify the implementation and convergence of the multi-grid scheme, the new wall distance solver was tested on an ice-accreted airfoil. In addition, the overset grid capabilities of the wall distance solver were verified on the McDonnell Douglas airfoil. Finally, the DLR-F6, a 3D case, was solved to show that the Eikonal equation can be extended to 3D meshes