A strategy for mapping unstructured mesh computational mechanics programs onto distributed memory parallel architectures

The motivation of this thesis was to develop strategies that would enable unstructured mesh based computational mechanics codes to exploit the computational advantages offered by distributed memory parallel processors. Strategies that successfully map structured mesh codes onto parallel machines have been developed over the previous decade and used to build a toolkit for automation of the parallelisation process. Extension of the capabilities of this toolkit to include unstructured mesh codes requires new strategies to be developed. This thesis examines the method of parallelisation by geometric domain decomposition using the single program multi data programming paradigm with explicit message passing. This technique involves splitting (decomposing) the problem definition into P parts that may be distributed over P processors in a parallel machine. Each processor runs the same program and operates only on its part of the problem. Messages passed between the processors allow data exchange to maintain consistency with the original algorithm. The strategies developed to parallelise unstructured mesh codes should meet a number of requirements: The algorithms are faithfully reproduced in parallel. The code is largely unaltered in the parallel version. The parallel efficiency is maximised. The techniques should scale to highly parallel systems. The parallelisation process should become automated. Techniques and strategies that meet these requirements are developed and tested in this dissertation using a state of the art integrated computational fluid dynamics and solid mechanics code. The results presented demonstrate the importance of the problem partition in the definition of inter-processor communication and hence parallel performance. The classical measure of partition quality based on the number of cut edges in the mesh partition can be inadequate for real parallel machines. Consideration of the topology of the parallel machine in the mesh partition is demonstrated to be a more significant factor than the number of cut edges in the achieved parallel efficiency. It is shown to be advantageous to allow an increase in the volume of communication in order to achieve an efficient mapping dominated by localised communications. The limitation to parallel performance resulting from communication startup latency is clearly revealed together with strategies to minimise the effect. The generic application of the techniques to other unstructured mesh codes is discussed in the context of automation of the parallelisation process. Automation of parallelisation based on the developed strategies is presented as possible through the use of run time inspector loops to accurately determine the dependencies that define the necessary inter-processor communication

A class of multilevel parallel preconditioning strategies

In this paper, we introduce a class of recursive multilevel preconditioning strategies suited for solving large sparse linear systems of equations on modern day architectures. They are based on a reordering of the input matrix into a nested bordered block diagonal form, which allows a nested formulation of the preconditioners. The first one, which we refer to as nested SSOR (NSSOR), requires only the factorization of diagonal blocks at the innermost level of the recursive formulation. Hence, its construction is embarassingly parallel, and the memory requirements are very limited. Next two are nested versions of Modified ILU preconditioner with row sum (NMILUR) and colsum (NMILUC) property. We compare these methods in terms of iteration number, memory requirements, and overall solve time, with ILU(0) with natural ordering and nested dissection ordering, and MILU. We find that NSSOR compares favorably with ILU(0) with nested dissection ordering, while NMILUR and NMILUC outperform the other methods for certain matrices in our test set. It is proved that the NSSOR method is convergent when the input matrix is SPD. The preconditioners are designed to be suitable for parallel computing.Dans ce papier nous décrivons une classe de préconditionneurs multiniveaux parallèles pour résoudre des systèmes linéaires de grande taille. Ils se basent sur une renumérotation de la matrice d'entrée en forme block diagonale bornée et emboitée, qui permet une définition emboitée des préconditionneurs. Nous prouvons qu'un des préconditionneurs, NSSOR, converge quand la matrice d'entrée est symmétrique et définie positive. Les préconditionneurs sont adaptés au calcul parallèle

Efficient and robust monolithic finite element multilevel Krylov subspace solvers for the solution of stationary incompressible Navier-Stokes equations

Multigrid methods belong to the best-known methods for solving linear systems arising from the discretization of elliptic partial differential equations. The main attraction of multigrid methods is that they have an asymptotically meshindependent convergence behavior. Multigrid with Vanka (or local multilevel pressure Schur complement method) as smoother have been frequently used for the construction of very effcient coupled monolithic solvers for the solution of the stationary incompressible Navier-Stokes equations in 2D and 3D. However, due to its innate Gauß-Seidel/Jacobi character, Vanka has a strong influence of the underlying mesh, and therefore, coupled multigrid solvers with Vanka smoothing very frequently face convergence issues on meshes with high aspect ratios. Moreover, even on very nice regular grids, these solvers may fail when the anisotropies are introduced from the differential operator. In this thesis, we develop a new class of robust and efficient monolithic finite element multilevel Krylov subspace methods (MLKM) for the solution of the stationary incompressible Navier-Stokes equations as an alternative to the coupled multigrid-based solvers. Different from multigrid, the MLKM utilizes a Krylov method as the basis in the error reduction process. The solver is based on the multilevel projection-based method of Erlangga and Nabben, which accelerates the convergence of the Krylov subspace methods by shifting the small eigenvalues of the system matrix, responsible for the slow convergence of the Krylov iteration, to the largest eigenvalue. Before embarking on the Navier-Stokes equations, we first test our implementation of the MLKM solver by solving scalar model problems, namely the convection-diffusion problem and the anisotropic diffusion problem. We validate the method by solving several standard benchmark problems. Next, we present the numerical results for the solution of the incompressible Navier-Stokes equations in two dimensions. The results show that the MLKM solvers produce asymptotically mesh-size independent, as well as Reynolds number independent convergence rates, for a moderate range of Reynolds numbers. Moreover, numerical simulations also show that the coupled MLKM solvers can handle (both mesh and operator based) anisotropies better than the coupled multigrid solvers

Preconditioning for spare matrices with applications

This thesis deals with the construction of preconditioners for systems of linear equations as they occur in a number of practical problems. Primarily, we consider preconditioning techniques based on an incomplete decomposition of A, because of their simple definition and high efficiency. ... Zie: Conclusions and suggestions for future research

Parallel mesh adaptive techniques for complex flow simulation

Dynamic mesh adaptation on unstructured grids, by localised refinement and derefinement, is a very efficient tool for enhancing solution accuracy and optimise computational time. One of the major drawbacks however resides in the projection of the new nodes created, during the refinement process, onto the boundary surfaces. This can be addressed by the introduction of a library capable of handling geometric properties given by a CAD (Computer Aided Design) description. This is of particular interest also to enhance the adaptation module when the mesh is being smoothed, and hence moved, to then re-project it onto the surface of the exact geometry. However, the above procedure is not always possibly due to either faulty or too complex designs, which require a higher level of complexity in the CAD library. It is therefore paramount to have a built-in algorithm able to place the new nodes, belonging to the boundary, closer to the geometric definition of it. Such a procedure is proposed in this work, based on the idea of interpolating subdivision. In order to efficiently and effectively adapt a mesh to a solution field, the criteria used for the adaptation process needs to be as accurate as possible. Due to the nature of the solution, which is obtained by discretisation of a continuum model, numerical error is intrinsic in the calculation. A posteriori error estimation allows us to somewhat assess the accuracy by using the computed solution itself. In particular, an a posteriori error estimator based on the Zienkievicz Zhu model is introduced. This can be used in the adaptation procedure to refine the mesh in those areas where the local error exceeds a set tolerance, hence further increasing the accuracy of the solution in those regions during the next computational step. Variants of this error estimator have also been studied and implemented. One of the important aspects of this project is the fact that the algorithmic concepts are developed thinking parallel, i.e. the algorithms take into account the possibility of multiprocessor implementation. Indeed these concepts require complex programming if one tries to parallelise them, once they have been devised serially. Another important and innovative aspect of this work is the consistency of the algorithms with parallel processor execution

Aspects of Unstructured Grids and Finite-Volume Solvers for the Euler and Navier-Stokes Equations

One of the major achievements in engineering science has been the development of computer algorithms for solving nonlinear differential equations such as the Navier-Stokes equations. In the past, limited computer resources have motivated the development of efficient numerical schemes in computational fluid dynamics (CFD) utilizing structured meshes. The use of structured meshes greatly simplifies the implementation of CFD algorithms on conventional computers. Unstructured grids on the other hand offer an alternative to modeling complex geometries. Unstructured meshes have irregular connectivity and usually contain combinations of triangles, quadrilaterals, tetrahedra, and hexahedra. The generation and use of unstructured grids poses new challenges in CFD. The purpose of this note is to present recent developments in the unstructured grid generation and flow solution technology