Parallel implementation of the finite element method on shared memory multiprocessors

PhD ThesisThe work presented in this thesis concerns parallel methods for finite element analysis. The research has been funded by British Gas and some of the presented material involves work on their software. Practical problems involving the finite element method can use a large amount of processing power and the execution times can be very large. It is consequently important to investigate the possibilities for the parallel implementation of the method. The research has been carried out on an Encore Multimax, a shared memory multiprocessor with 14 identical CPU's. We firstly experimented on autoparallelising a large British Gas finite element program (GASP4) using Encore's parallelising Fortran compiler (epf). The par- allel program generated by epj proved not to be efficient. The main reasons are the complexity of the code and small grain parallelism. Since the program is hard to analyse for the compiler at high levels, only small grain parallelism has been inserted automatically into the code. This involves a great deal of low level syn- chronisations which produce large overheads and cause inefficiency. A detailed analysis of the autoparallelised code has been made with a view to determining the reasons for the inefficiency. Suggestions have also been made about writing programs such that they are suitable for efficient autoparallelisation. The finite element method consists of the assembly of a stiffness matrix and the solution of a set of simultaneous linear equations. A sparse representation of the stiffness matrix has been used to allow experimentation on large problems. Parallel assembly techniques for the sparse representation have been developed. Some of these methods have proved to be very efficient giving speed ups that are near ideal. For the solution phase, we have used the preconditioned conjugate gradient method (PCG). An incomplete LU factorization ofthe stiffness matrix with no fill- in (ILU(O)) has been found to be an effective preconditioner. The factors can be obtained at a low cost. We have parallelised all the steps of the PCG method. The main bottleneck is the triangular solves (preconditioning operations) at each step. Two parallel methods of triangular solution have been implemented. One is based on level scheduling (row-oriented parallelism) and the other is a new approach called independent columns (column-oriented parallelism). The algorithms have been tested for row and red-black orderings of the nodal unknowns in the finite element meshes considered. The best speed ups obtained are 7.29 (on 12 processors) for level scheduling and 7.11 (on 12 processors) for independent columns. Red-black ordering gives rise to better parallel performance than row ordering in general. An analysis of methods for the improvement of the parallel efficiency has been made.British Ga

High-performance computing for impact-induced fracture analysis exploiting octree mesh patterns

The impact-induced fracture analysis has a wide range of engineering and defence applications, including aerospace, manufacturing and construction. An accurate simulation of impact events often requires modelling large-scale complex geometries along with dynamic stress waves and damage propagation. To perform such simulations in a timely manner, a highly efficient and scalable computational framework is necessary. This thesis aims to develop a high-performance computational framework for analysing large-scale structural problems pertaining to impact-induced fracture events. A hierarchical grid-based mesh containing octree cells is utilised for discretising the problem domain. The scaled boundary finite element method (SBFEM) is employed, which can efficiently handle the octree cells by eliminating the hanging node issues. The octree-mesh is used in balanced form with a limited number of octree cell patterns. The master element matrices of each pattern are pre-computed while the storage of the individual element matrices is avoided leading to a significant reduction in memory requirements, especially for large-scale models. Further, the advantages of octree cells are leveraged by automatic mesh generation and local refinement process, which enables efficient pre-processing of models with complex geometries. To handle the matrix operations associated with large-scale simulation, a pattern-by-pattern (PBP) approach is proposed. In this technique, the octree-patterns are exploited to recast a majority of the computational work into pattern-level dense matrix operations. This avoids global matrix assembly, allows better cache utilisation, and aids the associated memory-bandwidth limited computations, resulting in significant performance gains in matrix operations. The PBP approach also supports large-scale parallelism. In this work, the parallel computation is carried out using the mesh-partitioning strategy and implemented using the message passing technique. It is shown that the developed solvers can simulate large-scale and complex structural problems, e.g. delamination/fracture in sandwich panels with approximately a billion unknowns (or DOFs). A massive scaling can be achieved with more than ten thousand cores in a distributed computing environment, which reduces the computation time from months (on a single core) to a few minutes

Application of domain decomposition methods to problems in topology optimisation

Determination of the optimal layout of structures can be seen in everyday life, from nature to industry, with research dating back to the eighteenth century. The focus of this thesis involves investigation into the relatively modern field of topology optimisation, where the aim is to determine both the optimal shape and topology of structures. However, the inherent large-scale nature means that even problems defined using a relatively coarse finite element discretisation can be computationally demanding. This thesis aims to describe alternative approaches allowing for the practical use of topology optimisation on a large scale. Commonly used solution methods will be compared and scrutinised, with observations used in the application of a novel substructuring domain decomposition method for the subsequent large-scale linear systems. Numerical and analytical investigations involving the governing equations of linear elasticity will lead to the development of three different algorithms for compliance minimisation problems in topology optimisation. Each algorithm will involve an appropriate preconditioning strategy incorporating a matrix representation of a discrete interpolation norm, with numerical results indicating mesh independent performance

An adapted deflated conjugate gradient solver for robust extended/generalised finite element solutions of large scale, 3D crack propagation problems

An adapted deflation preconditioner is employed to accelerate the solution of linear systems resulting from the discretisation of fracture mechanics problems with well-conditioned extended/generalised finite elements. The deflation space typically used for linear elasticity problems is enriched with additional vectors, accounting for the enrichment functions used, thus effectively removing low frequency components of the error. To further improve performance, deflation is combined, in a multiplicative way, with a block-Jacobi preconditioner, which removes high frequency components of the error as well as near-linear dependencies introduced by enrichment. The resulting scheme is tested on a series of non-planar crack propagation problems and compared to alternative linear solvers in terms of performance

An upwind cell centred finite volume method for large strain explicit solid dynamics in OpenFOAM

Cotutela Universitat Politècnica de Catalunya i Swansea UniversityIn practical engineering applications involving extremely complex geometries, meshing typically constitutes a large portion of the overall design and analysis time. In the computational mechanics community, the ability to perform calculations on tetrahedral meshes has become increasingly important. For these reasons, automated tetrahedral mesh generation by means of Delaunay and advancing front techniques have recently received increasing attention in a number of applications, namely: crash impact simulations, cardiovascular modelling, blast and fracture modelling. Unfortunately, modern industry codes in solid mechanics typically rely on the use of traditional displacement based Finite Element formulations which possess several distinct disadvantages, namely: (1) reduced order of convergence for strains and stresses in comparison with displacements; (2) high frequency noise in the vicinity of shocks; and (3) numerical instabilities associated with shear locking, volumetric locking and pressure checker-boarding. In order to address the above mentioned shortcomings, a new mixed-based set of equations for solid dynamics formulated in a system of first order hyperbolic conservation laws was introduced. Crucially, the new set of conservation laws has a similar structure to that of the well known Euler equations in the context of Computational Fluid Dynamics (CFD). This enables us to borrow some of the available CFD technologies and to adapt the method in the context of solid dynamics. This thesis builds on the work carried out by Lee et al. 2013 by further developing the upwind cell centred finite volume framework for the numerical analysis of large strain explicit solid dynamics and its tailor-made implementation within the open source code OpenFOAM, extensively used in industrial and academic environments. The object oriented nature of OpenFOAM implementation provides a very efficient platform for future development. In this computational framework, the primary unknown variables are linear momentum and deformation gradient tensor of the system. Moreover, the formulation is further extended for an additional set of geometric strain measures comprising of the co-factor of deformation gradient tensor and the Jacobian of deformation, in order to simulate polyconvex constitutive models ensuring material stability. The domain is spatially discretised using a standard Godunov-type cell centred framework where second order accuracy is achieved by employing a linear reconstruction procedure in conjunction with a slope limiter. This leads to discontinuities in variables at the cell interface which motivate the use of a Riemann solver by introducing an upwind bias into the evaluation of numerical contact fluxes. The acoustic Riemann solver presented is further developed by applying preconditioned dissipation to improve its performance in the near incompressibility regime and extending its range to contact applications. Moreover, two evolutionary frameworks are proposed in this study to satisfy the underlying involutions (or compatibility conditions) of the system. Additionally, the spatial discretisation is alternatively represented through a nodal cell centred finite volume framework for comparison purposes. From a temporal discretisation point of view, a two stage Total Variation Diminishing Runge-Kutta time integrator is employed to ensure second order accuracy. Additionally, inclusion of a global posteriori angular momentum projection procedure enables preservation of angular momenta of the system. Finally, benchmark numerical examples are simulated to demonstrate various aspects of the formulation including mesh convergence, momentum preservation and the locking-free nature of the formulation on complex computational domains.En aplicaciones prácticas de ingeniería que implican geometrías extremadamente complejas, el mallado requiere típicamente una gran parte del tiempo total de diseño y análisis. En la comunidad de mecánica computacional, la capacidad de realizar cálculos sobre mallas tetraédricas está siendo cada vez más importante. Por estas razones, la generación automatizada de mallas tetraédricas por medio de técnicas de Delaunay y frente avanzado han recibido cada vez más atención en ciertas aplicaciones, a saber: simulaciones de impacto, modelado cardiovascular, modelado de explosión y fractura. Por desgracia, los códigos en la industria moderna para mecánica de sólidos se basan normalmente en el uso de formulaciones tradicionales de Elementos Finitos formulados en desplazamientos que poseen varias desventajas: (1) menor orden de convergencia para tensiones y deformaciones; (2) ruido de alta frecuencia cerca de las ondas de choque; y (3) inestabilidades numéricas asociadas con el bloqueo a cortante, el bloqueo volumétrico y oscilaciones de presión. Con el fin de abordar estas deficiencias, se introduce un nuevo conjunto de ecuaciones para mecánica del sólido formulada como un sistema de leyes de conservación de primer orden basada en una formulación mixta. Fundamentalmente, el nuevo sistema de leyes de conservación tiene una estructura similar a la de las famosas ecuaciones de Euler en el contexto de la Dinámica de Fluidos Computacional (CFD). Esto nos permite aprovechar algunas de las tecnologías CFD disponibles y adaptar el método en el contexto de la Mecánica de Sólidos. Esta tesis se basa en el trabajo realizado en Lee et al. 2013 mediante el desarrollo de la estructura de volúmenes finitos centrados en celdas upwind para el análisis numérico de dinámica del sólido explícita en grandes deformaciones y su implementación específicamente diseñada dentro del software de código abierto OpenFOAM, ampliamente utilizado ámbito académico e industrial. Además, la naturaleza orientada a objetos de su implementación proporciona una plataforma muy eficiente para su desarrollo posterior. En este marco computacional, las incógnitas básicas de este sistema son el momento lineal y el tensor gradiente de deformación. Asimismo, la formulación se extiende adicionalmente para un conjunto adicional de medidas de deformación que comprenden el cofactor del tensor gradiente de deformación y el jacobiano de deformación, con el fin de simular modelos constitutivos policonvexos que aseguran la estabilidad del material. El dominio se discretiza espacialmente usando un marco centrado en células de tipo Godunov estándar, donde se consigue la precisión de segundo orden empleando un procedimiento de reconstrucción lineal junto con un limitador de pendiente. Esto conduce a discontinuidades en las variables en la interfase de la célula que motivan el uso de un solucionador de Riemann mediante la introducción de un sesgo contra el viento en la evaluación de flujos de contacto numéricos. El presente solucionador acústico de Riemann es posteriormente desarrollado aplicando disipación pre-condicionada para mejorar su rendimiento en el cercano pero incompresibilidad régimen y extender su gama a aplicaciones de contacto. Además, se proponen dos marcos evolutivos en este estudio para satisfacer las involuciones subyacentes (o condiciones de compatibilidad) del sistema. Además, la discretización espacial se representa alternativamente a través de un marco de volumen finito centrado en células nodales para fines de comparación. Desde el punto de vista de la discretización temporal, se emplea un integrador temporal de Runge-Kutta de dos etapas con Disminución de Variación Total para asegurar segundo orden de precision. Finalmente, se simulan ejemplos numéricos de referencia para demostrar varios aspectos de la formulación que incluyen convergencia de malla, conservación de momento y la naturaleza libre de bloqueo de la formulación en dominios computacionales complejos.Postprint (published version

An upwind cell centred finite volume method for large strain explicit solid dynamics in OpenFOAM

In practical engineering applications involving extremely complex geometries, meshing typically constitutes a large portion of the overall design and analysis time. In the computational mechanics community, the ability to perform calculations on tetrahedral meshes has become increasingly important. For these reasons, automated tetrahedral mesh generation by means of Delaunay and advancing front techniques have recently received increasing attention in a number of applications, namely: crash impact simulations, cardiovascular modelling, blast and fracture modelling. Unfortunately, modern industry codes in solid mechanics typically rely on the use of traditional displacement based Finite Element formulations which possess several distinct disadvantages, namely: (1) reduced order of convergence for strains and stresses in comparison with displacements; (2) high frequency noise in the vicinity of shocks; and (3) numerical instabilities associated with shear locking, volumetric locking and pressure checker-boarding. In order to address the above mentioned shortcomings, a new mixed-based set of equations for solid dynamics formulated in a system of first order hyperbolic conservation laws was introduced. Crucially, the new set of conservation laws has a similar structure to that of the well known Euler equations in the context of Computational Fluid Dynamics (CFD). This enables us to borrow some of the available CFD technologies and to adapt the method in the context of solid dynamics. This thesis builds on the work carried out by Lee et al. 2013 by further developing the upwind cell centred finite volume framework for the numerical analysis of large strain explicit solid dynamics and its tailor-made implementation within the open source code OpenFOAM, extensively used in industrial and academic environments. The object oriented nature of OpenFOAM implementation provides a very efficient platform for future development. In this computational framework, the primary unknown variables are linear momentum and deformation gradient tensor of the system. Moreover, the formulation is further extended for an additional set of geometric strain measures comprising of the co-factor of deformation gradient tensor and the Jacobian of deformation, in order to simulate polyconvex constitutive models ensuring material stability. The domain is spatially discretised using a standard Godunov-type cell centred framework where second order accuracy is achieved by employing a linear reconstruction procedure in conjunction with a slope limiter. This leads to discontinuities in variables at the cell interface which motivate the use of a Riemann solver by introducing an upwind bias into the evaluation of numerical contact fluxes. The acoustic Riemann solver presented is further developed by applying preconditioned dissipation to improve its performance in the near incompressibility regime and extending its range to contact applications. Moreover, two evolutionary frameworks are proposed in this study to satisfy the underlying involutions (or compatibility conditions) of the system. Additionally, the spatial discretisation is alternatively represented through a nodal cell centred finite volume framework for comparison purposes. From a temporal discretisation point of view, a two stage Total Variation Diminishing Runge-Kutta time integrator is employed to ensure second order accuracy. Additionally, inclusion of a global posteriori angular momentum projection procedure enables preservation of angular momenta of the system. Finally, benchmark numerical examples are simulated to demonstrate various aspects of the formulation including mesh convergence, momentum preservation and the locking-free nature of the formulation on complex computational domains.En aplicaciones prácticas de ingeniería que implican geometrías extremadamente complejas, el mallado requiere típicamente una gran parte del tiempo total de diseño y análisis. En la comunidad de mecánica computacional, la capacidad de realizar cálculos sobre mallas tetraédricas está siendo cada vez más importante. Por estas razones, la generación automatizada de mallas tetraédricas por medio de técnicas de Delaunay y frente avanzado han recibido cada vez más atención en ciertas aplicaciones, a saber: simulaciones de impacto, modelado cardiovascular, modelado de explosión y fractura. Por desgracia, los códigos en la industria moderna para mecánica de sólidos se basan normalmente en el uso de formulaciones tradicionales de Elementos Finitos formulados en desplazamientos que poseen varias desventajas: (1) menor orden de convergencia para tensiones y deformaciones; (2) ruido de alta frecuencia cerca de las ondas de choque; y (3) inestabilidades numéricas asociadas con el bloqueo a cortante, el bloqueo volumétrico y oscilaciones de presión. Con el fin de abordar estas deficiencias, se introduce un nuevo conjunto de ecuaciones para mecánica del sólido formulada como un sistema de leyes de conservación de primer orden basada en una formulación mixta. Fundamentalmente, el nuevo sistema de leyes de conservación tiene una estructura similar a la de las famosas ecuaciones de Euler en el contexto de la Dinámica de Fluidos Computacional (CFD). Esto nos permite aprovechar algunas de las tecnologías CFD disponibles y adaptar el método en el contexto de la Mecánica de Sólidos. Esta tesis se basa en el trabajo realizado en Lee et al. 2013 mediante el desarrollo de la estructura de volúmenes finitos centrados en celdas upwind para el análisis numérico de dinámica del sólido explícita en grandes deformaciones y su implementación específicamente diseñada dentro del software de código abierto OpenFOAM, ampliamente utilizado ámbito académico e industrial. Además, la naturaleza orientada a objetos de su implementación proporciona una plataforma muy eficiente para su desarrollo posterior. En este marco computacional, las incógnitas básicas de este sistema son el momento lineal y el tensor gradiente de deformación. Asimismo, la formulación se extiende adicionalmente para un conjunto adicional de medidas de deformación que comprenden el cofactor del tensor gradiente de deformación y el jacobiano de deformación, con el fin de simular modelos constitutivos policonvexos que aseguran la estabilidad del material. El dominio se discretiza espacialmente usando un marco centrado en células de tipo Godunov estándar, donde se consigue la precisión de segundo orden empleando un procedimiento de reconstrucción lineal junto con un limitador de pendiente. Esto conduce a discontinuidades en las variables en la interfase de la célula que motivan el uso de un solucionador de Riemann mediante la introducción de un sesgo contra el viento en la evaluación de flujos de contacto numéricos. El presente solucionador acústico de Riemann es posteriormente desarrollado aplicando disipación pre-condicionada para mejorar su rendimiento en el cercano pero incompresibilidad régimen y extender su gama a aplicaciones de contacto. Además, se proponen dos marcos evolutivos en este estudio para satisfacer las involuciones subyacentes (o condiciones de compatibilidad) del sistema. Además, la discretización espacial se representa alternativamente a través de un marco de volumen finito centrado en células nodales para fines de comparación. Desde el punto de vista de la discretización temporal, se emplea un integrador temporal de Runge-Kutta de dos etapas con Disminución de Variación Total para asegurar segundo orden de precision. Finalmente, se simulan ejemplos numéricos de referencia para demostrar varios aspectos de la formulación que incluyen convergencia de malla, conservación de momento y la naturaleza libre de bloqueo de la formulación en dominios computacionales complejos

Numerical Solution of Systems with Stochastic Uncertainties: A General Purpose Framework for Stochastic Finite Elements

This work develops numerical techniques for the simulation of systems with stochastic parameters, modelled by stochastic partial differential equations (SPDEs). After treating the theory of linear and nonlinear elliptic SPDEs, discretisation techniques are presented. The spatial discretisation is performed by existing simulation software and the stochastic discretisation is carried out by directly integrating statistics or by expansions in tensor products of finite element shape functions times stochastic functions. Monte Carlo and Smolyak integration techniques are employed for the direct integration of statistics, whereas the discretisation by series expansions is realised either by orthogonal projections or by Galerkin methods, which yield large systems of coupled block equations. For the solution of linear SPDEs, efficient representations of the linear block equations are developed and used in iterative solvers. Due to the size of the equations, a parallel solver is supplied. The solution of nonlinear SPDEs is performed by approximate and by quasi-Newton methods. An adaptive refinement of the stochastic ansatz-spaces is implemented based on the solution of dual problems. The numerical techniques described in this thesis are implemented in a general purpose software for stochastic finite elements that allows to introduce stochastic uncertainties into existing simulation codes and that permits to propagate the input uncertainties to the system response.Inhalt der Arbeit ist die numerische Simulation von Systemen mit stochastischen Parametern, die durch stochastische partielle Differentialgleichungen (SPDGLn) beschrieben werden. Es werden die Theorie linearer und nichtlinearer elliptischer SPDGLn sowie Diskretisierungsverfahren beschrieben. Für die räumliche Diskretisierung wird eine existierende Simulationssoftware verwendet, während die stochastische Diskretisierung durch die direkte numerische Integration von Statistiken unter Verwendung von Monte Carlo- und Smolyak-Quadraturverfahren oder durch Reihenentwicklungen in Tensorprodukten finiter Elemente und stochastischer Ansatzfunktionen erfolgt. Die Reihenentwicklung wird dabei durch orthogonale Projektionen oder durch Galerkinverfahren gewonnen. Bei der Anwendung stochastischer Galerkinvervahren entstehen große Systeme gekoppelter Blockgleichungssysteme, welche hier durch iterative Verfahren gelöst werden. Zur Lösung linearer SPDGln werden effiziente Darstellungen der Gleichungssysteme und iterative Löser entwickelt. Aufgrund der Größe der entstehenden Gleichungssysteme wird ein paralleler Löser bereitgestellt. Die Lösung nichtlinearer SPDGLn geschieht durch approximative und Quasi-Newtonverfahren. Ein duales Verfahren ermöglicht die adaptive Verfeinerung der Lösung. Diese Verfahren werden in einer Allzwecksoftware für stochastische finite Elemente implementiert, die es erlaubt, existierende Simulationscodes um stochastische Unsicherheiten zu erweitern

Robust and stable discrete adjoint solver development for shape optimisation of incompressible flows with industrial applications

PhD, 156ppThis thesis investigates stabilisation of the SIMPLE-family discretisations for incompressible flow and their discrete adjoint counterparts. The SIMPLE method is presented from typical \prediction-correction" point of view, but also using a pressure Schur complement approach, which leads to a wider class of schemes. A novel semicoupled implicit solver with velocity coupling is proposed to improve stability. Skewness correction methods are applied to enhance solver accuracy on non-orthogonal grids. An algebraic multi grid linear solver from the HYPRE library is linked to flow and discrete adjoint solvers to further stabilise the computation and improve the convergence rate. With the improved implementation, both of flow and discrete adjoint solvers can be applied to a wide range of 2D and 3D test cases. Results show that the semi-coupled implicit solver is more robust compared to the standard SIMPLE solver. A shape optimisation of a S-bend air flow duct from a VW Golf vehicle is studied using a CAD-based parametrisation for two Reynolds numbers. The optimised shapes and their flows are analysed to con rm the physical nature of the improvement. A first application of the new stabilised discrete adjoint method to a reverse osmosis (RO) membrane channel flow is presented. A CFD model of the RO membrane process with a membrane boundary condition is added. Two objective functions, pressure drop and permeate flux, are evaluated for various spacer geometries such as open channel, cavity, submerged and zigzag spacer arrangements. The flow and the surface sensitivity of these two objective functions is computed and analysed for these geometries. An optimisation with a node-base parametrisation approach is carried out for the zigzag con guration channel flow in order to reduce the pressure drop. Results indicate that the pressure loss can be reduced by 24% with a slight reduction in permeate flux by 0.43%.Queen Mary-China Scholarship Council Co-funded Scholarship No. 201206280018