MPI+X: task-based parallelization and dynamic load balance of finite element assembly

The main computing tasks of a finite element code(FE) for solving partial differential equations (PDE's) are the algebraic system assembly and the iterative solver. This work focuses on the first task, in the context of a hybrid MPI+X paradigm. Although we will describe algorithms in the FE context, a similar strategy can be straightforwardly applied to other discretization methods, like the finite volume method. The matrix assembly consists of a loop over the elements of the MPI partition to compute element matrices and right-hand sides and their assemblies in the local system to each MPI partition. In a MPI+X hybrid parallelism context, X has consisted traditionally of loop parallelism using OpenMP. Several strategies have been proposed in the literature to implement this loop parallelism, like coloring or substructuring techniques to circumvent the race condition that appears when assembling the element system into the local system. The main drawback of the first technique is the decrease of the IPC due to bad spatial locality. The second technique avoids this issue but requires extensive changes in the implementation, which can be cumbersome when several element loops should be treated. We propose an alternative, based on the task parallelism of the element loop using some extensions to the OpenMP programming model. The taskification of the assembly solves both aforementioned problems. In addition, dynamic load balance will be applied using the DLB library, especially efficient in the presence of hybrid meshes, where the relative costs of the different elements is impossible to estimate a priori. This paper presents the proposed methodology, its implementation and its validation through the solution of large computational mechanics problems up to 16k cores

Radiative Transfer with Finite Elements II. Ly-alpha Line Transfer in Moving Media

A finite element method for solving the resonance line transfer problem in moving media is presented. The algorithm works in three spatial dimensions on unstructured grids which are adaptively refined by means of an a posteriori error indicator. Frequency discretization is implemented via a first-order Euler scheme. We discuss the resulting matrix structure for coherent isotropic scattering and complete redistribution. The solution is performed using an iterative procedure, where monochromatic radiative transfer problems are successively solved. The present implementation is applicable for arbitrary model configurations with an optical depth up to 10^(3-4). Results of Ly-alpha line transfer calculations for a spherically symmetric model, a disk-like configuration, and a halo containing three source regions are discussed. We find the characteristic double-peaked Ly-alpha line profile for all models with an optical depth > 1. In general, the blue peak of the profile is enhanced for models with infall motion and the red peak for models with outflow motion. Both velocity fields produce a triangular shape in the two-dimensional Ly-alpha spectra, whereas rotation creates a shear pattern. Frequency-resolved Ly-alpha images may help to find the number and position of multiple Ly-alpha sources located in a single halo. A qualitative comparison with observations of extended Ly-alpha halos associated with high redshift galaxies shows that even models with lower hydrogen column densities than required from profile fitting yield results which reproduce many features in the observed line profiles and two-dimensional spectra.Comment: 13 pages, accepted for publication in A&

A REAL TIME NEURAL NETWORK BASED FINITE ELEMENT ANALYSIS OF SHELL STRUCTURE

In recent years, the finite element method has been widely used as a powerful tool in the analysis of engineering problems. In the simulation of deformable objects using the finite element method, a complex system of nodes which make a mesh grid is used. The FEM model includes material and structural properties, which altogether determine the model’s response to certain loading conditions. A reliable simulation is supposed to provide for an easier, faster and less expensive development of structures. The real-time simulation of shell-type deformable objects using the finite element method for a non-linear analysis is a challenging task because of the need for fast systems that do not demand high computational cost. In this paper, we present an efficient method based on neural networks for simulating the real-time behavior of a thin walled structure modeled by the finite element method in the commercial FE software. Using the finite element method, the structures displacements are computed offline, by applying forces in the specified range. In the online application mode, a trained neural network is used for obtaining required results for specified loads

An Object-oriented Environment for Developing Finite Element Codes for Multi-disciplinary Applications

The objective of this work is to describe the design and implementation of a framework for building multi-disciplinary finite element programs. The main goals are generality, reusability, extendibility, good performance and memory efficiency. Another objective is preparing the code structure for team development to ensure the easy collaboration of experts in different fields in the development of multi-disciplinary applications. Kratos, the framework described in this work, contains several tools for the easy implementation of finite element applications and also provides a common platform for the natural interaction of different applications. To achieve this, an innovative variable base interface is designed and implemented. This interface is used at different levels of abstraction and showed to be very clear and extendible. A very efficient and flexible data structure and an extensible IO are created to overcome difficulties in dealing with multi-disciplinary problems. Several other concepts in existing works are also collected and adapted to coupled problems. The use of an interpreter, of different data layouts and variable number of dofs per node are examples of such approach. In order to minimize the possible conflicts arising in the development, a kernel and application approach is used. The code is structured in layers to reflect the working space of developers with different fields of expertise. Details are given on the approach chosen to increase performance and efficiency. Examples of application of Kratos to different multidisciplinary problems are presented in order to demonstrate the applicability and efficiency of the new object oriented environment

A framework for developing finite element codes for multi- disciplinary applications

The world of computing simulation has experienced great progresses in recent years and requires more exigent multidisciplinary challenges to satisfy the new upcoming demands. Increasing the importance of solving multi-disciplinary problems makes developers put more attention to these problems and deal with difficulties involved in developing software in this area. Conventional finite element codes have several difficulties in dealing with multi-disciplinary problems. Many of these codes are designed and implemented for solving a certain type of problems, generally involving a single field. Extending these codes to deal with another field of analysis usually consists of several problems and large amounts of modifications and implementations. Some typical difficulties are: predefined set of degrees of freedom per node, data structure with fixed set of defined variables, global list of variables for all entities, domain based interfaces, IO restriction in reading new data and writing new results and algorithm definition inside the code. A common approach is to connect different solvers via a master program which implements the interaction algorithms and also transfers data from one solver to another. This approach has been used successfully in practice but results duplicated implementation and redundant overhead of data storing and transferring which may be significant depending to the solvers data structure. The objective of this work is to design and implement a framework for building multi-disciplinary finite element programs. Generality, reusability, extendibility, good performance and memory efficiency are considered to be the main points in design and implementation of this framework. Preparing the structure for team development is another objective because usually a team of experts in different fields are involved in the development of multi-disciplinary code. Kratos, the framework created in this work, provides several tools for easy implementation of finite element applications and also provides a common platform for natural interaction of its applications in different ways. This is done not only by a number of innovations but also by collecting and reusing several existing works. In this work an innovative variable base interface is designed and implemented which is used at different levels of abstraction and showed to be very clear and extendible. Another innovation is a very efficient and flexible data structure which can be used to store any type of data in a type-safe manner. An extendible IO is also created to overcome another bottleneck in dealing with multi-disciplinary problems. Collecting different concepts of existing works and adapting them to coupled problems is considered to be another innovation in this work. Examples are using an interpreter, different data organizations and variable number of dofs per node. The kernel and application approach is used to reduce the possible conflicts arising between developers of different fields and layers are designed to reflect the working space of different developers also considering their programming knowledge. Finally several technical details are applied in order to increase the performance and efficiency of Kratos which makes it practically usable. This work is completed by demonstrating the framework’s functionality in practice. First some classical single field applications like thermal, fluid and structural applications are implemented and used as benchmark to prove its performance. These applications are used to solve coupled problems in order to demonstrate the natural interaction facility provided by the framework. Finally some less classical coupled finite element algorithms are implemented to show its high flexibility and extendibility

Development of a computational platform for the simulation of low Prandtl number turbulent flows

Mathematical modeling of physical phenomena is at the basis of many scientific field researches. Complex systems show multiscale and multiphysics aspects that cannot be always taken into account in detail. In the past many numerical codes have been developed and specialized to solve different aspects of turbulence and, in general, fluid motion for a very wide range of engineering applications. Nowadays, numerical code coupling and computational platforms are gaining a lot of interest for the simulation of very complex phenomena. This PhD study focuses on modeling physical systems with coupled simulations, in particular turbulent heat transfer for liquid metals. This type of fluids, known as low Prandtl number fluids, requires more sophisticated turbulent heat transfer models since those used to simulate fluids such as air or water lead to a sensible heat transfer overestimation. Seeking an increased numerical stability, a four logarithmic parameter turbulence model is proposed, starting from a model that has already been validated with simulations of Lead-Bismuth-Eutectic (LBE) fully developed turbulent flows. The turbulence model has been implemented in the finite element code FEMuS to perform an extensive validation by comparing obtained results with Direct Numerical Simulations and experimental data. Many simulations are performed, for fully developed turbulent flows in plane channels, cylindrical pipes and 19 pin nuclear reactor bundles and for turbulent forced and mixed convection over a backward facing step. When conservation equations of mass, momentum and energy need be coupled with dynamic two-equation or thermal turbulence four-equation models the use of numerical coupling becomes important. In order to dispose of a greater choice of dynamical turbulence models, a computational platform containing OpenFOAM and FEMuS codes has been developed

Bridge-in-a-Backpack(TM). Task 2.3 : low-rise arch study with soil-structure interaction and spread footing foundation.

PDFTech ReportME 15-0320111223*2878Arch bridgesConcrete bridgesFiber reinforced polymersBridge constructionBridge designCostsFinite element methodGeometric configurations and shapesMaineMaine. Dept. of TransportationRofes, XeniaGoslin, KeenanUniversity of Maine. Advanced Structures and Composites CenterUS Transportation CollectionThis report includes fulfillment of Task 2.3 of a multi-task contract to further enhance concrete filled FRP tubes, orthe Bridge in a Backpack. Task 2 is an investigation of alternative shapes for the FRP tubes with varying radii. Task2.3 explores the effects of decreasing the rise (R) of an arch for a constant span (S) with a set of different earthcovers. It uses the finite element code (FE Code) by the University of Maine Advanced Structures and CompositesCenter (The Center) that takes into consideration soil-structure interaction (Clapp and Davids, 2011).A parametric study on a set of four bridge geometries given by rise-to-span (R/S) ratios of 0.30, 0.25, 0.2and 0.15 was selected. In addition, two sets of cover were investigated for each geometry, 4 ft., and 8 ft.respectively.For a 40 ft. span bridge, decreasing the rise by 50% from 12 ft. to 6 ft. could increase the overall bridge cost by asmuch as 16%

Bridge-in-a-Backpack(TM). Task 2.1 and 2.2 : investigate alternative shapes with varying radii.

PDFTech ReportME 15-0220111223*2878Arch bridgesConcrete bridgesFiber reinforced polymersBridge constructionBridge designComputer modelsLaboratory testsGeometric configurations and shapesLive loadsMaineMaine. Dept. of TransportationGoslin, KeenanRofes, XeniaUniversity of Maine. Advanced Structures and Composites CenterUS Transportation CollectionThis report includes fulfillment of Tasks 2.1 and 2.2 of a multi-task contract to further enhance concrete filled FRPtubes, or the Bridge in a Backpack. Task 2 is an investigation of alternative shapes for the FRP tubes with varyingradii. Task 2.1 develops the computer model for FRP tubes with high gradients of fabric braid angle. Task 2.2completes FRP tube shape optimization and laboratory testing of multi-radius arches.UMaine has developed and licensed a hybrid composite arch bridge system. The main structural bridge elementsutilize a tubular braided composite fabric that can be bent to a desired geometry. To date, only arches of constantradius have been fully analyzed, which sometimes pose a handicap due to clearance and/or hydraulic opening designrequirements. This project uses the finite element code (FE Code) written by the University of Maine -AdvancedStructures and Composites Center (The Center) developed in previous work (Clapp & Davids 2011, Clapp &Davids 2011) that takes into consideration soil-structure interaction amongst other things, and extends it toincorporate the effects unique to a multi-radius arch.This project includes a parametric study, optimization of shapes and laboratory testing of multi-radius arches

Zastosowanie sztucznych sieci neuronowych w modelowaniu numerycznym kompozytów przy pomocy metody elementów skończonych.

An application of Artificial Neural Networks for a definition of the effective constitutive law for a composite is described in the paper. First, a classical homogenisation procedure is directly interpreted with a use of this numerical tool. Next, a self-learning Finite Element code (FE with ANN inside) is used in the case when the effective constitutive law is deduced from a numerical experiment (substituting here a purely phenomenological approach). The new contribution to the classical self-learning procedure consists of its adaptation to a case of a non-monotonic loading (non-to-one load-deformation curve). This new ability of the method is principally due to the incremental form of the constitutive equation and the respective scheme of the neural network structure. Also an organisation of a constitutive data-base containing learning patterns is suitably modified. It is shown by examples that the training process is very quick. The error of this method is smaller, comparing to other schemes of data acquisition.W artykule opisano zastosowanie sztucznych sieci neuronowych do określenia efektywnego związku konstytutywnego dla kompozytów. To narzędzie numeryczne użyte zostało dwojako: do bezpośredniego zapisu wyników otrzymanych w ramach klasycznej metody homogenizacji oraz do wnioskowania o własnościach efektywnych na podstawie eksperymentu numerycznego (zastępującego eksperyment rzeczywisty) wykonanego na małej, lecz reprezentatywnej próbce kompozytu. W tym drugim przypadku zastosowano schemat "samouczącego się" programu metody elementów skończonych, w którym związek konstytutywny opisany jest siecią neuronową. Schemat ten zaadaptowano tak, że może być użyty w przypadku obciążeń niemonotonicznych oraz wtedy, gdy zależność: miara odkształcenia-miara naprężenia nie jest wzajemnie jednoznaczna. Te nowe możliwości uzyskane zostały dzięki przedstawieniu związku konstytutywnego w formie przyrostowej oraz opracowania odpowiedniej do tego budowy sieci neuronowej. Schemat "samouczącego się" programu MES charakteryzuje się tym, że proces formułowania nieznanego związku konstytutywnego jest szybki, a zgodność modelu numerycznego z eksperymentem większa niż dla innych metod