Clinckemaillie, J

Elsner, B

Lonsdale, G

Meliciani, S

Vlachoutsis, S

CERN Document Server

1Parallel and distributed crashworthiness simulationG. Lonsdale C&C Research Laboratories, NEC Europe Ltd.,St. Augustin, GermanyJ. Clinckemaillie ESI SA, Rungis, FranceS. Meliciani ESI SA, Rungis, FranceS. Vlachoutsis ESI SA, Rungis, FranceB. Elsner ESI GmbH, Eschborn, GermanyAbstractThis paper focusses on recent developments and on-going research related to paralleland distributed approaches for crashworthiness simulation, using the PAM-CRASHcode from ESI as an example. The developments described here relate directly to thetake-up in the automotive industry of the latest High Performance Computing (HPC)technology, since crash and safety analysis is well established in the automotivedesign process.Keywords: Parallel, crash simulation, PAM-CRASH, message-passing, HPF1 IntroductionOver the last decade, crashworthiness simulation, and more recently simulation of occupantsafety and the associated systems, has established itself within automotive design. Advances inboth the modelling of physical phenomena and performance (arising from computer architectureadvances and more efficient numerical algorithms) have led to an increase in the detail andcomplexity of models which can be handled, to the point where the position of numericalsimulation has “moved-forward” in the design cycle. The numerical simulations are being used toinfluence the early designs of new automobiles, for example, rather than being used to makeimprovements to the design at a much later stage, when possibilities for modification are morerestricted. With numerical simulation offering a more detailed analysis of the whole structurewithin a crash event, the experimental testing is often used only as a validation of the simulation.An overview of typical simulations, modelling aspects and computational requirements is given in[5]. An example of a simulation at the current high-end of computational requirements is shownin Figure 1.Despite the advances made, both car manufacturers and their suppliers have an increasingdemand for computational performance. One reason for this is the expansion in the detail ofinvestigation being carried out: as an example, the occupant safety simulation is constantlyincreasing the detail of the dummy models being employed, with the expectation that certainbiomechanic (human-like) parts will be included in the near future. Another reason comes fromthe introduction of more restrictive crashworthiness regulations, which results in the need for ahigher number of different simulations to be performed. These demands have led to thedevelopment of message-passing versions of crash codes for parallel and distributed HPCarchitectures.2Figure 1: Car-to-car crash simulation Courtesy of BMWAlthough shared-memory parallel implementations of the PAM-CRASH code have been inexistence since 1987, the exploitation of distributed-memory parallel architectures has onlyrecently been possible. In the ESPRIT project CAMAS and its extension to link to theEUROPORT action (see [12,13]), a prototype, portable message-passing version of the PAM-CRASH code was developed and benchmarked using industrially relevant models. Theperformance of this code version - even on a shared-memory architecture - has led to theacceptance of the, meanwhile matured, version as a viable 'production-use' design tool byautomotive manufacturers who already have access to parallel HPC systems. Beyond this,current investigations are focussed on the efficient use of the message-passing version onnetworked workstations. Not only do networked workstations provide the “economic entry-point”to parallel computing for the smaller automotive suppliers, but they also allow the auto-manufacturers the possibility to both reduce the load on the dedicated parallel platform and toexploit a large computational resource which would otherwise remain idle outside office-hours -for example, the large number of CAD-stations otherwise used only for interactive, graphicalwork.The central discussion of the paper is contained in Section 2, which gives an overview of thealgorithms used within the PAM-CRASH code and the significant features of its message-passing, mesh-partitioned parallelization; in particular, describing the areas of difficulty andrequirements for future developments. The degree of scalability achieved with the message-passing approach will be illustrated and a comparison with a shared-memory implementationmade.Although the message-passing approach does provide a high-performance solution, thecomplexity of the code implementation poses maintenance difficulties. A high-level approachwould therefore be favoured, and the possibilities for using High Performance Fortran (HPF) arebeing investigated. Since the available constructs of HPF-1 (as defined in [7]) were adjudgedinadequate for the requirements of irregular, unstructured industrial applications (see [1,2]), theESPRIT project HPF+ ([9]) is investigating the extension of the language to enable efficientimplementations of codes like PAM-CRASH. A brief overview of the project and the status of thePAM-CRASH related developments will be given in Section 3.32 Message-passing PAM-CRASHPAM-CRASH is an explicit time-marching Finite Element program used for the numericalsimulation of the highly nonlinear, dynamic phenomena arising in short-duration contact-impactproblems. It uses a central difference explicit time-marching scheme with unstructured meshescomprised of mechanical elements, of various types (e.g. thin-shells, beams, bars and specialelements to model features such as rivets) and associated with a range of material properties,which model the behaviour of the structure under consideration. The Lagrangian formulationused has the following basic computational components: the time-integration at nodal points,“force calculations” on the elements defined by those nodal points. The time-integrationcalculates the accelerations, velocities and finally new-coordinates of the mesh based on theexisting forces at the nodal points, created/generated by the movement of the points at theprevious time-step. The major computational costs of the algorithm comes in the calculation ofthe forces at the nodal points. These force calculations can be broken down into (essentially1)stress-strain calculations and contact-impact calculations - the two having very different levels ofdata locality, when considering parallel implementations.The stress-strain calculations are performed over the elements. The calculation on each elementrequires as input the latest co-ordinates and velocities from only those nodal points defining theelement. Once calculated, the force on the element is distributed as individual forces at the nodalpoints. Since these calculations produce the largest contribution to the overall computational cost(between 60% and 80%, depending on the particular model) and exhibit a high degree of datalocality, the message-passing parallelization employed an element-wise mesh-partitioning.In contrast to the stress-strain calculations, the contact-impact algorithms used within the codehave, in terms of data access, a pseudo-global nature. These contact algorithms serve to detectand correct penetration of structural components. This is achieved by first performing aproximity and penetration search, followed by a penetration correction procedure. Within thePAM-CRASH code, the latter is currently a penalty method, whereby contact forces areintroduced at the impacting node and at the nodal points of the impacted segment. Animplementation (or practical usage) issue which affects parallelization is that the contactcalculations are performed only within user-defined (and not necessarily disjunct) areas.In the following, we will give a brief overview of the message-passing parallelization. Furtherinformation can be found in the references:In the papers [10,11], the parallelization of the PAM-CRASH code using static domainpartitioning was detailed and, most importantly, the problems arising due to the inclusion of thecontact-impact algorithms was discussed: the pseudo-global nature of the communicationspatterns, the static and dynamic load imbalance, the necessity to remove even small parts of non-scalable code. An analysis of load-balancing issues in the message-passing version and theperformance bottle-necks occurring in the shared-memory or symmetric multiprocessing (SMP)version is including in the more recent publication [3].As indicated above, a static mesh-partitioniong approach, based on a partitioning of elements,was employed. This static partition was calculated prior to the simulation. For the benchmarkingresults for EUROPORT, the multilevel recursive spectral bisection algorithm of the DDT tool([4]) of the University of Southampton (produced in the ESPRIT project CAMAS) was used. Thepartitions employed a static partition balancing based on contact surface definitions (see [3]).With the unique assignment of elements to processes, the non-contact components of the solution                                               1 A simplification is taken here, though the bulk of the computation is covered. A description of a range ofspecial features, such as the handling of nodal constraint sets, rigid-body interactions or airbag calculations, isbeyond the scope of this paper.4scheme (time-integration + stress-strain force calculations) can be efficiently parallelized by theprovision of sub-domain interface communication. In order to enable contact search procedures tobe performed locally, more complex communication constructs need to be introduced - based onthe dynamically varying results of the search routines. Experience with the EUROPORTbenchmark models has shown that load imbalance created by the contact-impact calculations isthe major factor governing the scalability of the algorithms. Future research will thus beaddressing the use of dynamic mesh re-partitioning, based on the changing computational load asthe simulation progresses.We conclude this section with an illustration of the performance of the message-passing PAM-CRASH protoype on a distributed-memory machine (the IBM SP2) and an SMP machine (theSilicon Graphics Power Challenge). Details of the EUROPORT benchmark models and furtherresults can be found in [3,11]. The models themselves are illustrated, by their final deformedstates, in Figure 2.  Figure 2: CAMAS-Link EUROPORT Benchmark models: AUDI-Crash30, BMW-Crash15 and BMW-Crash50, respectively. Courtesy of Audi, BMWThe BMW-Crash50 model causes the greatest loss in scalability of the three, since it includes thehigh percentage of elements within contact calculations (45,497 contact segments from the61,039 elements). Nevertheless, the results on the IBM SP2 shown in Figure 3 demonstrate animpressive speed-up, with sequential supercomputer performance achieved at relatively lowprocessor numbers.0500001000001500002000002500003000003500000 8 16 24 32Number ofprocessesElapsed time (s)IBM SP2CRAY YMPFigure 3: BMW-Crash50 PerformanceBoth the code portability and the advantage of the message-passing parallelization approach wereclearly demonstrated by the benchmarking performed on the SGI Power Challenge. Tests with avariety of models, using numbers of contact segments ranging from 10% to over 70% of the totalnumber of elements in the model, have shown that the message-passing version of PAM-CRASH(in this case using PVM) not only outperforms the shared-memory version at low processor0481216200 8 16 24 32Number of processesSpeed-up5numbers but also scales significantly better. This is exemplified by the speed-ups obtained on an18-processor machine shown in Figure 4. The cases tested are two of the above modelssupplemented by a third BMW model, which is very similar to the BMW-Crash50 case but ofslighly smaller size and involving a frontal rather than an offset crash. This additional model isdenoted BMW-xtra in the figure.14916AUDI-30 SMPBMW-15 SMPBMW-xtra SMPAUDI-30 PVMBMW-15 PVMBMW-xtra PVM02468101214Speed-UpNo. ProcessorsFigure 4: Speed-up for Shared versus Distributed Code Versions on SGI3 HPF+ Project and PAM-CRASH investigationThe central aim of the ESPRIT Long Term Research project HPF+ is to support and acceleratethe development of the HPF language in order that a much wider range of applications may beefficiently parallelised using the high-level language approach. To do this, the project hasfocussed on complex, scientific applications which involve irregular constructs: unstructuredmeshes, irregular data structures, computational tasks with dynamically changing costs and dataaccesses. A sequence of benchmark kernels representing various features and levels ofcompleteness from the codes FIRE (from AVL), IFS (from ECMWF) and PAM-CRASH (fromESI) will be developed and used to define language extensions necessary for these types ofirregular codes. A close cooperation with the HPF-2 effort ([8]) will guarantee that HPF+ willnot divert from the standard except where this is necessary for providing functionality not beingconsidered under HPF-2. The HPF+ Consortium consists of the application developers listedabove2 together with both academic and commercial language, compiler and tool developers fromthe Universities of Vienna and Pavia and NA Software.At the time of writing, the project has completed the definition of first benchmark kernels and aninitial definition of the HPF+ language extensions. For the PAM-CRASH -related investigation,concentration has been on the “stress-strain” calculations as described in Section 2, the definitionof kernels representing the more complex contact-impact tasks will be handled in the forthcomingproject periods. Due to space restrictions, detailed examples of code fragments will not beincluded here. However, the following discussion gives an indication of the areas of difficulty and                                               2 C&C Research Laboratories, NEC Europe are contributing to the project via a collaboration with ESI6(draft) language extensions to solve them. Since only language extensions are discussed, aknowledge of HPF-1 is assumed.The dual requirements of the unstructured finite-element mesh - basic variables stored andcalculated at nodal points interacting with element-based force computations as described inSection 2 - means that the regular data distributions of HPF-1 are unlikely ever to result inefficient parallel code. Building on experience with the mesh-partitioning approach, anappropriate solution would be the clustering together of elements together with correspondingnodes, since this should minimise necessary communication for the initial kernel. A naturalimplementation of such would exploit the DYNAMIC and REDISTRIBUTE features, since thepartitioning will be computed or read at run-time, coupled with either a general block distribution(GEN_BLOCK) or a completely irregular (array )element-to-processor mapping, INDIRECT (see,for example, [1,2]). Turning then to the force calculations, the simplest ‘generic’, sequentialkernel features a loop over elements within which necessary nodal data is gathered, a subroutineis called to calculate the force on the element, corresponding forces are scattered back to theconstituent nodal points. Indirect addressing is used for the gather-scatter operations. The HPF+parallelization of this kernel (currently) focusses on the use of the INDEPENDENT directive, linkedto the do loop, supplemented by constructs to: specify the distribution of the computation over theprocessors, ON HOME; indicate that a reduction operation takes place, REDUCTION; specify thatthe subroutine execution neither influences other do loop iterations nor introducescommunication; indicate to the compiler, that the communication pattern which it may generate isinvariant (i.e. it will not change for all executions of the particular loop), REUSE_SCHEDULE.Whereas, the former options may be found (perhaps in a modified form) in the HPF-2developments, the latter belongs to the work-in-progess category.4 Concluding RemarksInvestigations into the use of HPF for irregular codes such as PAM-CRASH are in their infancy,but it is already clear that a further development of the language will be necessary if efficientcode is to be generated. Although, the simplified code maintenance offered by a high-levelprogramming approach is most desirable, HPF does lag behind message-passing in terms ofexploitability and performance for complex, irregular industrial codes. The message-passingversion of the PAM-CRASH code has, thanks (to a large extent) to the EUROPORTdemonstration of performance, achieved industrial acceptance as a design tool. Although theobtained scalability is not perfect and subject to improvement, a nearly linear speed-up isobtained for processor numbers far beyond what is required to match traditional vectorsupercomputer performance and this at considerably lower cost. Besides allowing a decreasedturn-around, the possibility has now been created to solve larger problems with acceptable delays.In comparison with the shared-memory programming model, the message-passing code versionincreases the effective peak performance of the machine, allowing greater flexibility in itsexploitation.AcknowledgementsThe authors would like to acknowledge the collaboration of their colleagues in the ESPRITprojects CAMAS, CAMAS-Link and HPF+, which led to the results reported on in this paper -the financial support of the European Commission for those projects is also gratefullyacknowledged.7References1. Chapman, B. et.al., ‘High Performance Fortran Languages: Advanced applications and theirimplementation’ Future Generation Computer Systems 11 (1995)2. Chapman, B. et.al., ‘Extending HPF for advanced data parallel applications’, World-Wide Webdocument:http://www.vcpc.univie.ac.at/activities/r_and_d/Publications.html3. Clinckemaillie, J. et. al., ‘Performance issues of the parallel PAM-CRASH code’, Int. J.Supercomputer Applications, (accepted for publication)4. Floros, N. et.al., ‘Comparitive efficiencies of domain decompositions’, Parallel Computing 21, , pp.1823-1835 (1995)5. Haug, E. et. al., ‘Transport vehicle crash, safety and manufacturing simulation in the perspective ofhigh performance computing and networking’, Future Generation Computer Systems 10, pp.173-181 (1994)6. Hertzberger, B. & Serazzi, G. (Eds), Proceedings of the HPCN '95 Conference, Lecture Notes inComputer Science 919, Springer-Verlag (1995).7. High Performance Fortran Forum. High Performance Fortran Language Specification Version 1.1,World-Wide Web document:http://www.crpc.rice.edu/HPFF/hpf18. High Performance Fortran Forum. HPF-2, World-Wide Web document:http://www.crpc.rice.edu/HPFF/hpf29. HPF+ ESPRIT LTR Project No. 21033. Optimizing HPF for Advanced Applications.  World-WideWeb document:http://www.par.univie.ac.at/hpf+10. Lonsdale, G. et al., ‘Communication requirements in parallel crashworthiness simulation’,Proceedings of the HPCN '94 Conference, Lecture Notes in Computer Science 796, Springer-Verlag, pp. 55-61. (1994)11. Lonsdale, G. et al., ‘Experiences with industrial crashworthiness simulations using the portable,message-passing PAM-CRASH code’, pp. 856-862 in [6]12. Mierendorff, H. et.al., ‘Europort-1: Porting industrial codes to parallel architectures’,pp. 806-812 in [6]13. Stüben, K. et.al., ‘Parallel industrial Fluid Dynamics and Structural Mechanics codes”, Proceedingsof the HPCN '96 Conference, Lecture Notes in Computer Science 1067, Springer-Verlag,pp.90-98 (1996)

Parallel and Distributed Crashworthiness Simulation

Abstract

Similar works

Full text

Available Versions

CERN Document Server