International audienceWe present a parallel unstructured mesh adaptation algorithm based on iterative remeshing and mesh repartitioning. The algorithm rests on a two-level parallelization scheme allowing to tweak the mesh group size for remeshing, and on a mesh repartitioning scheme based on interface displacement by front advancement. The numerical procedure is implemented in the open source ParMmg software package. It enables the reuse of existing sequential remeshing libraries, a non-intrusive linkage with thirdparty solvers, and a tunable exploitation of distributed parallel environments. We show the efficiency of the approach by comparing interface displacement repartitioning with graph-based repartitioning, and by showing isotropic weak-scaling tests and preliminary anisotropic tests

Cirrottola, Luca

Froehly, Algiane

We present a parallel unstructured mesh adaptation algorithm based on iterative remeshing and mesh repartitioning. The algorithm rests on a two-level parallelization scheme allowing to tweak the mesh group size for remeshing, and on a mesh repartitioning scheme based on interface displacement by front advancement. The numerical procedure is implemented in the open source ParMmg software package. It enables the reuse of existing sequential remeshing libraries, a non-intrusive linkage with thirdparty solvers, and a tunable exploitation of distributed parallel environments. We show the efficiency of the approach by comparing interface displacement repartitioning with graph-based repartitioning, and by showing isotropic weak-scaling tests and preliminary anisotropic tests

CIRROTTOLA, Luca

FROEHLY, Algiane

Oskar Bordeaux

Parallel Unstructured Mesh Adaptation Based on Iterative Remershing and Repartitioning

Cirrottola, L.

Froehly, A.

Scipedia

14th World Congress on Computational Mechanics (WCCM)ECCOMAS Congress 2020)Virtual Congress: 11-–15 January 2021F. Chinesta, R. Abgrall, O. Allix and M. Kaliske (Eds)PARALLEL UNSTRUCTURED MESH ADAPTATION BASED ONITERATIVE REMESHING AND REPARTITIONINGLuca Cirrottola1*, Algiane Froehly21 INRIA, Université de Bordeaux, CNRS, Bordeaux INP, IMB UMR 5251, 200 Avenue de la VieilleTour, 33405 Talence cedex, France, luca.cirrottola@inria.fr2 INRIA Direction Générale Déléguée à l’Innovation, InriaSoft, Mmg Consortium,algiane.froehly@inria.frKey words: Parallel mesh adaptation, unstructured meshes, mesh migrationAbstract. We present a parallel unstructured mesh adaptation algorithm based on iterative remeshingand mesh repartitioning. The algorithm rests on a two-level parallelization scheme allowing to tweak themesh group size for remeshing, and on a mesh repartitioning scheme based on interface displacement byfront advancement. The numerical procedure is implemented in the open source ParMmg software pack-age. It enables the reuse of existing sequential remeshing libraries, a non-intrusive linkage with third-party solvers, and a tunable exploitation of distributed parallel environments. We show the efficiency ofthe approach by comparing interface displacement repartitioning with graph-based repartitioning, and byshowing isotropic weak-scaling tests and preliminary anisotropic tests.1 INTRODUCTIONModern computational mechanics solvers nowadays routinely exploit parallel, distributed memory com-puter architectures. Even if a parallel solver could in principle use a sequential remesher by gatheringthe distributed mesh on a single process, due to memory limitations it is not always possible to guaranteethat a large distributed mesh can be gathered on a single computing node. Beside this primary feasibilityconsideration, sequential remeshing in a parallel simulation also represents a significant performancebottleneck [1]. Parallel remeshing is thus becoming increasingly demanded.Among the many previous works on parallel remeshing (for example [2] [3] [4] [5] [6] [7] [8] [9]), abroad classification can be made into methods that aim at parallelizing the remeshing kernel itself, andmethods that aim at employing a sequential remeshing kernel in a parallel framework by leveraging aparallel repartitioning phase. We have opted for a modular approach by adopting an iterative remeshing-repartitioning scheme that does not modify the adaptation kernel [9], allowing the reuse of an existingsequential remeshing kernel. Our method is implemented into the parallel ParMmg application and li-brary [10], built on top of the sequential Mmg3d remesher [11] for tetrahedral mesh adaptation. Bothsoftware are free and open source.The rest of this paper is organized as follows. Section 2 briefly recalls the parallelization algorithm, whichis detailed in [12]. Section 3 discusses the parallel performances at the current level of implementation,while section 4 sums up our observations and future research lines.1Luca Cirrottola, Algiane Froehly2 METHODAs described in depth in [12], at each iteration of the parallel algorithm the sequential remeshing kernelis applied on the interior partition on each process, while maintaining fixed (non-adapted) parallel inter-faces. Then the adapted mesh is repartitioned in order to move the non-adapted frontiers to the interiorof the partitions at the next iteration, thus eliminating the presence of non-adapted zones as the iterationsprogress. Although the repartitioning step can be accomplished through standard mesh partitioning li-braries, differently from [9] we have explored the usage of a direct parallel interface displacement method(or diffusion algorithm) to explicitly prioritize interface displacement over load balancing.3 PARALLEL PERFORMANCE ASSESSMENTWe show the performances of the parallel algorithm on two academic edge sizemaps, by means of a weakscaling and a strong scaling test. Both the weak and strong scaling tests have been performed with therelease v1.3.0 of ParMmg[10].3.1 Weak scalingA weak scaling test is shown in table 1. A sphere of radius 10 is uniformly refined while keeping theworkload of each process as constant as possible as the number of processes is increased. The test isperformed on the bora nodes of the PlaFRIM cluster1. The weak scaling performances are shown infigure 1a. The slow increase in the time spent in the repartitioning and redistribution phase still leavesspace for some implementation optimization. Anyway, a major improvement is visible with respect tothe preliminary results shown in [12], where a significant performance degradation was present on morethan 64 cores. This improvement is due to an algorithmic optimization concerning the pre-computationof the pair of processes interacting in the communication phase, which replaces a direct probing on thepresence of data to exchange whose cost grew quadratically.3.2 Strong scalingA strong scaling test is performed with an isotropic sizemap h describing a double Archimedean spiralh(x,y) = min(1.6+ |ρ−aθ1|+0.005,1.6+ |ρ+aθ2|+0.0125) (1)withθ1 = φ+π(1+floor( ρ2πa))θ2 = φ−π(1+floor( ρ2πa)) (2)and φ = atan2(y,x), ρ = s√x2 + y2, a = 0.6, s = 0.5, into a sphere of radius 10 with uniform unit edgelength. The surfacic adaptation and a volumic cut of the adapted meshes on 1 and 1024 processesare shown in figure 2. The resulting edge length statistics are shown in table 2. The strong scalingperformances are shown in logarithmic scale in figure 1b, where the speedup Sp over p processes is1Supported by Inria, CNRS (LABRI and IMB), Université de Bordeaux, Bordeaux INP and Conseil Régional d’Aquitaine(see https://www.plafrim.fr/)2Luca Cirrottola, Algiane Froehlyp ninv /p noutv /p noutv /ninv noutv nine /p noute /p noute /nine noute2 3625 1293637 356.81 2587274 18876 7780974 412.21 155619484 3467 1341637 386.88 5366549 18798 8072081 429.39 322883258 3346 1380055 412.38 11040444 18666 8306084 444.98 6644867516 3264 1412516 432.66 22600269 18599 8503695 457.2 13605912932 3190 1437267 450.46 45992552 18557 8654569 466.36 27694621064 3214 1431186 445.2 91595935 18625 8619098 462.75 551622317128 3215 1444674 449.31 184918370 18878 8701524 460.91 1113795077256 3345 1468905 439.01 376039759 19705 8848464 449.03 2265206835512 3375 1446532 428.52 740624790 19998 8714450 435.74 44617987091024 3335 1449215 434.54 1483996788 19821 8731162 440.49 8940710661Table 1: Mesh statistics for the ParMmg weak scaling test. Number of vertices nv and tetrahedra ne ininput and output using p processors.p N(0,0.3] N(0.3,0.6] N(0.6,0.7] N(0.71,0.9] N(0.9,1.3] N(1.3,1.41] N(1.41,2] N(2,5] N>51 1.34 % 35.34 % 16.89 % 25.49 % 20.15 % 0.63 % 0.16 % 0 02 1.14 % 34.87 % 16.97 % 25.79 % 20.45 % 0.63 % 0.15 % 0 04 1.04 % 34.20 % 17.03 % 26.08 % 20.85 % 0.64 % 0.16 % 0 08 0.98 % 33.66 % 17.06 % 26.30 % 21.18 % 0.66 % 0.16 % 0 016 1.07 % 32.41 % 17.06 % 26.78 % 21.84 % 0.68 % 0.16 % 0 032 0.91 % 30.78 % 17.29 % 27.59 % 22.57 % 0.70 % 0.16 % 0 064 0.93 % 30.22 % 17.20 % 27.76 % 23.01 % 0.72 % 0.17 % 0 0128 0.88 % 29.48 % 17.20 % 28.06 % 23.48 % 0.73 % 0.17 % < 0.01 % 0256 0.70 % 27.63 % 17.20 % 28.84 % 24.67 % 0.77 % 0.18 % < 0.01 % < 0.01 %512 0.66 % 27.19 % 17.14 % 28.96 % 25.04 % 0.80 % 0.20 % < 0.01 % < 0.01 %1024 0.69 % 26.14 % 16.91 % 29.03 % 25.99 % 0.92 % 0.30 % 0.02 % < 0.01 %Table 2: Mesh statistics for the ParMmg strong scaling test. Percentage of edges N(a,b] whose length inthe assigned metrics falls in the interval lM ∈ (a,b], for each simulation on p processors.defined as the ratio between the wall time on 1 process and the wall time on p processesSp =T1Tp(3)except for the speedup of the redistribution part of the program, which is defined with respect to theredistribution time on 2 processes instead of 1 (as there is no redistribution on 1 process). As the numberof interfaces increases with the number of processes, it can be noticed both in table 2 and in figure 1b thatthere is still a trend for a more pronounced propagation of large edge sizes from the fixed interfaces asthe number of processes increases, as well as a performance reduction in the redistribution phase. Theseissues are the subject of current improvement efforts.3Luca Cirrottola, Algiane Froehly2 4 8 16 32 64 128 256 512 102402004006008001,0001,2001,400Number of processesWalltime(s)IdealTotalRemeshingRedistribution(a) Weak scaling performances for the uniform refinement of a sphere.1 2 4 8 16 32 64 128 256 512 102412481632641282565121024Number of processesSpeedupIdealSequentialTotalRemeshingRedistribution(b) Strong scaling performances for the double Archimedean spiral in a sphere.Figure 1: Weak and strong scaling performances of ParMmg for two different tests.4Luca Cirrottola, Algiane Froehly(a) Surface adaptation on 1 process. (b) Surface adaptation on 1024 processes.(c) Volume adaptation on 1 process. (d) Volume adaptation on 1024 processesFigure 2: Examples of adaptation to the double Archimedean spiral on 1 and 1024 processes.5Luca Cirrottola, Algiane Froehly3.3 Extension to anisotropic metricsAs already shown in [12], the application to anisotropic metrics can suffer from a more serious sizediffusion effect than the one shown in the previous section, if specific precautions are not taken whendealing with the metric specification on parallel interfaces. This is the subject of current studies.4 CONCLUSIONSWe have shown a performance assessment of a parallel remeshing algorithm for unstructured meshesbased on iterative remeshing-repartitioning first described in [12]. Parallel iterations are performed byalternating remeshing on interior process partitions, with fixed parallel interfaces, to mesh repartition-ing aiming at moving previous interfaces on the interior of the next partitions to be remeshed. A keypoint for the choice of this specific parallelization strategy rests in software modularity, since building aparallelization framework on top of an existing sequential remeshing kernel allows the parallel softwareapplication to benefit from the continuous development of its sequential kernel. Modularity also appliesto the choice of the mesh repartitioning strategy, since it is independent from the internal mechanics ofthe remeshing kernel. While weak scaling results show that meshes of billions of tetrahedra are easilyachievable through the current framework, they also show that care needs to be taken in the design ofthe parallel redistribution phase, as more data and more processes are involved in the communicationphase. Strong scaling results show that the parallel application exhibits an overhead with respect to itssequential counterpart, due to the fact that several remeshing iterations are performed. Although thisoverhead is quickly catched up as the number of processes increases, it has to be kept in mind when thecoupling with an external solver is envisaged, as its relative weight on overall performances can dependon the performances of the external solver itself, on the target mesh size, and on the parallel communi-cator size. Future research lines involve further improvement of the scalability of the mesh redistributionphase, which could be achieved by further optimizing the software implementation of the parallel com-munication, as well as the reduction of the parallel overhead, which could be achieved by optimizingthe number of iterations or the interface displacement phase. Future developments also concern the fullsupport of non-smooth boundary surfaces and anisotropic metrics.References[1] Michael A Park et al. “Unstructured Grid Adaptation: Status, Potential Impacts, and Recom-mended Investments Towards CFD 2030”. In: AIAA Fluid Dynamics Conference, AIAA AVIATIONForum. Washington DC, United States, 2016. URL: https://hal.inria.fr/hal-01438667.[2] José G. Castaños and John E. Savage. “The Dynamic Adaptation of Parallel Mesh-Based Compu-tation”. In: PPSC. 1997.[3] H. L. De Cougny and M. S. Shephard. “Parallel refinement and coarsening of tetrahedral meshes”.In: International Journal for Numerical Methods in Engineering 46.7 (1999), pp. 1101–1125.[4] Leonid Oliker, Rupak Biswas, and Harold N Gabow. “Parallel tetrahedral mesh adaptation withdynamic load balancing”. In: Parallel Computing 26.12 (2000). Graph Partitioning and ParallelComputing, pp. 1583 –1608. URL: http://www.sciencedirect.com/science/article/pii/S0167819100000478.6Luca Cirrottola, Algiane Froehly[5] Nikos Chrisochoides and Démian Nave. “Parallel Delaunay mesh generation kernel”. In: Interna-tional Journal for Numerical Methods in Engineering 58.2 (2003), pp. 161–176. eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/nme.765. URL: https://onlinelibrary.wiley.com/doi/abs/10.1002/nme.765.[6] J.E. Flaherty et al. “Parallel structures and dynamic load balancing for adaptive finite elementcomputation”. In: Applied Numerical Mathematics 26.1 (1998), pp. 241 –263. URL: http://www.sciencedirect.com/science/article/pii/S0168927497000949.[7] Peter A. Cavallo, Neeraj Sinha, and Gregory M. Feldman. “Parallel Unstructured Mesh AdaptationMethod for Moving Body Applications”. In: AIAA Journal 43.9 (2005), pp. 1937–1945. eprint:https://doi.org/10.2514/1.7818. URL: https://doi.org/10.2514/1.7818.[8] Hugues Digonnet et al. “Massively parallel anisotropic mesh adaptation”. In: International Jour-nal of High Performance Computing Applications (Mar. 2017). URL: https://hal.archives-ouvertes.fr/hal-01487424.[9] Pierre Benard et al. “Mesh adaptation for large-eddy simulations in complex geometries”. In: In-ternational Journal for Numerical Methods in Fluids 81.12 (2016), pp. 719–740. eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/fld.4204. URL: https://onlinelibrary.wiley.com/doi/abs/10.1002/fld.4204.[10] ParMmg version 1.3.0. SWHID:swh:1:rel:bf45d6314a386455d53a6c351be771d6252e0d43; REPOS-ITORY: https://github.com/MmgTools/ParMmg.[11] Mmg version 5.5.2. SWHID: swh:1:rel:fe173a75f45f079d363d5a82204c9737550c5d79; REPOSI-TORY: https://github.com/MmgTools/mmg.[12] Luca Cirrottola and Algiane Froehly. Parallel unstructured mesh adaptation using iterative remesh-ing and repartitioning. Research Report RR-9307. INRIA Bordeaux, équipe CARDAMOM, Nov.2019. URL: https://hal.inria.fr/hal-02386837.7

Parallel Unstructured Mesh Adaptation Based on Iterative Remeshing and Repartitioning

INRIA a CCSD electronic archive server

HAL Id: hal-03208569https://hal.inria.fr/hal-03208569Submitted on 26 Apr 2021HAL is a multi-disciplinary open accessarchive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come fromteaching and research institutions in France orabroad, or from public or private research centers.L’archive ouverte pluridisciplinaire HAL, estdestinée au dépôt et à la diffusion de documentsscientifiques de niveau recherche, publiés ou non,émanant des établissements d’enseignement et derecherche français ou étrangers, des laboratoirespublics ou privés.Parallel Unstructured Mesh Adaptation Based onIterative Remershing and RepartitioningLuca Cirrottola, Algiane FroehlyTo cite this version:Luca Cirrottola, Algiane Froehly. Parallel Unstructured Mesh Adaptation Based on Iterative Remer-shing and Repartitioning. WCCM-Eccomas 2020 - 14th World Congress on Computational Mechanic,Jan 2021, Paris / Virtual, France. ￿hal-03208569￿14th World Congress on Computational Mechanics (WCCM)ECCOMAS Congress 2020)Virtual Congress: 11-–15 January 2021F. Chinesta, R. Abgrall, O. Allix and M. Kaliske (Eds)PARALLEL UNSTRUCTURED MESH ADAPTATION BASED ONITERATIVE REMESHING AND REPARTITIONINGLuca Cirrottola1*, Algiane Froehly21 INRIA, Université de Bordeaux, CNRS, Bordeaux INP, IMB UMR 5251, 200 Avenue de la VieilleTour, 33405 Talence cedex, France, luca.cirrottola@inria.fr2 INRIA Direction Générale Déléguée à l’Innovation, InriaSoft, Mmg Consortium,algiane.froehly@inria.frKey words: Parallel mesh adaptation, unstructured meshes, mesh migrationAbstract. We present a parallel unstructured mesh adaptation algorithm based on iterative remeshingand mesh repartitioning. The algorithm rests on a two-level parallelization scheme allowing to tweak themesh group size for remeshing, and on a mesh repartitioning scheme based on interface displacement byfront advancement. The numerical procedure is implemented in the open source ParMmg software pack-age. It enables the reuse of existing sequential remeshing libraries, a non-intrusive linkage with third-party solvers, and a tunable exploitation of distributed parallel environments. We show the efficiency ofthe approach by comparing interface displacement repartitioning with graph-based repartitioning, and byshowing isotropic weak-scaling tests and preliminary anisotropic tests.1 INTRODUCTIONModern computational mechanics solvers nowadays routinely exploit parallel, distributed memory com-puter architectures. Even if a parallel solver could in principle use a sequential remesher by gatheringthe distributed mesh on a single process, due to memory limitations it is not always possible to guaranteethat a large distributed mesh can be gathered on a single computing node. Beside this primary feasibilityconsideration, sequential remeshing in a parallel simulation also represents a significant performancebottleneck [1]. Parallel remeshing is thus becoming increasingly demanded.Among the many previous works on parallel remeshing (for example [2] [3] [4] [5] [6] [7] [8] [9]), abroad classification can be made into methods that aim at parallelizing the remeshing kernel itself, andmethods that aim at employing a sequential remeshing kernel in a parallel framework by leveraging aparallel repartitioning phase. We have opted for a modular approach by adopting an iterative remeshing-repartitioning scheme that does not modify the adaptation kernel [9], allowing the reuse of an existingsequential remeshing kernel. Our method is implemented into the parallel ParMmg application and li-brary [10], built on top of the sequential Mmg3d remesher [11] for tetrahedral mesh adaptation. Bothsoftware are free and open source.The rest of this paper is organized as follows. Section 2 briefly recalls the parallelization algorithm, whichis detailed in [12]. Section 3 discusses the parallel performances at the current level of implementation,while section 4 sums up our observations and future research lines.1Luca Cirrottola, Algiane Froehly2 METHODAs described in depth in [12], at each iteration of the parallel algorithm the sequential remeshing kernelis applied on the interior partition on each process, while maintaining fixed (non-adapted) parallel inter-faces. Then the adapted mesh is repartitioned in order to move the non-adapted frontiers to the interiorof the partitions at the next iteration, thus eliminating the presence of non-adapted zones as the iterationsprogress. Although the repartitioning step can be accomplished through standard mesh partitioning li-braries, differently from [9] we have explored the usage of a direct parallel interface displacement method(or diffusion algorithm) to explicitly prioritize interface displacement over load balancing.3 PARALLEL PERFORMANCE ASSESSMENTWe show the performances of the parallel algorithm on two academic edge sizemaps, by means of a weakscaling and a strong scaling test. Both the weak and strong scaling tests have been performed with therelease v1.3.0 of ParMmg[10].3.1 Weak scalingA weak scaling test is shown in table 1. A sphere of radius 10 is uniformly refined while keeping theworkload of each process as constant as possible as the number of processes is increased. The test isperformed on the bora nodes of the PlaFRIM cluster1. The weak scaling performances are shown infigure 1a. The slow increase in the time spent in the repartitioning and redistribution phase still leavesspace for some implementation optimization. Anyway, a major improvement is visible with respect tothe preliminary results shown in [12], where a significant performance degradation was present on morethan 64 cores. This improvement is due to an algorithmic optimization concerning the pre-computationof the pair of processes interacting in the communication phase, which replaces a direct probing on thepresence of data to exchange whose cost grew quadratically.3.2 Strong scalingA strong scaling test is performed with an isotropic sizemap h describing a double Archimedean spiralh(x,y) = min(1.6+ |ρ−aθ1|+0.005,1.6+ |ρ+aθ2|+0.0125) (1)withθ1 = φ+π(1+floor( ρ2πa))θ2 = φ−π(1+floor( ρ2πa)) (2)and φ = atan2(y,x), ρ = s√x2 + y2, a = 0.6, s = 0.5, into a sphere of radius 10 with uniform unit edgelength. The surfacic adaptation and a volumic cut of the adapted meshes on 1 and 1024 processesare shown in figure 2. The resulting edge length statistics are shown in table 2. The strong scalingperformances are shown in logarithmic scale in figure 1b, where the speedup Sp over p processes is1Supported by Inria, CNRS (LABRI and IMB), Université de Bordeaux, Bordeaux INP and Conseil Régional d’Aquitaine(see https://www.plafrim.fr/)2Luca Cirrottola, Algiane Froehlyp ninv /p noutv /p noutv /ninv noutv nine /p noute /p noute /nine noute2 3625 1293637 356.81 2587274 18876 7780974 412.21 155619484 3467 1341637 386.88 5366549 18798 8072081 429.39 322883258 3346 1380055 412.38 11040444 18666 8306084 444.98 6644867516 3264 1412516 432.66 22600269 18599 8503695 457.2 13605912932 3190 1437267 450.46 45992552 18557 8654569 466.36 27694621064 3214 1431186 445.2 91595935 18625 8619098 462.75 551622317128 3215 1444674 449.31 184918370 18878 8701524 460.91 1113795077256 3345 1468905 439.01 376039759 19705 8848464 449.03 2265206835512 3375 1446532 428.52 740624790 19998 8714450 435.74 44617987091024 3335 1449215 434.54 1483996788 19821 8731162 440.49 8940710661Table 1: Mesh statistics for the ParMmg weak scaling test. Number of vertices nv and tetrahedra ne ininput and output using p processors.p N(0,0.3] N(0.3,0.6] N(0.6,0.7] N(0.71,0.9] N(0.9,1.3] N(1.3,1.41] N(1.41,2] N(2,5] N>51 1.34 % 35.34 % 16.89 % 25.49 % 20.15 % 0.63 % 0.16 % 0 02 1.14 % 34.87 % 16.97 % 25.79 % 20.45 % 0.63 % 0.15 % 0 04 1.04 % 34.20 % 17.03 % 26.08 % 20.85 % 0.64 % 0.16 % 0 08 0.98 % 33.66 % 17.06 % 26.30 % 21.18 % 0.66 % 0.16 % 0 016 1.07 % 32.41 % 17.06 % 26.78 % 21.84 % 0.68 % 0.16 % 0 032 0.91 % 30.78 % 17.29 % 27.59 % 22.57 % 0.70 % 0.16 % 0 064 0.93 % 30.22 % 17.20 % 27.76 % 23.01 % 0.72 % 0.17 % 0 0128 0.88 % 29.48 % 17.20 % 28.06 % 23.48 % 0.73 % 0.17 % < 0.01 % 0256 0.70 % 27.63 % 17.20 % 28.84 % 24.67 % 0.77 % 0.18 % < 0.01 % < 0.01 %512 0.66 % 27.19 % 17.14 % 28.96 % 25.04 % 0.80 % 0.20 % < 0.01 % < 0.01 %1024 0.69 % 26.14 % 16.91 % 29.03 % 25.99 % 0.92 % 0.30 % 0.02 % < 0.01 %Table 2: Mesh statistics for the ParMmg strong scaling test. Percentage of edges N(a,b] whose length inthe assigned metrics falls in the interval lM ∈ (a,b], for each simulation on p processors.defined as the ratio between the wall time on 1 process and the wall time on p processesSp =T1Tp(3)except for the speedup of the redistribution part of the program, which is defined with respect to theredistribution time on 2 processes instead of 1 (as there is no redistribution on 1 process). As the numberof interfaces increases with the number of processes, it can be noticed both in table 2 and in figure 1b thatthere is still a trend for a more pronounced propagation of large edge sizes from the fixed interfaces asthe number of processes increases, as well as a performance reduction in the redistribution phase. Theseissues are the subject of current improvement efforts.3Luca Cirrottola, Algiane Froehly2 4 8 16 32 64 128 256 512 102402004006008001,0001,2001,400Number of processesWalltime(s)IdealTotalRemeshingRedistribution(a) Weak scaling performances for the uniform refinement of a sphere.1 2 4 8 16 32 64 128 256 512 102412481632641282565121024Number of processesSpeedupIdealSequentialTotalRemeshingRedistribution(b) Strong scaling performances for the double Archimedean spiral in a sphere.Figure 1: Weak and strong scaling performances of ParMmg for two different tests.4Luca Cirrottola, Algiane Froehly(a) Surface adaptation on 1 process. (b) Surface adaptation on 1024 processes.(c) Volume adaptation on 1 process. (d) Volume adaptation on 1024 processesFigure 2: Examples of adaptation to the double Archimedean spiral on 1 and 1024 processes.5Luca Cirrottola, Algiane Froehly3.3 Extension to anisotropic metricsAs already shown in [12], the application to anisotropic metrics can suffer from a more serious sizediffusion effect than the one shown in the previous section, if specific precautions are not taken whendealing with the metric specification on parallel interfaces. This is the subject of current studies.4 CONCLUSIONSWe have shown a performance assessment of a parallel remeshing algorithm for unstructured meshesbased on iterative remeshing-repartitioning first described in [12]. Parallel iterations are performed byalternating remeshing on interior process partitions, with fixed parallel interfaces, to mesh repartition-ing aiming at moving previous interfaces on the interior of the next partitions to be remeshed. A keypoint for the choice of this specific parallelization strategy rests in software modularity, since building aparallelization framework on top of an existing sequential remeshing kernel allows the parallel softwareapplication to benefit from the continuous development of its sequential kernel. Modularity also appliesto the choice of the mesh repartitioning strategy, since it is independent from the internal mechanics ofthe remeshing kernel. While weak scaling results show that meshes of billions of tetrahedra are easilyachievable through the current framework, they also show that care needs to be taken in the design ofthe parallel redistribution phase, as more data and more processes are involved in the communicationphase. Strong scaling results show that the parallel application exhibits an overhead with respect to itssequential counterpart, due to the fact that several remeshing iterations are performed. Although thisoverhead is quickly catched up as the number of processes increases, it has to be kept in mind when thecoupling with an external solver is envisaged, as its relative weight on overall performances can dependon the performances of the external solver itself, on the target mesh size, and on the parallel communi-cator size. Future research lines involve further improvement of the scalability of the mesh redistributionphase, which could be achieved by further optimizing the software implementation of the parallel com-munication, as well as the reduction of the parallel overhead, which could be achieved by optimizingthe number of iterations or the interface displacement phase. Future developments also concern the fullsupport of non-smooth boundary surfaces and anisotropic metrics.References[1] Michael A Park et al. “Unstructured Grid Adaptation: Status, Potential Impacts, and Recom-mended Investments Towards CFD 2030”. In: AIAA Fluid Dynamics Conference, AIAA AVIATIONForum. Washington DC, United States, 2016. URL: https://hal.inria.fr/hal-01438667.[2] José G. Castaños and John E. Savage. “The Dynamic Adaptation of Parallel Mesh-Based Compu-tation”. In: PPSC. 1997.[3] H. L. De Cougny and M. S. Shephard. “Parallel refinement and coarsening of tetrahedral meshes”.In: International Journal for Numerical Methods in Engineering 46.7 (1999), pp. 1101–1125.[4] Leonid Oliker, Rupak Biswas, and Harold N Gabow. “Parallel tetrahedral mesh adaptation withdynamic load balancing”. In: Parallel Computing 26.12 (2000). Graph Partitioning and ParallelComputing, pp. 1583 –1608. URL: http://www.sciencedirect.com/science/article/pii/S0167819100000478.6Luca Cirrottola, Algiane Froehly[5] Nikos Chrisochoides and Démian Nave. “Parallel Delaunay mesh generation kernel”. In: Interna-tional Journal for Numerical Methods in Engineering 58.2 (2003), pp. 161–176. eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/nme.765. URL: https://onlinelibrary.wiley.com/doi/abs/10.1002/nme.765.[6] J.E. Flaherty et al. “Parallel structures and dynamic load balancing for adaptive finite elementcomputation”. In: Applied Numerical Mathematics 26.1 (1998), pp. 241 –263. URL: http://www.sciencedirect.com/science/article/pii/S0168927497000949.[7] Peter A. Cavallo, Neeraj Sinha, and Gregory M. Feldman. “Parallel Unstructured Mesh AdaptationMethod for Moving Body Applications”. In: AIAA Journal 43.9 (2005), pp. 1937–1945. eprint:https://doi.org/10.2514/1.7818. URL: https://doi.org/10.2514/1.7818.[8] Hugues Digonnet et al. “Massively parallel anisotropic mesh adaptation”. In: International Jour-nal of High Performance Computing Applications (Mar. 2017). URL: https://hal.archives-ouvertes.fr/hal-01487424.[9] Pierre Benard et al. “Mesh adaptation for large-eddy simulations in complex geometries”. In: In-ternational Journal for Numerical Methods in Fluids 81.12 (2016), pp. 719–740. eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/fld.4204. URL: https://onlinelibrary.wiley.com/doi/abs/10.1002/fld.4204.[10] ParMmg version 1.3.0. SWHID:swh:1:rel:bf45d6314a386455d53a6c351be771d6252e0d43; REPOS-ITORY: https://github.com/MmgTools/ParMmg.[11] Mmg version 5.5.2. SWHID: swh:1:rel:fe173a75f45f079d363d5a82204c9737550c5d79; REPOSI-TORY: https://github.com/MmgTools/mmg.[12] Luca Cirrottola and Algiane Froehly. Parallel unstructured mesh adaptation using iterative remesh-ing and repartitioning. Research Report RR-9307. INRIA Bordeaux, équipe CARDAMOM, Nov.2019. URL: https://hal.inria.fr/hal-02386837.7

https://hal.inria.fr/hal-03208569/file/p1957.pdf

Parallel Unstructured Mesh Adaptation Based on Iterative Remershing and Repartitioning

Abstract

Similar works

Full text

Available Versions

Oskar Bordeaux

Scipedia

INRIA a CCSD electronic archive server