HERMESH : a geometrical domain composition method in computational mechanics

With this thesis we present the HERMESH method which has been classified by us as a a composition domain method. This term comes from the idea that HERMESH obtains a global solution of the problem from two independent meshes as a result of the mesh coupling. The global mesh maintains the same number of degrees of freedom as the sum of the independent meshes, which are coupled in the interfaces via new elements referred to by us as extension elements. For this reason we enunciate that the domain composition method is geometrical. The result of the global mesh is a non-conforming mesh in the interfaces between independent meshes due to these new connectivities formed with existing nodes and represented by the new extension elements. The first requirements were that the method be implicit, be valid for any partial differential equation and not imply any additional effort or loss in efficiency in the parallel performance of the code in which the method has been implemented. In our opinion, these properties constitute the main contribution in mesh coupling for the computational mechanics framework. From these requirements, we have been able to develop an automatic and topology-independent tool to compose independent meshes. The method can couple overlapping meshes with minimal intervention on the user's part. The overlapping can be partial or complete in the sense of overset meshes. The meshes can be disjoint with or without a gap between them. And we have demonstrated the flexibility of the method in the relative mesh size. In this work we present a detailed description of HERMESH which has been implemented in a high-performance computing computational mechanics code within the framework of the finite element methods. This code is called Alya. The numerical properties will be proved with different benchmark-type problems and the manufactured solution technique. Finally, the results in complex problems solved with HERMESH will be presented, clearly showing the versatility of the method.En este trabajo presentamos el metodo HERMESH al que hemos catalogado como un método de composición de dominios puesto que a partir de mallas independientes se obtiene una solución global del problema como la unión de los subproblemas que forman las mallas independientes. Como resultado, la malla global mantiene el mismo número de grados de libertad que la suma de los grados de libertad de las mallas independientes, las cuales se acoplan en las interfases internas a través de nuevos elementos a los que nos referimos como elementos de extensión. Por este motivo decimos que el método de composición de dominio es geométrico. El resultado de la malla global es una malla que no es conforme en las interfases entre las distintas mallas debido a las nuevas conectividades generadas sobre los nodos existentes. Los requerimientos de partida fueron que el método se implemente de forma implícita, sea válido para cualquier PDE y no implique ningún esfuerzo adicional ni perdida de eficiencia para el funcionamiento paralelo del código de altas prestaciones en el que ha sido implementado. Creemos que estas propiedades son las principales aportaciones de esta tesis dentro del marco de acoplamiento de mallas en mecánica computacional. A partir de estas premisas, hemos conseguido una herramienta automática e independiente de la topología para componer mallas. Es capaz de acoplar sin necesidad de intervención del usuario, mallas con solapamiento parcial o total así como mallas disjuntas con o sin "gap" entre ellas. También hemos visto que ofrece cierta flexibilidad en relación al tamaños relativos entre las mallas siendo un método válido como técnica de remallado local. Presentamos una descripción detallada de la implementación de esta técnica, llevada a cabo en un código de altas prestaciones de mecánica computacional en el contexto de elementos finitos, Alya. Se demostrarán todas las propiedades numéricas que ofrece el métodos a través de distintos problemas tipo benchmark y el método de la solución manufacturada. Finalmente se mostrarán los resultados en problemas complejos resueltos con el método HERMESH, que a su vez es una prueba de la gran flexibilidad que nos brinda

Computational Fluid and Particle Dynamics Simulations for Respiratory System: Runtime Optimization on an Arm Cluster

Computational fluid and particle dynamics simulations (CFPD) are of paramount importance for studying and improving drug effectiveness. Computational requirements of CFPD codes involves high-performance computing (HPC) resources. For these reasons we introduce and evaluate in this paper system software techniques for improving performance and tolerate load imbalance on a state-of-the-art production CFPD code. We demonstrate benefits of these techniques on both Intel- and Arm-based HPC clusters showing the importance of using mechanisms applied at runtime to improve the performance independently of the underlying architecture. We run a real CFPD simulation of particle tracking on the human respiratory system, showing performance improvements of up to 2X, keeping the computational resources constant.This work is partially supported by the Spanish Government (SEV-2015-0493), by the Spanish Ministry of Science and Technology project (TIN2015-65316-P), by the Generalitat de Catalunya (2017-SGR-1414), and by the European Mont-Blanc projects (288777, 610402 and 671697).Peer ReviewedPostprint (author's final draft

Runtime Mechanisms to Survive New HPC Architectures: A Use-Case in Human Respiratory Simulations

Computational Fluid and Particle Dynamics (CFPD) simulations are of paramount importance for studying and improving drug effectiveness. Computational requirements of CFPD codes demand high-performance computing (HPC) resources. For these reasons we introduce and evaluate in this paper system software techniques for improving performance and tolerate load imbalance on a state-of-the-art production CFPD code. We demonstrate benefits of these techniques on Intel-, IBM-, and Arm-based HPC technologies ranked in the Top500 supercomputers, showing the importance of using mechanisms applied at runtime to improve the performance independently of the underlying architecture. We run a real CFPD simulation of particle tracking on the human respiratory system, showing performance improvements of up to 2x, across different architectures, while applying runtime techniques and keeping constant the computational resources.This work is partially supported by the Spanish Government (SEV-2015-0493), by the Spanish Ministry of Science and Technology project (TIN2015-65316-P), by the Generalitat de Catalunya (2017-SGR-1414), and by the European Mont-Blanc projects (288777, 610402 and 671697).Peer ReviewedPreprin

Flow features and micro-particle deposition in a human respiratory system during sniffing

As we inhale, the air drawn through our nose undergoes successive accelerations and decelerations as it is turned, split and recombined before splitting again at the end of the trachea as it enters the bronchi. Fully describing the dynamic behaviour of the airflow and how it transports inhaled particles poses a severe challenge to computational simulations. In this paper we explore two aspects: the dynamic behaviour of airflow during a rapid inhalation (or sniff) and the transport of inhaled aerosols. The development of flow unsteadiness from a laminar state at entry to the nose through to the turbulent character of tracheal flow is resolved using accurate numerical models with high performance computing-based large scale simulations. Combining the flow solution with a Lagrangian computation reveals the effects of flow behaviour and airway geometry on the deposition of inhaled microparticles. Improved modelling of airflow and delivery of therapeutic aerosols could be applied to improve diagnosis and treatment

The OTree: multidimensional indexing with efficient data sampling for HPC

Spatial big data is considered an essential trend in future scientific and business applications. Indeed, research instruments, medical devices, and social networks generate hundreds of petabytes of spatial data per year. However, many authors have pointed out that the lack of specialized frameworks for multidimensional Big Data is limiting possible applications and precluding many scientific breakthroughs. Paramount in achieving High-Performance Data Analytics is to optimize and reduce the I/O operations required to analyze large data sets. To do so, we need to organize and index the data according to its multidimensional attributes. At the same time, to enable fast and interactive exploratory analysis, it is vital to generate approximate representations of large datasets efficiently. In this paper, we propose the Outlook Tree (or OTree), a novel Multidimensional Indexing with efficient data Sampling (MIS) algorithm. The OTree enables exploratory analysis of large multidimensional datasets with arbitrary precision, a vital missing feature in current distributed data management solutions. Our algorithm reduces the indexing overhead and achieves high performance even for write-intensive HPC applications. Indeed, we use the OTree to store the scientific results of a study on the efficiency of drug inhalers. Then we compare the OTree implementation on Apache Cassandra, named Qbeast, with PostgreSQL and plain storage. Lastly, we demonstrate that our proposal delivers better performance and scalability.Peer ReviewedPostprint (author's final draft

Design and execution of a verification, validation, and uncertainty quantification plan for a numerical model of left ventricular flow after LVAD implantation

BACKGROUND: Left ventricular assist devices (LVADs) are implantable pumps that act as a life support therapy for patients with severe heart failure. Despite improving the survival rate, LVAD therapy can carry major complications. Particularly, the flow distortion introduced by the LVAD in the left ventricle (LV) may induce thrombus formation. While previous works have used numerical models to study the impact of multiple variables in the intra-LV stagnation regions, a comprehensive validation analysis has never been executed. The main goal of this work is to present a model of the LV-LVAD system and to design and follow a verification, validation and uncertainty quantification (VVUQ) plan based on the ASME V&V40 and V&V20 standards to ensure credible predictions. METHODS: The experiment used to validate the simulation is the SDSU cardiac simulator, a bench mock-up of the cardiovascular system that allows mimicking multiple operation conditions for the heart-LVAD system. The numerical model is based on Alya, the BSC’s in-house platform for numerical modelling. Alya solves the Navier-Stokes equation with an Arbitrary Lagrangian-Eulerian (ALE) formulation in a deformable ventricle and includes pressure-driven valves, a 0D Windkessel model for the arterial output and a LVAD boundary condition modeled through a dynamic pressure-flow performance curve. The designed VVUQ plan involves: (a) a risk analysis and the associated credibility goals; (b) a verification stage to ensure correctness in the numerical solution procedure; (c) a sensitivity analysis to quantify the impact of the inputs on the four quantities of interest (QoIs) (average aortic root flow , maximum aortic root flow , average LVAD flow , and maximum LVAD flow ); (d) an uncertainty quantification using six validation experiments that include extreme operating conditions. RESULTS: Numerical code verification tests ensured correctness of the solution procedure and numerical calculation verification showed a grid convergence index (GCI)95% <3.3%. The total Sobol indices obtained during the sensitivity analysis demonstrated that the ejection fraction, the heart rate, and the pump performance curve coefficients are the most impactful inputs for the analysed QoIs. The Minkowski norm is used as validation metric for the uncertainty quantification. It shows that the midpoint cases have more accurate results when compared to the extreme cases. The total computational cost of the simulations was above 100 [core-years] executed in around three weeks time span in Marenostrum IV supercomputer. Conclusions This work details a novel numerical model for the LV-LVAD system, that is supported by the design and execution of a VVUQ plan created following recognised international standards. We present a methodology demonstrating that stringent VVUQ according to ASME standards is feasible but computationally expensive.This project was funded in part by the FDA Critical Path Initiative and by an appointment to the Research Participation Program at the Division of Biomedical Physics, Office of Science and Engineering Laboratories, Center for Devices and Radiological Health, U.S. Food and Drug Administration, administered by the Oak Ridge Institute for Science, and Education through an interagency agreement between the U.S. Department of Energy and FDA to RAG. MV and AS acknowledge the funding from the project CompBioMed2 (H2020-EU.1.4.1.3. Grant number: 823712), SilicoFCM (H2020-EU.3.1.5. Grant number: 777204), and NEOTEC 2019 - "Generador de Corazones Virtuales" (“Ministerio de Economía y competititvidad”, EXP - 00123159 / SNEO-20191113). AS salary is partially funded by the “Ministerio de Economía y competititvidad” under the Torres Quevedo Program (grant number: PTQ2019-010528). CB salary is partially funded by the Torres Quevedo Program (grant number: PTQ2018-010290). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.Peer ReviewedPostprint (published version

A Parallel CFD Model for Wind Farms

We present a Computational Fluid Dynamics (CFD) modeling strategy for onshore wind farms aimed at predicting and opti- mizing the production of farms using a CFD model that includes meteorological data assimilation, complex terrain and wind turbine effects. The model involves the solution of the Reynolds-Averaged Navier-Stokes (RANS) equations together with a k-ɛ turbulence model specially designed for the Atmospheric Boundary Layer (ABL). The model involves automatic meshing and generation of boundary conditions with atmospheric boundary layer shape for the entering wind flow. As the integration of the model up to the ground surface is still not viable for complex terrains, a specific law of the wall including roughness effects is implemented. The wake effects and the aerodynamic behavior of the wind turbines are described using the actuator disk model, upon which a volumetric force is included in the momentum equations. The placement of the wind turbines and a mesh refinement for the near wakes is done by means of a Chimera method. The model is implemented in Alya, a High Performance Computing (HPC) multi physics parallel solver based on finite elements and developed at Barcelona Supercomputing Center.The research of G. Houzeaux is being partly done under a I3 contract with the Spanish Ministerio de Ciencia e Inovación. The work of B. Eguzkitza is financed by a scholarship from the Fundación IBERDROLA supporting the project ”Optimization of wind farms using computational fluid dynamics”.Peer ReviewedPostprint (published version

1st BSC Doctoral Symposium : book of abstracts

Peer Reviewe

A parallel coupling strategy for the chimera and domain decomposition methods in computational mechanics

Domain Decomposition Methods (DDMs) are techniques that divide the solution of a PDE on a domain into smaller solutions on smaller subdomains coupling them using a certain strategy. They are used for essentially two purposes: designing parallel solvers and/or coupling subdomains with different meshes, different numerical approximations, etc. In this paper we are interested in this last category. One example of application is the Chimera method. In that sense, the Chimera method can be viewed as a preprocess technique plus a DDM on overlapping and non-conforming subdomains. The coupling technique of DDM is usually achieved via transmission conditions to impose the continuities of the unknown and its flux across the subdomain boundaries. We propose in this work an alternative coupling strategy, intervening as a preprocess method. It consists in connecting the nodes of one subdomain with the nodes of the adjacent subdomains via newly created elements. In this way, the multi-domain character of a DDM disappears, making it a parallel, implicit and versatile method. We discuss in this paper the relation between the proposed method and the existing coupling strategies. We also present some convergence results as well as some applications to the Navier-Stokes equations and other PDE's. © 2012 Elsevier Ltd.This work has been partly carried out in the framework of FP7 European project W2Plastics. The research of Dr. Houzeaux has been partly done under a I3 contract with the Spanish Ministerio de Ciencia e InovaciónPeer Reviewe

A Chimera method for the incompressible Navier-Stokes equations

The Chimera method was developed three decades ago as a meshing simplification tool. Different components are meshed independently and then glued together using a domain decomposition technique to couple the equations solved on each component. This coupling is achieved via transmission conditions (in the finite element context) or by imposing the continuity of fluxes (in the finite volume context). Historically, the method has then been used extensively to treat moving objects, as the independent meshes are free to move with respect to the others. At each time step, the main task consists in recomputing the interpolation of the transmission conditions or fluxes. This paper presents a Chimera method applied to the Navier-Stokes equations. After an introduction on the Chimera method, we describe in two different sections the two independent steps of the method: the hole cutting to create the interfaces of the subdomains, and the coupling of the subdomains. Then we present the Navier-Stokes solver considered in this work. Implementation aspects are then detailed in order to apply efficiently the method to this specific parallel Navier-Stokes solver. We conclude with some examples to demonstrate the reliability and the application of the proposed method.This work has been partly carried out in the framework of the FP7 European projects W2Plastics. Part of the work of Beatriz Eguzkitza was sponsored by an IBERDROLA grant.Peer Reviewe