Computational Fluid and Particle Dynamics Simulations for Respiratory System: Runtime Optimization on an Arm Cluster

Computational fluid and particle dynamics simulations (CFPD) are of paramount importance for studying and improving drug effectiveness. Computational requirements of CFPD codes involves high-performance computing (HPC) resources. For these reasons we introduce and evaluate in this paper system software techniques for improving performance and tolerate load imbalance on a state-of-the-art production CFPD code. We demonstrate benefits of these techniques on both Intel- and Arm-based HPC clusters showing the importance of using mechanisms applied at runtime to improve the performance independently of the underlying architecture. We run a real CFPD simulation of particle tracking on the human respiratory system, showing performance improvements of up to 2X, keeping the computational resources constant.This work is partially supported by the Spanish Government (SEV-2015-0493), by the Spanish Ministry of Science and Technology project (TIN2015-65316-P), by the Generalitat de Catalunya (2017-SGR-1414), and by the European Mont-Blanc projects (288777, 610402 and 671697).Peer ReviewedPostprint (author's final draft

Runtime Mechanisms to Survive New HPC Architectures: A Use-Case in Human Respiratory Simulations

Computational Fluid and Particle Dynamics (CFPD) simulations are of paramount importance for studying and improving drug effectiveness. Computational requirements of CFPD codes demand high-performance computing (HPC) resources. For these reasons we introduce and evaluate in this paper system software techniques for improving performance and tolerate load imbalance on a state-of-the-art production CFPD code. We demonstrate benefits of these techniques on Intel-, IBM-, and Arm-based HPC technologies ranked in the Top500 supercomputers, showing the importance of using mechanisms applied at runtime to improve the performance independently of the underlying architecture. We run a real CFPD simulation of particle tracking on the human respiratory system, showing performance improvements of up to 2x, across different architectures, while applying runtime techniques and keeping constant the computational resources.This work is partially supported by the Spanish Government (SEV-2015-0493), by the Spanish Ministry of Science and Technology project (TIN2015-65316-P), by the Generalitat de Catalunya (2017-SGR-1414), and by the European Mont-Blanc projects (288777, 610402 and 671697).Peer ReviewedPreprin

The OTree: multidimensional indexing with efficient data sampling for HPC

Spatial big data is considered an essential trend in future scientific and business applications. Indeed, research instruments, medical devices, and social networks generate hundreds of petabytes of spatial data per year. However, many authors have pointed out that the lack of specialized frameworks for multidimensional Big Data is limiting possible applications and precluding many scientific breakthroughs. Paramount in achieving High-Performance Data Analytics is to optimize and reduce the I/O operations required to analyze large data sets. To do so, we need to organize and index the data according to its multidimensional attributes. At the same time, to enable fast and interactive exploratory analysis, it is vital to generate approximate representations of large datasets efficiently. In this paper, we propose the Outlook Tree (or OTree), a novel Multidimensional Indexing with efficient data Sampling (MIS) algorithm. The OTree enables exploratory analysis of large multidimensional datasets with arbitrary precision, a vital missing feature in current distributed data management solutions. Our algorithm reduces the indexing overhead and achieves high performance even for write-intensive HPC applications. Indeed, we use the OTree to store the scientific results of a study on the efficiency of drug inhalers. Then we compare the OTree implementation on Apache Cassandra, named Qbeast, with PostgreSQL and plain storage. Lastly, we demonstrate that our proposal delivers better performance and scalability.Peer ReviewedPostprint (author's final draft

HERMESH : a geometrical domain composition method in computational mechanics

With this thesis we present the HERMESH method which has been classified by us as a a composition domain method. This term comes from the idea that HERMESH obtains a global solution of the problem from two independent meshes as a result of the mesh coupling. The global mesh maintains the same number of degrees of freedom as the sum of the independent meshes, which are coupled in the interfaces via new elements referred to by us as extension elements. For this reason we enunciate that the domain composition method is geometrical. The result of the global mesh is a non-conforming mesh in the interfaces between independent meshes due to these new connectivities formed with existing nodes and represented by the new extension elements. The first requirements were that the method be implicit, be valid for any partial differential equation and not imply any additional effort or loss in efficiency in the parallel performance of the code in which the method has been implemented. In our opinion, these properties constitute the main contribution in mesh coupling for the computational mechanics framework. From these requirements, we have been able to develop an automatic and topology-independent tool to compose independent meshes. The method can couple overlapping meshes with minimal intervention on the user's part. The overlapping can be partial or complete in the sense of overset meshes. The meshes can be disjoint with or without a gap between them. And we have demonstrated the flexibility of the method in the relative mesh size. In this work we present a detailed description of HERMESH which has been implemented in a high-performance computing computational mechanics code within the framework of the finite element methods. This code is called Alya. The numerical properties will be proved with different benchmark-type problems and the manufactured solution technique. Finally, the results in complex problems solved with HERMESH will be presented, clearly showing the versatility of the method.En este trabajo presentamos el metodo HERMESH al que hemos catalogado como un método de composición de dominios puesto que a partir de mallas independientes se obtiene una solución global del problema como la unión de los subproblemas que forman las mallas independientes. Como resultado, la malla global mantiene el mismo número de grados de libertad que la suma de los grados de libertad de las mallas independientes, las cuales se acoplan en las interfases internas a través de nuevos elementos a los que nos referimos como elementos de extensión. Por este motivo decimos que el método de composición de dominio es geométrico. El resultado de la malla global es una malla que no es conforme en las interfases entre las distintas mallas debido a las nuevas conectividades generadas sobre los nodos existentes. Los requerimientos de partida fueron que el método se implemente de forma implícita, sea válido para cualquier PDE y no implique ningún esfuerzo adicional ni perdida de eficiencia para el funcionamiento paralelo del código de altas prestaciones en el que ha sido implementado. Creemos que estas propiedades son las principales aportaciones de esta tesis dentro del marco de acoplamiento de mallas en mecánica computacional. A partir de estas premisas, hemos conseguido una herramienta automática e independiente de la topología para componer mallas. Es capaz de acoplar sin necesidad de intervención del usuario, mallas con solapamiento parcial o total así como mallas disjuntas con o sin "gap" entre ellas. También hemos visto que ofrece cierta flexibilidad en relación al tamaños relativos entre las mallas siendo un método válido como técnica de remallado local. Presentamos una descripción detallada de la implementación de esta técnica, llevada a cabo en un código de altas prestaciones de mecánica computacional en el contexto de elementos finitos, Alya. Se demostrarán todas las propiedades numéricas que ofrece el métodos a través de distintos problemas tipo benchmark y el método de la solución manufacturada. Finalmente se mostrarán los resultados en problemas complejos resueltos con el método HERMESH, que a su vez es una prueba de la gran flexibilidad que nos brinda

Performance Assessment of an Electrostatic Filter-Diverter Stent Cerebrovascular Protection Device: Evaluation of a Range of Potential Electrostatic Fields Focusing on Small Particles

Silent Brain Infarction (SBI) is increasingly recognized in patients with cardiac conditions, particularly Atrial Fibrillation (AF) in elderly patients and those undergoing Transcatheter Aortic Valve Implantation (TAVI). While these infarcts often go unnoticed due to a lack of acute symptoms, they are associated with a threefold increase in stroke risk and are considered a precursor to ischemic stroke. Moreover, accumulating evidence suggests that SBI may contribute to the development of dementia, depression, and cognitive decline, particularly in the elderly population. The burden of SBI is substantial, with studies showing that up to 11 million Americans may experience a silent stroke annually. In AF patients, silent brain infarcts are common and can lead to progressive brain damage, even in those receiving anticoagulation therapy. The use of cerebral embolic protection devices (CEPDs) during TAVI has been explored to mitigate the risk of stroke; however, their efficacy remains under debate. Despite advancements in TAVI technology, cerebrovascular events, including silent brain lesions, continue to pose significant challenges, underscoring the need for improved preventive strategies and therapeutic approaches. We propose a device consisting of a strut structure placed at the base of the treated artery to model the potential risk of cerebral embolisms caused by atrial fibrillation, thromboembolism, or dislodged debris of varying potential TAVI patients. The study has been carried out in two stages. Both are based on computational fluid dynamics (CFD) coupled with the Lagrangian particle tracking method. The first stage of the work evaluates a variety of strut thicknesses and inter-strut spacings, contrasting with the device-free baseline geometry. The analysis is carried out by imposing flow rate waveforms characteristic of healthy and AF patients. Boundary conditions are calibrated to reproduce physiological flow rates and pressures in a patient’s aortic arch. In the second stage, the optimal geometric design from the first stage was employed, with the addition of lateral struts to prevent the filtration of particles and electronegatively charged strut surfaces, studying the effect of electrical forces on the clots if they are considered charged. Flowrate boundary conditions were used to emulate both healthy and AF conditions. Results from numerical simulations coming from the first stage indicate that the device blocks particles of sizes larger than the inter-strut spacing. It was found that lateral strut space had the highest impact on efficacy. Based on the results of the second stage, deploying the electronegatively charged device in all three aortic arch arteries, the number of particles entering these arteries was reduced on average by 62.6 % and 51.2 %, for the healthy and diseased models respectively, matching or surpassing current oral anticoagulant efficacy. In conclusion, the device demonstrated a two-fold mechanism for filtering emboli: (1) while the smallest particles are deflected by electrostatic repulsion, avoiding micro embolisms, which could lead to cognitive impairment, the largest ones are mechanically filtered since they cannot fit in between the struts, effectively blocking the full range of particle sizes analyzed in this study. The device presented in this manuscript offers an anticoagulant-free method to prevent stroke and SBIs, imperative given the growing population of AF and elderly patients.This work was partially financed by the European Project “An ecosystem for digital twins in healthcare (EDITH)” under the call: DIGITAL-2021-DEPLOY-01-TWINS-HEALTH, Ref: 101083771. The research has been also supported by the European Union’s Horizon 2020 research and innovation program under the Marie Sklodowska-Curie grant agreement No. 713673, by the project CompBioMed2 (H2020-EU.1.4.1.3. Grant No. 823712), by CREXDATA project (HORIZON-CL4-2022-DATA-01. Grant No. 101092749), and by the EIC Accelerator grant “ELVIS—ELEM Virtual Heart Populations for Supercomputers” Grant ID 190134524.Peer ReviewedPostprint (published version

Flow features and micro-particle deposition in a human respiratory system during sniffing

As we inhale, the air drawn through our nose undergoes successive accelerations and decelerations as it is turned, split and recombined before splitting again at the end of the trachea as it enters the bronchi. Fully describing the dynamic behaviour of the airflow and how it transports inhaled particles poses a severe challenge to computational simulations. In this paper we explore two aspects: the dynamic behaviour of airflow during a rapid inhalation (or sniff) and the transport of inhaled aerosols. The development of flow unsteadiness from a laminar state at entry to the nose through to the turbulent character of tracheal flow is resolved using accurate numerical models with high performance computing-based large scale simulations. Combining the flow solution with a Lagrangian computation reveals the effects of flow behaviour and airway geometry on the deposition of inhaled microparticles. Improved modelling of airflow and delivery of therapeutic aerosols could be applied to improve diagnosis and treatment

Design and execution of a verification, validation, and uncertainty quantification plan for a numerical model of left ventricular flow after LVAD implantation

BACKGROUND: Left ventricular assist devices (LVADs) are implantable pumps that act as a life support therapy for patients with severe heart failure. Despite improving the survival rate, LVAD therapy can carry major complications. Particularly, the flow distortion introduced by the LVAD in the left ventricle (LV) may induce thrombus formation. While previous works have used numerical models to study the impact of multiple variables in the intra-LV stagnation regions, a comprehensive validation analysis has never been executed. The main goal of this work is to present a model of the LV-LVAD system and to design and follow a verification, validation and uncertainty quantification (VVUQ) plan based on the ASME V&V40 and V&V20 standards to ensure credible predictions. METHODS: The experiment used to validate the simulation is the SDSU cardiac simulator, a bench mock-up of the cardiovascular system that allows mimicking multiple operation conditions for the heart-LVAD system. The numerical model is based on Alya, the BSC’s in-house platform for numerical modelling. Alya solves the Navier-Stokes equation with an Arbitrary Lagrangian-Eulerian (ALE) formulation in a deformable ventricle and includes pressure-driven valves, a 0D Windkessel model for the arterial output and a LVAD boundary condition modeled through a dynamic pressure-flow performance curve. The designed VVUQ plan involves: (a) a risk analysis and the associated credibility goals; (b) a verification stage to ensure correctness in the numerical solution procedure; (c) a sensitivity analysis to quantify the impact of the inputs on the four quantities of interest (QoIs) (average aortic root flow , maximum aortic root flow , average LVAD flow , and maximum LVAD flow ); (d) an uncertainty quantification using six validation experiments that include extreme operating conditions. RESULTS: Numerical code verification tests ensured correctness of the solution procedure and numerical calculation verification showed a grid convergence index (GCI)95% <3.3%. The total Sobol indices obtained during the sensitivity analysis demonstrated that the ejection fraction, the heart rate, and the pump performance curve coefficients are the most impactful inputs for the analysed QoIs. The Minkowski norm is used as validation metric for the uncertainty quantification. It shows that the midpoint cases have more accurate results when compared to the extreme cases. The total computational cost of the simulations was above 100 [core-years] executed in around three weeks time span in Marenostrum IV supercomputer. Conclusions This work details a novel numerical model for the LV-LVAD system, that is supported by the design and execution of a VVUQ plan created following recognised international standards. We present a methodology demonstrating that stringent VVUQ according to ASME standards is feasible but computationally expensive.This project was funded in part by the FDA Critical Path Initiative and by an appointment to the Research Participation Program at the Division of Biomedical Physics, Office of Science and Engineering Laboratories, Center for Devices and Radiological Health, U.S. Food and Drug Administration, administered by the Oak Ridge Institute for Science, and Education through an interagency agreement between the U.S. Department of Energy and FDA to RAG. MV and AS acknowledge the funding from the project CompBioMed2 (H2020-EU.1.4.1.3. Grant number: 823712), SilicoFCM (H2020-EU.3.1.5. Grant number: 777204), and NEOTEC 2019 - "Generador de Corazones Virtuales" (“Ministerio de Economía y competititvidad”, EXP - 00123159 / SNEO-20191113). AS salary is partially funded by the “Ministerio de Economía y competititvidad” under the Torres Quevedo Program (grant number: PTQ2019-010528). CB salary is partially funded by the Torres Quevedo Program (grant number: PTQ2018-010290). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.Peer ReviewedPostprint (published version

Alya: Computational Solid Mechanics for Supercomputers

While solid mechanics codes are now conventional tools both in industry and research, the increasingly more exigent requirements of both sectors are fuelling the need for more computational power and more advanced algorithms. For obvious reasons, commercial codes are lagging behind academic codes often dedicated either to the implementation of one new technique, or the upscaling of current conventional codes to tackle massively large scale computational problems. Only in a few cases, both approaches have been followed simultaneously. In this article, a solid mechanics simulation strategy for parallel supercomputers based on a hybrid approach is presented. Hybrid parallelization exploits the thread-level parallelism of multicore architectures, combining MPI tasks with OpenMP threads. This paper describes the proposed strategy, programmed in Alya, a parallel multi-physics code. Hybrid parallelization is specially well suited for the current trend of supercomputers, namely large clusters of multicores. The strategy is assessed through transient non-linear solid mechanics problems, both for explicit and implicit schemes, running on thousands of cores. In order to demonstrate the flexibility of the proposed strategy under advance algorithmic evolution of computational mechanics, a non-local parallel overset meshes method (Chimera-like) is implemented and the conservation of the scalability is demonstrated.D.D.T and A.J acknowledge funding through SIMUCOMP and ERA-NET MATERA+ project financed by the Consejería de Educación y Empleo of the Comunidad de Madrid and by the European Union’s Seventh Framework Programme (FP7/2007-2013). Thisworkwas partially supported by the grant SEV-2011-00067, Severo Ochoa Program, awarded by the Spanish Government. The authors’ would like to acknowledge PRACE infrastructure support.Peer ReviewedPostprint (published version

1st BSC Doctoral Symposium : book of abstracts

Peer Reviewe

An exploration of the taken for granted world of the cancer nurse in relation to stress and the person witn cancer

SIGLEAvailable from British Library Document Supply Centre- DSC:DX171316 / BLDSC - British Library Document Supply CentreGBUnited Kingdo