Analyzing and Modeling the Performance of the HemeLB Lattice-Boltzmann Simulation Environment

We investigate the performance of the HemeLB lattice-Boltzmann simulator for cerebrovascular blood flow, aimed at providing timely and clinically relevant assistance to neurosurgeons. HemeLB is optimised for sparse geometries, supports interactive use, and scales well to 32,768 cores for problems with ~81 million lattice sites. We obtain a maximum performance of 29.5 billion site updates per second, with only an 11% slowdown for highly sparse problems (5% fluid fraction). We present steering and visualisation performance measurements and provide a model which allows users to predict the performance, thereby determining how to run simulations with maximum accuracy within time constraints.Comment: Accepted by the Journal of Computational Science. 33 pages, 16 figures, 7 table

Towards Blood Flow in the Virtual Human: Efficient Self-Coupling of HemeLB

Many scientific and medical researchers are working towards the creation of a virtual human - a personalised digital copy of an individual - that will assist in a patient's diagnosis, treatment and recovery. The complex nature of living systems means that the development of this remains a major challenge. We describe progress in enabling the HemeLB lattice Boltzmann code to simulate 3D macroscopic blood flow on a full human scale. Significant developments in memory management and load balancing allow near linear scaling performance of the code on hundreds of thousands of computer cores. Integral to the construction of a virtual human, we also outline the implementation of a self-coupling strategy for HemeLB. This allows simultaneous simulation of arterial and venous vascular trees based on human-specific geometries.Comment: 30 pages, 10 figures, To be published in Interface Focus (https://royalsocietypublishing.org/journal/rsfs

Performance of distributed multiscale simulations

Multiscale simulations model phenomena across natural scales using monolithic or component-based code, running on local or distributed resources. In this work, we investigate the performance of distributed multiscale computing of component-based models, guided by six multiscale applications with different characteristics and from several disciplines. Three modes of distributed multiscale computing are identified: supplementing local dependencies with large-scale resources, load distribution over multiple resources, and load balancing of small- and large-scale resources. We find that the first mode has the apparent benefit of increasing simulation speed, and the second mode can increase simulation speed if local resources are limited. Depending on resource reservation and model coupling topology, the third mode may result in a reduction of resource consumption

High fidelity blood flow in a patient-specific arteriovenous fistula

An arteriovenous fistula, created by artificially connecting segments of a patient's vasculature, is the preferred way to gain access to the bloodstream for kidney dialysis. The increasing power and availability of supercomputing infrastructure means that it is becoming more realistic to use simulations to help identify the best type and location of a fistula for a specific patient. We describe a 3D fistula model that uses the lattice Boltzmann method to simultaneously resolve blood flow in patient-specific arteries and veins. The simulations conducted here, comprising vasculatures of the whole forearm, demonstrate qualified validation against clinical data. Ongoing research to further encompass complex biophysics on realistic time scales will permit the use of human-scale physiological models for basic and clinical medicine

HEMELB Acceleration and Visualization for Cerebral Aneurysms

A weakness in the wall of a cerebral artery causing a dilation or ballooning of the blood vessel is known as a cerebral aneurysm. Optimal treatment requires fast and accurate diagnosis of the aneurysm. HemeLB is a fluid dynamics solver for complex geometries developed to provide neurosurgeons with information related to the flow of blood in and around aneurysms. On a cost efficient platform, HemeLB could be employed in hospitals to provide surgeons with the simulation results in real-time. In this work, we developed an improved version of HemeLB for GPU implementation and result visualization. A visualization platform for smooth interaction with end users is also presented. Finally, a comprehensive evaluation of this implementation is reported. The results demonstrate that the proposed implementation achieves a maximum performance of 15,168,964 site updates per second, and is capable of speeding up HemeLB for deployment in hospitals and clinical investigations

Lattice-Boltzmann interactive blood flow simulation pipeline.

PURPOSE:Cerebral aneurysms are one of the prevalent cerebrovascular disorders in adults worldwide and caused by a weakness in the brain artery. The most impressive treatment for a brain aneurysm is interventional radiology treatment, which is extremely dependent on the skill level of the radiologist. Hence, accurate detection and effective therapy for cerebral aneurysms still remain important clinical challenges. In this work, we have introduced a pipeline for cerebral blood flow simulation and real-time visualization incorporating all aspects from medical image acquisition to real-time visualization and steering. METHODS:We have developed and employed an improved version of HemeLB as the main computational core of the pipeline. HemeLB is a massive parallel lattice-Boltzmann fluid solver optimized for sparse and complex geometries. The visualization component of this pipeline is based on the ray marching method implemented on CUDA capable GPU cores. RESULTS:The proposed visualization engine is evaluated comprehensively and the reported results demonstrate that it achieves significantly higher scalability and sites updates per second, indicating higher update rate of geometry sites' values, in comparison with the original HemeLB. This proposed engine is more than two times faster and capable of 3D visualization of the results by processing more than 30 frames per second. CONCLUSION:A reliable modeling and visualizing environment for measuring and displaying blood flow patterns in vivo, which can provide insight into the hemodynamic characteristics of cerebral aneurysms, is presented in this work. This pipeline increases the speed of visualization and maximizes the performance of the processing units to do the tasks by breaking them into smaller tasks and working with GPU to render the images. Hence, the proposed pipeline can be applied as part of clinical routines to provide the clinicians with the real-time cerebral blood flow-related information

Zynq SoC based acceleration of the lattice Boltzmann method

Cerebral aneurysm is a life‐threatening condition. It is a weakness in a blood vessel that may enlarge and bleed into the surrounding area. In order to understand the surrounding environmental conditions during the interventions or surgical procedures, a simulation of blood flow in cerebral arteries is needed. One of the effective simulation approaches is to use the lattice Boltzmann (LB) method. Due to the computational complexity of the algorithm, the simulation is usually performed on high performance computers. In this paper, efficient hardware architectures of the LB method on a Zynq system‐on‐chip (SoC) are designed and implemented. The proposed architectures have first been simulated in Vivado HLS environment and later implemented on a ZedBoard using the software‐defined SoC (SDSoC) development environment. In addition, a set of evaluations of different hardware architectures of the LB implementation is discussed in this paper. The experimental results show that the proposed implementation is able to accelerate the processing speed by a factor of 52 compared to a dual‐core ARM processor‐based software implementation

Heterogeneous System-on-Chip based Lattice-Boltzmann Visual Simulation System

Cerebral aneurysm is a cerebrovascular disorder caused by a weakness in the wall of an artery or vein, that causes a localised dilation or ballooning of the blood vessel. It is life-threatening, hence an early and accurate diagnosis would be a great aid to medical professionals in making the correct choice of treatment. HemeLB is a massively parallel lattice-Boltzmann simulation software which is designed to provide the radiologist with estimates of flow rates, pressures and shear stresses throughout the relevant vascular structures, intended to eventually permit greater precision in the choice of therapeutic intervention. However, in order to allow surgeries and doctors to view and visualise the results in real-time at medical environments, a cost-efficient, practical platform is needed. In this paper, we have developed and evaluated a version of HemeLB on various heterogeneous system-on-chip platforms, allowing users to run HemeLB on a low cost embedded platform and to visualise the simulation results in real-time. A comprehensive evaluation of implementation on the Zynq SoC and Jetson TX1 embedded graphic processing unit platforms are reported. The achieved results show that the proposed Jetson TX1 implementation outperforms the Zynq implementation by a factor of 19 in terms of site updates per second

Parametric analysis of an efficient boundary condition to control outlet flow rates in large arterial networks

Substantial effort is being invested in the creation of a virtual human-a model which will improve our understanding of human physiology and diseases and assist clinicians in the design of personalised medical treatments. A central challenge of achieving blood flow simulations at full-human scale is the development of an efficient and accurate approach to imposing boundary conditions on many outlets. A previous study proposed an efficient method for implementing the two-element Windkessel model to control the flow rate ratios at outlets. Here we clarify the general role of the resistance and capacitance in this approach and conduct a parametric sweep to examine how to choose their values for complex geometries. We show that the error of the flow rate ratios decreases exponentially as the resistance increases. The errors fall below 4% in a simple five-outlets model and 7% in a human artery model comprising ten outlets. Moreover, the flow rate ratios converge faster and suffer from weaker fluctuations as the capacitance decreases. Our findings also establish constraints on the parameters controlling the numerical stability of the simulations. The findings from this work are directly applicable to larger and more complex vascular domains encountered at full-human scale