Accelerating Cardiac Bidomain Simulations Using Graphics Processing Units

Anatomically realistic and biophysically detailed multiscale computer models of the heart are playing an increasingly important role in advancing our understanding of integrated cardiac function in health and disease. Such detailed simulations, however, are computationally vastly demanding, which is a limiting factor for a wider adoption of in-silico modeling. While current trends in high-performance computing (HPC) hardware promise to alleviate this problem, exploiting the potential of such architectures remains challenging since strongly scalable algorithms are necessitated to reduce execution times. Alternatively, acceleration technologies such as graphics processing units (GPUs) are being considered. While the potential of GPUs has been demonstrated in various applications, benefits in the context of bidomain simulations where large sparse linear systems have to be solved in parallel with advanced numerical techniques are less clear. In this study, the feasibility of multi-GPU bidomain simulations is demonstrated by running strong scalability benchmarks using a state-of-the-art model of rabbit ventricles. The model is spatially discretized using the finite element methods (FEM) on fully unstructured grids. The GPU code is directly derived from a large pre-existing code, the Cardiac Arrhythmia Research Package (CARP), with very minor perturbation of the code base. Overall, bidomain simulations were sped up by a factor of 11.8 to 16.3 in benchmarks running on 6-20 GPUs compared to the same number of CPU cores. To match the fastest GPU simulation which engaged 20 GPUs, 476 CPU cores were required on a national supercomputing facility

Modeling and simulation of the electric activity of the heart using graphic processing units

Mathematical modelling and simulation of the electric activity of the heart (cardiac electrophysiology) offers and ideal framework to combine clinical and experimental data in order to help understanding the underlying mechanisms behind the observed respond under physiological and pathological conditions. In this regard, solving the electric activity of the heart possess a big challenge, not only because of the structural complexities inherent to the heart tissue, but also because of the complex electric behaviour of the cardiac cells. The multi- scale nature of the electrophysiology problem makes difficult its numerical solution, requiring temporal and spatial resolutions of 0.1 ms and 0.2 mm respectively for accurate simulations, leading to models with millions degrees of freedom that need to be solved for thousand time steps. Solution of this problem requires the use of algorithms with higher level of parallelism in multi-core platforms. In this regard the newer programmable graphic processing units (GPU) has become a valid alternative due to their tremendous computational horsepower. This thesis develops around the implementation of an electrophysiology simulation software entirely developed in Compute Unified Device Architecture (CUDA) for GPU computing. The software implements fully explicit and semi-implicit solvers for the monodomain model, using operator splitting and the finite element method for space discretization. Performance is compared with classical multi-core MPI based solvers operating on dedicated high-performance computer clusters. Results obtained with the GPU based solver show enormous potential for this technology with accelerations over 50× for three-dimensional problems when using an implicit scheme for the parabolic equation, whereas accelerations reach values up to 100× for the explicit implementation. The implemented solver has been applied to study pro-arrhythmic mechanisms during acute ischemia. In particular, we investigate on how hyperkalemia affects the vulnerability window to reentry and the reentry patterns in the heterogeneous substrate caused by acute regional ischemia using an anatomically and biophysically detailed human biventricular model. A three dimensional geometrically and anatomically accurate regionally ischemic human heart model was created. The ischemic region was located in the inferolateral and posterior side of the left ventricle mimicking the occlusion of the circumflex artery, and the presence of a washed-out zone not affected by ischemia at the endocardium has been incorporated. Realistic heterogeneity and fi er anisotropy has also been considered in the model. A highly electrophysiological detailed action potential model for human has been adapted to make it suitable for modeling ischemic conditions (hyperkalemia, hipoxia, and acidic conditions) by introducing a formulation of the ATP-sensitive K+ current. The model predicts the generation of sustained re-entrant activity in the form single and double circus around a blocked area within the ischemic zone for K+ concentrations bellow 9mM, with the reentrant activity associated with ventricular tachycardia in all cases. Results suggest the washed-out zone as a potential pro-arrhythmic substrate factor helping on establishing sustained ventricular tachycardia.Colli-Franzone P, Pavarino L. A parallel solver for reaction-diffusion systems in computational electrocardiology, Math. Models Methods Appl. Sci. 14 (06):883-911, 2004.Colli-Franzone P, Deu hard P, Erdmann B, Lang J, Pavarino L F. Adaptivity in space and time for reaction-diffusion systems in electrocardiology, SIAM J. Sci. Comput. 28 (3):942-962, 2006.Ferrero J M(Jr), Saiz J, Ferrero J M, Thakor N V. Simulation of action potentials from metabolically impaired cardiac myocytes: Role of atp-sensitive K+ current. Circ Res, 79(2):208-221, 1996.Ferrero J M (Jr), Trenor B. Rodriguez B, Saiz J. Electrical acticvity and reentry during acute regional myocardial ischemia: Insights from simulations.Int J Bif Chaos, 13:3703-3715, 2003.Heidenreich E, Ferrero J M, Doblare M, Rodriguez J F. Adaptive macro finite elements for the numerical solution of monodomain equations in cardiac electrophysiology, Ann. Biomed. Eng. 38 (7):2331-2345, 2010.Janse M J, Kleber A G. Electrophysiological changes and ventricular arrhythmias in the early phase of regional myocardial ischemia. Circ. Res. 49:1069-1081, 1981.ten Tusscher K HWJ, Panlov A V. Alternans and spiral breakup in a human ventricular tissue model. Am. J.Physiol. Heart Circ. Physiol. 291(3):1088-1100, 2006.<br /

Adaptive step ODE algorithms for the 3D simulation of electric heart activity with graphics processing units

In this paper we studied the implementation and performance of adaptive step methods for large systems of ordinary differential equations systems in graphics processing units, focusing on the simulation of three-dimensional electric cardiac activity. The Rush-Larsen method was applied in all the implemented solvers to improve efficiency. We compared the adaptive methods with the fixed step methods, and we found that the fixed step methods can be faster while the adaptive step methods are better in terms of accuracy and robustness. (c) 2013 Elsevier Ltd. All rights reserved.This work has been partially funded by Universitat Politecnica de Valencia through Programa de Apoyo a la InvestigaciOn y Desarrollo (PAID-06-11) and (PAID-05-12), by Generalitat Valenciana through projects PROMETEO/2009/013 and Ayudas para la realizacion de proyectos de I+D para grupos de investigacion emergentes GV/2012/039, and by Ministerio Espafiol de Economia y Competitividad through project TEC2012-38142-004.García Mollá, VM.; Liberos Mascarell, A.; Vidal Maciá, AM.; Guillem Sánchez, MS.; Millet Roig, J.; González Salvador, A.; Martínez Zaldívar, FJ.... (2014). Adaptive step ODE algorithms for the 3D simulation of electric heart activity with graphics processing units. Computers in Biology and Medicine. 44:15-26. https://doi.org/10.1016/j.compbiomed.2013.10.023S15264

Using delay differential equations in models of cardiac electrophysiology

In cardiac physiology, electrical alternans is a phenomenon characterized by long-short alternations in the action potential duration of cardiac myocytes that give rise to complex spatiotemporal dynamics in tissue. Experiments and clinical measurements indicate that alternans can be a precursor of life-threatening arrhythmias, such as cardiac _brillation. Despite the importance of alternans in the study of cardiac disease, many mathematical models developed to describe cardiac electrophysiology at the cellular level are not able to produce this phenomenon. As a potential remedy to this de_ciency, we introduce short time-delays in some formulations of existing cardiac cell models that are based on Ordinary Di_erential Equations (ODEs). Many processes within cardiac cells involve delays in sensing and responding to changes. In addition, delay di_erential equations (DDEs) are known to give rise to complex dynamical properties in mathematical models. In biological modeling, DDEs have been applied to epidemiology, population dynamics, immunology, and neural networks. Therefore, DDEs can potentially represent mechanisms that result in complex dynamics both at the cellular level and at the tissue level. In this thesis, we propose DDE-based formulations for ion channel models based on the Hodgkin-Huxley formalism that can induce alternans in single-cell simulations in many models found in the literature. We also show that these modi_cations can destabilize spiral waves and produce spiral breakups in two-dimensional simulations, which is a typical model of cardiac _brillation. However, the new DDE-based formulations introduce new computational challenges due to the need for storing and retrieving past values of variables. Therefore, we present novel numerical methods to overcome these challenges and enable e_cient DDE-based studies at the tissue level in standard computational environments. We _nd that the proposed methods decrease memory usage by up to 95% in cardiac tissue simulations compared to straightforward history management algorithms available in widely used DDE solvers.Em fisiologia cardíaca, alternans elétrica _e um fenômeno caracterizado pela alternância entre potenciais de ação longos e curtos que dá origem a complexos comportamentos espaço-temporais em tecido. Experimentos e medições clínicas indicam que alternans pode ser um precursor de perigosas arritmias, como fibrilação ventricular ou morte súbita. Apesar da importância do alternans no estudo de doenças cardíacas, muitos modelos matemáticos para a eletrofisiologia de células cardíacas não são capazes de reproduzir este fenômeno. Como um potencial remédio para esta deficiência, introduzimos curtos atrasos de tempo em algumas formulações de modelos preexistentes para células cardíacas que são baseados em Equações Diferenciais Ordinárias (EDOs). Vários processos em células cardíacas envolvem atrasos de sensibilidade e de resposta a mudanças em variáveis fisiológicas. Além disso, equações diferenciais com atraso (DDEs) são conhecidas por dar origem a complexas propriedades dinâmicas em modelos matemáticos. Em modelagem biológica, DDEs têm sido aplicadas em epidemiologia, dinâmica populacional, imunologia e redes neurais. Portanto, DDEs podem representar mecanismos que resultam em dinâmicas complexas tanto no nível celular, quanto no nível do tecido. Nesta tese, propomos formulações baseadas em DDEs para modelos de canais iônicos descritos pelo formalismo de Hodgkin-Huxley. Tais formulações são capazes de induzir alternans em simulações celulares envolvendo vários modelos encontrados na literatura. Nós também mostramos que essas modificações podem desestabilizar e quebrar ondas espirais em simulações bidimensionais de propagação elétrica, o que é típico de fibrilação cardíaca. Entretanto, as formulações propostas introduzem novos desafios computacionais devido à necessidade de armazenar e recuperar valores passados de variáveis. Deste modo, nós apresentamos novos métodos numéricos para superar tais desafios e permitir a eficiente simulação de modelos baseados em DDEs no nível do tecido cardíaco. Os métodos propostos foram capazes de diminuir o uso de memória em até 95% em comparação aos algoritmos largamente utilizados na solução numérica de DDEs. Assim, os novos modelos baseados em DDEs e os eficientes métodos numéricos propostos nesta tese contribuem para o estudo de arritmias cardíacas fatais através de modelagem computacional

Scalable and Accurate ECG Simulation for Reaction-Diffusion Models of the Human Heart

International audienceRealistic electrocardiogram (ECG) simulation with numerical models is important for research linking cellular and molecular physiology to clinically observable signals, and crucial for patient tailoring of numerical heart models. However, ECG simulation with a realistic torso model is computationally much harder than simulation of cardiac activity itself, so that many studies with sophisticated heart models have resorted to crude approximations of the ECG. This paper shows how the classical concept of electrocardiographic lead fields can be used for an ECG simulation method that matches the realism of modern heart models. The accuracy and resource requirements were compared to those of a full-torso solution for the potential and scaling was tested up to 14,336 cores with a heart model consisting of 11 million nodes. Reference ECGs were computed on a 3.3 billion-node heart-torso mesh at 0.2 mm resolution. The results show that the lead-field method is more efficient than a full-torso solution when the number of simulated samples is larger than the number of computed ECG leads. While the initial computation of the lead fields remains a hard and poorly scalable problem, the ECG computation itself scales almost perfectly and, even for several hundreds of ECG leads, takes much less time than the underlying simulation of cardiac activity

A domain decomposition strategy for a very high-order finite volumes scheme applied to cardiac electrophysiology

International audienceIn this paper, a domain decomposition technique for a very high-order finite volumes scheme is proposed. The objective is to obtain an efficient way to perform numerical simulations in cardiac electrophysiology. The aim is to extend a very high-order numerical scheme previously designed, where large stencils are used for polynomial reconstructions. Therefore, a particular attention has to be paid to maintain the scalability in parallel. Here, we propose to constrain the stencils inside the subdomains or their first layer of neighbors. The method is shown to remain accurate and to scale perfectly up to the level where there are not enough cells in the subdomains. Hence, these high-order schemes are proved to be efficient tools to perform realistic simulations in cardiac electrophysiology