High-performance tsunami modelling with modern GPU technology

PhD ThesisEarthquake-induced tsunamis commonly propagate in the deep ocean as long waves and develop into sharp-fronted surges moving rapidly coastward, which may be effectively simulated by hydrodynamic models solving the nonlinear shallow water equations (SWEs). Tsunamis can cause substantial economic and human losses, which could be mitigated through early warning systems given efficient and accurate modelling. Most existing tsunami models require long simulation times for real-world applications. This thesis presents a graphics processing unit (GPU) accelerated finite volume hydrodynamic model using the compute unified device architecture (CUDA) for computationally efficient tsunami simulations. Compared with a standard PC, the model is able to reduce run-time by a factor of > 40. The validated model is used to reproduce the 2011 Japan tsunami. Two source models were tested, one based on tsunami waveform inversion and another using deep-ocean tsunameters. Vertical sea surface displacement is computed by the Okada model, assuming instantaneous sea-floor deformation. Both source models can reproduce the wave propagation at offshore and nearshore gauges, but the tsunameter-based model better simulates the first wave amplitude. Effects of grid resolutions between 450-3600 m, slope limiters, and numerical accuracy are also investigated for the simulation of the 2011 Japan tsunami. Grid resolutions of 1-2 km perform well with a proper limiter; the Sweby limiter is optimal for coarser resolutions, recovers wave peaks better than minmod, and is more numerically stable than Superbee. One hour of tsunami propagation can be predicted in 50 times on a regular low-cost PC-hosted GPU, compared to a single CPU. For 450 m resolution on a larger-memory server-hosted GPU, performance increased by ~70 times. Finally, two adaptive mesh refinement (AMR) techniques including simplified dynamic adaptive grids on CPU and a static adaptive grid on GPU are introduced to provide multi-scale simulations. Both can reduce run-time by ~3 times while maintaining acceptable accuracy. The proposed computationally-efficient tsunami model is expected to provide a new practical tool for tsunami modelling for different purposes, including real-time warning, evacuation planning, risk management and city planning

High Resolution 3D Ultrasonic Breast Imaging by Time-Domain Full Waveform Inversion

Ultrasound tomography (UST) scanners allow quantitative images of the human breast's acoustic properties to be derived with potential applications in screening, diagnosis and therapy planning. Time domain full waveform inversion (TD-FWI) is a promising UST image formation technique that fits the parameter fields of a wave physics model by gradient-based optimization. For high resolution 3D UST, it holds three key challenges: Firstly, its central building block, the computation of the gradient for a single US measurement, has a restrictively large memory footprint. Secondly, this building block needs to be computed for each of the $10^3-10^4$ measurements, resulting in a massive parallel computation usually performed on large computational clusters for days. Lastly, the structure of the underlying optimization problem may result in slow progression of the solver and convergence to a local minimum. In this work, we design and evaluate a comprehensive computational strategy to overcome these challenges: Firstly, we introduce a novel gradient computation based on time reversal that dramatically reduces the memory footprint at the expense of one additional wave simulation per source. Secondly, we break the dependence on the number of measurements by using source encoding (SE) to compute stochastic gradient estimates. Also we describe a more accurate, TD-specific SE technique with a finer variance control and use a state-of-the-art stochastic LBFGS method. Lastly, we design an efficient TD multi-grid scheme together with preconditioning to speed up the convergence while avoiding local minima. All components are evaluated in extensive numerical proof-of-concept studies simulating a bowl-shaped 3D UST breast scanner prototype. Finally, we demonstrate that their combination allows us to obtain an accurate 442x442x222 voxel image with a resolution of 0.5mm using Matlab on a single GPU within 24h

Experimental validation of k-Wave: Nonlinear wave propagation in layered, absorbing fluid media

Models of ultrasound propagation in biologically relevant media have applications in planning and verification of ultrasound therapies and computational dosimetry. To be effective, the models must be able to accurately predict both the spatial distribution and amplitude of the acoustic pressure. This requires that the models are validated in absolute terms, which for arbitrarily heterogeneous media should be performed by comparison with measurements of the acoustic field. In this study, simulations performed using the open-source k-Wave acoustics toolbox, with a measurement-based source definition, were quantitatively validated against measurements of acoustic pressure in water and layered absorbing fluid media. In water, the measured and simulated spatial peak pressures agreed to within 3% under linear conditions and 7% under non-linear conditions. After propagation through a planar or wedge shaped glycerol-filled phantom, the difference in spatial peak pressure was 8.5% and 10.7%, respectively. These differences are within or close to the expected uncertainty of the acoustic pressure measurement. The -6 dB width and length of the focus agreed to within 4% in all cases, and the focal positions were within 0.7 mm for the planar phantom and 1.2 mm for the wedge shaped phantom. These results demonstrate that when the acoustic medium properties and geometry are well known, accurate quantitative predictions of the acoustic field can be made using k-Wave

MATLAB

This excellent book represents the final part of three-volumes regarding MATLAB-based applications in almost every branch of science. The book consists of 19 excellent, insightful articles and the readers will find the results very useful to their work. In particular, the book consists of three parts, the first one is devoted to mathematical methods in the applied sciences by using MATLAB, the second is devoted to MATLAB applications of general interest and the third one discusses MATLAB for educational purposes. This collection of high quality articles, refers to a large range of professional fields and can be used for science as well as for various educational purposes

Evaluation of the Suitability of Intel Xeon Phi Clusters for the Simulation of Ultrasound Wave Propagation Using Pseudospectral Methods

The ability to perform large-scale ultrasound simulations using Fourier pseudospectral methods has generated significant interest in medical ultrasonics, including for treatment planning in therapeutic ultrasound and image reconstruction in photoacoustic tomography. However, the routine execution of such simulations is computationally very challenging. Nowadays, the trend in parallel computing is towards the use of accelerated clusters where computationally intensive parts are offloaded from processors to accelerators. During last five years, Intel has released two generations of Xeon Phi accelerators. The goal of this paper is to investigate the performance on both architectures with respect to current processors, and evaluate the suitability of accelerated clusters for the distributed simulation of ultrasound propagation using Fourier-based methods. The paper reveals that the former version of Xeon Phis, the Knight’s Corner architecture, suffers from several flaws that reduce the performance far below the Haswell processors. On the other hand, the second generation called Knight’s Landing shows very promising performance comparable with current processors

High resolution 3D ultrasonic breast imaging by time-domain full waveform inversion

Ultrasound tomography (UST) scanners allow quantitative images of the human breast's acoustic properties to be derived with potential applications in screening, diagnosis and therapy planning. Time domain full waveform inversion (TD-FWI) is a promising UST image formation technique that fits the parameter fields of a wave physics model by gradient-based optimization. For high resolution 3D UST, it holds three key challenges: firstly, its central building block, the computation of the gradient for a single US measurement, has a restrictively large memory footprint. Secondly, this building block needs to be computed for each of the 1000 to 10000 measurements, resulting in a massive parallel computation usually performed on large computational clusters for days. Lastly, the structure of the underlying optimization problem may result in slow progression of the solver and convergence to a local minimum. In this work, we design and evaluate a comprehensive computational strategy to overcome these challenges: firstly, we exploit a gradient computation based on time reversal that dramatically reduces the memory footprint at the expense of one additional wave simulation per source. Secondly, we break the dependence on the number of measurements by using source encoding (SE) to compute stochastic gradient estimates. Also we describe a more accurate, TD-specific SE technique with a finer variance control and use a state-of-the-art stochastic LBFGS method. Lastly, we design an efficient TD multi-grid scheme together with preconditioning to speed up the convergence while avoiding local minima. All components are evaluated in extensive numerical proof-of-concept studies simulating a bowl-shaped 3D UST breast scanner prototype. Finally, we demonstrate that their combination allows us to obtain an accurate 442 × 442 × 222 voxel image with a resolution of 0.5 mm using Matlab on a single GPU within 24 h

The Inertial Range of Turbulence in the Inner Heliosheath and in the Local Interstellar Medium

The governing mechanisms of magnetic field annihilation in the outer heliosphere is an intriguing topic. It is currently believed that the turbulent fluctuations pervade the inner heliosheath (IHS) and the Local Interstellar Medium (LISM). Turbulence, magnetic reconnection, or their reciprocal link may be responsible for magnetic energy conversion in the IHS.   As 1-day averaged data are typically used, the present literature mainly concerns large-scale analysis and does not describe inertial-cascade dynamics of turbulence in the IHS. Moreover, lack of spectral analysis make IHS dynamics remain critically understudied. Our group showed that 48-s MAG data from the Voyager mission are appropriate for a power spectral analysis over a frequency range of five decades, from 5e-8 Hz to 1e-2 Hz [Gallana et al., JGR 121 (2016)]. Special spectral estimation techniques are used to deal with the large amount of missing data (70%). We provide the first clear evidence of an inertial-cascade range of turbulence (spectral index is between -2 and -1.5). A spectral break at about 1e-5 Hz is found to separate the inertial range from the enegy-injection range (1/f energy decay). Instrumental noise bounds our investigation to frequencies lower than 5e-4 Hz. By considering several consecutive periods after 2009 at both V1 and V2, we show that the extension and the spectral energy decay of these two regimes may be indicators of IHS regions governed by different physical processes. We describe fluctuations’ regimes in terms of spectral energy density, anisotropy, compressibility, and statistical analysis of intermittency.   In the LISM, it was theorized that pristine interstellar turbulence may coexist with waves from the IHS, however this is still a debated topic. We observe that the fluctuating magnetic energy cascades as a power law with spectral index in the range [-1.35, -1.65] in the whole range of frequencies unaffected by noise. No spectral break is observed, nor decaying turbulence

Roadmap on Electronic Structure Codes in the Exascale Era

Electronic structure calculations have been instrumental in providing many important insights into a range of physical and chemical properties of various molecular and solid-state systems. Their importance to various fields, including materials science, chemical sciences, computational chemistry and device physics, is underscored by the large fraction of available public supercomputing resources devoted to these calculations. As we enter the exascale era, exciting new opportunities to increase simulation numbers, sizes, and accuracies present themselves. In order to realize these promises, the community of electronic structure software developers will however first have to tackle a number of challenges pertaining to the efficient use of new architectures that will rely heavily on massive parallelism and hardware accelerators. This roadmap provides a broad overview of the state-of-the-art in electronic structure calculations and of the various new directions being pursued by the community. It covers 14 electronic structure codes, presenting their current status, their development priorities over the next five years, and their plans towards tackling the challenges and leveraging the opportunities presented by the advent of exascale computing.Comment: Submitted as a roadmap article to Modelling and Simulation in Materials Science and Engineering; Address any correspondence to Vikram Gavini ([email protected]) and Danny Perez ([email protected]