Deep generative models for solving geophysical inverse problems

My thesis presents several novel methods to facilitate solving large-scale inverse problems by utilizing recent advances in machine learning, and particularly deep generative modeling. Inverse problems involve reliably estimating unknown parameters of a physical model from indirect observed data that are noisy. Solving inverse problems presents primarily two challenges. The first challenge is to capture and incorporate prior knowledge into ill-posed inverse problems whose solutions cannot be uniquely identified. The second challenge is the computational complexity of solving inverse problems, particularly the cost of quantifying uncertainty. The main goal of this thesis is to address these issues by developing practical data-driven methods that are scalable to geophysical applications in which access to high-quality training data is often limited. There are six papers included in this thesis. A majority of these papers focus on addressing computational challenges associated with Bayesian inference and uncertainty quantification, while others focus on developing regularization techniques to improve inverse problem solution quality and accelerate the solution process. These papers demonstrate the applicability of the proposed methods to seismic imaging, a large-scale geophysical inverse problem with a computationally expensive forward operator for which sufficiently capturing the variability in the Earth's heterogeneous subsurface through a training dataset is challenging. The first two papers present computationally feasible methods of applying a class of methods commonly referred to as deep priors to seismic imaging and uncertainty quantification. I also present a systematic Bayesian approach to translate uncertainty in seismic imaging to uncertainty in downstream tasks performed on the image. The next two papers aim to address the reliability concerns surrounding data-driven methods for solving Bayesian inverse problems by leveraging variational inference formulations that offer the benefits of fully-learned posteriors while being directly informed by physics and data. The last two papers are concerned with correcting forward modeling errors where the first proposes an adversarially learned postprocessing step to attenuate numerical dispersion artifacts in wave-equation simulations due to coarse finite-difference discretizations, while the second trains a Fourier neural operator surrogate forward model in order to accelerate the qualification of uncertainty due to errors in the forward model parameterization.Ph.D

Feature engineering and symbolic regression methods for detecting hidden physics from sparse sensors

In this study we put forth a modular approach for distilling hidden flow physics in discrete and sparse observations. To address functional expressiblity, a key limitation of the black-box machine learning methods, we have exploited the use of symbolic regression as a principle for identifying relations and operators that are related to the underlying processes. This approach combines evolutionary computation with feature engineering to provide a tool to discover hidden parameterizations embedded in the trajectory of fluid flows in the Eulerian frame of reference. Our approach in this study mainly involves gene expression programming (GEP) and sequential threshold ridge regression (STRidge) algorithms. We demonstrate our results in three different applications: (i) equation discovery, (ii) truncation error analysis, and (iii) hidden physics discovery, for which we include both predicting unknown source terms from a set of sparse observations and discovering subgrid scale closure models. We illustrate that both GEP and STRidge algorithms are able to distill the Smagorinsky model from an array of tailored features in solving the Kraichnan turbulence problem. Our results demonstrate the huge potential of these techniques in complex physics problems, and reveal the importance of feature selection and feature engineering in model discovery approaches

Advanced techniques for subsurface imaging Bayesian neural networks and Marchenko methods

Estimation of material properties such as density and velocity of the Earth’s subsurface are important in resource exploration, waste and CO2 storage and for monitoring changes underground. These properties can be used to create structural images of the subsurface or for resource characterisation. Seismic data are often the main source of information from which these estimates are derived. However the complex nature of the Earth, limitations in data acquisition and in resolution of images, and various types of noise all mean that estimates of material parameters also come with a level of uncertainty. The physics relating these material parameters to recorded seismic data is usually non-linear, necessitating the use of Monte Carlo inversion methods to solve the estimation problem in a fully probabilistic sense. Such methods are computationally expensive which usually prohibits their use over areas with many data, or for subsurface models that involve many parameters. Furthermore multiple unknown material parameters can be jointly dependent on each datum so trade-offs between parameters deteriorate parameter estimates and increase uncertainty in the results. In this thesis various types of neural networks are trained to provide probabilistic estimates of the subsurface velocity structure. A trained network can rapidly invert data in near real- time, much more rapidly than any traditional non-linear sampling method such as Monte Carlo. The thesis also shows how the density estimation problem can be reformulated to avoid direct trade-offs with velocity, by using a combination of seismic interferometry and Marchenko methods. First this thesis shows how neural networks can provide a full probability density function describing the uncertainty in parameters of interest, by using a form of network called a mixture density network. This type of network uses a weighted sum of kernel distributions (in our case Gaussians) to model the Bayesian posterior probability density function. The method is demonstrated by inverting localised phase velocity dispersion curves for shear-wave velocity profiles at the scale of a subsurface fluid reservoir, and is applied to field data from the North Sea. This work shows that when the data contain significant noise, including data uncertainties in the network gives more reliable mean velocity estimates. Whilst the post-training inversion process is rapid using neural networks, the method to estimate localised phase velocities in the first place is significantly slower. Therefore a computationally cheap method is demonstrated that combines gradiometry to estimate phase velocities and mixture density networks to invert for subsurface velocity-depth structure, the whole process taking a matter of minutes. This opens the possibility of real-time monitoring using spatially dense surface seismic arrays. For some monitoring situations a dense array is not available and gradiometry therefore cannot be applied to estimate phase velocities. In a third application this thesis uses mixture density networks to invert travel-time data for 2D localised velocity maps with associated uncertainty estimates. The importance of prior information in high dimensional inverse problems is also demonstrated. A new method is then developed to estimate density in the subsurface using a formulation of seismic interferometry that contains a linear dependence of seismic data on subsurface density, avoiding the usual direct trade-off between density and velocity. When wavefields cannot be measured directly in the subsurface, the method requires the use of a technique called Marchenko redatuming that can estimate the Green’s function from a virtual source or receiver inside a medium to the surface. This thesis shows that critical to implementing this work would be the development of more robust methods to scale the amplitude of Green’s function estimates from Marchenko methods. Finally the limitations of the methods presented in this thesis are discussed, as are suggestions for further research, and alternative applications for some of the methods. Overall this thesis proposes several new ways to monitor the subsurface efficiently using probabilistic machine learning techniques, discusses a novel way to estimate subsurface density, and demonstrates the methods on a mixture of synthetic and field data

Safety and Reliability - Safe Societies in a Changing World

The contributions cover a wide range of methodologies and application areas for safety and reliability that contribute to safe societies in a changing world. These methodologies and applications include: - foundations of risk and reliability assessment and management - mathematical methods in reliability and safety - risk assessment - risk management - system reliability - uncertainty analysis - digitalization and big data - prognostics and system health management - occupational safety - accident and incident modeling - maintenance modeling and applications - simulation for safety and reliability analysis - dynamic risk and barrier management - organizational factors and safety culture - human factors and human reliability - resilience engineering - structural reliability - natural hazards - security - economic analysis in risk managemen