Efficient probabilistic inversion of geophysical data

Estimation of uncertainties is critical for subsequent decision making in all applications of geosciences such as geological hazard analysis and risk mitigation, management and exploitation of subsurface resources, and environmental waste disposal. More efficient probabilistic inversion methods in geosciences are vital to making rapid and improved predictions of geological hazards and estimation of subsurface resources from geophysical data, and estimation of associated uncertainties. While this thesis focuses on seismic data inversion for the estimation of geological properties, the methods developed may find a wide variety of applications in all fields of research that involve spatial data analysis. New concepts, models and methods are developed to perform more efficient probabilistic inversion by making use of the latest developments in machine learning and Bayesian inverse theory to solve geophysical inverse problems. The major contribution of this thesis is the development of efficient geostatistical inversion methods for approximate inference for structured inverse problems where probabilistic dependence between unknown model parameters may be expressed as a Markov random field (MRF). These methods are many orders of magnitude faster than the corresponding sampling based methods in such types of inverse problems. Further, some of the commonly used but avoidable assumptions in conventional geostatistical inversion methods are progressively relaxed and finally removed in this research. The faster inversion methods allow more complex models to be evaluated for more accurate predictions and improved estimation of uncertainty for given compute power and time. Most existing geostatistical inversion methods are based on the localized likelihoods assumption, whereby the seismic data at a location are assumed to depend on the geology only at that location. Such an assumption is unrealistic because of imperfect seismic data acquisition and processing, and fundamental limitations of seismic imaging methods. It is also assumed in most such previous research that the data are completely free of any correlated noise or errors. Although these requirements are almost never met in reality, existing methods use these assumptions to make solutions computationally tractable. Both of these assumptions are progressively removed in this thesis while still allowing computationally tractable solutions to be found for suitably structured problems. The class of problems considered here spans a broad range of spatial data analysis and geosciences, where geology at a location is assumed to depend directly only on the geology within some pre-specified neighbourhood of that location – the so called Markovian assumption – which is the core assumption across the entire literature of geostatistics and has been proven to be valid for all practical purposes. Exact Bayesian inference is intractable in most models of practical interest because it requires normalization of the posterior distribution by integrating model parameters over a very high dimensional space. Therefore, approximate inference is used in practice. Stochastic sampling (e.g., by using Markov-chain Monte Carlo – McMC) is the most commonly used approximate inference method but is computationally expensive and detection of its convergence is often based on subjective criteria and hence is unreliable. New Bayesian inversion methods are introduced that estimate the spatial distribution of geological properties from attributes of seismic data, by showing how the usual probabilistic inverse problem can be solved using an optimization framework while still providing full probabilistic results – the so called variational inference approach. The intractable posterior distribution is replaced by a tractable approximation in the variational approach. Inference can then be performed using the approximate distribution in an optimization framework, thus circumventing the need for sampling, while still providing probabilistic results. The methods developed in this thesis infer the post-inversion (posterior) probability density of the unknown model parameters from seismic data and geological prior information. These methods are shown to be robust against weak prior information and correlated noise in the data. The methods are computationally efficient, and are expected to be applicable to 3D models of realistic size on modern computers without incurring any significant computational limitations

Non-parametric Bayesian models for structured output prediction

Structured output prediction is a machine learning tasks in which an input object is not just assigned a single class, as in classification, but multiple, interdependent labels. This means that the presence or value of a given label affects the other labels, for instance in text labelling problems, where output labels are applied to each word, and their interdependencies must be modelled. Non-parametric Bayesian (NPB) techniques are probabilistic modelling techniques which have the interesting property of allowing model capacity to grow, in a controllable way, with data complexity, while maintaining the advantages of Bayesian modelling. In this thesis, we develop NPB algorithms to solve structured output problems. We first study a map-reduce implementation of a stochastic inference method designed for the infinite hidden Markov model, applied to a computational linguistics task, part-of-speech tagging. We show that mainstream map-reduce frameworks do not easily support highly iterative algorithms. The main contribution of this thesis consists in a conceptually novel discriminative model, GPstruct. It is motivated by labelling tasks, and combines attractive properties of conditional random fields (CRF), structured support vector machines, and Gaussian process (GP) classifiers. In probabilistic terms, GPstruct combines a CRF likelihood with a GP prior on factors; it can also be described as a Bayesian kernelized CRF. To train this model, we develop a Markov chain Monte Carlo algorithm based on elliptical slice sampling and investigate its properties. We then validate it on real data experiments, and explore two topologies: sequence output with text labelling tasks, and grid output with semantic segmentation of images. The latter case poses scalability issues, which are addressed using likelihood approximations and an ensemble method which allows distributed inference and prediction. The experimental validation demonstrates: (a) the model is flexible and its constituent parts are modular and easy to engineer; (b) predictive performance and, most crucially, the probabilistic calibration of predictions are better than or equal to that of competitor models, and (c) model hyperparameters can be learnt from data