Bayesian Variational Time-lapse Full-waveform Inversion

Time-lapse seismic full-waveform inversion (FWI) provides estimates of dynamic changes in the subsurface by performing multiple seismic surveys at different times. Since FWI problems are highly non-linear and non-unique, it is important to quantify uncertainties in such estimates to allow robust decision making. Markov chain Monte Carlo (McMC) methods have been used for this purpose, but due to their high computational cost, those studies often require an accurate baseline model and estimates of the locations of potential velocity changes, and neglect uncertainty in the baseline velocity model. Such detailed and accurate prior information is not always available in practice. In this study we use an efficient optimization method called stochastic Stein variational gradient descent (sSVGD) to solve time-lapse FWI problems without assuming such prior knowledge, and to estimate uncertainty both in the baseline velocity model and the velocity change. We test two Bayesian strategies: separate Bayesian inversions for each seismic survey, and a single join inversion for baseline and repeat surveys, and compare the methods with the standard linearised double difference inversion. The results demonstrate that all three methods can produce accurate velocity change estimates in the case of having fixed (exactly repeatable) acquisition geometries, but that the two Bayesian methods generate more accurate results when the acquisition geometry changes between surveys. Furthermore the joint inversion provides the most accurate velocity change and uncertainty estimates in all cases. We therefore conclude that Bayesian time-lapse inversion, especially adopting a joint inversion strategy, may be useful to image and monitor the subsurface changes, in particular where uncertainty in the results might lead to significantly different decisions

3D Bayesian Variational Full Waveform Inversion

Seismic full-waveform inversion (FWI) provides high resolution images of the subsurface by exploiting information in the recorded seismic waveforms. This is achieved by solving a highly nonnlinear and nonunique inverse problem. Bayesian inference is therefore used to quantify uncertainties in the solution. Variational inference is a method that provides probabilistic, Bayesian solutions efficiently using optimization. The method has been applied to 2D FWI problems to produce full Bayesian posterior distributions. However, due to higher dimensionality and more expensive computational cost, the performance of the method in 3D FWI problems remains unknown. We apply three variational inference methods to 3D FWI and analyse their performance. Specifically we apply automatic differential variational inference (ADVI), Stein variational gradient descent (SVGD) and stochastic SVGD (sSVGD), to a 3D FWI problem, and compare their results and computational cost. The results show that ADVI is the most computationally efficient method but systematically underestimates the uncertainty. The method can therefore be used to provide relatively rapid but approximate insights into the subsurface together with a lower bound estimate of the uncertainty. SVGD demands the highest computational cost, and still produces biased results. In contrast, by including a randomized term in the SVGD dynamics, sSVGD becomes a Markov chain Monte Carlo method and provides the most accurate results at intermediate computational cost. We thus conclude that 3D variational full-waveform inversion is practically applicable, at least in small problems, and can be used to image the Earth's interior and to provide reasonable uncertainty estimates on those images

Adversarial Machine Learning: Bayesian Perspectives

Adversarial Machine Learning (AML) is emerging as a major field aimed at protecting machine learning (ML) systems against security threats: in certain scenarios there may be adversaries that actively manipulate input data to fool learning systems. This creates a new class of security vulnerabilities that ML systems may face, and a new desirable property called adversarial robustness essential to trust operations based on ML outputs. Most work in AML is built upon a game-theoretic modelling of the conflict between a learning system and an adversary, ready to manipulate input data. This assumes that each agent knows their opponent's interests and uncertainty judgments, facilitating inferences based on Nash equilibria. However, such common knowledge assumption is not realistic in the security scenarios typical of AML. After reviewing such game-theoretic approaches, we discuss the benefits that Bayesian perspectives provide when defending ML-based systems. We demonstrate how the Bayesian approach allows us to explicitly model our uncertainty about the opponent's beliefs and interests, relaxing unrealistic assumptions, and providing more robust inferences. We illustrate this approach in supervised learning settings, and identify relevant future research problems

Stein variational gradient descent: Many-particle and long-time asymptotics

Stein variational gradient descent (SVGD) refers to a class of methods for Bayesian inference based on interacting particle systems. In this paper, we consider the originally proposed deterministic dynamics as well as a stochastic variant, each of which represent one of the two main paradigms in Bayesian computational statistics: emphvariational inference and emphMarkov chain Monte Carlo. As it turns out, these are tightly linked through a correspondence between gradient flow structures and large-deviation principles rooted in statistical physics. To expose this relationship, we develop the cotangent space construction for the Stein geometry, prove its basic properties, and determine the large-deviation functional governing the many-particle limit for the empirical measure. Moreover, we identify the emphStein-Fisher information (or emphkernelised Stein discrepancy) as its leading order contribution in the long-time and many-particle regime in the sense of $Gamma$-convergence, shedding some light on the finite-particle properties of SVGD. Finally, we establish a comparison principle between the Stein-Fisher information and RKHS-norms that might be of independent interest