Bayesian probabilistic numerical methods

The increasing complexity of computer models used to solve contemporary inference problems has been set against a decreasing rate of improvement in processor speed in recent years. As a result, in many of these problems numerical error is a challenge for practitioners. However, while there has been a recent push towards rigorous quantification of uncertainty in inference problems based upon computer models, numerical error is still largely required to be driven down to a level at which its impact on inferences is negligible. Probabilistic numerical methods have been proposed to alleviate this; these are a class of numerical methods that return probabilistic uncertainty quantification for their numerical error. The attraction of such methods is clear: if numerical error in the computer model and uncertainty in an inference problem are quantified in a unified framework then careful tuning of numerical methods to mitigate the impact of numerical error on inferences could become unnecessary. In this thesis we introduce the class of Bayesian probabilistic numerical methods, whose uncertainty has a strict and rigorous Bayesian interpretation. A number of examples of conjugate Bayesian probabilistic numerical methods are presented before we present analysis and algorithms for the general case, in which the posterior distribution does not posess a closed form. We conclude by studying how these methods can be rigorously composed to yield Bayesian pipelines of computation. Throughout we present applications of the developed methods to real-world inference problems, and indicate that the uncertainty quantification provided by these methods can be of significant practical use

Bayesian Probabilistic Numerical Methods in Time-Dependent State Estimation for Industrial Hydrocyclone Equipment

The use of high-power industrial equipment, such as large-scale mixing equipment or a hydrocyclone for separation of particles in liquid suspension, demands careful monitoring to ensure correct operation. The fundamental task of state-estimation for the liquid suspension can be posed as a time-evolving inverse problem and solved with Bayesian statistical methods. In this article, we extend Bayesian methods to incorporate statistical models for the error that is incurred in the numerical solution of the physical governing equations. This enables full uncertainty quantification within a principled computation-precision trade-off, in contrast to the over-confident inferences that are obtained when all sources of numerical error are ignored. The method is cast within a sequential Monte Carlo framework and an optimized implementation is provided in Python

Bayesian Quadrature for Multiple Related Integrals

Bayesian probabilistic numerical methods are a set of tools providing posterior distributions on the output of numerical methods. The use of these methods is usually motivated by the fact that they can represent our uncertainty due to incomplete/finite information about the continuous mathematical problem being approximated. In this paper, we demonstrate that this paradigm can provide additional advantages, such as the possibility of transferring information between several numerical methods. This allows users to represent uncertainty in a more faithful manner and, as a by-product, provide increased numerical efficiency. We propose the first such numerical method by extending the well-known Bayesian quadrature algorithm to the case where we are interested in computing the integral of several related functions. We then prove convergence rates for the method in the well-specified and misspecified cases, and demonstrate its efficiency in the context of multi-fidelity models for complex engineering systems and a problem of global illumination in computer graphics.Comment: Proceedings of the 35th International Conference on Machine Learning (ICML), PMLR 80:5369-5378, 201

Bayesian Numerical Integration with Neural Networks

Bayesian probabilistic numerical methods for numerical integration offer significant advantages over their non-Bayesian counterparts: they can encode prior information about the integrand, and can quantify uncertainty over estimates of an integral. However, the most popular algorithm in this class, Bayesian quadrature, is based on Gaussian process models and is therefore associated with a high computational cost. To improve scalability, we propose an alternative approach based on Bayesian neural networks which we call Bayesian Stein networks. The key ingredients are a neural network architecture based on Stein operators, and an approximation of the Bayesian posterior based on the Laplace approximation. We show that this leads to orders of magnitude speed-ups on the popular Genz functions benchmark, and on challenging problems arising in the Bayesian analysis of dynamical systems, and the prediction of energy production for a large-scale wind farm