Posterior Contraction Rates for the Bayesian Approach to Linear Ill-Posed Inverse Problems

We consider a Bayesian nonparametric approach to a family of linear inverse problems in a separable Hilbert space setting with Gaussian noise. We assume Gaussian priors, which are conjugate to the model, and present a method of identifying the posterior using its precision operator. Working with the unbounded precision operator enables us to use partial differential equations (PDE) methodology to obtain rates of contraction of the posterior distribution to a Dirac measure centered on the true solution. Our methods assume a relatively weak relation between the prior covariance, noise covariance and forward operator, allowing for a wide range of applications

Gaussian processes for Bayesian inverse problems associated with linear partial differential equations

This work is concerned with the use of Gaussian surrogate models for Bayesian inverse problems associated with linear partial differential equations. A particular focus is on the regime where only a small amount of training data is available. In this regime the type of Gaussian prior used is of critical importance with respect to how well the surrogate model will perform in terms of Bayesian inversion. We extend the framework of Raissi et. al. (2017) to construct PDE-informed Gaussian priors that we then use to construct different approximate posteriors. A number of different numerical experiments illustrate the superiority of the PDE-informed Gaussian priors over more traditional priors

Gaussian Process Priors for Systems of Linear Partial Differential Equations with Constant Coefficients

Partial differential equations (PDEs) are important tools to model physical systems and including them into machine learning models is an important way of incorporating physical knowledge. Given any system of linear PDEs with constant coefficients, we propose a family of Gaussian process (GP) priors, which we call EPGP, such that all realizations are exact solutions of this system. We apply the Ehrenpreis-Palamodov fundamental principle, which works as a non-linear Fourier transform, to construct GP kernels mirroring standard spectral methods for GPs. Our approach can infer probable solutions of linear PDE systems from any data such as noisy measurements, or pointwise defined initial and boundary conditions. Constructing EPGP-priors is algorithmic, generally applicable, and comes with a sparse version (S-EPGP) that learns the relevant spectral frequencies and works better for big data sets. We demonstrate our approach on three families of systems of PDEs, the heat equation, wave equation, and Maxwell's equations, where we improve upon the state of the art in computation time and precision, in some experiments by several orders of magnitude.Comment: 26 pages, 8 figures; ICML 2023 (oral); updated with expanded appendices and ancillary files. Code available at https://github.com/haerski/EPGP. For animations, see https://mathrepo.mis.mpg.de/EPGP/index.html. For a presentation see https://icml.cc/virtual/2023/oral/25571. The paper and all ancillary files are released under CC-B

Efficient State-Space Inference of Periodic Latent Force Models

Latent force models (LFM) are principled approaches to incorporating solutions to differential equations within non-parametric inference methods. Unfortunately, the development and application of LFMs can be inhibited by their computational cost, especially when closed-form solutions for the LFM are unavailable, as is the case in many real world problems where these latent forces exhibit periodic behaviour. Given this, we develop a new sparse representation of LFMs which considerably improves their computational efficiency, as well as broadening their applicability, in a principled way, to domains with periodic or near periodic latent forces. Our approach uses a linear basis model to approximate one generative model for each periodic force. We assume that the latent forces are generated from Gaussian process priors and develop a linear basis model which fully expresses these priors. We apply our approach to model the thermal dynamics of domestic buildings and show that it is effective at predicting day-ahead temperatures within the homes. We also apply our approach within queueing theory in which quasi-periodic arrival rates are modelled as latent forces. In both cases, we demonstrate that our approach can be implemented efficiently using state-space methods which encode the linear dynamic systems via LFMs. Further, we show that state estimates obtained using periodic latent force models can reduce the root mean squared error to 17% of that from non-periodic models and 27% of the nearest rival approach which is the resonator model.Comment: 61 pages, 13 figures, accepted for publication in JMLR. Updates from earlier version occur throughout article in response to JMLR review

Nonparametric Bayesian methods for one-dimensional diffusion models

In this paper we review recently developed methods for nonparametric Bayesian inference for one-dimensional diffusion models. We discuss different possible prior distributions, computational issues, and asymptotic results

Indifference Pricing and Hedging in a Multiple-Priors Model with Trading Constraints

This paper considers utility indifference valuation of derivatives under model uncertainty and trading constraints, where the utility is formulated as an additive stochastic differential utility of both intertemporal consumption and terminal wealth, and the uncertain prospects are ranked according to a multiple-priors model of Chen and Epstein (2002). The price is determined by two optimal stochastic control problems (mixed with optimal stopping time in the case of American option) of forward-backward stochastic differential equations. By means of backward stochastic differential equation and partial differential equation methods, we show that both bid and ask prices are closely related to the Black-Scholes risk-neutral price with modified dividend rates. The two prices will actually coincide with each other if there is no trading constraint or the model uncertainty disappears. Finally, two applications to European option and American option are discussed.Comment: 28 pages in Science China Mathematics, 201