Delayed acceptance ABC-SMC

Approximate Bayesian computation (ABC) is now an established technique for statistical inference used in cases where the likelihood function is computationally expensive or not available. It relies on the use of a~model that is specified in the form of a~simulator, and approximates the likelihood at a~parameter value $\theta$ by simulating auxiliary data sets $x$ and evaluating the distance of $x$ from the true data $y$. However, ABC is not computationally feasible in cases where using the simulator for each $\theta$ is very expensive. This paper investigates this situation in cases where a~cheap, but approximate, simulator is available. The approach is to employ delayed acceptance Markov chain Monte Carlo (MCMC) within an ABC sequential Monte Carlo (SMC) sampler in order to, in a~first stage of the kernel, use the cheap simulator to rule out parts of the parameter space that are not worth exploring, so that the ``true'' simulator is only run (in the second stage of the kernel) where there is a~reasonable chance of accepting proposed values of $\theta$. We show that this approach can be used quite automatically, with few tuning parameters. Applications to stochastic differential equation models and latent doubly intractable distributions are presented

Approximate Bayesian Computations to fit and compare insurance loss models

Approximate Bayesian Computation (ABC) is a statistical learning technique to calibrate and select models by comparing observed data to simulated data. This technique bypasses the use of the likelihood and requires only the ability to generate synthetic data from the models of interest. We apply ABC to fit and compare insurance loss models using aggregated data. We present along the way how to use ABC for the more common claim counts and claim sizes data. A state-of-the-art ABC implementation in Python is proposed. It uses sequential Monte Carlo to sample from the posterior distribution and the Wasserstein distance to compare the observed and synthetic data

Ensemble MCMC: Accelerating Pseudo-Marginal MCMC for State Space Models using the Ensemble Kalman Filter

Particle Markov chain Monte Carlo (pMCMC) is now a popular method for performing Bayesian statistical inference on challenging state space models (SSMs) with unknown static parameters. It uses a particle filter (PF) at each iteration of an MCMC algorithm to unbiasedly estimate the likelihood for a given static parameter value. However, pMCMC can be computationally intensive when a large number of particles in the PF is required, such as when the data are highly informative, the model is misspecified and/or the time series is long. In this paper we exploit the ensemble Kalman filter (EnKF) developed in the data assimilation literature to speed up pMCMC. We replace the unbiased PF likelihood with the biased EnKF likelihood estimate within MCMC to sample over the space of the static parameter. On a wide class of different non-linear SSM models, we demonstrate that our extended ensemble MCMC (eMCMC) methods can significantly reduce the computational cost whilst maintaining reasonable accuracy. We also propose several extensions of the vanilla eMCMC algorithm to further improve computational efficiency. Computer code to implement our methods on all the examples can be downloaded from https://github.com/cdrovandi/Ensemble-MCMC

ABC Samplers

This Chapter, "ABC Samplers", is to appear in the forthcoming Handbook of Approximate Bayesian Computation (2018). It details the main ideas and algorithms used to sample from the ABC approximation to the posterior distribution, including methods based on rejection/importance sampling, MCMC and sequential Monte Carlo

A sequential Monte Carlo algorithm with transformations for Bayesian model exploration: applications in population genetics

Given a statistical model that attempts to explain the data, calculating the Bayes’ posterior distribution of the models parameters is desirable. The marginal likelihood of the model is also of interest, which is used for model comparison. However, for most applications, only estimates of these two measurements can be obtained with a class of methods that give consistent estimates being Monte Carlo algorithms. This thesis attempts to improve both the process in inferring a high-dimensional posterior distribution and the corresponding model marginal likelihood, on the condition that we can define an ordered set of statistical models in which deterministic transformations between each adjacent model can be applied. We propose an adaption of the sequential Monte Carlo algorithm, which we term the “transformation Sequential Monte Carlo” algorithm. The key feature of this algorithm is by defining a series of target distributions, that make use of said mentioned model transformations, we aim to infer high dimensional models by using easier to estimate posteriors from lower dimensional models with a model transformation applied. Our proposed algorithm has advantages over many established MC methods. One notable advantage is that we can tailor the algorithm if we wish to update a posterior distribution by including additional observations, but these observations also correspond to a new parameter set that needs to be inferred. Alternatively it is useful where the parameter space can become too large to explore using basic MC methods, for example if there exists an exponential or factorial relationship with observation size and the number of discrete values, but using a lower dimensional model and incorporating it into the model exploration assists with convergence. We test these strengths of tSMC under three applications, which include two population genetics applications being ancestral reconstruction under the coalescent and the other being the Structure algorithm