Birth/birth-death processes and their computable transition probabilities with biological applications

Birth-death processes track the size of a univariate population, but many biological systems involve interaction between populations, necessitating models for two or more populations simultaneously. A lack of efficient methods for evaluating finite-time transition probabilities of bivariate processes, however, has restricted statistical inference in these models. Researchers rely on computationally expensive methods such as matrix exponentiation or Monte Carlo approximation, restricting likelihood-based inference to small systems, or indirect methods such as approximate Bayesian computation. In this paper, we introduce the birth(death)/birth-death process, a tractable bivariate extension of the birth-death process. We develop an efficient and robust algorithm to calculate the transition probabilities of birth(death)/birth-death processes using a continued fraction representation of their Laplace transforms. Next, we identify several exemplary models arising in molecular epidemiology, macro-parasite evolution, and infectious disease modeling that fall within this class, and demonstrate advantages of our proposed method over existing approaches to inference in these models. Notably, the ubiquitous stochastic susceptible-infectious-removed (SIR) model falls within this class, and we emphasize that computable transition probabilities newly enable direct inference of parameters in the SIR model. We also propose a very fast method for approximating the transition probabilities under the SIR model via a novel branching process simplification, and compare it to the continued fraction representation method with application to the 17th century plague in Eyam. Although the two methods produce similar maximum a posteriori estimates, the branching process approximation fails to capture the correlation structure in the joint posterior distribution

Particle MCMC algorithms and architectures for accelerating inference in state-space models.

Particle Markov Chain Monte Carlo (pMCMC) is a stochastic algorithm designed to generate samples from a probability distribution, when the density of the distribution does not admit a closed form expression. pMCMC is most commonly used to sample from the Bayesian posterior distribution in State-Space Models (SSMs), a class of probabilistic models used in numerous scientific applications. Nevertheless, this task is prohibitive when dealing with complex SSMs with massive data, due to the high computational cost of pMCMC and its poor performance when the posterior exhibits multi-modality. This paper aims to address both issues by: 1) Proposing a novel pMCMC algorithm (denoted ppMCMC), which uses multiple Markov chains (instead of the one used by pMCMC) to improve sampling efficiency for multi-modal posteriors, 2) Introducing custom, parallel hardware architectures, which are tailored for pMCMC and ppMCMC. The architectures are implemented on Field Programmable Gate Arrays (FPGAs), a type of hardware accelerator with massive parallelization capabilities. The new algorithm and the two FPGA architectures are evaluated using a large-scale case study from genetics. Results indicate that ppMCMC achieves 1.96x higher sampling efficiency than pMCMC when using sequential CPU implementations. The FPGA architecture of pMCMC is 12.1x and 10.1x faster than state-of-the-art, parallel CPU and GPU implementations of pMCMC and up to 53x more energy efficient; the FPGA architecture of ppMCMC increases these speedups to 34.9x and 41.8x respectively and is 173x more power efficient, bringing previously intractable SSM-based data analyses within reach.The authors would like to thank the Wellcome Trust (Grant reference 097816/Z/11/A) and the EPSRC (Grant reference EP/I012036/1) for the financial support given to this research project

Computational Inference Algorithms for Spatiotemporal Processes and Other Complex Models

Data analysis can be carried out based on a stochastic model that reflects the analyst's understanding of how the system in question behaves. The stochastic model describes where in the system randomness is present and how the randomness plays a role in generating data. The likelihood of the data defined by the model summarizes the evidence provided by observations of the system. Drawing inference from the likelihood of the data, however, can be far from being simple or straightforward, especially in modern statistical data analyses. Complex probability models and big data call for new computational methods to translate the likelihood of data into inference results. In this thesis, I present two innovations in computational inference for complex stochastic models. The first innovation lies in the development of a method that enables inference on coupled dynamic systems that are partially observed. The high dimensionality of the model that defines the joint distribution of the coupled dynamic processes makes computational inference a challenge. I focus on the case where the probability model is not analytically tractable, which makes the computational inference even more challenging. A mechanistic model of a dynamic process that is defined via a simulation algorithm can lead to analytically intractable models. I show that algorithms that utilize the Markov structure and the mixing property of stochastic dynamic systems can enable fully likelihood based inference for these high dimensional analytically intractable models. I demonstrate theoretically that these algorithms can substantially reduce the computational cost for inference, and the reduction may be orders of magnitude in practice. Spatiotemporal dynamics of measles transmission are inferred from data collected at linked geographic locations, as an illustration that this algorithm can offer an advance in scientific inference. The second innovation involves a generalization of the framework in which samples from a probability distribution with unnormalized density are drawn using Markov chain Monte Carlo algorithms. The new framework generalizes the widely used Metropolis-Hastings acceptance or rejection strategy. The resulting method is straightforward to implement in a broad range of MCMC algorithms, including the most frequently used ones such as random walk Metropolis, Metropolis adjusted Langevin, Hamiltonian Monte Carlo, or the bouncy particle sampler. Numerical studies show that this new framework enables flexible tuning of parameters and facilitates faster mixing of the Markov chain, especially when the target probability density has complex structure.PHDStatisticsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/145801/1/joonhap_1.pd