On inference for ABC approximations of time series models

Markov chain Monte Carlo (MCMC) and Sequential Monte Carlo (SMC) are well-studied simulation based methods that have been very successful for a variety of complex inference problems. However, standard MCMC and SMC cannot deal with problems where the corresponding likelihood is intractable. By intractable we mean that the density of the likelihood cannot be calculated analytically or it is too computationally expensive to do so. This motivated the use of Approximate Bayesian Computation (ABC) as far back as thirty years ago, but only in the last ten years has its popularity surged due to the higher computational power available in modern computers. The focus of this thesis is to use simulation to estimate the density of the intractable likelihoods as in ABC methods and combine with MCMC and SMC methods. The flexibility and accuracy of the latter can be used to address general issues regarding inference for time series and ABC techniques can be used to deal with the intractable conditional observation likelihoods. The times series we will investigate in this thesis are Hidden Markov Models (HMMs) and observation driven time series models. SMC and ABC methods have been used to perform filtering and smoothing for intractable HMMs [Jasra et al., 2012]. We will present a method that combines SMC and ABC, and uses gradient ascent to deliver maximum likelihood estimates for the model parameters. The accuracy of the method will be illustrated by theoretical results and challenging numerical examples. In regard to estimating model parameters for other time series such as intractable observation driven models, we consider methods that use MCMC together with ABC approximations. These methods can be used to perform Bayesian estimation of the model parameters. However applying ideas from ABC in this context naively will prevent the exploration of difficult regions in the parameter space. Difficult regions arise, for example, when attempting ABC approximations very close to the true model. Based on recent ideas from [Lee, 2012], we identify a new unbiased estimator for the ABC likelihood which is more robust and able to explore the difficult state-space regions. The advantage of the resulting algorithm is that it is capable of producing very accurate inference. Often one is interested in estimating the initial condition of dynamic systems based on noisy observations of its evolution. These types of inverse problems are often formulated as Bayesian inference problems. The prior used in Bayesian inference can be interpreted also as a regularization term to ensure the problem is well-posed. We show how SMC together with ABC approximations can be used for estimating the initial condition of deterministic dynamical systems that are observed with additive noise whose density is intractable. We present a method which uses SMC samplers as in [Chopin, 2002, Del Moral et al., 2006] with ABC approximations, and is applied to challenging numerical examples from data assimilation.Open Acces

A Markovian approach to the analysis and optimization of a portfolio of credit card accounts

Master'sMASTER OF ENGINEERIN

Bayesian Inference of Stochastic Degradation Models: A Likelihood-Free Approach

The structural integrity and system performance of large engineering systems are adversely affected by various forms of degradation mechanisms. Modeling of such mechanisms is accomplished by collecting degradation data from periodic in-service inspections of structures and components. Subsequently, the degradation prediction is transformed into system and component lifetimes that are necessary inputs into the risk-based life-cycle management of critical structures. Stochastic degradation models are widely applicable for predicting degradation growths in structural components. The statistical estimation of such models is often challenged by various uncertainties, such as inherent randomness of a degradation process, parameter uncertainty due to noise in measurements, coverage issues, probe signal loss, the limited resolution of the inspection probe, and small sample size. The Bayesian inference method can be used to quantify the uncertainties of the model parameters. However, degradation data of engineering structures are often contaminated by a significant amount of inspection errors added by various inspection tools. As a result, the likelihood function becomes analytically intractable and computationally expensive to a degree that any traditional likelihood-based Bayesian inference scheme (e.g., Gibbs Sampler, Metropolis-Hastings sampler) turns difficult for practical use. This study proposes a practical likelihood-free approach for parameter estimation based on the approximate Bayesian computation (ABC) method. ABC is a simulation-based approach that does not require an explicit formulation of the likelihood function. Three advanced computational algorithms, namely, ABC using Markov chain Monte Carlo (ABC-MCMC), ABC using Hamiltonian Monte Carlo (ABC-HMC), and ABC using subset simulation (ABC-SS), are developed and implemented for the parameter estimation task. In the context of degradation modeling, various implementation issues of these algorithms are discussed in detail. To improve the mixing properties of ABC-MCMC, a new ABC algorithm is derived based on the HMC sampler that uses the Hamiltonian dynamics to simulate new samples from its seed samples. Its non-random walk behavior helps to explore the target probability space more effectively and efficiently than the standard random-walk MCMC method. The convergence of the proposed ABC-HMC algorithm is proved by satisfying the detailed balance equation, and its efficacy is verified using a numerical example. Furthermore, A new sequential ABC algorithm is proposed to deal with highly diffused priors in a Bayesian inference problem. The proposed ABC algorithm is based on the subset simulation method and a modified HMC algorithm. With faster convergence, the new algorithm turns out to be a powerful method to sample from a complex multi-modal target density as shown by a numerical example. The applicability of the proposed algorithm is further extended by transforming it into a likelihood-free Bayesian model selection tool. The proposed likelihood-free approach for Bayesian inference is applied to analyze practical data sets from the Canadian nuclear power plants. The practical data consist of two types of degradation measurements: (1) wall thickness data of the feeder pipes that are affected by the flow-accelerated corrosion (FAC) and (2) data from the steam generator tubes affected by the pitting corrosion. Four popular stochastic degradation models are considered, namely, the random rate model, the gamma process model, the mixed-effects regression model, and the Poisson process model, for characterizing the degradation processes under study. In the modeling process, various inspection uncertainties, such as the sizing error, the coverage error, and the probability of detection (POD) error are taken into account. The numerical results demonstrate that, in comparison to the likelihood-based approach, the proposed likelihood-free approach notably reduces computational time while accurately estimating the model parameters. This study finds that these intuitive and easy-to-implement likelihood-free algorithms are versatile tools for Bayesian inference of stochastic degradation models and a promising alternative to the traditional Bayesian estimation methods