Accelerating Asymptotically Exact MCMC for Computationally Intensive Models via Local Approximations

We construct a new framework for accelerating Markov chain Monte Carlo in posterior sampling problems where standard methods are limited by the computational cost of the likelihood, or of numerical models embedded therein. Our approach introduces local approximations of these models into the Metropolis-Hastings kernel, borrowing ideas from deterministic approximation theory, optimization, and experimental design. Previous efforts at integrating approximate models into inference typically sacrifice either the sampler's exactness or efficiency; our work seeks to address these limitations by exploiting useful convergence characteristics of local approximations. We prove the ergodicity of our approximate Markov chain, showing that it samples asymptotically from the \emph{exact} posterior distribution of interest. We describe variations of the algorithm that employ either local polynomial approximations or local Gaussian process regressors. Our theoretical results reinforce the key observation underlying this paper: when the likelihood has some \emph{local} regularity, the number of model evaluations per MCMC step can be greatly reduced without biasing the Monte Carlo average. Numerical experiments demonstrate multiple order-of-magnitude reductions in the number of forward model evaluations used in representative ODE and PDE inference problems, with both synthetic and real data.Comment: A major update of the theory and example

Economic regimes identification using machine learning technics

43 páginas.Trabajo de Máster en Economía, Finanzas y Computación. Director: Dr. José Manuel Bravo Caro. Economic conditions over long time periods can be distinguished by regimes. Regime identification has been object of numerous investigations in economics and financial modeling for years. Recently, new machine learning technics such as decision trees, support vector machines and neural networks, among others, followed by alternative datasets and cheap computational processing power became available, allowing for alternative ways to model complex economic relationships. In the present work, we develop a supervised machine learning classifier using Random Forest technic to identify economic regimes using the S&P 500 stock market index series.Las condiciones económicas durante largos períodos de tiempo pueden distinguirse por regímenes. La identificación del régimen ha sido objeto de numerosas investigaciones en economía y modelos financieros durante años. Recientemente, se pusieron a disposición nuevas técnicas de aprendizaje automático, como árboles de decisión, máquinas de suporte vectorial y redes neuronales, entre otras, seguidas de conjuntos de datos alternativos y una capacidad de procesamiento computacional barata, que permite formas alternativas de modelar relaciones económicas complejas. En el presente trabajo, desarrollamos un clasificador de aprendizaje automático supervisado utilizando la técnica de Random Forest para identificar regímenes económicos utilizando la serie del índices de mercado S&P 500

Nonparametric likelihood based estimation of linear filters for point processes

We consider models for multivariate point processes where the intensity is given nonparametrically in terms of functions in a reproducing kernel Hilbert space. The likelihood function involves a time integral and is consequently not given in terms of a finite number of kernel evaluations. The main result is a representation of the gradient of the log-likelihood, which we use to derive computable approximations of the log-likelihood and the gradient by time discretization. These approximations are then used to minimize the approximate penalized log-likelihood. For time and memory efficiency the implementation relies crucially on the use of sparse matrices. As an illustration we consider neuron network modeling, and we use this example to investigate how the computational costs of the approximations depend on the resolution of the time discretization. The implementation is available in the R package ppstat.Comment: 10 pages, 3 figure

Function estimation with locally adaptive dynamic models

We present a nonparametric Bayesian method for fitting unsmooth and highly oscillating functions, which is based on a locally adaptive hierarchical extension of standard dynamic or state space models. The main idea is to introduce locally varying variances in the state equations and to add a further smoothness prior for this variance function. Estimation is fully Bayesian and carried out by recent MCMC techniques. The whole approach can be understood as an alternative to other nonparametric function estimators, such as local or penalized regression with variable bandwidth or smoothing parameter selection. Performance is illustrated with simulated data, including unsmooth examples constructed for wavelet shrinkage, and by an application to sales data. Although the approach is developed for classical Gaussian nonparametric regression, it can be extended to more complex regression problems