7,994 research outputs found
Spectral Density-Based and Measure-Preserving ABC for partially observed diffusion processes. An illustration on Hamiltonian SDEs
Approximate Bayesian Computation (ABC) has become one of the major tools of
likelihood-free statistical inference in complex mathematical models.
Simultaneously, stochastic differential equations (SDEs) have developed to an
established tool for modelling time dependent, real world phenomena with
underlying random effects. When applying ABC to stochastic models, two major
difficulties arise. First, the derivation of effective summary statistics and
proper distances is particularly challenging, since simulations from the
stochastic process under the same parameter configuration result in different
trajectories. Second, exact simulation schemes to generate trajectories from
the stochastic model are rarely available, requiring the derivation of suitable
numerical methods for the synthetic data generation. To obtain summaries that
are less sensitive to the intrinsic stochasticity of the model, we propose to
build up the statistical method (e.g., the choice of the summary statistics) on
the underlying structural properties of the model. Here, we focus on the
existence of an invariant measure and we map the data to their estimated
invariant density and invariant spectral density. Then, to ensure that these
model properties are kept in the synthetic data generation, we adopt
measure-preserving numerical splitting schemes. The derived property-based and
measure-preserving ABC method is illustrated on the broad class of partially
observed Hamiltonian type SDEs, both with simulated data and with real
electroencephalography (EEG) data. The proposed ingredients can be incorporated
into any type of ABC algorithm and directly applied to all SDEs that are
characterised by an invariant distribution and for which a measure-preserving
numerical method can be derived.Comment: 35 pages, 21 figure
Simulation and inference algorithms for stochastic biochemical reaction networks: from basic concepts to state-of-the-art
Stochasticity is a key characteristic of intracellular processes such as gene
regulation and chemical signalling. Therefore, characterising stochastic
effects in biochemical systems is essential to understand the complex dynamics
of living things. Mathematical idealisations of biochemically reacting systems
must be able to capture stochastic phenomena. While robust theory exists to
describe such stochastic models, the computational challenges in exploring
these models can be a significant burden in practice since realistic models are
analytically intractable. Determining the expected behaviour and variability of
a stochastic biochemical reaction network requires many probabilistic
simulations of its evolution. Using a biochemical reaction network model to
assist in the interpretation of time course data from a biological experiment
is an even greater challenge due to the intractability of the likelihood
function for determining observation probabilities. These computational
challenges have been subjects of active research for over four decades. In this
review, we present an accessible discussion of the major historical
developments and state-of-the-art computational techniques relevant to
simulation and inference problems for stochastic biochemical reaction network
models. Detailed algorithms for particularly important methods are described
and complemented with MATLAB implementations. As a result, this review provides
a practical and accessible introduction to computational methods for stochastic
models within the life sciences community
Stochastic Volatility Filtering with Intractable Likelihoods
This paper is concerned with particle filtering for -stable
stochastic volatility models. The -stable distribution provides a
flexible framework for modeling asymmetry and heavy tails, which is useful when
modeling financial returns. An issue with this distributional assumption is the
lack of a closed form for the probability density function. To estimate the
volatility of financial returns in this setting, we develop a novel auxiliary
particle filter. The algorithm we develop can be easily applied to any hidden
Markov model for which the likelihood function is intractable or
computationally expensive. The approximate target distribution of our auxiliary
filter is based on the idea of approximate Bayesian computation (ABC). ABC
methods allow for inference on posterior quantities in situations when the
likelihood of the underlying model is not available in closed form, but
simulating samples from it is possible. The ABC auxiliary particle filter
(ABC-APF) that we propose provides not only a good alternative to state
estimation in stochastic volatility models, but it also improves on the
existing ABC literature. It allows for more flexibility in state estimation
while improving on the accuracy through better proposal distributions in cases
when the optimal importance density of the filter is unavailable in closed
form. We assess the performance of the ABC-APF on a simulated dataset from the
-stable stochastic volatility model and compare it to other currently
existing ABC filters
Patterns of Scalable Bayesian Inference
Datasets are growing not just in size but in complexity, creating a demand
for rich models and quantification of uncertainty. Bayesian methods are an
excellent fit for this demand, but scaling Bayesian inference is a challenge.
In response to this challenge, there has been considerable recent work based on
varying assumptions about model structure, underlying computational resources,
and the importance of asymptotic correctness. As a result, there is a zoo of
ideas with few clear overarching principles.
In this paper, we seek to identify unifying principles, patterns, and
intuitions for scaling Bayesian inference. We review existing work on utilizing
modern computing resources with both MCMC and variational approximation
techniques. From this taxonomy of ideas, we characterize the general principles
that have proven successful for designing scalable inference procedures and
comment on the path forward
Scalable Inference for Markov Processes with Intractable Likelihoods
Bayesian inference for Markov processes has become increasingly relevant in
recent years. Problems of this type often have intractable likelihoods and
prior knowledge about model rate parameters is often poor. Markov Chain Monte
Carlo (MCMC) techniques can lead to exact inference in such models but in
practice can suffer performance issues including long burn-in periods and poor
mixing. On the other hand approximate Bayesian computation techniques can allow
rapid exploration of a large parameter space but yield only approximate
posterior distributions. Here we consider the combined use of approximate
Bayesian computation (ABC) and MCMC techniques for improved computational
efficiency while retaining exact inference on parallel hardware
Considerate Approaches to Achieving Sufficiency for ABC model selection
For nearly any challenging scientific problem evaluation of the likelihood is
problematic if not impossible. Approximate Bayesian computation (ABC) allows us
to employ the whole Bayesian formalism to problems where we can use simulations
from a model, but cannot evaluate the likelihood directly. When summary
statistics of real and simulated data are compared --- rather than the data
directly --- information is lost, unless the summary statistics are sufficient.
Here we employ an information-theoretical framework that can be used to
construct (approximately) sufficient statistics by combining different
statistics until the loss of information is minimized. Such sufficient sets of
statistics are constructed for both parameter estimation and model selection
problems. We apply our approach to a range of illustrative and real-world model
selection problems
Approximate Bayesian Computation for a Class of Time Series Models
In the following article we consider approximate Bayesian computation (ABC)
for certain classes of time series models. In particular, we focus upon
scenarios where the likelihoods of the observations and parameter are
intractable, by which we mean that one cannot evaluate the likelihood even
up-to a positive unbiased estimate. This paper reviews and develops a class of
approximation procedures based upon the idea of ABC, but, specifically
maintains the probabilistic structure of the original statistical model. This
idea is useful, in that it can facilitate an analysis of the bias of the
approximation and the adaptation of established computational methods for
parameter inference. Several existing results in the literature are surveyed
and novel developments with regards to computation are given
Auxiliary Likelihood-Based Approximate Bayesian Computation in State Space Models
A computationally simple approach to inference in state space models is
proposed, using approximate Bayesian computation (ABC). ABC avoids evaluation
of an intractable likelihood by matching summary statistics for the observed
data with statistics computed from data simulated from the true process, based
on parameter draws from the prior. Draws that produce a 'match' between
observed and simulated summaries are retained, and used to estimate the
inaccessible posterior. With no reduction to a low-dimensional set of
sufficient statistics being possible in the state space setting, we define the
summaries as the maximum of an auxiliary likelihood function, and thereby
exploit the asymptotic sufficiency of this estimator for the auxiliary
parameter vector. We derive conditions under which this approach - including a
computationally efficient version based on the auxiliary score - achieves
Bayesian consistency. To reduce the well-documented inaccuracy of ABC in
multi-parameter settings, we propose the separate treatment of each parameter
dimension using an integrated likelihood technique. Three stochastic volatility
models for which exact Bayesian inference is either computationally
challenging, or infeasible, are used for illustration. We demonstrate that our
approach compares favorably against an extensive set of approximate and exact
comparators. An empirical illustration completes the paper.Comment: This paper is forthcoming at the Journal of Computational and
Graphical Statistics. It also supersedes the earlier arXiv paper "Approximate
Bayesian Computation in State Space Models" (arXiv:1409.8363
Statistical Inference for Partially Observed Markov Processes via the R Package pomp
Partially observed Markov process (POMP) models, also known as hidden Markov
models or state space models, are ubiquitous tools for time series analysis.
The R package pomp provides a very flexible framework for Monte Carlo
statistical investigations using nonlinear, non-Gaussian POMP models. A range
of modern statistical methods for POMP models have been implemented in this
framework including sequential Monte Carlo, iterated filtering, particle Markov
chain Monte Carlo, approximate Bayesian computation, maximum synthetic
likelihood estimation, nonlinear forecasting, and trajectory matching. In this
paper, we demonstrate the application of these methodologies using some simple
toy problems. We also illustrate the specification of more complex POMP models,
using a nonlinear epidemiological model with a discrete population,
seasonality, and extra-demographic stochasticity. We discuss the specification
of user-defined models and the development of additional methods within the
programming environment provided by pomp.Comment: In press at the Journal of Statistical Software. A version of this
paper is provided at the pomp package website: http://kingaa.github.io/pom
- …