117 research outputs found
Approximate maximum likelihood estimation using data-cloning ABC
A maximum likelihood methodology for a general class of models is presented,
using an approximate Bayesian computation (ABC) approach. The typical target of
ABC methods are models with intractable likelihoods, and we combine an ABC-MCMC
sampler with so-called "data cloning" for maximum likelihood estimation.
Accuracy of ABC methods relies on the use of a small threshold value for
comparing simulations from the model and observed data. The proposed
methodology shows how to use large threshold values, while the number of
data-clones is increased to ease convergence towards an approximate maximum
likelihood estimate. We show how to exploit the methodology to reduce the
number of iterations of a standard ABC-MCMC algorithm and therefore reduce the
computational effort, while obtaining reasonable point estimates. Simulation
studies show the good performance of our approach on models with intractable
likelihoods such as g-and-k distributions, stochastic differential equations
and state-space models.Comment: 25 pages. Minor revision. It includes a parametric bootstrap for the
exact MLE for the first example; includes mean bias and RMSE calculations for
the third example. Forthcoming in Computational Statistics and Data Analysi
Bayesian inference for stochastic differential equation mixed effects models of a tumor xenography study
We consider Bayesian inference for stochastic differential equation mixed
effects models (SDEMEMs) exemplifying tumor response to treatment and regrowth
in mice. We produce an extensive study on how a SDEMEM can be fitted using both
exact inference based on pseudo-marginal MCMC and approximate inference via
Bayesian synthetic likelihoods (BSL). We investigate a two-compartments SDEMEM,
these corresponding to the fractions of tumor cells killed by and survived to a
treatment, respectively. Case study data considers a tumor xenography study
with two treatment groups and one control, each containing 5-8 mice. Results
from the case study and from simulations indicate that the SDEMEM is able to
reproduce the observed growth patterns and that BSL is a robust tool for
inference in SDEMEMs. Finally, we compare the fit of the SDEMEM to a similar
ordinary differential equation model. Due to small sample sizes, strong prior
information is needed to identify all model parameters in the SDEMEM and it
cannot be determined which of the two models is the better in terms of
predicting tumor growth curves. In a simulation study we find that with a
sample of 17 mice per group BSL is able to identify all model parameters and
distinguish treatment groups.Comment: Minor revision: posterior predictive checks for BSL have ben updated
(both theory and results). Code on GitHub has ben revised accordingl
Partially Exchangeable Networks and Architectures for Learning Summary Statistics in Approximate Bayesian Computation
We present a novel family of deep neural architectures, named partially
exchangeable networks (PENs) that leverage probabilistic symmetries. By design,
PENs are invariant to block-switch transformations, which characterize the
partial exchangeability properties of conditionally Markovian processes.
Moreover, we show that any block-switch invariant function has a PEN-like
representation. The DeepSets architecture is a special case of PEN and we can
therefore also target fully exchangeable data. We employ PENs to learn summary
statistics in approximate Bayesian computation (ABC). When comparing PENs to
previous deep learning methods for learning summary statistics, our results are
highly competitive, both considering time series and static models. Indeed,
PENs provide more reliable posterior samples even when using less training
data.Comment: Forthcoming on the Proceedings of ICML 2019. New comparisons with
several different networks. We now use the Wasserstein distance to produce
comparisons. Code available on GitHub. 16 pages, 5 figures, 21 table
Accelerating delayed-acceptance Markov chain Monte Carlo algorithms
Delayed-acceptance Markov chain Monte Carlo (DA-MCMC) samples from a
probability distribution via a two-stages version of the Metropolis-Hastings
algorithm, by combining the target distribution with a "surrogate" (i.e. an
approximate and computationally cheaper version) of said distribution. DA-MCMC
accelerates MCMC sampling in complex applications, while still targeting the
exact distribution. We design a computationally faster, albeit approximate,
DA-MCMC algorithm. We consider parameter inference in a Bayesian setting where
a surrogate likelihood function is introduced in the delayed-acceptance scheme.
When the evaluation of the likelihood function is computationally intensive,
our scheme produces a 2-4 times speed-up, compared to standard DA-MCMC.
However, the acceleration is highly problem dependent. Inference results for
the standard delayed-acceptance algorithm and our approximated version are
similar, indicating that our algorithm can return reliable Bayesian inference.
As a computationally intensive case study, we introduce a novel stochastic
differential equation model for protein folding data.Comment: 40 pages, 21 figures, 10 table
Sequential Neural Posterior and Likelihood Approximation
We introduce the sequential neural posterior and likelihood approximation
(SNPLA) algorithm. SNPLA is a normalizing flows-based algorithm for inference
in implicit models, and therefore is a simulation-based inference method that
only requires simulations from a generative model. SNPLA avoids Markov chain
Monte Carlo sampling and correction-steps of the parameter proposal function
that are introduced in similar methods, but that can be numerically unstable or
restrictive. By utilizing the reverse KL divergence, SNPLA manages to learn
both the likelihood and the posterior in a sequential manner. Over four
experiments, we show that SNPLA performs competitively when utilizing the same
number of model simulations as used in other methods, even though the inference
problem for SNPLA is more complex due to the joint learning of posterior and
likelihood function. Due to utilizing normalizing flows SNPLA generates
posterior draws much faster (4 orders of magnitude) than MCMC-based methods.Comment: 28 pages, 8 tables, 14 figures. The supplementary material is
attached to the main pape
Statistical modeling of diabetic neuropathy: Exploring the dynamics of nerve mortality
Diabetic neuropathy is a disorder characterized by impaired nerve function
and reduction of the number of epidermal nerve fibers per epidermal surface.
Additionally, as neuropathy related nerve fiber loss and regrowth progresses
over time, the two-dimensional spatial arrangement of the nerves becomes more
clustered. These observations suggest that with development of neuropathy, the
spatial pattern of diminished skin innervation is defined by a thinning process
which remains incompletely characterized. We regard samples obtained from
healthy controls and subjects suffering from diabetic neuropathy as
realisations of planar point processes consisting of nerve entry points and
nerve endings, and propose point process models based on spatial thinning to
describe the change as neuropathy advances. Initially, the hypothesis that the
nerve removal occurs completely at random is tested using independent random
thinning of healthy patterns. Then, a dependent parametric thinning model that
favors the removal of isolated nerve trees is proposed. Approximate Bayesian
computation is used to infer the distribution of the model parameters, and the
goodness-of-fit of the models is evaluated using both non-spatial and spatial
summary statistics. Our findings suggest that the nerve mortality process
changes behaviour as neuropathy advances
Coupling stochastic EM and Approximate Bayesian computation for parameter inference in state-space models
International audienceWe study the class of state-space models (or hidden Markov models) and perform maximum likelihood inference on the model parameters. We consider a stochastic approximation expectation-maximization (SAEM) algorithm to maximize the likelihood function with the novelty of using approximate Bayesian computation (ABC) within SAEM. The task is to provide each iteration of SAEM with a filtered state of the system and this is achieved using ABC-SMC, that is we used an approximate sequential Monte Carlo (SMC) sampler for the hidden state. Three simulation studies are presented, first a nonlinear Gaussian state-space model then a state-space model having dynamics expressed by a stochastic differential equation, finally a stochastic volatility model. In our examples, ten iterations of our SAEM-ABC-SMC strategy were enough to return sensible parameter estimates. Comparisons with results using SAEM coupled with a standard, non-ABC, SMC sampler show that the ABC algorithm can be calibrated to return accurate solutions
Statistical modeling of diabetic neuropathy: Exploring the dynamics of nerve mortality
Diabetic neuropathy is a disorder characterized by impaired nerve function and reduction of the number of epidermal nerve fibers per epidermal surface. Additionally, as neuropathy related nerve fiber loss and regrowth progresses over time, the two-dimensional spatial arrangement of the nerves becomes more clustered. These observations suggest that with development of neuropathy, the spatial pattern of diminished skin innervation is defined by a thinning process which remains incompletely characterized. We regard samples obtained from healthy controls and subjects suffering from diabetic neuropathy as realisations of planar point processes consisting of nerve entry points and nerve endings, and propose point process models based on spatial thinning to describe the change as neuropathy advances. Initially, the hypothesis that the nerve removal occurs completely at random is tested using independent random thinning of healthy patterns. Then, a dependent parametric thinning model that favors the removal of isolated nerve trees is proposed. Approximate Bayesian computation is used to infer the distribution of the model parameters, and the goodness-of-fit of the models is evaluated using both non-spatial and spatial summary statistics. Our findings suggest that the nerve mortality process changes as neuropathy advances
JANA: Jointly Amortized Neural Approximation of Complex Bayesian Models
This work proposes ''jointly amortized neural approximation'' (JANA) of
intractable likelihood functions and posterior densities arising in Bayesian
surrogate modeling and simulation-based inference. We train three complementary
networks in an end-to-end fashion: 1) a summary network to compress individual
data points, sets, or time series into informative embedding vectors; 2) a
posterior network to learn an amortized approximate posterior; and 3) a
likelihood network to learn an amortized approximate likelihood. Their
interaction opens a new route to amortized marginal likelihood and posterior
predictive estimation -- two important ingredients of Bayesian workflows that
are often too expensive for standard methods. We benchmark the fidelity of JANA
on a variety of simulation models against state-of-the-art Bayesian methods and
propose a powerful and interpretable diagnostic for joint calibration. In
addition, we investigate the ability of recurrent likelihood networks to
emulate complex time series models without resorting to hand-crafted summary
statistics
- …