13 research outputs found
Efficient Probabilistic Inference in the Quest for Physics Beyond the Standard Model
We present a novel probabilistic programming framework that couples directly
to existing large-scale simulators through a cross-platform probabilistic
execution protocol, which allows general-purpose inference engines to record
and control random number draws within simulators in a language-agnostic way.
The execution of existing simulators as probabilistic programs enables highly
interpretable posterior inference in the structured model defined by the
simulator code base. We demonstrate the technique in particle physics, on a
scientifically accurate simulation of the tau lepton decay, which is a key
ingredient in establishing the properties of the Higgs boson. Inference
efficiency is achieved via inference compilation where a deep recurrent neural
network is trained to parameterize proposal distributions and control the
stochastic simulator in a sequential importance sampling scheme, at a fraction
of the computational cost of a Markov chain Monte Carlo baseline.Comment: 20 pages, 9 figure
Distilling importance sampling
The two main approaches to Bayesian inference are sampling and optimisation
methods. However many complicated posteriors are difficult to approximate by
either. Therefore we propose a novel approach combining features of both. We
use a flexible parameterised family of densities, such as a normalising flow.
Given a density from this family approximating the posterior, we use importance
sampling to produce a weighted sample from a more accurate posterior
approximation. This sample is then used in optimisation to update the
parameters of the approximate density, which we view as distilling the
importance sampling results. We iterate these steps and gradually improve the
quality of the posterior approximation. We illustrate our method in two
challenging examples: a queueing model and a stochastic differential equation
model.Comment: This version adds a second application, and fixes some minor error
Amortized Rejection Sampling in Universal Probabilistic Programming
Existing approaches to amortized inference in probabilistic programs with
unbounded loops can produce estimators with infinite variance. An instance of
this is importance sampling inference in programs that explicitly include
rejection sampling as part of the user-programmed generative procedure. In this
paper we develop a new and efficient amortized importance sampling estimator.
We prove finite variance of our estimator and empirically demonstrate our
method's correctness and efficiency compared to existing alternatives on
generative programs containing rejection sampling loops and discuss how to
implement our method in a generic probabilistic programming framework
Divide, Conquer, and Combine: a New Inference Strategy for Probabilistic Programs with Stochastic Support
Universal probabilistic programming systems (PPSs) provide a powerful
framework for specifying rich probabilistic models. They further attempt to
automate the process of drawing inferences from these models, but doing this
successfully is severely hampered by the wide range of non--standard models
they can express. As a result, although one can specify complex models in a
universal PPS, the provided inference engines often fall far short of what is
required. In particular, we show that they produce surprisingly unsatisfactory
performance for models where the support varies between executions, often doing
no better than importance sampling from the prior. To address this, we
introduce a new inference framework: Divide, Conquer, and Combine, which
remains efficient for such models, and show how it can be implemented as an
automated and generic PPS inference engine. We empirically demonstrate
substantial performance improvements over existing approaches on three
examples.Comment: Published at the 37th International Conference on Machine Learning
(ICML 2020
Planning as Inference in Epidemiological Models
In this work we demonstrate how existing software tools can be used to
automate parts of infectious disease-control policy-making via performing
inference in existing epidemiological dynamics models. The kind of inference
tasks undertaken include computing, for planning purposes, the posterior
distribution over putatively controllable, via direct policy-making choices,
simulation model parameters that give rise to acceptable disease progression
outcomes. Neither the full capabilities of such inference automation software
tools nor their utility for planning is widely disseminated at the current
time. Timely gains in understanding about these tools and how they can be used
may lead to more fine-grained and less economically damaging policy
prescriptions, particularly during the current COVID-19 pandemic.Comment: minor typos correcte