54,031 research outputs found
Bayesian Optimization for Probabilistic Programs
We present the first general purpose framework for marginal maximum a
posteriori estimation of probabilistic program variables. By using a series of
code transformations, the evidence of any probabilistic program, and therefore
of any graphical model, can be optimized with respect to an arbitrary subset of
its sampled variables. To carry out this optimization, we develop the first
Bayesian optimization package to directly exploit the source code of its
target, leading to innovations in problem-independent hyperpriors, unbounded
optimization, and implicit constraint satisfaction; delivering significant
performance improvements over prominent existing packages. We present
applications of our method to a number of tasks including engineering design
and parameter optimization
Delayed Sampling and Automatic Rao-Blackwellization of Probabilistic Programs
We introduce a dynamic mechanism for the solution of analytically-tractable
substructure in probabilistic programs, using conjugate priors and affine
transformations to reduce variance in Monte Carlo estimators. For inference
with Sequential Monte Carlo, this automatically yields improvements such as
locally-optimal proposals and Rao-Blackwellization. The mechanism maintains a
directed graph alongside the running program that evolves dynamically as
operations are triggered upon it. Nodes of the graph represent random
variables, edges the analytically-tractable relationships between them. Random
variables remain in the graph for as long as possible, to be sampled only when
they are used by the program in a way that cannot be resolved analytically. In
the meantime, they are conditioned on as many observations as possible. We
demonstrate the mechanism with a few pedagogical examples, as well as a
linear-nonlinear state-space model with simulated data, and an epidemiological
model with real data of a dengue outbreak in Micronesia. In all cases one or
more variables are automatically marginalized out to significantly reduce
variance in estimates of the marginal likelihood, in the final case
facilitating a random-weight or pseudo-marginal-type importance sampler for
parameter estimation. We have implemented the approach in Anglican and a new
probabilistic programming language called Birch.Comment: 13 pages, 4 figure
Probabilistic Constraint Logic Programming
This paper addresses two central problems for probabilistic processing
models: parameter estimation from incomplete data and efficient retrieval of
most probable analyses. These questions have been answered satisfactorily only
for probabilistic regular and context-free models. We address these problems
for a more expressive probabilistic constraint logic programming model. We
present a log-linear probability model for probabilistic constraint logic
programming. On top of this model we define an algorithm to estimate the
parameters and to select the properties of log-linear models from incomplete
data. This algorithm is an extension of the improved iterative scaling
algorithm of Della-Pietra, Della-Pietra, and Lafferty (1995). Our algorithm
applies to log-linear models in general and is accompanied with suitable
approximation methods when applied to large data spaces. Furthermore, we
present an approach for searching for most probable analyses of the
probabilistic constraint logic programming model. This method can be applied to
the ambiguity resolution problem in natural language processing applications.Comment: 35 pages, uses sfbart.cl
A Compilation Target for Probabilistic Programming Languages
Forward inference techniques such as sequential Monte Carlo and particle
Markov chain Monte Carlo for probabilistic programming can be implemented in
any programming language by creative use of standardized operating system
functionality including processes, forking, mutexes, and shared memory.
Exploiting this we have defined, developed, and tested a probabilistic
programming language intermediate representation language we call probabilistic
C, which itself can be compiled to machine code by standard compilers and
linked to operating system libraries yielding an efficient, scalable, portable
probabilistic programming compilation target. This opens up a new hardware and
systems research path for optimizing probabilistic programming systems.Comment: In Proceedings of the 31st International Conference on Machine
Learning (ICML), 201
Evaluating probabilistic programming languages for simulating quantum correlations
This article explores how probabilistic programming can be used to simulate
quantum correlations in an EPR experimental setting. Probabilistic programs are
based on standard probability which cannot produce quantum correlations. In
order to address this limitation, a hypergraph formalism was programmed which
both expresses the measurement contexts of the EPR experimental design as well
as associated constraints. Four contemporary open source probabilistic
programming frameworks were used to simulate an EPR experiment in order to shed
light on their relative effectiveness from both qualitative and quantitative
dimensions. We found that all four probabilistic languages successfully
simulated quantum correlations. Detailed analysis revealed that no language was
clearly superior across all dimensions, however, the comparison does highlight
aspects that can be considered when using probabilistic programs to simulate
experiments in quantum physics.Comment: 24 pages, 8 figures, code is available at
https://github.com/askoj/bell-ppl
- …