343 research outputs found
Amortized Rejection Sampling in Universal Probabilistic Programming
Existing approaches to amortized inference in probabilistic programs with
unbounded loops can produce estimators with infinite variance. An instance of
this is importance sampling inference in programs that explicitly include
rejection sampling as part of the user-programmed generative procedure. In this
paper we develop a new and efficient amortized importance sampling estimator.
We prove finite variance of our estimator and empirically demonstrate our
method's correctness and efficiency compared to existing alternatives on
generative programs containing rejection sampling loops and discuss how to
implement our method in a generic probabilistic programming framework
Efficient Probabilistic Inference in the Quest for Physics Beyond the Standard Model
We present a novel probabilistic programming framework that couples directly
to existing large-scale simulators through a cross-platform probabilistic
execution protocol, which allows general-purpose inference engines to record
and control random number draws within simulators in a language-agnostic way.
The execution of existing simulators as probabilistic programs enables highly
interpretable posterior inference in the structured model defined by the
simulator code base. We demonstrate the technique in particle physics, on a
scientifically accurate simulation of the tau lepton decay, which is a key
ingredient in establishing the properties of the Higgs boson. Inference
efficiency is achieved via inference compilation where a deep recurrent neural
network is trained to parameterize proposal distributions and control the
stochastic simulator in a sequential importance sampling scheme, at a fraction
of the computational cost of a Markov chain Monte Carlo baseline.Comment: 20 pages, 9 figure
Training deep neural density estimators to identify mechanistic models of neural dynamics
Mechanistic modeling in neuroscience aims to explain observed phenomena in terms of underlying causes. However, determining which model parameters agree with complex and stochastic neural data presents a significant challenge. We address this challenge with a machine learning tool which uses deep neural density estimators-- trained using model simulations-- to carry out Bayesian inference and retrieve the full space of parameters compatible with raw data or selected data features. Our method is scalable in parameters and data features, and can rapidly analyze new data after initial training. We demonstrate the power and flexibility of our approach on receptive fields, ion channels, and Hodgkin-Huxley models. We also characterize the space of circuit configurations giving rise to rhythmic activity in the crustacean stomatogastric ganglion, and use these results to derive hypotheses for underlying compensation mechanisms. Our approach will help close the gap between data-driven and theory-driven models of neural dynamics
Some models are useful, but how do we know which ones? Towards a unified Bayesian model taxonomy
Probabilistic (Bayesian) modeling has experienced a surge of applications in
almost all quantitative sciences and industrial areas. This development is
driven by a combination of several factors, including better probabilistic
estimation algorithms, flexible software, increased computing power, and a
growing awareness of the benefits of probabilistic learning. However, a
principled Bayesian model building workflow is far from complete and many
challenges remain. To aid future research and applications of a principled
Bayesian workflow, we ask and provide answers for what we perceive as two
fundamental questions of Bayesian modeling, namely (a) "What actually is a
Bayesian model?" and (b) "What makes a good Bayesian model?". As an answer to
the first question, we propose the PAD model taxonomy that defines four basic
kinds of Bayesian models, each representing some combination of the assumed
joint distribution of all (known or unknown) variables (P), a posterior
approximator (A), and training data (D). As an answer to the second question,
we propose ten utility dimensions according to which we can evaluate Bayesian
models holistically, namely, (1) causal consistency, (2) parameter
recoverability, (3) predictive performance, (4) fairness, (5) structural
faithfulness, (6) parsimony, (7) interpretability, (8) convergence, (9)
estimation speed, and (10) robustness. Further, we propose two example utility
decision trees that describe hierarchies and trade-offs between utilities
depending on the inferential goals that drive model building and testing
Sampling for Bayesian program learning
Towards learning programs from data, we introduce the problem of sampling programs from posterior distributions conditioned on that data. Within this setting, we propose an algorithm that uses a symbolic solver to efficiently sample programs. The proposal combines constraint-based program synthesis with sampling via random parity constraints. We give theoretical guarantees on how well the samples approximate the true posterior, and have empirical results showing the algorithm is efficient in practice, evaluating our approach on 22 program learning problems in the domains of text editing and computer-aided programming.National Science Foundation (U.S.) (Award NSF-1161775)United States. Air Force Office of Scientific Research (Award FA9550-16-1-0012
- …