4,701 research outputs found
Probabilistic Programming Concepts
A multitude of different probabilistic programming languages exists today,
all extending a traditional programming language with primitives to support
modeling of complex, structured probability distributions. Each of these
languages employs its own probabilistic primitives, and comes with a particular
syntax, semantics and inference procedure. This makes it hard to understand the
underlying programming concepts and appreciate the differences between the
different languages. To obtain a better understanding of probabilistic
programming, we identify a number of core programming concepts underlying the
primitives used by various probabilistic languages, discuss the execution
mechanisms that they require and use these to position state-of-the-art
probabilistic languages and their implementation. While doing so, we focus on
probabilistic extensions of logic programming languages such as Prolog, which
have been developed since more than 20 years
A Compilation Target for Probabilistic Programming Languages
Forward inference techniques such as sequential Monte Carlo and particle
Markov chain Monte Carlo for probabilistic programming can be implemented in
any programming language by creative use of standardized operating system
functionality including processes, forking, mutexes, and shared memory.
Exploiting this we have defined, developed, and tested a probabilistic
programming language intermediate representation language we call probabilistic
C, which itself can be compiled to machine code by standard compilers and
linked to operating system libraries yielding an efficient, scalable, portable
probabilistic programming compilation target. This opens up a new hardware and
systems research path for optimizing probabilistic programming systems.Comment: In Proceedings of the 31st International Conference on Machine
Learning (ICML), 201
Extending Stan for Deep Probabilistic Programming
Stan is a popular declarative probabilistic programming language with a
high-level syntax for expressing graphical models and beyond. Stan differs by
nature from generative probabilistic programming languages like Church,
Anglican, or Pyro. This paper presents a comprehensive compilation scheme to
compile any Stan model to a generative language and proves its correctness.
This sheds a clearer light on the relative expressiveness of different kinds of
probabilistic languages and opens the door to combining their mutual strengths.
Specifically, we use our compilation scheme to build a compiler from Stan to
Pyro and extend Stan with support for explicit variational inference guides and
deep probabilistic models. That way, users familiar with Stan get access to new
features without having to learn a fundamentally new language. Overall, our
paper clarifies the relationship between declarative and generative
probabilistic programming languages and is a step towards making deep
probabilistic programming easier
- …