8 research outputs found
Probabilistic Inference Modulo Theories
We present SGDPLL(T), an algorithm that solves (among many other problems)
probabilistic inference modulo theories, that is, inference problems over
probabilistic models defined via a logic theory provided as a parameter
(currently, propositional, equalities on discrete sorts, and inequalities, more
specifically difference arithmetic, on bounded integers). While many solutions
to probabilistic inference over logic representations have been proposed,
SGDPLL(T) is simultaneously (1) lifted, (2) exact and (3) modulo theories, that
is, parameterized by a background logic theory. This offers a foundation for
extending it to rich logic languages such as data structures and relational
data. By lifted, we mean algorithms with constant complexity in the domain size
(the number of values that variables can take). We also detail a solver for
summations with difference arithmetic and show experimental results from a
scenario in which SGDPLL(T) is much faster than a state-of-the-art
probabilistic solver.Comment: Submitted to StarAI-16 workshop as closely revised version of
IJCAI-16 pape
Approximate weighted model integration on DNF structures
Weighted model counting consists of computing the weighted sum of all satisfying assignments of a propositional formula. Weighted model counting is well-known to be #P-hard for exact solving, but admits a fully polynomial randomized approximation scheme when restricted to DNF structures. In this work, we study weighted model integration, a generalization of weighted model counting which involves real variables in addition to propositional variables, and pose the following question: Does weighted model integration on DNF structures admit a fully polynomial randomized approximation scheme? Building on classical results from approximate weighted model counting and approximate volume computation, we show that weighted model integration on DNF structures can indeed be approximated for a class of weight functions. Our approximation algorithm is based on three subroutines, each of which can be a weak (i.e., approximate), or a strong (i.e., exact) oracle, and in all cases, comes along with accuracy guarantees. We experimentally verify our approach over randomly generated DNF instances of varying sizes, and show that our algorithm scales to large problem instances, involving up to 1K variables, which are currently out of reach for existing, general-purpose weighted model integration solvers
Symbolic Variable Elimination for Discrete and Continuous Graphical Models
Probabilistic reasoning in the real-world often requires inference incontinuous variable graphical models, yet there are few methods for exact, closed-form inference when joint distributions are non-Gaussian. To address this inferential deficit, we introduce SVE -- a symbolic extension of the well-known variable elimination algorithm to perform exact inference in an expressive class of mixed discrete and continuous variable graphical models whose conditional probability functions can be well-approximated as piecewise combinations of polynomials with bounded support. Using this representation, we show that we can compute all of the SVE operations exactly and in closed-form, which crucially includes definite integration w.r.t. multivariate piecewise polynomial functions. To aid in the efficient computation and compact representation of this solution, we use an extended algebraic decision diagram (XADD) data structure that supports all SVE operations. We provide illustrative results for SVE on probabilistic inference queries inspired by robotics localization and tracking applications that mix various continuous distributions; this represents the first time a general closed-form exact solution has been proposed for this expressive class of discrete/continuous graphical models
Probabilistic Inference in Piecewise Graphical Models
In many applications of probabilistic inference the models
contain piecewise densities that are differentiable except at
partition boundaries. For instance, (1) some models may
intrinsically have finite support, being constrained to some
regions; (2) arbitrary density functions may be approximated by
mixtures of piecewise functions such as piecewise polynomials or
piecewise exponentials; (3) distributions derived from other
distributions (via random variable transformations) may be highly
piecewise; (4) in applications of Bayesian inference such as
Bayesian discrete classification and preference learning, the
likelihood functions may be piecewise; (5) context-specific
conditional probability density functions (tree-CPDs) are
intrinsically piecewise; (6) influence diagrams (generalizations
of Bayesian networks in which along with probabilistic inference,
decision making problems are modeled) are in many applications
piecewise; (7) in probabilistic programming, conditional
statements lead to piecewise models. As we will show, exact
inference on piecewise models is not often scalable (if
applicable) and the performance of the existing approximate
inference techniques on such models is usually quite poor.
This thesis fills this gap by presenting scalable and accurate
algorithms for inference in piecewise probabilistic graphical
models. Our first contribution is to present a variation of Gibbs
sampling algorithm that achieves an exponential sampling speedup
on a large class of models (including Bayesian models with
piecewise likelihood functions). As a second contribution, we
show that for a large range of models, the time-consuming Gibbs
sampling computations that are traditionally carried out per
sample, can be computed symbolically, once and prior to the
sampling process. Among many potential applications, the
resulting symbolic Gibbs sampler can be used for fully automated
reasoning in the presence of deterministic constraints among
random variables. As a third contribution, we are motivated by
the behavior of Hamiltonian dynamics in optics —in particular,
the reflection and refraction of light on the refractive
surfaces— to present a new Hamiltonian Monte Carlo method that
demonstrates a significantly improved performance on piecewise
models.
Hopefully, the present work represents a step towards scalable
and accurate inference in an important class of probabilistic
models that has largely been overlooked in the literature