16,237 research outputs found
Neural Probabilistic Logic Programming in Discrete-Continuous Domains
Neural-symbolic AI (NeSy) allows neural networks to exploit symbolic
background knowledge in the form of logic. It has been shown to aid learning in
the limited data regime and to facilitate inference on out-of-distribution
data. Probabilistic NeSy focuses on integrating neural networks with both logic
and probability theory, which additionally allows learning under uncertainty. A
major limitation of current probabilistic NeSy systems, such as DeepProbLog, is
their restriction to finite probability distributions, i.e., discrete random
variables. In contrast, deep probabilistic programming (DPP) excels in
modelling and optimising continuous probability distributions. Hence, we
introduce DeepSeaProbLog, a neural probabilistic logic programming language
that incorporates DPP techniques into NeSy. Doing so results in the support of
inference and learning of both discrete and continuous probability
distributions under logical constraints. Our main contributions are 1) the
semantics of DeepSeaProbLog and its corresponding inference algorithm, 2) a
proven asymptotically unbiased learning algorithm, and 3) a series of
experiments that illustrate the versatility of our approach.Comment: 27 pages, 9 figure
Inference in Probabilistic Logic Programs with Continuous Random Variables
Probabilistic Logic Programming (PLP), exemplified by Sato and Kameya's
PRISM, Poole's ICL, Raedt et al's ProbLog and Vennekens et al's LPAD, is aimed
at combining statistical and logical knowledge representation and inference. A
key characteristic of PLP frameworks is that they are conservative extensions
to non-probabilistic logic programs which have been widely used for knowledge
representation. PLP frameworks extend traditional logic programming semantics
to a distribution semantics, where the semantics of a probabilistic logic
program is given in terms of a distribution over possible models of the
program. However, the inference techniques used in these works rely on
enumerating sets of explanations for a query answer. Consequently, these
languages permit very limited use of random variables with continuous
distributions. In this paper, we present a symbolic inference procedure that
uses constraints and represents sets of explanations without enumeration. This
permits us to reason over PLPs with Gaussian or Gamma-distributed random
variables (in addition to discrete-valued random variables) and linear equality
constraints over reals. We develop the inference procedure in the context of
PRISM; however the procedure's core ideas can be easily applied to other PLP
languages as well. An interesting aspect of our inference procedure is that
PRISM's query evaluation process becomes a special case in the absence of any
continuous random variables in the program. The symbolic inference procedure
enables us to reason over complex probabilistic models such as Kalman filters
and a large subclass of Hybrid Bayesian networks that were hitherto not
possible in PLP frameworks. (To appear in Theory and Practice of Logic
Programming).Comment: 12 pages. arXiv admin note: substantial text overlap with
arXiv:1203.428
Probabilistic Integral Circuits
Continuous latent variables (LVs) are a key ingredient of many generative
models, as they allow modelling expressive mixtures with an uncountable number
of components. In contrast, probabilistic circuits (PCs) are hierarchical
discrete mixtures represented as computational graphs composed of input, sum
and product units. Unlike continuous LV models, PCs provide tractable inference
but are limited to discrete LVs with categorical (i.e. unordered) states. We
bridge these model classes by introducing probabilistic integral circuits
(PICs), a new language of computational graphs that extends PCs with integral
units representing continuous LVs. In the first place, PICs are symbolic
computational graphs and are fully tractable in simple cases where analytical
integration is possible. In practice, we parameterise PICs with light-weight
neural nets delivering an intractable hierarchical continuous mixture that can
be approximated arbitrarily well with large PCs using numerical quadrature. On
several distribution estimation benchmarks, we show that such PIC-approximating
PCs systematically outperform PCs commonly learned via expectation-maximization
or SGD
Probabilistic Inference from Arbitrary Uncertainty using Mixtures of Factorized Generalized Gaussians
This paper presents a general and efficient framework for probabilistic
inference and learning from arbitrary uncertain information. It exploits the
calculation properties of finite mixture models, conjugate families and
factorization. Both the joint probability density of the variables and the
likelihood function of the (objective or subjective) observation are
approximated by a special mixture model, in such a way that any desired
conditional distribution can be directly obtained without numerical
integration. We have developed an extended version of the expectation
maximization (EM) algorithm to estimate the parameters of mixture models from
uncertain training examples (indirect observations). As a consequence, any
piece of exact or uncertain information about both input and output values is
consistently handled in the inference and learning stages. This ability,
extremely useful in certain situations, is not found in most alternative
methods. The proposed framework is formally justified from standard
probabilistic principles and illustrative examples are provided in the fields
of nonparametric pattern classification, nonlinear regression and pattern
completion. Finally, experiments on a real application and comparative results
over standard databases provide empirical evidence of the utility of the method
in a wide range of applications
- …