261 research outputs found
On the Complexity and Approximation of Binary Evidence in Lifted Inference
Lifted inference algorithms exploit symmetries in probabilistic models to
speed up inference. They show impressive performance when calculating
unconditional probabilities in relational models, but often resort to
non-lifted inference when computing conditional probabilities. The reason is
that conditioning on evidence breaks many of the model's symmetries, which can
preempt standard lifting techniques. Recent theoretical results show, for
example, that conditioning on evidence which corresponds to binary relations is
#P-hard, suggesting that no lifting is to be expected in the worst case. In
this paper, we balance this negative result by identifying the Boolean rank of
the evidence as a key parameter for characterizing the complexity of
conditioning in lifted inference. In particular, we show that conditioning on
binary evidence with bounded Boolean rank is efficient. This opens up the
possibility of approximating evidence by a low-rank Boolean matrix
factorization, which we investigate both theoretically and empirically.Comment: To appear in Advances in Neural Information Processing Systems 26
(NIPS), Lake Tahoe, USA, December 201
Learning Logistic Circuits
This paper proposes a new classification model called logistic circuits. On
MNIST and Fashion datasets, our learning algorithm outperforms neural networks
that have an order of magnitude more parameters. Yet, logistic circuits have a
distinct origin in symbolic AI, forming a discriminative counterpart to
probabilistic-logical circuits such as ACs, SPNs, and PSDDs. We show that
parameter learning for logistic circuits is convex optimization, and that a
simple local search algorithm can induce strong model structures from data.Comment: Published in the Proceedings of the Thirty-Third AAAI Conference on
Artificial Intelligence (AAAI19
Tractability through Exchangeability: A New Perspective on Efficient Probabilistic Inference
Exchangeability is a central notion in statistics and probability theory. The
assumption that an infinite sequence of data points is exchangeable is at the
core of Bayesian statistics. However, finite exchangeability as a statistical
property that renders probabilistic inference tractable is less
well-understood. We develop a theory of finite exchangeability and its relation
to tractable probabilistic inference. The theory is complementary to that of
independence and conditional independence. We show that tractable inference in
probabilistic models with high treewidth and millions of variables can be
understood using the notion of finite (partial) exchangeability. We also show
that existing lifted inference algorithms implicitly utilize a combination of
conditional independence and partial exchangeability.Comment: In Proceedings of the 28th AAAI Conference on Artificial Intelligenc
On the Role of Canonicity in Bottom-up Knowledge Compilation
We consider the problem of bottom-up compilation of knowledge bases, which is
usually predicated on the existence of a polytime function for combining
compilations using Boolean operators (usually called an Apply function). While
such a polytime Apply function is known to exist for certain languages (e.g.,
OBDDs) and not exist for others (e.g., DNNF), its existence for certain
languages remains unknown. Among the latter is the recently introduced language
of Sentential Decision Diagrams (SDDs), for which a polytime Apply function
exists for unreduced SDDs, but remains unknown for reduced ones (i.e. canonical
SDDs). We resolve this open question in this paper and consider some of its
theoretical and practical implications. Some of the findings we report question
the common wisdom on the relationship between bottom-up compilation, language
canonicity and the complexity of the Apply function
Generating and Sampling Orbits for Lifted Probabilistic Inference
A key goal in the design of probabilistic inference algorithms is identifying
and exploiting properties of the distribution that make inference tractable.
Lifted inference algorithms identify symmetry as a property that enables
efficient inference and seek to scale with the degree of symmetry of a
probability model. A limitation of existing exact lifted inference techniques
is that they do not apply to non-relational representations like factor graphs.
In this work we provide the first example of an exact lifted inference
algorithm for arbitrary discrete factor graphs. In addition we describe a
lifted Markov-Chain Monte-Carlo algorithm that provably mixes rapidly in the
degree of symmetry of the distribution
Lifted Relax, Compensate and then Recover: From Approximate to Exact Lifted Probabilistic Inference
We propose an approach to lifted approximate inference for first-order
probabilistic models, such as Markov logic networks. It is based on performing
exact lifted inference in a simplified first-order model, which is found by
relaxing first-order constraints, and then compensating for the relaxation.
These simplified models can be incrementally improved by carefully recovering
constraints that have been relaxed, also at the first-order level. This leads
to a spectrum of approximations, with lifted belief propagation on one end, and
exact lifted inference on the other. We discuss how relaxation, compensation,
and recovery can be performed, all at the firstorder level, and show
empirically that our approach substantially improves on the approximations of
both propositional solvers and lifted belief propagation.Comment: Appears in Proceedings of the Twenty-Eighth Conference on Uncertainty
in Artificial Intelligence (UAI2012
Understanding the Complexity of Lifted Inference and Asymmetric Weighted Model Counting
In this paper we study lifted inference for the Weighted First-Order Model
Counting problem (WFOMC), which counts the assignments that satisfy a given
sentence in first-order logic (FOL); it has applications in Statistical
Relational Learning (SRL) and Probabilistic Databases (PDB). We present several
results. First, we describe a lifted inference algorithm that generalizes prior
approaches in SRL and PDB. Second, we provide a novel dichotomy result for a
non-trivial fragment of FO CNF sentences, showing that for each sentence the
WFOMC problem is either in PTIME or #P-hard in the size of the input domain; we
prove that, in the first case our algorithm solves the WFOMC problem in PTIME,
and in the second case it fails. Third, we present several properties of the
algorithm. Finally, we discuss limitations of lifted inference for symmetric
probabilistic databases (where the weights of ground literals depend only on
the relation name, and not on the constants of the domain), and prove the
impossibility of a dichotomy result for the complexity of probabilistic
inference for the entire language FOL
Symbolic Exact Inference for Discrete Probabilistic Programs
The computational burden of probabilistic inference remains a hurdle for
applying probabilistic programming languages to practical problems of interest.
In this work, we provide a semantic and algorithmic foundation for efficient
exact inference on discrete-valued finite-domain imperative probabilistic
programs. We leverage and generalize efficient inference procedures for
Bayesian networks, which exploit the structure of the network to decompose the
inference task, thereby avoiding full path enumeration. To do this, we first
compile probabilistic programs to a symbolic representation. Then we adapt
techniques from the probabilistic logic programming and artificial intelligence
communities in order to perform inference on the symbolic representation. We
formalize our approach, prove it sound, and experimentally validate it against
existing exact and approximate inference techniques. We show that our inference
approach is competitive with inference procedures specialized for Bayesian
networks, thereby expanding the class of probabilistic programs that can be
practically analyzed
Probabilistic Program Abstractions
Abstraction is a fundamental tool for reasoning about complex systems.
Program abstraction has been utilized to great effect for analyzing
deterministic programs. At the heart of program abstraction is the relationship
between a concrete program, which is difficult to analyze, and an abstract
program, which is more tractable. Program abstractions, however, are typically
not probabilistic. We generalize non-deterministic program abstractions to
probabilistic program abstractions by explicitly quantifying the
non-deterministic choices. Our framework upgrades key definitions and
properties of abstractions to the probabilistic context. We also discuss
preliminary ideas for performing inference on probabilistic abstractions and
general probabilistic programs
- …