1,176 research outputs found
Kernel Belief Propagation
We propose a nonparametric generalization of belief propagation, Kernel
Belief Propagation (KBP), for pairwise Markov random fields. Messages are
represented as functions in a reproducing kernel Hilbert space (RKHS), and
message updates are simple linear operations in the RKHS. KBP makes none of the
assumptions commonly required in classical BP algorithms: the variables need
not arise from a finite domain or a Gaussian distribution, nor must their
relations take any particular parametric form. Rather, the relations between
variables are represented implicitly, and are learned nonparametrically from
training data. KBP has the advantage that it may be used on any domain where
kernels are defined (Rd, strings, groups), even where explicit parametric
models are not known, or closed form expressions for the BP updates do not
exist. The computational cost of message updates in KBP is polynomial in the
training data size. We also propose a constant time approximate message update
procedure by representing messages using a small number of basis functions. In
experiments, we apply KBP to image denoising, depth prediction from still
images, and protein configuration prediction: KBP is faster than competing
classical and nonparametric approaches (by orders of magnitude, in some cases),
while providing significantly more accurate results
Bethe Projections for Non-Local Inference
Many inference problems in structured prediction are naturally solved by
augmenting a tractable dependency structure with complex, non-local auxiliary
objectives. This includes the mean field family of variational inference
algorithms, soft- or hard-constrained inference using Lagrangian relaxation or
linear programming, collective graphical models, and forms of semi-supervised
learning such as posterior regularization. We present a method to
discriminatively learn broad families of inference objectives, capturing
powerful non-local statistics of the latent variables, while maintaining
tractable and provably fast inference using non-Euclidean projected gradient
descent with a distance-generating function given by the Bethe entropy. We
demonstrate the performance and flexibility of our method by (1) extracting
structured citations from research papers by learning soft global constraints,
(2) achieving state-of-the-art results on a widely-used handwriting recognition
task using a novel learned non-convex inference procedure, and (3) providing a
fast and highly scalable algorithm for the challenging problem of inference in
a collective graphical model applied to bird migration.Comment: minor bug fix to appendix. appeared in UAI 201
Gaussian Belief with dynamic data and in dynamic network
In this paper we analyse Belief Propagation over a Gaussian model in a
dynamic environment. Recently, this has been proposed as a method to average
local measurement values by a distributed protocol ("Consensus Propagation",
Moallemi & Van Roy, 2006), where the average is available for read-out at every
single node. In the case that the underlying network is constant but the values
to be averaged fluctuate ("dynamic data"), convergence and accuracy are
determined by the spectral properties of an associated Ruelle-Perron-Frobenius
operator. For Gaussian models on Erdos-Renyi graphs, numerical computation
points to a spectral gap remaining in the large-size limit, implying
exceptionally good scalability. In a model where the underlying network also
fluctuates ("dynamic network"), averaging is more effective than in the dynamic
data case. Altogether, this implies very good performance of these methods in
very large systems, and opens a new field of statistical physics of large (and
dynamic) information systems.Comment: 5 pages, 7 figure
Quantum Graphical Models and Belief Propagation
Belief Propagation algorithms acting on Graphical Models of classical
probability distributions, such as Markov Networks, Factor Graphs and Bayesian
Networks, are amongst the most powerful known methods for deriving
probabilistic inferences amongst large numbers of random variables. This paper
presents a generalization of these concepts and methods to the quantum case,
based on the idea that quantum theory can be thought of as a noncommutative,
operator-valued, generalization of classical probability theory. Some novel
characterizations of quantum conditional independence are derived, and
definitions of Quantum n-Bifactor Networks, Markov Networks, Factor Graphs and
Bayesian Networks are proposed. The structure of Quantum Markov Networks is
investigated and some partial characterization results are obtained, along the
lines of the Hammersely-Clifford theorem. A Quantum Belief Propagation
algorithm is presented and is shown to converge on 1-Bifactor Networks and
Markov Networks when the underlying graph is a tree. The use of Quantum Belief
Propagation as a heuristic algorithm in cases where it is not known to converge
is discussed. Applications to decoding quantum error correcting codes and to
the simulation of many-body quantum systems are described.Comment: 58 pages, 9 figure
A Homological Approach to Belief Propagation and Bethe Approximations
We introduce a differential complex of local observables given a
decomposition of a global set of random variables into subsets. Its boundary
operator allows us to define a transport equation equivalent to Belief
Propagation. This definition reveals a set of conserved quantities under Belief
Propagation and gives new insight on the relationship of its equilibria with
the critical points of Bethe free energy.Comment: 14 pages, submitted for the 2019 Geometric Science of Information
colloquiu
- …