4,016 research outputs found
Quantum Graphical Models and Belief Propagation
Belief Propagation algorithms acting on Graphical Models of classical
probability distributions, such as Markov Networks, Factor Graphs and Bayesian
Networks, are amongst the most powerful known methods for deriving
probabilistic inferences amongst large numbers of random variables. This paper
presents a generalization of these concepts and methods to the quantum case,
based on the idea that quantum theory can be thought of as a noncommutative,
operator-valued, generalization of classical probability theory. Some novel
characterizations of quantum conditional independence are derived, and
definitions of Quantum n-Bifactor Networks, Markov Networks, Factor Graphs and
Bayesian Networks are proposed. The structure of Quantum Markov Networks is
investigated and some partial characterization results are obtained, along the
lines of the Hammersely-Clifford theorem. A Quantum Belief Propagation
algorithm is presented and is shown to converge on 1-Bifactor Networks and
Markov Networks when the underlying graph is a tree. The use of Quantum Belief
Propagation as a heuristic algorithm in cases where it is not known to converge
is discussed. Applications to decoding quantum error correcting codes and to
the simulation of many-body quantum systems are described.Comment: 58 pages, 9 figure
Pairwise MRF Calibration by Perturbation of the Bethe Reference Point
We investigate different ways of generating approximate solutions to the
pairwise Markov random field (MRF) selection problem. We focus mainly on the
inverse Ising problem, but discuss also the somewhat related inverse Gaussian
problem because both types of MRF are suitable for inference tasks with the
belief propagation algorithm (BP) under certain conditions. Our approach
consists in to take a Bethe mean-field solution obtained with a maximum
spanning tree (MST) of pairwise mutual information, referred to as the
\emph{Bethe reference point}, for further perturbation procedures. We consider
three different ways following this idea: in the first one, we select and
calibrate iteratively the optimal links to be added starting from the Bethe
reference point; the second one is based on the observation that the natural
gradient can be computed analytically at the Bethe point; in the third one,
assuming no local field and using low temperature expansion we develop a dual
loop joint model based on a well chosen fundamental cycle basis. We indeed
identify a subclass of planar models, which we refer to as \emph{Bethe-dual
graph models}, having possibly many loops, but characterized by a singly
connected dual factor graph, for which the partition function and the linear
response can be computed exactly in respectively O(N) and operations,
thanks to a dual weight propagation (DWP) message passing procedure that we set
up. When restricted to this subclass of models, the inverse Ising problem being
convex, becomes tractable at any temperature. Experimental tests on various
datasets with refined or regularization procedures indicate that
these approaches may be competitive and useful alternatives to existing ones.Comment: 54 pages, 8 figure. section 5 and refs added in V
Learning Deep Structured Models
Many problems in real-world applications involve predicting several random
variables which are statistically related. Markov random fields (MRFs) are a
great mathematical tool to encode such relationships. The goal of this paper is
to combine MRFs with deep learning algorithms to estimate complex
representations while taking into account the dependencies between the output
random variables. Towards this goal, we propose a training algorithm that is
able to learn structured models jointly with deep features that form the MRF
potentials. Our approach is efficient as it blends learning and inference and
makes use of GPU acceleration. We demonstrate the effectiveness of our
algorithm in the tasks of predicting words from noisy images, as well as
multi-class classification of Flickr photographs. We show that joint learning
of the deep features and the MRF parameters results in significant performance
gains.Comment: 11 pages including referenc
- …