76 research outputs found
Double-Edge Factor Graphs: Definition, Properties, and Examples
Some of the most interesting quantities associated with a factor graph are
its marginals and its partition sum. For factor graphs \emph{without cycles}
and moderate message update complexities, the sum-product algorithm (SPA) can
be used to efficiently compute these quantities exactly. Moreover, for various
classes of factor graphs \emph{with cycles}, the SPA has been successfully
applied to efficiently compute good approximations to these quantities. Note
that in the case of factor graphs with cycles, the local functions are usually
non-negative real-valued functions. In this paper we introduce a class of
factor graphs, called double-edge factor graphs (DE-FGs), which allow local
functions to be complex-valued and only require them, in some suitable sense,
to be positive semi-definite. We discuss various properties of the SPA when
running it on DE-FGs and we show promising numerical results for various
example DE-FGs, some of which have connections to quantum information
processing.Comment: Submitte
Zero-Temperature Limit of a Convergent Algorithm to Minimize the Bethe Free Energy
After the discovery that fixed points of loopy belief propagation coincide
with stationary points of the Bethe free energy, several researchers proposed
provably convergent algorithms to directly minimize the Bethe free energy.
These algorithms were formulated only for non-zero temperature (thus finding
fixed points of the sum-product algorithm) and their possible extension to zero
temperature is not obvious. We present the zero-temperature limit of the
double-loop algorithm by Heskes, which converges a max-product fixed point. The
inner loop of this algorithm is max-sum diffusion. Under certain conditions,
the algorithm combines the complementary advantages of the max-product belief
propagation and max-sum diffusion (LP relaxation): it yields good approximation
of both ground states and max-marginals.Comment: Research Repor
Clamping improves TRW and mean field approximations
We examine the effect of clamping variables for approximate inference in
undirected graphical models with pairwise relationships and discrete variables.
For any number of variable labels, we demonstrate that clamping and summing
approximate sub-partition functions can lead only to a decrease in the
partition function estimate for TRW, and an increase for the naive mean field
method, in each case guaranteeing an improvement in the approximation and
bound. We next focus on binary variables, add the Bethe approximation to
consideration and examine ways to choose good variables to clamp, introducing
new methods. We show the importance of identifying highly frustrated cycles,
and of checking the singleton entropy of a variable. We explore the value of
our methods by empirical analysis and draw lessons to guide practitioners.NICTA is funded by the Australian Government through the Department of Communications and the Australian Research Council through the ICT Centre of Excellence Program.This is the author accepted manuscript. It is currently under an indefinite embargo pending publication by MIT Press
On Sampling from the Gibbs Distribution with Random Maximum A-Posteriori Perturbations
In this paper we describe how MAP inference can be used to sample efficiently
from Gibbs distributions. Specifically, we provide means for drawing either
approximate or unbiased samples from Gibbs' distributions by introducing low
dimensional perturbations and solving the corresponding MAP assignments. Our
approach also leads to new ways to derive lower bounds on partition functions.
We demonstrate empirically that our method excels in the typical "high signal -
high coupling" regime. The setting results in ragged energy landscapes that are
challenging for alternative approaches to sampling and/or lower bounds
Inferning 2012
We consider the problem of inference in a graphical model with binary variables. While in theory it is arguably preferable to compute marginal probabilities, in practice researchers often use MAP inference due to the availability of efficient discrete optimization algorithms. We bridge the gap between the two approaches by introducing the Discrete Marginals technique in which approximate marginals are obtained by minimizing an objective function with unary and pairwise terms over a discretized domain. This allows the use of techniques originally developed for MAP-MRF inference and learning. We explore two ways to set up the objective function - by discretizing the Bethe free energy and by learning it from training data. Experimental results show that for certain types of graphs a learned function can outperform the Bethe approximation. We also establish a link between the Bethe free energy and submodular functions
- …