57 research outputs found
Sufficient conditions for convergence of the Sum-Product Algorithm
We derive novel conditions that guarantee convergence of the Sum-Product
algorithm (also known as Loopy Belief Propagation or simply Belief Propagation)
to a unique fixed point, irrespective of the initial messages. The
computational complexity of the conditions is polynomial in the number of
variables. In contrast with previously existing conditions, our results are
directly applicable to arbitrary factor graphs (with discrete variables) and
are shown to be valid also in the case of factors containing zeros, under some
additional conditions. We compare our bounds with existing ones, numerically
and, if possible, analytically. For binary variables with pairwise
interactions, we derive sufficient conditions that take into account local
evidence (i.e., single variable factors) and the type of pair interactions
(attractive or repulsive). It is shown empirically that this bound outperforms
existing bounds.Comment: 15 pages, 5 figures. Major changes and new results in this revised
version. Submitted to IEEE Transactions on Information Theor
Local stability of Belief Propagation algorithm with multiple fixed points
A number of problems in statistical physics and computer science can be
expressed as the computation of marginal probabilities over a Markov random
field. Belief propagation, an iterative message-passing algorithm, computes
exactly such marginals when the underlying graph is a tree. But it has gained
its popularity as an efficient way to approximate them in the more general
case, even if it can exhibits multiple fixed points and is not guaranteed to
converge. In this paper, we express a new sufficient condition for local
stability of a belief propagation fixed point in terms of the graph structure
and the beliefs values at the fixed point. This gives credence to the usual
understanding that Belief Propagation performs better on sparse graphs.Comment: arXiv admin note: substantial text overlap with arXiv:1101.417
The Role of Normalization in the Belief Propagation Algorithm
An important part of problems in statistical physics and computer science can
be expressed as the computation of marginal probabilities over a Markov Random
Field. The belief propagation algorithm, which is an exact procedure to compute
these marginals when the underlying graph is a tree, has gained its popularity
as an efficient way to approximate them in the more general case. In this
paper, we focus on an aspect of the algorithm that did not get that much
attention in the literature, which is the effect of the normalization of the
messages. We show in particular that, for a large class of normalization
strategies, it is possible to focus only on belief convergence. Following this,
we express the necessary and sufficient conditions for local stability of a
fixed point in terms of the graph structure and the beliefs values at the fixed
point. We also explicit some connexion between the normalization constants and
the underlying Bethe Free Energy
Inferning 2012
We consider the problem of inference in a graphical model with binary variables. While in theory it is arguably preferable to compute marginal probabilities, in practice researchers often use MAP inference due to the availability of efficient discrete optimization algorithms. We bridge the gap between the two approaches by introducing the Discrete Marginals technique in which approximate marginals are obtained by minimizing an objective function with unary and pairwise terms over a discretized domain. This allows the use of techniques originally developed for MAP-MRF inference and learning. We explore two ways to set up the objective function - by discretizing the Bethe free energy and by learning it from training data. Experimental results show that for certain types of graphs a learned function can outperform the Bethe approximation. We also establish a link between the Bethe free energy and submodular functions
Large scale probabilistic available bandwidth estimation
The common utilization-based definition of available bandwidth and many of
the existing tools to estimate it suffer from several important weaknesses: i)
most tools report a point estimate of average available bandwidth over a
measurement interval and do not provide a confidence interval; ii) the commonly
adopted models used to relate the available bandwidth metric to the measured
data are invalid in almost all practical scenarios; iii) existing tools do not
scale well and are not suited to the task of multi-path estimation in
large-scale networks; iv) almost all tools use ad-hoc techniques to address
measurement noise; and v) tools do not provide enough flexibility in terms of
accuracy, overhead, latency and reliability to adapt to the requirements of
various applications. In this paper we propose a new definition for available
bandwidth and a novel framework that addresses these issues. We define
probabilistic available bandwidth (PAB) as the largest input rate at which we
can send a traffic flow along a path while achieving, with specified
probability, an output rate that is almost as large as the input rate. PAB is
expressed directly in terms of the measurable output rate and includes
adjustable parameters that allow the user to adapt to different application
requirements. Our probabilistic framework to estimate network-wide
probabilistic available bandwidth is based on packet trains, Bayesian
inference, factor graphs and active sampling. We deploy our tool on the
PlanetLab network and our results show that we can obtain accurate estimates
with a much smaller measurement overhead compared to existing approaches.Comment: Submitted to Computer Network
- âŠ