1,440 research outputs found
Belief Propagation and LP Relaxation for Weighted Matching in General Graphs
Loopy belief propagation has been employed in a wide variety of applications with great empirical success, but it comes with few theoretical guarantees. In this paper, we analyze the performance of the max-product form of belief propagation for the weighted matching problem on general graphs. We show that the performance of max-product is exactly characterized by the natural linear programming (LP) relaxation of the problem. In particular, we first show that if the LP relaxation has no fractional optima then max-product always converges to the correct answer. This establishes the extension of the recent result by Bayati, Shah and Sharma, which considered bipartite graphs, to general graphs. Perhaps more interestingly, we also establish a tight converse, namely that the presence of any fractional LP optimum implies that max-product will fail to yield useful estimates on some of the edges. We extend our results to the weighted b-matching and r -edge-cover problems. We also demonstrate how to simplify the max-product message-update equations for weighted matching, making it easily deployable in distributed settings like wireless or sensor networks.National Science Foundation (U.S.) (Grant CAREER 0954059)National Science Foundation (U.S.) (Grant 0964391
Belief-Propagation for Weighted b-Matchings on Arbitrary Graphs and its Relation to Linear Programs with Integer Solutions
We consider the general problem of finding the minimum weight \bm-matching
on arbitrary graphs. We prove that, whenever the linear programming (LP)
relaxation of the problem has no fractional solutions, then the belief
propagation (BP) algorithm converges to the correct solution. We also show that
when the LP relaxation has a fractional solution then the BP algorithm can be
used to solve the LP relaxation. Our proof is based on the notion of graph
covers and extends the analysis of (Bayati-Shah-Sharma 2005 and Huang-Jebara
2007}.
These results are notable in the following regards: (1) It is one of a very
small number of proofs showing correctness of BP without any constraint on the
graph structure. (2) Variants of the proof work for both synchronous and
asynchronous BP; it is the first proof of convergence and correctness of an
asynchronous BP algorithm for a combinatorial optimization problem.Comment: 28 pages, 2 figures. Submitted to SIAM journal on Discrete
Mathematics on March 19, 2009; accepted for publication (in revised form)
August 30, 2010; published electronically July 1, 201
Belief Propagation for Linear Programming
Belief Propagation (BP) is a popular, distributed heuristic for performing
MAP computations in Graphical Models. BP can be interpreted, from a variational
perspective, as minimizing the Bethe Free Energy (BFE). BP can also be used to
solve a special class of Linear Programming (LP) problems. For this class of
problems, MAP inference can be stated as an integer LP with an LP relaxation
that coincides with minimization of the BFE at ``zero temperature". We
generalize these prior results and establish a tight characterization of the LP
problems that can be formulated as an equivalent LP relaxation of MAP
inference. Moreover, we suggest an efficient, iterative annealing BP algorithm
for solving this broader class of LP problems. We demonstrate the algorithm's
performance on a set of weighted matching problems by using it as a cutting
plane method to solve a sequence of LPs tightened by adding ``blossom''
inequalities.Comment: To appear in ISIT 201
Analysis of the Min-Sum Algorithm for Packing and Covering Problems via Linear Programming
Message-passing algorithms based on belief-propagation (BP) are successfully
used in many applications including decoding error correcting codes and solving
constraint satisfaction and inference problems. BP-based algorithms operate
over graph representations, called factor graphs, that are used to model the
input. Although in many cases BP-based algorithms exhibit impressive empirical
results, not much has been proved when the factor graphs have cycles.
This work deals with packing and covering integer programs in which the
constraint matrix is zero-one, the constraint vector is integral, and the
variables are subject to box constraints. We study the performance of the
min-sum algorithm when applied to the corresponding factor graph models of
packing and covering LPs.
We compare the solutions computed by the min-sum algorithm for packing and
covering problems to the optimal solutions of the corresponding linear
programming (LP) relaxations. In particular, we prove that if the LP has an
optimal fractional solution, then for each fractional component, the min-sum
algorithm either computes multiple solutions or the solution oscillates below
and above the fraction. This implies that the min-sum algorithm computes the
optimal integral solution only if the LP has a unique optimal solution that is
integral.
The converse is not true in general. For a special case of packing and
covering problems, we prove that if the LP has a unique optimal solution that
is integral and on the boundary of the box constraints, then the min-sum
algorithm computes the optimal solution in pseudo-polynomial time.
Our results unify and extend recent results for the maximum weight matching
problem by [Sanghavi et al.,'2011] and [Bayati et al., 2011] and for the
maximum weight independent set problem [Sanghavi et al.'2009]
Minimum Weight Perfect Matching via Blossom Belief Propagation
Max-product Belief Propagation (BP) is a popular message-passing algorithm
for computing a Maximum-A-Posteriori (MAP) assignment over a distribution
represented by a Graphical Model (GM). It has been shown that BP can solve a
number of combinatorial optimization problems including minimum weight
matching, shortest path, network flow and vertex cover under the following
common assumption: the respective Linear Programming (LP) relaxation is tight,
i.e., no integrality gap is present. However, when LP shows an integrality gap,
no model has been known which can be solved systematically via sequential
applications of BP. In this paper, we develop the first such algorithm, coined
Blossom-BP, for solving the minimum weight matching problem over arbitrary
graphs. Each step of the sequential algorithm requires applying BP over a
modified graph constructed by contractions and expansions of blossoms, i.e.,
odd sets of vertices. Our scheme guarantees termination in O(n^2) of BP runs,
where n is the number of vertices in the original graph. In essence, the
Blossom-BP offers a distributed version of the celebrated Edmonds' Blossom
algorithm by jumping at once over many sub-steps with a single BP. Moreover,
our result provides an interpretation of the Edmonds' algorithm as a sequence
of LPs
On the exactness of the cavity method for Weighted b-Matchings on Arbitrary Graphs and its Relation to Linear Programs
We consider the general problem of finding the minimum weight b-matching on
arbitrary graphs. We prove that, whenever the linear programming relaxation of
the problem has no fractional solutions, then the cavity or belief propagation
equations converge to the correct solution both for synchronous and
asynchronous updating
Approximating the Permanent with Fractional Belief Propagation
We discuss schemes for exact and approximate computations of permanents, and
compare them with each other. Specifically, we analyze the Belief Propagation
(BP) approach and its Fractional Belief Propagation (FBP) generalization for
computing the permanent of a non-negative matrix. Known bounds and conjectures
are verified in experiments, and some new theoretical relations, bounds and
conjectures are proposed. The Fractional Free Energy (FFE) functional is
parameterized by a scalar parameter , where
corresponds to the BP limit and corresponds to the exclusion
principle (but ignoring perfect matching constraints) Mean-Field (MF) limit.
FFE shows monotonicity and continuity with respect to . For every
non-negative matrix, we define its special value to be the
for which the minimum of the -parameterized FFE functional is
equal to the permanent of the matrix, where the lower and upper bounds of the
-interval corresponds to respective bounds for the permanent. Our
experimental analysis suggests that the distribution of varies for
different ensembles but always lies within the interval.
Moreover, for all ensembles considered the behavior of is highly
distinctive, offering an emprirical practical guidance for estimating
permanents of non-negative matrices via the FFE approach.Comment: 42 pages, 14 figure
- ā¦