5 research outputs found
Belief Propagation for Linear Programming
Belief Propagation (BP) is a popular, distributed heuristic for performing
MAP computations in Graphical Models. BP can be interpreted, from a variational
perspective, as minimizing the Bethe Free Energy (BFE). BP can also be used to
solve a special class of Linear Programming (LP) problems. For this class of
problems, MAP inference can be stated as an integer LP with an LP relaxation
that coincides with minimization of the BFE at ``zero temperature". We
generalize these prior results and establish a tight characterization of the LP
problems that can be formulated as an equivalent LP relaxation of MAP
inference. Moreover, we suggest an efficient, iterative annealing BP algorithm
for solving this broader class of LP problems. We demonstrate the algorithm's
performance on a set of weighted matching problems by using it as a cutting
plane method to solve a sequence of LPs tightened by adding ``blossom''
inequalities.Comment: To appear in ISIT 201
Minimum Weight Perfect Matching via Blossom Belief Propagation
Max-product Belief Propagation (BP) is a popular message-passing algorithm
for computing a Maximum-A-Posteriori (MAP) assignment over a distribution
represented by a Graphical Model (GM). It has been shown that BP can solve a
number of combinatorial optimization problems including minimum weight
matching, shortest path, network flow and vertex cover under the following
common assumption: the respective Linear Programming (LP) relaxation is tight,
i.e., no integrality gap is present. However, when LP shows an integrality gap,
no model has been known which can be solved systematically via sequential
applications of BP. In this paper, we develop the first such algorithm, coined
Blossom-BP, for solving the minimum weight matching problem over arbitrary
graphs. Each step of the sequential algorithm requires applying BP over a
modified graph constructed by contractions and expansions of blossoms, i.e.,
odd sets of vertices. Our scheme guarantees termination in O(n^2) of BP runs,
where n is the number of vertices in the original graph. In essence, the
Blossom-BP offers a distributed version of the celebrated Edmonds' Blossom
algorithm by jumping at once over many sub-steps with a single BP. Moreover,
our result provides an interpretation of the Edmonds' algorithm as a sequence
of LPs
Optimal Inference in Crowdsourced Classification via Belief Propagation
Crowdsourcing systems are popular for solving large-scale labelling tasks
with low-paid workers. We study the problem of recovering the true labels from
the possibly erroneous crowdsourced labels under the popular Dawid-Skene model.
To address this inference problem, several algorithms have recently been
proposed, but the best known guarantee is still significantly larger than the
fundamental limit. We close this gap by introducing a tighter lower bound on
the fundamental limit and proving that Belief Propagation (BP) exactly matches
this lower bound. The guaranteed optimality of BP is the strongest in the sense
that it is information-theoretically impossible for any other algorithm to
correctly label a larger fraction of the tasks. Experimental results suggest
that BP is close to optimal for all regimes considered and improves upon
competing state-of-the-art algorithms.Comment: This article is partially based on preliminary results published in
the proceeding of the 33rd International Conference on Machine Learning (ICML
2016
Max-Product Belief Propagation for Linear Programming: Applications to . . .
Max-product belief propagation (BP) is a popular message-passing algorithm for computing a maximum-a-posteriori (MAP) assignment in a joint distribution represented by a graphical model (GM). It has been shown that BP can solve a few classes of Linear Programming (LP) formulations to combinatorial optimization problems including maximum weight matching and shortest path, i.e., BP can be a distributed solver for certain LPs. However, those LPs and corresponding BP analysis are very sensitive to underlying problem setups, and it has been not clear what extent these results can be generalized to. In this paper, we obtain a generic criteria that BP converges to the optimal solution of given LP, and show that it is satisfied in LP formulations associated to many classical combinatorial optimization problems including maximum weight perfect matching, shortest path, traveling salesman, cycle packing and vertex cover. More importantly, our criteria can guide the BP design to compute fractional LP solutions, while most prior results focus on integral ones. Our results provide new tools on BP analysis and new directions on efficient solvers for large-scale LPs