87,667 research outputs found
Message-passing for Maximum Weight Independent Set
We investigate the use of message-passing algorithms for the problem of
finding the max-weight independent set (MWIS) in a graph. First, we study the
performance of the classical loopy max-product belief propagation. We show that
each fixed point estimate of max-product can be mapped in a natural way to an
extreme point of the LP polytope associated with the MWIS problem. However,
this extreme point may not be the one that maximizes the value of node weights;
the particular extreme point at final convergence depends on the initialization
of max-product. We then show that if max-product is started from the natural
initialization of uninformative messages, it always solves the correct LP -- if
it converges. This result is obtained via a direct analysis of the iterative
algorithm, and cannot be obtained by looking only at fixed points.
The tightness of the LP relaxation is thus necessary for max-product
optimality, but it is not sufficient. Motivated by this observation, we show
that a simple modification of max-product becomes gradient descent on (a
convexified version of) the dual of the LP, and converges to the dual optimum.
We also develop a message-passing algorithm that recovers the primal MWIS
solution from the output of the descent algorithm. We show that the MWIS
estimate obtained using these two algorithms in conjunction is correct when the
graph is bipartite and the MWIS is unique.
Finally, we show that any problem of MAP estimation for probability
distributions over finite domains can be reduced to an MWIS problem. We believe
this reduction will yield new insights and algorithms for MAP estimation.Comment: Submitted to IEEE Transactions on Information Theor
Typing Copyless Message Passing
We present a calculus that models a form of process interaction based on
copyless message passing, in the style of Singularity OS. The calculus is
equipped with a type system ensuring that well-typed processes are free from
memory faults, memory leaks, and communication errors. The type system is
essentially linear, but we show that linearity alone is inadequate, because it
leaves room for scenarios where well-typed processes leak significant amounts
of memory. We address these problems basing the type system upon an original
variant of session types.Comment: 50 page
Hybrid approximate message passing
Gaussian and quadratic approximations of message passing algorithms on graphs have attracted considerable recent attention due to their computational simplicity, analytic tractability, and wide applicability in optimization and statistical inference problems. This paper presents a systematic framework for incorporating such approximate message passing (AMP) methods in general graphical models. The key concept is a partition of dependencies of a general graphical model into strong and weak edges, with the weak edges representing interactions through aggregates of small, linearizable couplings of variables. AMP approximations based on the Central Limit Theorem can be readily applied to aggregates of many weak edges and integrated with standard message passing updates on the strong edges. The resulting algorithm, which we call hybrid generalized approximate message passing (HyGAMP), can yield significantly simpler implementations of sum-product and max-sum loopy belief propagation. By varying the partition of strong and weak edges, a performance--complexity trade-off can be achieved. Group sparsity and multinomial logistic regression problems are studied as examples of the proposed methodology.The work of S. Rangan was supported in part by the National Science Foundation under Grants 1116589, 1302336, and 1547332, and in part by the industrial affiliates of NYU WIRELESS. The work of A. K. Fletcher was supported in part by the National Science Foundation under Grants 1254204 and 1738286 and in part by the Office of Naval Research under Grant N00014-15-1-2677. The work of V. K. Goyal was supported in part by the National Science Foundation under Grant 1422034. The work of E. Byrne and P. Schniter was supported in part by the National Science Foundation under Grant CCF-1527162. (1116589 - National Science Foundation; 1302336 - National Science Foundation; 1547332 - National Science Foundation; 1254204 - National Science Foundation; 1738286 - National Science Foundation; 1422034 - National Science Foundation; CCF-1527162 - National Science Foundation; NYU WIRELESS; N00014-15-1-2677 - Office of Naval Research
Analysis of the Min-Sum Algorithm for Packing and Covering Problems via Linear Programming
Message-passing algorithms based on belief-propagation (BP) are successfully
used in many applications including decoding error correcting codes and solving
constraint satisfaction and inference problems. BP-based algorithms operate
over graph representations, called factor graphs, that are used to model the
input. Although in many cases BP-based algorithms exhibit impressive empirical
results, not much has been proved when the factor graphs have cycles.
This work deals with packing and covering integer programs in which the
constraint matrix is zero-one, the constraint vector is integral, and the
variables are subject to box constraints. We study the performance of the
min-sum algorithm when applied to the corresponding factor graph models of
packing and covering LPs.
We compare the solutions computed by the min-sum algorithm for packing and
covering problems to the optimal solutions of the corresponding linear
programming (LP) relaxations. In particular, we prove that if the LP has an
optimal fractional solution, then for each fractional component, the min-sum
algorithm either computes multiple solutions or the solution oscillates below
and above the fraction. This implies that the min-sum algorithm computes the
optimal integral solution only if the LP has a unique optimal solution that is
integral.
The converse is not true in general. For a special case of packing and
covering problems, we prove that if the LP has a unique optimal solution that
is integral and on the boundary of the box constraints, then the min-sum
algorithm computes the optimal solution in pseudo-polynomial time.
Our results unify and extend recent results for the maximum weight matching
problem by [Sanghavi et al.,'2011] and [Bayati et al., 2011] and for the
maximum weight independent set problem [Sanghavi et al.'2009]
- …