Entropy Message Passing

The paper proposes a new message passing algorithm for cycle-free factor graphs. The proposed "entropy message passing" (EMP) algorithm may be viewed as sum-product message passing over the entropy semiring, which has previously appeared in automata theory. The primary use of EMP is to compute the entropy of a model. However, EMP can also be used to compute expressions that appear in expectation maximization and in gradient descent algorithms.Comment: 5 pages, 1 figure, to appear in IEEE Transactions on Information Theor

Hybrid approximate message passing

Gaussian and quadratic approximations of message passing algorithms on graphs have attracted considerable recent attention due to their computational simplicity, analytic tractability, and wide applicability in optimization and statistical inference problems. This paper presents a systematic framework for incorporating such approximate message passing (AMP) methods in general graphical models. The key concept is a partition of dependencies of a general graphical model into strong and weak edges, with the weak edges representing interactions through aggregates of small, linearizable couplings of variables. AMP approximations based on the Central Limit Theorem can be readily applied to aggregates of many weak edges and integrated with standard message passing updates on the strong edges. The resulting algorithm, which we call hybrid generalized approximate message passing (HyGAMP), can yield significantly simpler implementations of sum-product and max-sum loopy belief propagation. By varying the partition of strong and weak edges, a performance--complexity trade-off can be achieved. Group sparsity and multinomial logistic regression problems are studied as examples of the proposed methodology.The work of S. Rangan was supported in part by the National Science Foundation under Grants 1116589, 1302336, and 1547332, and in part by the industrial affiliates of NYU WIRELESS. The work of A. K. Fletcher was supported in part by the National Science Foundation under Grants 1254204 and 1738286 and in part by the Office of Naval Research under Grant N00014-15-1-2677. The work of V. K. Goyal was supported in part by the National Science Foundation under Grant 1422034. The work of E. Byrne and P. Schniter was supported in part by the National Science Foundation under Grant CCF-1527162. (1116589 - National Science Foundation; 1302336 - National Science Foundation; 1547332 - National Science Foundation; 1254204 - National Science Foundation; 1738286 - National Science Foundation; 1422034 - National Science Foundation; CCF-1527162 - National Science Foundation; NYU WIRELESS; N00014-15-1-2677 - Office of Naval Research

Typing Copyless Message Passing

We present a calculus that models a form of process interaction based on copyless message passing, in the style of Singularity OS. The calculus is equipped with a type system ensuring that well-typed processes are free from memory faults, memory leaks, and communication errors. The type system is essentially linear, but we show that linearity alone is inadequate, because it leaves room for scenarios where well-typed processes leak significant amounts of memory. We address these problems basing the type system upon an original variant of session types.Comment: 50 page

A Note on Shared Randomness and Shared Entanglement in Communication

We consider several models of 1-round classical and quantum communication, some of these models have not been defined before. We "almost separate" the models of simultaneous quantum message passing with shared entanglement and the model of simultaneous quantum message passing with shared randomness. We define a relation which can be efficiently exactly solved in the first model but cannot be solved efficiently, either exactly or in 0-error setup in the second model. In fact, our relation is exactly solvable even in a more restricted model of simultaneous classical message passing with shared entanglement. As our second contribution we strengthen a result by Yao that a "very short" protocol from the model of simultaneous classical message passing with shared randomness can be simulated in the model of simultaneous quantum message passing: for a boolean function f, QII(f) \in exp(O(RIIp(f))) log n. We show a similar result for protocols from a (stronger) model of 1-way classical message passing with shared randomness: QII(f) \in exp(O(RIp(f))) log n. We demonstrate a problem whose efficient solution in the model of simultaneous quantum message passing follows from our result but not from Yao's.Comment: Stronger separation, minor changes and fixe

Message passing for vertex covers

Constructing a minimal vertex cover of a graph can be seen as a prototype for a combinatorial optimization problem under hard constraints. In this paper, we develop and analyze message passing techniques, namely warning and survey propagation, which serve as efficient heuristic algorithms for solving these computational hard problems. We show also, how previously obtained results on the typical-case behavior of vertex covers of random graphs can be recovered starting from the message passing equations, and how they can be extended.Comment: 25 pages, 9 figures - version accepted for publication in PR

Auto-Mobiles: Optimised Message-Passing

Some message-passing concurrent systems, such as occam 2, prohibit aliasing of data objects. Communicated data must thus be copied, which can be time-intensive for large data packets such as video frames. We introduce automatic mobility, a compiler optimisation that performs communications by reference and deduces when these communications can be performed without copying. We discuss bounds for speed-up and memory use, and benchmark the automatic mobility optimisation. We show that in the best case it can transform an operation from being linear with respect to packet size into constant-time

Message passing for quantified Boolean formulas

We introduce two types of message passing algorithms for quantified Boolean formulas (QBF). The first type is a message passing based heuristics that can prove unsatisfiability of the QBF by assigning the universal variables in such a way that the remaining formula is unsatisfiable. In the second type, we use message passing to guide branching heuristics of a Davis-Putnam Logemann-Loveland (DPLL) complete solver. Numerical experiments show that on random QBFs our branching heuristics gives robust exponential efficiency gain with respect to the state-of-art solvers. We also manage to solve some previously unsolved benchmarks from the QBFLIB library. Apart from this our study sheds light on using message passing in small systems and as subroutines in complete solvers.Comment: 14 pages, 7 figure

Interference Alignment via Message-Passing

We introduce an iterative solution to the problem of interference alignment (IA) over MIMO channels based on a message-passing formulation. We propose a parameterization of the messages that enables the computation of IA precoders by a min-sum algorithm over continuous variable spaces -- under this parameterization, suitable approximations of the messages can be computed in closed-form. We show that the iterative leakage minimization algorithm of Cadambe et al. is a special case of our message-passing algorithm, obtained for a particular schedule. Finally, we show that the proposed algorithm compares favorably to iterative leakage minimization in terms of convergence speed, and discuss a distributed implementation.Comment: Submitted to the IEEE International Conference on Communications (ICC) 201