60 research outputs found
Polynomial Linear Programming with Gaussian Belief Propagation
Interior-point methods are state-of-the-art algorithms for solving linear
programming (LP) problems with polynomial complexity. Specifically, the
Karmarkar algorithm typically solves LP problems in time O(n^{3.5}), where
is the number of unknown variables. Karmarkar's celebrated algorithm is known
to be an instance of the log-barrier method using the Newton iteration. The
main computational overhead of this method is in inverting the Hessian matrix
of the Newton iteration. In this contribution, we propose the application of
the Gaussian belief propagation (GaBP) algorithm as part of an efficient and
distributed LP solver that exploits the sparse and symmetric structure of the
Hessian matrix and avoids the need for direct matrix inversion. This approach
shifts the computation from realm of linear algebra to that of probabilistic
inference on graphical models, thus applying GaBP as an efficient inference
engine. Our construction is general and can be used for any interior-point
algorithm which uses the Newton method, including non-linear program solvers.Comment: 7 pages, 1 figure, appeared in the 46th Annual Allerton Conference on
Communication, Control and Computing, Allerton House, Illinois, Sept. 200
Distributed Large Scale Network Utility Maximization
Recent work by Zymnis et al. proposes an efficient primal-dual interior-point
method, using a truncated Newton method, for solving the network utility
maximization (NUM) problem. This method has shown superior performance relative
to the traditional dual-decomposition approach. Other recent work by Bickson et
al. shows how to compute efficiently and distributively the Newton step, which
is the main computational bottleneck of the Newton method, utilizing the
Gaussian belief propagation algorithm.
In the current work, we combine both approaches to create an efficient
distributed algorithm for solving the NUM problem. Unlike the work of Zymnis,
which uses a centralized approach, our new algorithm is easily distributed.
Using an empirical evaluation we show that our new method outperforms previous
approaches, including the truncated Newton method and dual-decomposition
methods. As an additional contribution, this is the first work that evaluates
the performance of the Gaussian belief propagation algorithm vs. the
preconditioned conjugate gradient method, for a large scale problem.Comment: In the International Symposium on Information Theory (ISIT) 200
A Low Density Lattice Decoder via Non-Parametric Belief Propagation
The recent work of Sommer, Feder and Shalvi presented a new family of codes
called low density lattice codes (LDLC) that can be decoded efficiently and
approach the capacity of the AWGN channel. A linear time iterative decoding
scheme which is based on a message-passing formulation on a factor graph is
given.
In the current work we report our theoretical findings regarding the relation
between the LDLC decoder and belief propagation. We show that the LDLC decoder
is an instance of non-parametric belief propagation and further connect it to
the Gaussian belief propagation algorithm. Our new results enable borrowing
knowledge from the non-parametric and Gaussian belief propagation domains into
the LDLC domain. Specifically, we give more general convergence conditions for
convergence of the LDLC decoder (under the same assumptions of the original
LDLC convergence analysis). We discuss how to extend the LDLC decoder from
Latin square to full rank, non-square matrices. We propose an efficient
construction of sparse generator matrix and its matching decoder. We report
preliminary experimental results which show our decoder has comparable symbol
to error rate compared to the original LDLC decoder.%Comment: Submitted for publicatio
The julia content distribution network
Abstract — Peer-to-peer content distribution networks are currently being used widely, drawing upon a large fraction of the Internet bandwidth. Unfortunately, these applications are not designed to be network-friendly. They optimize download time by using all available bandwidth. As a result, long haul bottleneck links are becoming congested and the load on the network is not well balanced. In this paper, we introduce the Julia content distribution network. The innovation of Julia is in its reduction of the overall communication cost, which in turn improves network load balance and reduces the usage of long haul links. Compared with the state-of-the-art BitTorrent content distribution network, we find that while Julia achieves slightly slower average finishing times relative to BitTorrent, Julia nevertheless reduces the total communication cost in the network by approximately 33%. Furthermore, the Julia protocol achieves a better load balancing of the network resources, especially over trans-Atlantic links. We evaluated the Julia protocol using real WAN deployment and by extensive simulation. The WAN experimentation was carried over the PlanetLab wide area testbed using over 250 machines. Simulations were performed using the the GT-ITM topology generator with 1200 nodes. A surprisingly good match was exhibited between the two evaluation methods (itself an interesting result), an encouraging indication of the ability of our simulation to predict scaling behavior. I
Gaussian Belief Propagation Based Multiuser Detection
In this work, we present a novel construction for solving the linear
multiuser detection problem using the Gaussian Belief Propagation algorithm.
Our algorithm yields an efficient, iterative and distributed implementation of
the MMSE detector. We compare our algorithm's performance to a recent result
and show an improved memory consumption, reduced computation steps and a
reduction in the number of sent messages. We prove that recent work by
Montanari et al. is an instance of our general algorithm, providing new
convergence results for both algorithms.Comment: 6 pages, 1 figures, appeared in the 2008 IEEE International Symposium
on Information Theory, Toronto, July 200
Fixing Convergence of Gaussian Belief Propagation
Gaussian belief propagation (GaBP) is an iterative message-passing algorithm
for inference in Gaussian graphical models. It is known that when GaBP
converges it converges to the correct MAP estimate of the Gaussian random
vector and simple sufficient conditions for its convergence have been
established. In this paper we develop a double-loop algorithm for forcing
convergence of GaBP. Our method computes the correct MAP estimate even in cases
where standard GaBP would not have converged. We further extend this
construction to compute least-squares solutions of over-constrained linear
systems. We believe that our construction has numerous applications, since the
GaBP algorithm is linked to solution of linear systems of equations, which is a
fundamental problem in computer science and engineering. As a case study, we
discuss the linear detection problem. We show that using our new construction,
we are able to force convergence of Montanari's linear detection algorithm, in
cases where it would originally fail. As a consequence, we are able to increase
significantly the number of users that can transmit concurrently.Comment: In the IEEE International Symposium on Information Theory (ISIT)
2009, Seoul, South Korea, July 200
- …