2,882 research outputs found
Coding in the Finite-Blocklength Regime: Bounds based on Laplace Integrals and their Asymptotic Approximations
In this paper we provide new compact integral expressions and associated
simple asymptotic approximations for converse and achievability bounds in the
finite blocklength regime. The chosen converse and random coding union bounds
were taken from the recent work of Polyanskyi-Poor-Verdu, and are investigated
under parallel AWGN channels, the AWGN channels, the BI-AWGN channel, and the
BSC. The technique we use, which is a generalization of some recent results
available from the literature, is to map the probabilities of interest into a
Laplace integral, and then solve (or approximate) the integral by use of a
steepest descent technique. The proposed results are particularly useful for
short packet lengths, where the normal approximation may provide unreliable
results.Comment: 29 pages, 10 figures. Submitted to IEEE Trans. on Information Theory.
Matlab code available from http://dgt.dei.unipd.it section Download->Finite
Blocklength Regim
Entropy Message Passing
The paper proposes a new message passing algorithm for cycle-free factor
graphs. The proposed "entropy message passing" (EMP) algorithm may be viewed as
sum-product message passing over the entropy semiring, which has previously
appeared in automata theory. The primary use of EMP is to compute the entropy
of a model. However, EMP can also be used to compute expressions that appear in
expectation maximization and in gradient descent algorithms.Comment: 5 pages, 1 figure, to appear in IEEE Transactions on Information
Theor
Polynomial Linear Programming with Gaussian Belief Propagation
Interior-point methods are state-of-the-art algorithms for solving linear
programming (LP) problems with polynomial complexity. Specifically, the
Karmarkar algorithm typically solves LP problems in time O(n^{3.5}), where
is the number of unknown variables. Karmarkar's celebrated algorithm is known
to be an instance of the log-barrier method using the Newton iteration. The
main computational overhead of this method is in inverting the Hessian matrix
of the Newton iteration. In this contribution, we propose the application of
the Gaussian belief propagation (GaBP) algorithm as part of an efficient and
distributed LP solver that exploits the sparse and symmetric structure of the
Hessian matrix and avoids the need for direct matrix inversion. This approach
shifts the computation from realm of linear algebra to that of probabilistic
inference on graphical models, thus applying GaBP as an efficient inference
engine. Our construction is general and can be used for any interior-point
algorithm which uses the Newton method, including non-linear program solvers.Comment: 7 pages, 1 figure, appeared in the 46th Annual Allerton Conference on
Communication, Control and Computing, Allerton House, Illinois, Sept. 200
MAP inference via Block-Coordinate Frank-Wolfe Algorithm
We present a new proximal bundle method for Maximum-A-Posteriori (MAP)
inference in structured energy minimization problems. The method optimizes a
Lagrangean relaxation of the original energy minimization problem using a multi
plane block-coordinate Frank-Wolfe method that takes advantage of the specific
structure of the Lagrangean decomposition. We show empirically that our method
outperforms state-of-the-art Lagrangean decomposition based algorithms on some
challenging Markov Random Field, multi-label discrete tomography and graph
matching problems
- …