17 research outputs found
Exact solution for the conditional entropy of Poissonian LDPC codes over the Binary Erasure Channel
We consider communication over a binary erasure channel with low density parity check codes and optimal maximum a posteriori decoding. It is known that the problem of computing the average conditional entropy, over such code ensembles, in the asymptotic limit of large block length is closely related to computing the free energy of a mean field spin glass in the thermodynamic limit. Tentative, but explicit, formulas for these quantities have been derived thanks to the replica method (of spin glass theory) and are generally conjectured to be exact. In this contribution we show that the replica formulas are indeed exact in the case of Poissonian low density parity check ensembles. Our methods use ideas coming from the recent progress in the rigorous analysis of the Sherrington-Kirkpatrick model and their applications to the theory of error correcting codes
Tight bounds for LDPC and LDGM codes under MAP decoding
A new method for analyzing low density parity check (LDPC) codes and low
density generator matrix (LDGM) codes under bit maximum a posteriori
probability (MAP) decoding is introduced. The method is based on a rigorous
approach to spin glasses developed by Francesco Guerra. It allows to construct
lower bounds on the entropy of the transmitted message conditional to the
received one. Based on heuristic statistical mechanics calculations, we
conjecture such bounds to be tight. The result holds for standard irregular
ensembles when used over binary input output symmetric channels. The method is
first developed for Tanner graph ensembles with Poisson left degree
distribution. It is then generalized to `multi-Poisson' graphs, and, by a
completion procedure, to arbitrary degree distribution.Comment: 28 pages, 9 eps figures; Second version contains a generalization of
the previous resul
Sharp Bounds for Optimal Decoding of Low Density Parity Check Codes
Consider communication over a binary-input memoryless output-symmetric
channel with low density parity check (LDPC) codes and maximum a posteriori
(MAP) decoding. The replica method of spin glass theory allows to conjecture an
analytic formula for the average input-output conditional entropy per bit in
the infinite block length limit. Montanari proved a lower bound for this
entropy, in the case of LDPC ensembles with convex check degree polynomial,
which matches the replica formula. Here we extend this lower bound to any
irregular LDPC ensemble. The new feature of our work is an analysis of the
second derivative of the conditional input-output entropy with respect to
noise. A close relation arises between this second derivative and correlation
or mutual information of codebits. This allows us to extend the realm of the
interpolation method, in particular we show how channel symmetry allows to
control the fluctuations of the overlap parameters.Comment: 40 Pages, Submitted to IEEE Transactions on Information Theor
Decay of Correlations for Sparse Graph Error Correcting Codes
The subject of this paper is transmission over a general class of
binary-input memoryless symmetric channels using error correcting codes based
on sparse graphs, namely low-density generator-matrix and low-density
parity-check codes. The optimal (or ideal) decoder based on the posterior
measure over the code bits, and its relationship to the sub-optimal belief
propagation decoder, are investigated. We consider the correlation (or
covariance) between two codebits, averaged over the noise realizations, as a
function of the graph distance, for the optimal decoder. Our main result is
that this correlation decays exponentially fast for fixed general low-density
generator-matrix codes and high enough noise parameter, and also for fixed
general low-density parity-check codes and low enough noise parameter. This has
many consequences. Appropriate performance curves - called GEXIT functions - of
the belief propagation and optimal decoders match in high/low noise regimes.
This means that in high/low noise regimes the performance curves of the optimal
decoder can be computed by density evolution. Another interpretation is that
the replica predictions of spin-glass theory are exact. Our methods are rather
general and use cluster expansions first developed in the context of
mathematical statistical mechanics.Comment: 40 pages, Submitted to SIAM Journal of Discrete Mathematic
Decay of Correlations: An Application to Low-Density Parity Check codes
Recently the decay of correlations between bits of low density generator matrix (LDGM) codes have been investigated by using high temperature expansions from statistical physics \cite{KuMa07}. In this work we apply these ideas to a special class of low density parity check codes (LDPC) on the binary input gaussian white noise channel (BIAWGNC). We give a rigorous derivation of the MAP GEXIT curve (the derivative with respect to the noise parameter of the input-output conditional entropy) for high values of the noise. Our result agrees with the formal expressions obtainable from replica calculations, and is the first result that fully justifies the replica formulas beyond the binary erasure channel (BEC). The ensemble of LDPC codes considered here is constructed by adding randomly a sufficient fraction of degree one variable nodes to a standard irregular LDPC Tanner graphs
Proof of replica formulas in the high noise regime for communication using LDGM codes
We consider communication over a binary input memoryless output symmetric channel with low density generator matrix codes and optimal maximum a posteriori decoding. It is known that the problem of computing the average conditional entropy, over such code ensembles in the asymptotic limit of large block length, is closely related to computing the free energy of a mean field spin glass in the thermodynamic limit. Tentative explicit formulas for these quantities have been derived thanks to the replica method (of spin glass theory) and are generally conjectured to be exact. In this contribution we show that the replica solution is indeed exact in the high noise regime, where it coincides with density evolution equations. Our method uses ideas coming from high temperature expansions in spin glass theory
Statistical physics methods for sparse graph codes
This thesis deals with the asymptotic analysis of coding systems based on sparse graph codes. The goal of this work is to analyze the decoder performance when transmitting over a general binary-input memoryless symmetric-output (BMS) channel. We consider the two most fundamental decoders, the optimal maximum a posteriori (MAP) decoder and the sub-optimal belief propagation (BP) decoder. The BP decoder has low-complexity and its performance analysis is, hence, of great interest. The MAP decoder, on the other hand, is computationally expensive. However, the MAP decoder analysis provides fundamental limits on the code performance. As a result, the MAP-decoding analysis is important in designing codes which achieve the ultimate Shannon limit. It would be fair to say that, over the binary erasure channel (BEC), the performance of the MAP and BP decoder has been thoroughly understood. However, much less is known in the case of transmission over general BMS channels. The combinatorial methods used for analyzing the case of BEC do not extend easily to the general case. The main goal of this thesis is to advance the analysis in the case of transmission over general BMS channels. To do this, we use the recent convergence of statistical physics and coding theory. Sparse graph codes can be mapped into appropriate statistical physics spin-glass models. This allows us to use sophisticated methods from rigorous statistical mechanics like the correlation inequalities, interpolation method and cluster expansions for the purpose of our analysis. One of the main results of this thesis is that in some regimes of noise, the BP decoder is optimal for a typical code in an ensemble of codes. This result is a pleasing extension of the same result for the case of BEC. An important consequence of our results is that the heuristic predictions of the replica and cavity methods of spin-glass theory are correct in the realm of sparse graph codes