17 research outputs found

    Exact solution for the conditional entropy of Poissonian LDPC codes over the Binary Erasure Channel

    Get PDF
    We consider communication over a binary erasure channel with low density parity check codes and optimal maximum a posteriori decoding. It is known that the problem of computing the average conditional entropy, over such code ensembles, in the asymptotic limit of large block length is closely related to computing the free energy of a mean field spin glass in the thermodynamic limit. Tentative, but explicit, formulas for these quantities have been derived thanks to the replica method (of spin glass theory) and are generally conjectured to be exact. In this contribution we show that the replica formulas are indeed exact in the case of Poissonian low density parity check ensembles. Our methods use ideas coming from the recent progress in the rigorous analysis of the Sherrington-Kirkpatrick model and their applications to the theory of error correcting codes

    Tight bounds for LDPC and LDGM codes under MAP decoding

    Full text link
    A new method for analyzing low density parity check (LDPC) codes and low density generator matrix (LDGM) codes under bit maximum a posteriori probability (MAP) decoding is introduced. The method is based on a rigorous approach to spin glasses developed by Francesco Guerra. It allows to construct lower bounds on the entropy of the transmitted message conditional to the received one. Based on heuristic statistical mechanics calculations, we conjecture such bounds to be tight. The result holds for standard irregular ensembles when used over binary input output symmetric channels. The method is first developed for Tanner graph ensembles with Poisson left degree distribution. It is then generalized to `multi-Poisson' graphs, and, by a completion procedure, to arbitrary degree distribution.Comment: 28 pages, 9 eps figures; Second version contains a generalization of the previous resul

    Sharp Bounds for Optimal Decoding of Low Density Parity Check Codes

    Full text link
    Consider communication over a binary-input memoryless output-symmetric channel with low density parity check (LDPC) codes and maximum a posteriori (MAP) decoding. The replica method of spin glass theory allows to conjecture an analytic formula for the average input-output conditional entropy per bit in the infinite block length limit. Montanari proved a lower bound for this entropy, in the case of LDPC ensembles with convex check degree polynomial, which matches the replica formula. Here we extend this lower bound to any irregular LDPC ensemble. The new feature of our work is an analysis of the second derivative of the conditional input-output entropy with respect to noise. A close relation arises between this second derivative and correlation or mutual information of codebits. This allows us to extend the realm of the interpolation method, in particular we show how channel symmetry allows to control the fluctuations of the overlap parameters.Comment: 40 Pages, Submitted to IEEE Transactions on Information Theor

    Decay of Correlations for Sparse Graph Error Correcting Codes

    Get PDF
    The subject of this paper is transmission over a general class of binary-input memoryless symmetric channels using error correcting codes based on sparse graphs, namely low-density generator-matrix and low-density parity-check codes. The optimal (or ideal) decoder based on the posterior measure over the code bits, and its relationship to the sub-optimal belief propagation decoder, are investigated. We consider the correlation (or covariance) between two codebits, averaged over the noise realizations, as a function of the graph distance, for the optimal decoder. Our main result is that this correlation decays exponentially fast for fixed general low-density generator-matrix codes and high enough noise parameter, and also for fixed general low-density parity-check codes and low enough noise parameter. This has many consequences. Appropriate performance curves - called GEXIT functions - of the belief propagation and optimal decoders match in high/low noise regimes. This means that in high/low noise regimes the performance curves of the optimal decoder can be computed by density evolution. Another interpretation is that the replica predictions of spin-glass theory are exact. Our methods are rather general and use cluster expansions first developed in the context of mathematical statistical mechanics.Comment: 40 pages, Submitted to SIAM Journal of Discrete Mathematic

    Decay of Correlations: An Application to Low-Density Parity Check codes

    Get PDF
    Recently the decay of correlations between bits of low density generator matrix (LDGM) codes have been investigated by using high temperature expansions from statistical physics \cite{KuMa07}. In this work we apply these ideas to a special class of low density parity check codes (LDPC) on the binary input gaussian white noise channel (BIAWGNC). We give a rigorous derivation of the MAP GEXIT curve (the derivative with respect to the noise parameter of the input-output conditional entropy) for high values of the noise. Our result agrees with the formal expressions obtainable from replica calculations, and is the first result that fully justifies the replica formulas beyond the binary erasure channel (BEC). The ensemble of LDPC codes considered here is constructed by adding randomly a sufficient fraction pp of degree one variable nodes to a standard irregular LDPC(Λ,P)(\Lambda, P) Tanner graphs

    Proof of replica formulas in the high noise regime for communication using LDGM codes

    Get PDF
    We consider communication over a binary input memoryless output symmetric channel with low density generator matrix codes and optimal maximum a posteriori decoding. It is known that the problem of computing the average conditional entropy, over such code ensembles in the asymptotic limit of large block length, is closely related to computing the free energy of a mean field spin glass in the thermodynamic limit. Tentative explicit formulas for these quantities have been derived thanks to the replica method (of spin glass theory) and are generally conjectured to be exact. In this contribution we show that the replica solution is indeed exact in the high noise regime, where it coincides with density evolution equations. Our method uses ideas coming from high temperature expansions in spin glass theory

    Statistical physics methods for sparse graph codes

    Get PDF
    This thesis deals with the asymptotic analysis of coding systems based on sparse graph codes. The goal of this work is to analyze the decoder performance when transmitting over a general binary-input memoryless symmetric-output (BMS) channel. We consider the two most fundamental decoders, the optimal maximum a posteriori (MAP) decoder and the sub-optimal belief propagation (BP) decoder. The BP decoder has low-complexity and its performance analysis is, hence, of great interest. The MAP decoder, on the other hand, is computationally expensive. However, the MAP decoder analysis provides fundamental limits on the code performance. As a result, the MAP-decoding analysis is important in designing codes which achieve the ultimate Shannon limit. It would be fair to say that, over the binary erasure channel (BEC), the performance of the MAP and BP decoder has been thoroughly understood. However, much less is known in the case of transmission over general BMS channels. The combinatorial methods used for analyzing the case of BEC do not extend easily to the general case. The main goal of this thesis is to advance the analysis in the case of transmission over general BMS channels. To do this, we use the recent convergence of statistical physics and coding theory. Sparse graph codes can be mapped into appropriate statistical physics spin-glass models. This allows us to use sophisticated methods from rigorous statistical mechanics like the correlation inequalities, interpolation method and cluster expansions for the purpose of our analysis. One of the main results of this thesis is that in some regimes of noise, the BP decoder is optimal for a typical code in an ensemble of codes. This result is a pleasing extension of the same result for the case of BEC. An important consequence of our results is that the heuristic predictions of the replica and cavity methods of spin-glass theory are correct in the realm of sparse graph codes
    corecore