Statistical mechanics of low-density parity-check codes

We review recent theoretical progress on the statistical mechanics of error correcting codes, focusing on low-density parity-check (LDPC) codes in general, and on Gallager and MacKay-Neal codes in particular. By exploiting the relation between LDPC codes and Ising spin systems with multispin interactions, one can carry out a statistical mechanics based analysis that determines the practical and theoretical limitations of various code constructions, corresponding to dynamical and thermodynamical transitions, respectively, as well as the behaviour of error-exponents averaged over the corresponding code ensemble as a function of channel noise. We also contrast the results obtained using methods of statistical mechanics with those derived in the information theory literature, and show how these methods can be generalized to include other channel types and related communication problems

Statistical mechanics of error exponents for error-correcting codes

Error exponents characterize the exponential decay, when increasing message length, of the probability of error of many error-correcting codes. To tackle the long standing problem of computing them exactly, we introduce a general, thermodynamic, formalism that we illustrate with maximum-likelihood decoding of low-density parity-check (LDPC) codes on the binary erasure channel (BEC) and the binary symmetric channel (BSC). In this formalism, we apply the cavity method for large deviations to derive expressions for both the average and typical error exponents, which differ by the procedure used to select the codes from specified ensembles. When decreasing the noise intensity, we find that two phase transitions take place, at two different levels: a glass to ferromagnetic transition in the space of codewords, and a paramagnetic to glass transition in the space of codes.Comment: 32 pages, 13 figure

Low density parity check codes: a statistical physics perspective

The modem digital communication systems are made transmission reliable by employing error correction technique for the redundancies. Codes in the low-density parity-check work along the principles of Hamming code, and the parity-check matrix is very sparse, and multiple errors can be corrected. The sparseness of the matrix allows for the decoding process to be carried out by probability propagation methods similar to those employed in Turbo codes. The relation between spin systems in statistical physics and digital error correcting codes is based on the existence of a simple isomorphism between the additive Boolean group and the multiplicative binary group. Shannon proved general results on the natural limits of compression and error-correction by setting up the framework known as information theory. Error-correction codes are based on mapping the original space of words onto a higher dimensional space in such a way that the typical distance between encoded words increases

Applications of iterative decoding to magnetic recording channels.

Finally, Q-ary LDPC (Q-LDPC) codes are considered for MRCs. Belief propagation decoding for binary LDPC codes is extended to Q-LDPC codes and a reduced-complexity decoding algorithm for Q-LDPC codes is developed. Q-LDPC coded systems perform very well with random noise as well as with burst erasures. Simulations show that Q-LDPC systems outperform RS systems.Secondly, binary low-density parity-check (LDPC) codes are proposed for MRCs. Random binary LDPC codes, finite-geometry LDPC codes and irregular LDPC codes are considered. With belief propagation decoding, LDPC systems are shown to have superior performance over current Reed-Solomon (RS) systems at the range possible for computer simulation. The issue of RS-LDPC concatenation is also addressed.Three coding schemes are investigated for magnetic recording systems. Firstly, block turbo codes, including product codes and parallel block turbo codes, are considered on MRCs. Product codes with other types of component codes are briefly discussed.Magnetic recoding channels (MRCs) are subject to noise contamination and error-correcting codes (ECCs) are used to keep the integrity of the data. Conventionally, hard decoding of the ECCs is performed. In this dissertation, systems using soft iterative decoding techniques are presented and their improved performance is established