Weight Distributions of Regular Low-Density Parity-Check Codes over Finite Fields

The average weight distribution of a regular low-density parity-check (LDPC) code ensemble over a finite field is thoroughly analyzed. In particular, a precise asymptotic approximation of the average weight distribution is derived for the small-weight case, and a series of fundamental qualitative properties of the asymptotic growth rate of the average weight distribution are proved. Based on this analysis, a general result, including all previous results as special cases, is established for the minimum distance of individual codes in a regular LDPC code ensemble.Comment: 15 pages, 5 figures, accepted for publication in IEEE Transactions on Information Theory, July 201

Second-Order Weight Distributions

A fundamental property of codes, the second-order weight distribution, is proposed to solve the problems such as computing second moments of weight distributions of linear code ensembles. A series of results, parallel to those for weight distributions, is established for second-order weight distributions. In particular, an analogue of MacWilliams identities is proved. The second-order weight distributions of regular LDPC code ensembles are then computed. As easy consequences, the second moments of weight distributions of regular LDPC code ensembles are obtained. Furthermore, the application of second-order weight distributions in random coding approach is discussed. The second-order weight distributions of the ensembles generated by a so-called 2-good random generator or parity-check matrix are computed, where a 2-good random matrix is a kind of generalization of the uniformly distributed random matrix over a finite filed and is very useful for solving problems that involve pairwise or triple-wise properties of sequences. It is shown that the 2-good property is reflected in the second-order weight distribution, which thus plays a fundamental role in some well-known problems in coding theory and combinatorics. An example of linear intersecting codes is finally provided to illustrate this fact.Comment: 10 pages, accepted for publication in IEEE Transactions on Information Theory, May 201

Circulant Arrays on Cyclic Subgroups of Finite Fields: Rank Analysis and Construction of Quasi-Cyclic LDPC Codes

This paper consists of three parts. The first part presents a large class of new binary quasi-cyclic (QC)-LDPC codes with girth of at least 6 whose parity-check matrices are constructed based on cyclic subgroups of finite fields. Experimental results show that the codes constructed perform well over the binary-input AWGN channel with iterative decoding using the sum-product algorithm (SPA). The second part analyzes the ranks of the parity-check matrices of codes constructed based on finite fields with characteristic of 2 and gives combinatorial expressions for these ranks. The third part identifies a subclass of constructed QC-LDPC codes that have large minimum distances. Decoding of codes in this subclass with the SPA converges very fast.Comment: 26 pages, 6 figures, submitted to IEEE Transaction on Communication

Low-Density Arrays of Circulant Matrices: Rank and Row-Redundancy Analysis, and Quasi-Cyclic LDPC Codes

This paper is concerned with general analysis on the rank and row-redundancy of an array of circulants whose null space defines a QC-LDPC code. Based on the Fourier transform and the properties of conjugacy classes and Hadamard products of matrices, we derive tight upper bounds on rank and row-redundancy for general array of circulants, which make it possible to consider row-redundancy in constructions of QC-LDPC codes to achieve better performance. We further investigate the rank of two types of construction of QC-LDPC codes: constructions based on Vandermonde Matrices and Latin Squares and give combinatorial expression of the exact rank in some specific cases, which demonstrates the tightness of the bound we derive. Moreover, several types of new construction of QC-LDPC codes with large row-redundancy are presented and analyzed.Comment: arXiv admin note: text overlap with arXiv:1004.118

Multiplicatively Repeated Non-Binary LDPC Codes

We propose non-binary LDPC codes concatenated with multiplicative repetition codes. By multiplicatively repeating the (2,3)-regular non-binary LDPC mother code of rate 1/3, we construct rate-compatible codes of lower rates 1/6, 1/9, 1/12,... Surprisingly, such simple low-rate non-binary LDPC codes outperform the best low-rate binary LDPC codes so far. Moreover, we propose the decoding algorithm for the proposed codes, which can be decoded with almost the same computational complexity as that of the mother code.Comment: To appear in IEEE Transactions on Information Theor

Statistical mechanics of error exponents for error-correcting codes

Error exponents characterize the exponential decay, when increasing message length, of the probability of error of many error-correcting codes. To tackle the long standing problem of computing them exactly, we introduce a general, thermodynamic, formalism that we illustrate with maximum-likelihood decoding of low-density parity-check (LDPC) codes on the binary erasure channel (BEC) and the binary symmetric channel (BSC). In this formalism, we apply the cavity method for large deviations to derive expressions for both the average and typical error exponents, which differ by the procedure used to select the codes from specified ensembles. When decreasing the noise intensity, we find that two phase transitions take place, at two different levels: a glass to ferromagnetic transition in the space of codewords, and a paramagnetic to glass transition in the space of codes.Comment: 32 pages, 13 figure

The Dynamic Phase Transition for Decoding Algorithms

The state-of-the-art error correcting codes are based on large random constructions (random graphs, random permutations, ...) and are decoded by linear-time iterative algorithms. Because of these features, they are remarkable examples of diluted mean-field spin glasses, both from the static and from the dynamic points of view. We analyze the behavior of decoding algorithms using the mapping onto statistical-physics models. This allows to understand the intrinsic (i.e. algorithm independent) features of this behavior.Comment: 40 pages, 29 eps figure

Bit-Metric Decoding of Non-Binary LDPC Codes with Probabilistic Amplitude Shaping

A new approach for combining non-binary low-density parity-check (NB-LDPC) codes with higher-order modulation and probabilistic amplitude shaping (PAS) is presented. Instead of symbol-metric decoding (SMD), a bit-metric decoder (BMD) is used so that matching the field order of the non-binary code to the constellation size is not needed, which increases the flexibility of the coding scheme. Information rates, density evolution thresholds and finite-length simulations show that the flexibility comes at no loss of performance if PAS is used.Comment: Accepted for IEEE Communication Letter

Modern Coding Theory: The Statistical Mechanics and Computer Science Point of View

These are the notes for a set of lectures delivered by the two authors at the Les Houches Summer School on `Complex Systems' in July 2006. They provide an introduction to the basic concepts in modern (probabilistic) coding theory, highlighting connections with statistical mechanics. We also stress common concepts with other disciplines dealing with similar problems that can be generically referred to as `large graphical models'. While most of the lectures are devoted to the classical channel coding problem over simple memoryless channels, we present a discussion of more complex channel models. We conclude with an overview of the main open challenges in the field.Comment: Lectures at Les Houches Summer School on `Complex Systems', July 2006, 44 pages, 25 ps figure