A Simple Proof of Maxwell Saturation for Coupled Scalar Recursions

Low-density parity-check (LDPC) convolutional codes (or spatially-coupled codes) were recently shown to approach capacity on the binary erasure channel (BEC) and binary-input memoryless symmetric channels. The mechanism behind this spectacular performance is now called threshold saturation via spatial coupling. This new phenomenon is characterized by the belief-propagation threshold of the spatially-coupled ensemble increasing to an intrinsic noise threshold defined by the uncoupled system. In this paper, we present a simple proof of threshold saturation that applies to a wide class of coupled scalar recursions. Our approach is based on constructing potential functions for both the coupled and uncoupled recursions. Our results actually show that the fixed point of the coupled recursion is essentially determined by the minimum of the uncoupled potential function and we refer to this phenomenon as Maxwell saturation. A variety of examples are considered including the density-evolution equations for: irregular LDPC codes on the BEC, irregular low-density generator matrix codes on the BEC, a class of generalized LDPC codes with BCH component codes, the joint iterative decoding of LDPC codes on intersymbol-interference channels with erasure noise, and the compressed sensing of random vectors with i.i.d. components.Comment: This article is an extended journal version of arXiv:1204.5703 and has now been accepted to the IEEE Transactions on Information Theory. This version adds additional explanation for some details and also corrects a number of small typo

Spatially-Coupled MacKay-Neal Codes and Hsu-Anastasopoulos Codes

Kudekar et al. recently proved that for transmission over the binary erasure channel (BEC), spatial coupling of LDPC codes increases the BP threshold of the coupled ensemble to the MAP threshold of the underlying LDPC codes. One major drawback of the capacity-achieving spatially-coupled LDPC codes is that one needs to increase the column and row weight of parity-check matrices of the underlying LDPC codes. It is proved, that Hsu-Anastasopoulos (HA) codes and MacKay-Neal (MN) codes achieve the capacity of memoryless binary-input symmetric-output channels under MAP decoding with bounded column and row weight of the parity-check matrices. The HA codes and the MN codes are dual codes each other. The aim of this paper is to present an empirical evidence that spatially-coupled MN (resp. HA) codes with bounded column and row weight achieve the capacity of the BEC. To this end, we introduce a spatial coupling scheme of MN (resp. HA) codes. By density evolution analysis, we will show that the resulting spatially-coupled MN (resp. HA) codes have the BP threshold close to the Shannon limit.Comment: Corrected typos in degree distributions \nu and \mu of MN and HA code

Spatially-Coupled Precoded Rateless Codes

Raptor codes are rateless codes that achieve the capacity on the binary erasure channels. However the maximum degree of optimal output degree distribution is unbounded. This leads to a computational complexity problem both at encoders and decoders. Aref and Urbanke investigated the potential advantage of universal achieving-capacity property of proposed spatially-coupled (SC) low-density generator matrix (LDGM) codes. However the decoding error probability of SC-LDGM codes is bounded away from 0. In this paper, we investigate SC-LDGM codes concatenated with SC low-density parity-check codes. The proposed codes can be regarded as SC Hsu-Anastasopoulos rateless codes. We derive a lower bound of the asymptotic overhead from stability analysis for successful decoding by density evolution. The numerical calculation reveals that the lower bound is tight. We observe that with a sufficiently large number of information bits, the asymptotic overhead and the decoding error rate approach 0 with bounded maximum degree

Spatially-Coupled Precoded Rateless Codes with Bounded Degree Achieve the Capacity of BEC under BP decoding

Raptor codes are known as precoded rateless codes that achieve the capacity of BEC. However the maximum degree of Raptor codes needs to be unbounded to achieve the capacity. In this paper, we prove that spatially-coupled precoded rateless codes achieve the capacity with bounded degree under BP decoding

Approaching the Rate-Distortion Limit with Spatial Coupling, Belief propagation and Decimation

We investigate an encoding scheme for lossy compression of a binary symmetric source based on simple spatially coupled Low-Density Generator-Matrix codes. The degree of the check nodes is regular and the one of code-bits is Poisson distributed with an average depending on the compression rate. The performance of a low complexity Belief Propagation Guided Decimation algorithm is excellent. The algorithmic rate-distortion curve approaches the optimal curve of the ensemble as the width of the coupling window grows. Moreover, as the check degree grows both curves approach the ultimate Shannon rate-distortion limit. The Belief Propagation Guided Decimation encoder is based on the posterior measure of a binary symmetric test-channel. This measure can be interpreted as a random Gibbs measure at a "temperature" directly related to the "noise level of the test-channel". We investigate the links between the algorithmic performance of the Belief Propagation Guided Decimation encoder and the phase diagram of this Gibbs measure. The phase diagram is investigated thanks to the cavity method of spin glass theory which predicts a number of phase transition thresholds. In particular the dynamical and condensation "phase transition temperatures" (equivalently test-channel noise thresholds) are computed. We observe that: (i) the dynamical temperature of the spatially coupled construction saturates towards the condensation temperature; (ii) for large degrees the condensation temperature approaches the temperature (i.e. noise level) related to the information theoretic Shannon test-channel noise parameter of rate-distortion theory. This provides heuristic insight into the excellent performance of the Belief Propagation Guided Decimation algorithm. The paper contains an introduction to the cavity method

Threshold Saturation for Spatially Coupled LDPC and LDGM Codes on BMS Channels

Spatially-coupled low-density parity-check (LDPC) codes, which were first introduced as LDPC convolutional codes, have been shown to exhibit excellent performance under low-complexity belief-propagation decoding. This phenomenon is now termed threshold saturation via spatial coupling. Spatially-coupled codes have been successfully applied in numerous areas. In particular, it was proven that spatially-coupled regular LDPC codes universally achieve capacity over the class of binary memoryless symmetric (BMS) channels under belief-propagation decoding. Recently, potential functions have been used to simplify threshold saturation proofs for scalar and vector recursions. In this paper, potential functions are used to prove threshold saturation for irregular LDPC and low-density generator-matrix codes on BMS channels, extending the simplified proof technique to BMS channels. The corresponding potential functions are closely related to the average Bethe free entropy of the ensembles in the large-system limit. These functions also appear in statistical physics when the replica method is used to analyze optimal decoding

Capacity-Achieving Coding Mechanisms: Spatial Coupling and Group Symmetries

The broad theme of this work is in constructing optimal transmission mechanisms for a wide variety of communication systems. In particular, this dissertation provides a proof of threshold saturation for spatially-coupled codes, low-complexity capacity-achieving coding schemes for side-information problems, a proof that Reed-Muller and primitive narrow-sense BCH codes achieve capacity on erasure channels, and a mathematical framework to design delay sensitive communication systems. Spatially-coupled codes are a class of codes on graphs that are shown to achieve capacity universally over binary symmetric memoryless channels (BMS) under belief-propagation decoder. The underlying phenomenon behind spatial coupling, known as “threshold saturation via spatial coupling”, turns out to be general and this technique has been applied to a wide variety of systems. In this work, a proof of the threshold saturation phenomenon is provided for irregular low-density parity-check (LDPC) and low-density generator-matrix (LDGM) ensembles on BMS channels. This proof is far simpler than published alternative proofs and it remains as the only technique to handle irregular and LDGM codes. Also, low-complexity capacity-achieving codes are constructed for three coding problems via spatial coupling: 1) rate distortion with side-information, 2) channel coding with side-information, and 3) write-once memory system. All these schemes are based on spatially coupling compound LDGM/LDPC ensembles. Reed-Muller and Bose-Chaudhuri-Hocquengham (BCH) are well-known algebraic codes introduced more than 50 years ago. While these codes are studied extensively in the literature it wasn’t known whether these codes achieve capacity. This work introduces a technique to show that Reed-Muller and primitive narrow-sense BCH codes achieve capacity on erasure channels under maximum a posteriori (MAP) decoding. Instead of relying on the weight enumerators or other precise details of these codes, this technique requires that these codes have highly symmetric permutation groups. In fact, any sequence of linear codes with increasing blocklengths whose rates converge to a number between 0 and 1, and whose permutation groups are doubly transitive achieve capacity on erasure channels under bit-MAP decoding. This pro-vides a rare example in information theory where symmetry alone is sufficient to achieve capacity. While the channel capacity provides a useful benchmark for practical design, communication systems of the day also demand small latency and other link layer metrics. Such delay sensitive communication systems are studied in this work, where a mathematical framework is developed to provide insights into the optimal design of these systems