Modern Coding Theory: The Statistical Mechanics and Computer Science Point of View

These are the notes for a set of lectures delivered by the two authors at the Les Houches Summer School on `Complex Systems' in July 2006. They provide an introduction to the basic concepts in modern (probabilistic) coding theory, highlighting connections with statistical mechanics. We also stress common concepts with other disciplines dealing with similar problems that can be generically referred to as `large graphical models'. While most of the lectures are devoted to the classical channel coding problem over simple memoryless channels, we present a discussion of more complex channel models. We conclude with an overview of the main open challenges in the field.Comment: Lectures at Les Houches Summer School on `Complex Systems', July 2006, 44 pages, 25 ps figure

Numerical Techniques for Finding the Distances of Quantum Codes

We survey the existing techniques for calculating code distances of classical codes and apply these techniques to generic quantum codes. For classical and quantum LDPC codes, we also present a new linked-cluster technique. It reduces complexity exponent of all existing deterministic techniques designed for codes with small relative distances (which include all known families of quantum LDPC codes), and also surpasses the probabilistic technique for sufficiently high code rates.Comment: 5 pages, 1 figure, to appear in Proceedings of ISIT 2014 - IEEE International Symposium on Information Theory, Honolul

Entanglement-assisted quantum turbo codes

An unexpected breakdown in the existing theory of quantum serial turbo coding is that a quantum convolutional encoder cannot simultaneously be recursive and non-catastrophic. These properties are essential for quantum turbo code families to have a minimum distance growing with blocklength and for their iterative decoding algorithm to converge, respectively. Here, we show that the entanglement-assisted paradigm simplifies the theory of quantum turbo codes, in the sense that an entanglement-assisted quantum (EAQ) convolutional encoder can possess both of the aforementioned desirable properties. We give several examples of EAQ convolutional encoders that are both recursive and non-catastrophic and detail their relevant parameters. We then modify the quantum turbo decoding algorithm of Poulin et al., in order to have the constituent decoders pass along only "extrinsic information" to each other rather than a posteriori probabilities as in the decoder of Poulin et al., and this leads to a significant improvement in the performance of unassisted quantum turbo codes. Other simulation results indicate that entanglement-assisted turbo codes can operate reliably in a noise regime 4.73 dB beyond that of standard quantum turbo codes, when used on a memoryless depolarizing channel. Furthermore, several of our quantum turbo codes are within 1 dB or less of their hashing limits, so that the performance of quantum turbo codes is now on par with that of classical turbo codes. Finally, we prove that entanglement is the resource that enables a convolutional encoder to be both non-catastrophic and recursive because an encoder acting on only information qubits, classical bits, gauge qubits, and ancilla qubits cannot simultaneously satisfy them.Comment: 31 pages, software for simulating EA turbo codes is available at http://code.google.com/p/ea-turbo/ and a presentation is available at http://markwilde.com/publications/10-10-EA-Turbo.ppt ; v2, revisions based on feedback from journal; v3, modification of the quantum turbo decoding algorithm that leads to improved performance over results in v2 and the results of Poulin et al. in arXiv:0712.288

Self-concatenated code design and its application in power-efficient cooperative communications

In this tutorial, we have focused on the design of binary self-concatenated coding schemes with the help of EXtrinsic Information Transfer (EXIT) charts and Union bound analysis. The design methodology of future iteratively decoded self-concatenated aided cooperative communication schemes is presented. In doing so, we will identify the most important milestones in the area of channel coding, concatenated coding schemes and cooperative communication systems till date and suggest future research directions

Sparse Graph Codes for Quantum Error-Correction

We present sparse graph codes appropriate for use in quantum error-correction. Quantum error-correcting codes based on sparse graphs are of interest for three reasons. First, the best codes currently known for classical channels are based on sparse graphs. Second, sparse graph codes keep the number of quantum interactions associated with the quantum error correction process small: a constant number per quantum bit, independent of the blocklength. Third, sparse graph codes often offer great flexibility with respect to blocklength and rate. We believe some of the codes we present are unsurpassed by previously published quantum error-correcting codes.Comment: Version 7.3e: 42 pages. Extended version, Feb 2004. A shortened version was resubmitted to IEEE Transactions on Information Theory Jan 20, 200

Binary Message Passing Decoding of Product-like Codes

We propose a novel binary message passing decoding algorithm for product-like codes based on bounded distance decoding (BDD) of the component codes. The algorithm, dubbed iterative BDD with scaled reliability (iBDD-SR), exploits the channel reliabilities and is therefore soft in nature. However, the messages exchanged by the component decoders are binary (hard) messages, which significantly reduces the decoder data flow. The exchanged binary messages are obtained by combining the channel reliability with the BDD decoder output reliabilities, properly conveyed by a scaling factor applied to the BDD decisions. We perform a density evolution analysis for generalized low-density parity-check (GLDPC) code ensembles and spatially coupled GLDPC code ensembles, from which the scaling factors of the iBDD-SR for product and staircase codes, respectively, can be obtained. For the white additive Gaussian noise channel, we show performance gains up to $0.29$ dB and $0.31$ dB for product and staircase codes compared to conventional iterative BDD (iBDD) with the same decoder data flow. Furthermore, we show that iBDD-SR approaches the performance of ideal iBDD that prevents miscorrections.Comment: Accepted for publication in the IEEE Transactions on Communication

Mathematical Programming Decoding of Binary Linear Codes: Theory and Algorithms

Mathematical programming is a branch of applied mathematics and has recently been used to derive new decoding approaches, challenging established but often heuristic algorithms based on iterative message passing. Concepts from mathematical programming used in the context of decoding include linear, integer, and nonlinear programming, network flows, notions of duality as well as matroid and polyhedral theory. This survey article reviews and categorizes decoding methods based on mathematical programming approaches for binary linear codes over binary-input memoryless symmetric channels.Comment: 17 pages, submitted to the IEEE Transactions on Information Theory. Published July 201