Modern Coding Theory: The Statistical Mechanics and Computer Science Point of View

These are the notes for a set of lectures delivered by the two authors at the Les Houches Summer School on `Complex Systems' in July 2006. They provide an introduction to the basic concepts in modern (probabilistic) coding theory, highlighting connections with statistical mechanics. We also stress common concepts with other disciplines dealing with similar problems that can be generically referred to as `large graphical models'. While most of the lectures are devoted to the classical channel coding problem over simple memoryless channels, we present a discussion of more complex channel models. We conclude with an overview of the main open challenges in the field.Comment: Lectures at Les Houches Summer School on `Complex Systems', July 2006, 44 pages, 25 ps figure

Multi-Way Relay Networks: Orthogonal Uplink, Source-Channel Separation and Code Design

We consider a multi-way relay network with an orthogonal uplink and correlated sources, and we characterise reliable communication (in the usual Shannon sense) with a single-letter expression. The characterisation is obtained using a joint source-channel random-coding argument, which is based on a combination of Wyner et al.'s "Cascaded Slepian-Wolf Source Coding" and Tuncel's "Slepian-Wolf Coding over Broadcast Channels". We prove a separation theorem for the special case of two nodes; that is, we show that a modular code architecture with separate source and channel coding functions is (asymptotically) optimal. Finally, we propose a practical coding scheme based on low-density parity-check codes, and we analyse its performance using multi-edge density evolution.Comment: Authors' final version (accepted and to appear in IEEE Transactions on Communications

Error-correction on non-standard communication channels

Many communication systems are poorly modelled by the standard channels assumed in the information theory literature, such as the binary symmetric channel or the additive white Gaussian noise channel. Real systems suffer from additional problems including time-varying noise, cross-talk, synchronization errors and latency constraints. In this thesis, low-density parity-check codes and codes related to them are applied to non-standard channels. First, we look at time-varying noise modelled by a Markov channel. A low-density parity-check code decoder is modified to give an improvement of over 1dB. Secondly, novel codes based on low-density parity-check codes are introduced which produce transmissions with Pr(bit = 1) ≠ Pr(bit = 0). These non-linear codes are shown to be good candidates for multi-user channels with crosstalk, such as optical channels. Thirdly, a channel with synchronization errors is modelled by random uncorrelated insertion or deletion events at unknown positions. Marker codes formed from low-density parity-check codewords with regular markers inserted within them are studied. It is shown that a marker code with iterative decoding has performance close to the bounds on the channel capacity, significantly outperforming other known codes. Finally, coding for a system with latency constraints is studied. For example, if a telemetry system involves a slow channel some error correction is often needed quickly whilst the code should be able to correct remaining errors later. A new code is formed from the intersection of a convolutional code with a high rate low-density parity-check code. The convolutional code has good early decoding performance and the high rate low-density parity-check code efficiently cleans up remaining errors after receiving the entire block. Simulations of the block code show a gain of 1.5dB over a standard NASA code