141 research outputs found
Approaching the Rate-Distortion Limit with Spatial Coupling, Belief propagation and Decimation
We investigate an encoding scheme for lossy compression of a binary symmetric
source based on simple spatially coupled Low-Density Generator-Matrix codes.
The degree of the check nodes is regular and the one of code-bits is Poisson
distributed with an average depending on the compression rate. The performance
of a low complexity Belief Propagation Guided Decimation algorithm is
excellent. The algorithmic rate-distortion curve approaches the optimal curve
of the ensemble as the width of the coupling window grows. Moreover, as the
check degree grows both curves approach the ultimate Shannon rate-distortion
limit. The Belief Propagation Guided Decimation encoder is based on the
posterior measure of a binary symmetric test-channel. This measure can be
interpreted as a random Gibbs measure at a "temperature" directly related to
the "noise level of the test-channel". We investigate the links between the
algorithmic performance of the Belief Propagation Guided Decimation encoder and
the phase diagram of this Gibbs measure. The phase diagram is investigated
thanks to the cavity method of spin glass theory which predicts a number of
phase transition thresholds. In particular the dynamical and condensation
"phase transition temperatures" (equivalently test-channel noise thresholds)
are computed. We observe that: (i) the dynamical temperature of the spatially
coupled construction saturates towards the condensation temperature; (ii) for
large degrees the condensation temperature approaches the temperature (i.e.
noise level) related to the information theoretic Shannon test-channel noise
parameter of rate-distortion theory. This provides heuristic insight into the
excellent performance of the Belief Propagation Guided Decimation algorithm.
The paper contains an introduction to the cavity method
How to Achieve the Capacity of Asymmetric Channels
We survey coding techniques that enable reliable transmission at rates that
approach the capacity of an arbitrary discrete memoryless channel. In
particular, we take the point of view of modern coding theory and discuss how
recent advances in coding for symmetric channels help provide more efficient
solutions for the asymmetric case. We consider, in more detail, three basic
coding paradigms.
The first one is Gallager's scheme that consists of concatenating a linear
code with a non-linear mapping so that the input distribution can be
appropriately shaped. We explicitly show that both polar codes and spatially
coupled codes can be employed in this scenario. Furthermore, we derive a
scaling law between the gap to capacity, the cardinality of the input and
output alphabets, and the required size of the mapper.
The second one is an integrated scheme in which the code is used both for
source coding, in order to create codewords distributed according to the
capacity-achieving input distribution, and for channel coding, in order to
provide error protection. Such a technique has been recently introduced by
Honda and Yamamoto in the context of polar codes, and we show how to apply it
also to the design of sparse graph codes.
The third paradigm is based on an idea of B\"ocherer and Mathar, and
separates the two tasks of source coding and channel coding by a chaining
construction that binds together several codewords. We present conditions for
the source code and the channel code, and we describe how to combine any source
code with any channel code that fulfill those conditions, in order to provide
capacity-achieving schemes for asymmetric channels. In particular, we show that
polar codes, spatially coupled codes, and homophonic codes are suitable as
basic building blocks of the proposed coding strategy.Comment: 32 pages, 4 figures, presented in part at Allerton'14 and published
in IEEE Trans. Inform. Theor
Threshold Saturation in Spatially Coupled Constraint Satisfaction Problems
We consider chains of random constraint satisfaction models that are
spatially coupled across a finite window along the chain direction. We
investigate their phase diagram at zero temperature using the survey
propagation formalism and the interpolation method. We prove that the SAT-UNSAT
phase transition threshold of an infinite chain is identical to the one of the
individual standard model, and is therefore not affected by spatial coupling.
We compute the survey propagation complexity using population dynamics as well
as large degree approximations, and determine the survey propagation threshold.
We find that a clustering phase survives coupling. However, as one increases
the range of the coupling window, the survey propagation threshold increases
and saturates towards the phase transition threshold. We also briefly discuss
other aspects of the problem. Namely, the condensation threshold is not
affected by coupling, but the dynamic threshold displays saturation towards the
condensation one. All these features may provide a new avenue for obtaining
better provable algorithmic lower bounds on phase transition thresholds of the
individual standard model
Capacity-Achieving Coding Mechanisms: Spatial Coupling and Group Symmetries
The broad theme of this work is in constructing optimal transmission mechanisms for a wide variety of communication systems. In particular, this dissertation provides a proof of threshold saturation for spatially-coupled codes, low-complexity capacity-achieving coding schemes for side-information problems, a proof that Reed-Muller and primitive narrow-sense BCH codes achieve capacity on erasure channels, and a mathematical framework to design delay sensitive communication systems.
Spatially-coupled codes are a class of codes on graphs that are shown to achieve capacity universally over binary symmetric memoryless channels (BMS) under belief-propagation decoder. The underlying phenomenon behind spatial coupling, known as âthreshold saturation via spatial couplingâ, turns out to be general and this technique has been applied to a wide variety of systems. In this work, a proof of the threshold saturation phenomenon is provided for irregular low-density parity-check (LDPC) and low-density generator-matrix (LDGM) ensembles on BMS channels. This proof is far simpler than published alternative proofs and it remains as the only technique to handle irregular and LDGM codes. Also, low-complexity capacity-achieving codes are constructed for three coding problems via spatial coupling: 1) rate distortion with side-information, 2) channel coding with side-information, and 3) write-once memory system. All these schemes are based on spatially coupling compound LDGM/LDPC ensembles.
Reed-Muller and Bose-Chaudhuri-Hocquengham (BCH) are well-known algebraic codes introduced more than 50 years ago. While these codes are studied extensively in the literature it wasnât known whether these codes achieve capacity. This work introduces a technique to show that Reed-Muller and primitive narrow-sense BCH codes achieve capacity on erasure channels under maximum a posteriori (MAP) decoding. Instead of relying on the weight enumerators or other precise details of these codes, this technique requires that these codes have highly symmetric permutation groups. In fact, any sequence of linear codes with increasing blocklengths whose rates converge to a number between 0 and 1, and whose permutation groups are doubly transitive achieve capacity on erasure channels under bit-MAP decoding. This pro-vides a rare example in information theory where symmetry alone is suïŹcient to achieve capacity.
While the channel capacity provides a useful benchmark for practical design, communication systems of the day also demand small latency and other link layer metrics. Such delay sensitive communication systems are studied in this work, where a mathematical framework is developed to provide insights into the optimal design of these systems
On privacy amplification, lossy compression, and their duality to channel coding
We examine the task of privacy amplification from information-theoretic and
coding-theoretic points of view. In the former, we give a one-shot
characterization of the optimal rate of privacy amplification against classical
adversaries in terms of the optimal type-II error in asymmetric hypothesis
testing. This formulation can be easily computed to give finite-blocklength
bounds and turns out to be equivalent to smooth min-entropy bounds by Renner
and Wolf [Asiacrypt 2005] and Watanabe and Hayashi [ISIT 2013], as well as a
bound in terms of the divergence by Yang, Schaefer, and Poor
[arXiv:1706.03866 [cs.IT]]. In the latter, we show that protocols for privacy
amplification based on linear codes can be easily repurposed for channel
simulation. Combined with known relations between channel simulation and lossy
source coding, this implies that privacy amplification can be understood as a
basic primitive for both channel simulation and lossy compression. Applied to
symmetric channels or lossy compression settings, our construction leads to
proto- cols of optimal rate in the asymptotic i.i.d. limit. Finally, appealing
to the notion of channel duality recently detailed by us in [IEEE Trans. Info.
Theory 64, 577 (2018)], we show that linear error-correcting codes for
symmetric channels with quantum output can be transformed into linear lossy
source coding schemes for classical variables arising from the dual channel.
This explains a "curious duality" in these problems for the (self-dual) erasure
channel observed by Martinian and Yedidia [Allerton 2003; arXiv:cs/0408008] and
partly anticipates recent results on optimal lossy compression by polar and
low-density generator matrix codes.Comment: v3: updated to include equivalence of the converse bound with smooth
entropy formulations. v2: updated to include comparison with the one-shot
bounds of arXiv:1706.03866. v1: 11 pages, 4 figure
A Simple Proof of Maxwell Saturation for Coupled Scalar Recursions
Low-density parity-check (LDPC) convolutional codes (or spatially-coupled
codes) were recently shown to approach capacity on the binary erasure channel
(BEC) and binary-input memoryless symmetric channels. The mechanism behind this
spectacular performance is now called threshold saturation via spatial
coupling. This new phenomenon is characterized by the belief-propagation
threshold of the spatially-coupled ensemble increasing to an intrinsic noise
threshold defined by the uncoupled system. In this paper, we present a simple
proof of threshold saturation that applies to a wide class of coupled scalar
recursions. Our approach is based on constructing potential functions for both
the coupled and uncoupled recursions. Our results actually show that the fixed
point of the coupled recursion is essentially determined by the minimum of the
uncoupled potential function and we refer to this phenomenon as Maxwell
saturation. A variety of examples are considered including the
density-evolution equations for: irregular LDPC codes on the BEC, irregular
low-density generator matrix codes on the BEC, a class of generalized LDPC
codes with BCH component codes, the joint iterative decoding of LDPC codes on
intersymbol-interference channels with erasure noise, and the compressed
sensing of random vectors with i.i.d. components.Comment: This article is an extended journal version of arXiv:1204.5703 and
has now been accepted to the IEEE Transactions on Information Theory. This
version adds additional explanation for some details and also corrects a
number of small typo
Threshold Saturation in Spatially Coupled Constraint Satisfaction Problems
We consider chains of random constraint satisfaction models that are spatially coupled across a finite window along the chain direction. We investigate their phase diagram at zero temperature using the survey propagation formalism and the interpolation method. We prove that the SAT-UNSAT phase transition threshold of an infinite chain is identical to the one of the individual standard model, and is therefore not affected by spatial coupling. We compute the survey propagation complexity using population dynamics as well as large degree approximations, and determine the survey propagation threshold. We find that a clustering phase survives coupling. However, as one increases the range of the coupling window, the survey propagation threshold increases and saturates towards the phase transition threshold. We also briefly discuss other aspects of the problem. Namely, the condensation threshold is not affected by coupling, but the dynamic threshold displays saturation towards the condensation one. All these features may provide a new avenue for obtaining better provable algorithmic lower bounds on phase transition thresholds of the individual standard mode
- âŠ