109 research outputs found
Statistical mechanics of error exponents for error-correcting codes
Error exponents characterize the exponential decay, when increasing message
length, of the probability of error of many error-correcting codes. To tackle
the long standing problem of computing them exactly, we introduce a general,
thermodynamic, formalism that we illustrate with maximum-likelihood decoding of
low-density parity-check (LDPC) codes on the binary erasure channel (BEC) and
the binary symmetric channel (BSC). In this formalism, we apply the cavity
method for large deviations to derive expressions for both the average and
typical error exponents, which differ by the procedure used to select the codes
from specified ensembles. When decreasing the noise intensity, we find that two
phase transitions take place, at two different levels: a glass to ferromagnetic
transition in the space of codewords, and a paramagnetic to glass transition in
the space of codes.Comment: 32 pages, 13 figure
Capacity-Achieving Codes with Bounded Graphical Complexity on Noisy Channels
We introduce a new family of concatenated codes with an outer low-density
parity-check (LDPC) code and an inner low-density generator matrix (LDGM) code,
and prove that these codes can achieve capacity under any memoryless
binary-input output-symmetric (MBIOS) channel using maximum-likelihood (ML)
decoding with bounded graphical complexity, i.e., the number of edges per
information bit in their graphical representation is bounded. In particular, we
also show that these codes can achieve capacity on the binary erasure channel
(BEC) under belief propagation (BP) decoding with bounded decoding complexity
per information bit per iteration for all erasure probabilities in (0, 1). By
deriving and analyzing the average weight distribution (AWD) and the
corresponding asymptotic growth rate of these codes with a rate-1 inner LDGM
code, we also show that these codes achieve the Gilbert-Varshamov bound with
asymptotically high probability. This result can be attributed to the presence
of the inner rate-1 LDGM code, which is demonstrated to help eliminate high
weight codewords in the LDPC code while maintaining a vanishingly small amount
of low weight codewords.Comment: 17 pages, 2 figures. This paper is to be presented in the 43rd Annual
Allerton Conference on Communication, Control and Computing, Monticello, IL,
USA, Sept. 28-30, 200
Strong Secrecy for Erasure Wiretap Channels
We show that duals of certain low-density parity-check (LDPC) codes, when
used in a standard coset coding scheme, provide strong secrecy over the binary
erasure wiretap channel (BEWC). This result hinges on a stopping set analysis
of ensembles of LDPC codes with block length and girth , for some
. We show that if the minimum left degree of the ensemble is
, the expected probability of block error is
\calO(\frac{1}{n^{\lceil l_\mathrm{min} k /2 \rceil - k}}) when the erasure
probability , where
depends on the degree distribution of the ensemble. As long as and , the dual of this LDPC code provides strong secrecy over a
BEWC of erasure probability greater than .Comment: Submitted to the Information Theory Workship (ITW) 2010, Dubli
How to Achieve the Capacity of Asymmetric Channels
We survey coding techniques that enable reliable transmission at rates that
approach the capacity of an arbitrary discrete memoryless channel. In
particular, we take the point of view of modern coding theory and discuss how
recent advances in coding for symmetric channels help provide more efficient
solutions for the asymmetric case. We consider, in more detail, three basic
coding paradigms.
The first one is Gallager's scheme that consists of concatenating a linear
code with a non-linear mapping so that the input distribution can be
appropriately shaped. We explicitly show that both polar codes and spatially
coupled codes can be employed in this scenario. Furthermore, we derive a
scaling law between the gap to capacity, the cardinality of the input and
output alphabets, and the required size of the mapper.
The second one is an integrated scheme in which the code is used both for
source coding, in order to create codewords distributed according to the
capacity-achieving input distribution, and for channel coding, in order to
provide error protection. Such a technique has been recently introduced by
Honda and Yamamoto in the context of polar codes, and we show how to apply it
also to the design of sparse graph codes.
The third paradigm is based on an idea of B\"ocherer and Mathar, and
separates the two tasks of source coding and channel coding by a chaining
construction that binds together several codewords. We present conditions for
the source code and the channel code, and we describe how to combine any source
code with any channel code that fulfill those conditions, in order to provide
capacity-achieving schemes for asymmetric channels. In particular, we show that
polar codes, spatially coupled codes, and homophonic codes are suitable as
basic building blocks of the proposed coding strategy.Comment: 32 pages, 4 figures, presented in part at Allerton'14 and published
in IEEE Trans. Inform. Theor
- …