117 research outputs found
Almost Optimal Scaling of Reed-Muller Codes on BEC and BSC Channels
Consider a binary linear code of length , minimum distance
, transmission over the binary erasure channel with parameter
or the binary symmetric channel with parameter , and block-MAP decoding. It was shown by Tillich and Zemor that in
this case the error probability of the block-MAP decoder transitions "quickly"
from to for any if the minimum distance is
large. In particular the width of the transition is of order
. We strengthen this result by showing that under
suitable conditions on the weight distribution of the code, the transition
width can be as small as , for any ,
even if the minimum distance of the code is not linear. This condition applies
e.g., to Reed-Mueller codes. Since is the smallest
transition possible for any code, we speak of "almost" optimal scaling. We
emphasize that the width of the transition says nothing about the location of
the transition. Therefore this result has no bearing on whether a code is
capacity-achieving or not. As a second contribution, we present a new estimate
on the derivative of the EXIT function, the proof of which is based on the
Blowing-Up Lemma.Comment: Submitted to ISIT 201
Reed-Muller codes for random erasures and errors
This paper studies the parameters for which Reed-Muller (RM) codes over
can correct random erasures and random errors with high probability,
and in particular when can they achieve capacity for these two classical
channels. Necessarily, the paper also studies properties of evaluations of
multi-variate polynomials on random sets of inputs.
For erasures, we prove that RM codes achieve capacity both for very high rate
and very low rate regimes. For errors, we prove that RM codes achieve capacity
for very low rate regimes, and for very high rates, we show that they can
uniquely decode at about square root of the number of errors at capacity.
The proofs of these four results are based on different techniques, which we
find interesting in their own right. In particular, we study the following
questions about , the matrix whose rows are truth tables of all
monomials of degree in variables. What is the most (resp. least)
number of random columns in that define a submatrix having full column
rank (resp. full row rank) with high probability? We obtain tight bounds for
very small (resp. very large) degrees , which we use to show that RM codes
achieve capacity for erasures in these regimes.
Our decoding from random errors follows from the following novel reduction.
For every linear code of sufficiently high rate we construct a new code
, also of very high rate, such that for every subset of coordinates, if
can recover from erasures in , then can recover from errors in .
Specializing this to RM codes and using our results for erasures imply our
result on unique decoding of RM codes at high rate.
Finally, two of our capacity achieving results require tight bounds on the
weight distribution of RM codes. We obtain such bounds extending the recent
\cite{KLP} bounds from constant degree to linear degree polynomials
Reed-Muller codes polarize
Reed-Muller (RM) codes and polar codes are generated by the same matrix
but using different subset of rows. RM codes select simply rows having largest
weights. Polar codes select instead rows having the largest conditional mutual
information proceeding top to down in ; while this is a more elaborate and
channel-dependent rule, the top-to-down ordering has the advantage of making
the conditional mutual information polarize, giving directly a
capacity-achieving code on any binary memoryless symmetric channel (BMSC). RM
codes are yet to be proved to have such property.
In this paper, we reconnect RM codes to polarization theory. It is shown that
proceeding in the RM code ordering, i.e., not top-to-down but from the lightest
to the heaviest rows in , the conditional mutual information again
polarizes. We further demonstrate that it does so faster than for polar codes.
This implies that contains another code, different than the polar code
and called here the twin code, that is provably capacity-achieving on any BMSC.
This proves a necessary condition for RM codes to achieve capacity on BMSCs. It
further gives a sufficient condition if the rows with the largest conditional
mutual information correspond to the heaviest rows, i.e., if the twin code is
the RM code. We show here that the two codes bare similarity with each other
and give further evidence that they are likely the same
Recursive projection-aggregation decoding of Reed-Muller codes
We propose a new class of efficient decoding algorithms for Reed-Muller (RM)
codes over binary-input memoryless channels. The algorithms are based on
projecting the code on its cosets, recursively decoding the projected codes
(which are lower-order RM codes), and aggregating the reconstructions (e.g.,
using majority votes). We further provide extensions of the algorithms using
list-decoding.
We run our algorithm for AWGN channels and Binary Symmetric Channels at the
short code length () regime for a wide range of code rates.
Simulation results show that in both low code rate and high code rate regimes,
the new algorithm outperforms the widely used decoder for polar codes (SCL+CRC)
with the same parameters. The performance of the new algorithm for RM codes in
those regimes is in fact close to that of the maximal likelihood decoder.
Finally, the new decoder naturally allows for parallel implementations
A proof that Reed-Muller codes achieve Shannon capacity on symmetric channels
Reed-Muller codes were introduced in 1954, with a simple explicit
construction based on polynomial evaluations, and have long been conjectured to
achieve Shannon capacity on symmetric channels. Major progress was made towards
a proof over the last decades; using combinatorial weight enumerator bounds, a
breakthrough on the erasure channel from sharp thresholds, hypercontractivity
arguments, and polarization theory. Another major progress recently established
that the bit error probability vanishes slowly below capacity. However, when
channels allow for errors, the results of Bourgain-Kalai do not apply for
converting a vanishing bit to a vanishing block error probability, neither do
the known weight enumerator bounds. The conjecture that RM codes achieve
Shannon capacity on symmetric channels, with high probability of recovering the
codewords, has thus remained open.
This paper closes the conjecture's proof. It uses a new recursive boosting
framework, which aggregates the decoding of codeword restrictions on
`subspace-sunflowers', handling their dependencies via an Boolean Fourier
analysis, and using a list-decoding argument with a weight enumerator bound
from Sberlo-Shpilka. The proof does not require a vanishing bit error
probability for the base case, but only a non-trivial probability, obtained
here for general symmetric codes. This gives in particular a shortened and
tightened argument for the vanishing bit error probability result of
Reeves-Pfister, and with prior works, it implies the strong wire-tap secrecy of
RM codes on pure-state classical-quantum channels
Channel Coding at Low Capacity
Low-capacity scenarios have become increasingly important in the technology
of the Internet of Things (IoT) and the next generation of mobile networks.
Such scenarios require efficient and reliable transmission of information over
channels with an extremely small capacity. Within these constraints, the
performance of state-of-the-art coding techniques is far from optimal in terms
of either rate or complexity. Moreover, the current non-asymptotic laws of
optimal channel coding provide inaccurate predictions for coding in the
low-capacity regime. In this paper, we provide the first comprehensive study of
channel coding in the low-capacity regime. We will investigate the fundamental
non-asymptotic limits for channel coding as well as challenges that must be
overcome for efficient code design in low-capacity scenarios.Comment: 39 pages, 5 figure
From Polar to Reed-Muller Codes:Unified Scaling, Non-standard Channels, and a Proven Conjecture
The year 2016, in which I am writing these words, marks the centenary of Claude Shannon, the father of information theory. In his landmark 1948 paper "A Mathematical Theory of Communication", Shannon established the largest rate at which reliable communication is possible, and he referred to it as the channel capacity. Since then, researchers have focused on the design of practical coding schemes that could approach such a limit. The road to channel capacity has been almost 70 years long and, after many ideas, occasional detours, and some rediscoveries, it has culminated in the description of low-complexity and provably capacity-achieving coding schemes, namely, polar codes and iterative codes based on sparse graphs. However, next-generation communication systems require an unprecedented performance improvement and the number of transmission settings relevant in applications is rapidly increasing. Hence, although Shannon's limit seems finally close at hand, new challenges are just around the corner. In this thesis, we trace a road that goes from polar to Reed-Muller codes and, by doing so, we investigate three main topics: unified scaling, non-standard channels, and capacity via symmetry. First, we consider unified scaling. A coding scheme is capacity-achieving when, for any rate smaller than capacity, the error probability tends to 0 as the block length becomes increasingly larger. However, the practitioner is often interested in more specific questions such as, "How much do we need to increase the block length in order to halve the gap between rate and capacity?". We focus our analysis on polar codes and develop a unified framework to rigorously analyze the scaling of the main parameters, i.e., block length, rate, error probability, and channel quality. Furthermore, in light of the recent success of a list decoding algorithm for polar codes, we provide scaling results on the performance of list decoders. Next, we deal with non-standard channels. When we say that a coding scheme achieves capacity, we typically consider binary memoryless symmetric channels. However, practical transmission scenarios often involve more complicated settings. For example, the downlink of a cellular system is modeled as a broadcast channel, and the communication on fiber links is inherently asymmetric. We propose provably optimal low-complexity solutions for these settings. In particular, we present a polar coding scheme that achieves the best known rate region for the broadcast channel, and we describe three paradigms to achieve the capacity of asymmetric channels. To do so, we develop general coding "primitives", such as the chaining construction that has already proved to be useful in a variety of communication problems. Finally, we show how to achieve capacity via symmetry. In the early days of coding theory, a popular paradigm consisted in exploiting the structure of algebraic codes to devise practical decoding algorithms. However, proving the optimality of such coding schemes remained an elusive goal. In particular, the conjecture that Reed-Muller codes achieve capacity dates back to the 1960s. We solve this open problem by showing that Reed-Muller codes and, in general, codes with sufficient symmetry are capacity-achieving over erasure channels under optimal MAP decoding. As the proof does not rely on the precise structure of the codes, we are able to show that symmetry alone guarantees optimal performance
- …