1,445 research outputs found
Fast Polarization for Processes with Memory
Fast polarization is crucial for the performance guarantees of polar codes.
In the memoryless setting, the rate of polarization is known to be exponential
in the square root of the block length. A complete characterization of the rate
of polarization for models with memory has been missing. Namely, previous works
have not addressed fast polarization of the high entropy set under memory. We
consider polar codes for processes with memory that are characterized by an
underlying ergodic finite-state Markov chain. We show that the rate of
polarization for these processes is the same as in the memoryless setting, both
for the high and for the low entropy sets.Comment: 17 pages, 3 figures. Submitted to IEEE Transactions on Information
Theor
Capacity of Locally Recoverable Codes
Motivated by applications in distributed storage, the notion of a locally
recoverable code (LRC) was introduced a few years back. In an LRC, any
coordinate of a codeword is recoverable by accessing only a small number of
other coordinates. While different properties of LRCs have been well-studied,
their performance on channels with random erasures or errors has been mostly
unexplored. In this note, we analyze the performance of LRCs over such
stochastic channels. In particular, for input-symmetric discrete memoryless
channels, we give a tight characterization of the gap to Shannon capacity when
LRCs are used over the channel.Comment: Invited paper to the Information Theory Workshop (ITW) 201
Broadcast Capacity Region of Two-Phase Bidirectional Relaying
In a three-node network a half-duplex relay node enables bidirectional
communication between two nodes with a spectral efficient two phase protocol.
In the first phase, two nodes transmit their message to the relay node, which
decodes the messages and broadcast a re-encoded composition in the second
phase. In this work we determine the capacity region of the broadcast phase. In
this scenario each receiving node has perfect information about the message
that is intended for the other node. The resulting set of achievable rates of
the two-phase bidirectional relaying includes the region which can be achieved
by applying XOR on the decoded messages at the relay node. We also prove the
strong converse for the maximum error probability and show that this implies
that the [\eps_1,\eps_2]-capacity region defined with respect to the average
error probability is constant for small values of error parameters \eps_1,
\eps_2.Comment: 25 pages, 2 figures, submitted to IEEE Transactions on Information
Theor
Re-proving Channel Polarization Theorems: An Extremality and Robustness Analysis
The general subject considered in this thesis is a recently discovered coding
technique, polar coding, which is used to construct a class of error correction
codes with unique properties. In his ground-breaking work, Ar{\i}kan proved
that this class of codes, called polar codes, achieve the symmetric capacity
--- the mutual information evaluated at the uniform input distribution ---of
any stationary binary discrete memoryless channel with low complexity encoders
and decoders requiring in the order of operations in the
block-length . This discovery settled the long standing open problem left by
Shannon of finding low complexity codes achieving the channel capacity.
Polar coding settled an open problem in information theory, yet opened plenty
of challenging problems that need to be addressed. A significant part of this
thesis is dedicated to advancing the knowledge about this technique in two
directions. The first one provides a better understanding of polar coding by
generalizing some of the existing results and discussing their implications,
and the second one studies the robustness of the theory over communication
models introducing various forms of uncertainty or variations into the
probabilistic model of the channel.Comment: Preview of my PhD Thesis, EPFL, Lausanne, 2014. For the full version,
see http://people.epfl.ch/mine.alsan/publication
Distributed Channel Synthesis
Two familiar notions of correlation are rediscovered as the extreme operating
points for distributed synthesis of a discrete memoryless channel, in which a
stochastic channel output is generated based on a compressed description of the
channel input. Wyner's common information is the minimum description rate
needed. However, when common randomness independent of the input is available,
the necessary description rate reduces to Shannon's mutual information. This
work characterizes the optimal trade-off between the amount of common
randomness used and the required rate of description. We also include a number
of related derivations, including the effect of limited local randomness, rate
requirements for secrecy, applications to game theory, and new insights into
common information duality.
Our proof makes use of a soft covering lemma, known in the literature for its
role in quantifying the resolvability of a channel. The direct proof
(achievability) constructs a feasible joint distribution over all parts of the
system using a soft covering, from which the behavior of the encoder and
decoder is inferred, with no explicit reference to joint typicality or binning.
Of auxiliary interest, this work also generalizes and strengthens this soft
covering tool.Comment: To appear in IEEE Trans. on Information Theory (submitted Aug., 2012,
accepted July, 2013), 26 pages, using IEEEtran.cl
The Generalized Area Theorem and Some of its Consequences
There is a fundamental relationship between belief propagation and maximum a
posteriori decoding. The case of transmission over the binary erasure channel
was investigated in detail in a companion paper. This paper investigates the
extension to general memoryless channels (paying special attention to the
binary case). An area theorem for transmission over general memoryless channels
is introduced and some of its many consequences are discussed. We show that
this area theorem gives rise to an upper-bound on the maximum a posteriori
threshold for sparse graph codes. In situations where this bound is tight, the
extrinsic soft bit estimates delivered by the belief propagation decoder
coincide with the correct a posteriori probabilities above the maximum a
posteriori threshold. More generally, it is conjectured that the fundamental
relationship between the maximum a posteriori and the belief propagation
decoder which was observed for transmission over the binary erasure channel
carries over to the general case. We finally demonstrate that in order for the
design rate of an ensemble to approach the capacity under belief propagation
decoding the component codes have to be perfectly matched, a statement which is
well known for the special case of transmission over the binary erasure
channel.Comment: 27 pages, 46 ps figure
- …