342 research outputs found
Asymptotic Estimates in Information Theory with Non-Vanishing Error Probabilities
This monograph presents a unified treatment of single- and multi-user
problems in Shannon's information theory where we depart from the requirement
that the error probability decays asymptotically in the blocklength. Instead,
the error probabilities for various problems are bounded above by a
non-vanishing constant and the spotlight is shone on achievable coding rates as
functions of the growing blocklengths. This represents the study of asymptotic
estimates with non-vanishing error probabilities.
In Part I, after reviewing the fundamentals of information theory, we discuss
Strassen's seminal result for binary hypothesis testing where the type-I error
probability is non-vanishing and the rate of decay of the type-II error
probability with growing number of independent observations is characterized.
In Part II, we use this basic hypothesis testing result to develop second- and
sometimes, even third-order asymptotic expansions for point-to-point
communication. Finally in Part III, we consider network information theory
problems for which the second-order asymptotics are known. These problems
include some classes of channels with random state, the multiple-encoder
distributed lossless source coding (Slepian-Wolf) problem and special cases of
the Gaussian interference and multiple-access channels. Finally, we discuss
avenues for further research.Comment: Further comments welcom
Beta-Beta Bounds: Finite-Blocklength Analog of the Golden Formula
It is well known that the mutual information between two random variables can
be expressed as the difference of two relative entropies that depend on an
auxiliary distribution, a relation sometimes referred to as the golden formula.
This paper is concerned with a finite-blocklength extension of this relation.
This extension consists of two elements: 1) a finite-blocklength channel-coding
converse bound by Polyanskiy and Verd\'{u} (2014), which involves the ratio of
two Neyman-Pearson functions (beta-beta converse bound); and 2) a novel
beta-beta channel-coding achievability bound, expressed again as the ratio of
two Neyman-Pearson functions.
To demonstrate the usefulness of this finite-blocklength extension of the
golden formula, the beta-beta achievability and converse bounds are used to
obtain a finite-blocklength extension of Verd\'{u}'s (2002) wideband-slope
approximation. The proof parallels the derivation of the latter, with the
beta-beta bounds used in place of the golden formula.
The beta-beta (achievability) bound is also shown to be useful in cases where
the capacity-achieving output distribution is not a product distribution due
to, e.g., a cost constraint or structural constraints on the codebook, such as
orthogonality or constant composition. As an example, the bound is used to
characterize the channel dispersion of the additive exponential-noise channel
and to obtain a finite-blocklength achievability bound (the tightest to date)
for multiple-input multiple-output Rayleigh-fading channels with perfect
channel state information at the receiver.Comment: to appear in IEEE Transactions on Information Theor
Random Access Channel Coding in the Finite Blocklength Regime
Consider a random access communication scenario over a channel whose
operation is defined for any number of possible transmitters. Inspired by the
model recently introduced by Polyanskiy for the Multiple Access Channel (MAC)
with a fixed, known number of transmitters, we assume that the channel is
invariant to permutations on its inputs, and that all active transmitters
employ identical encoders. Unlike Polyanskiy, we consider a scenario where
neither the transmitters nor the receiver know which transmitters are active.
We refer to this agnostic communication setup as the Random Access Channel, or
RAC. Scheduled feedback of a finite number of bits is used to synchronize the
transmitters. The decoder is tasked with determining from the channel output
the number of active transmitters () and their messages but not which
transmitter sent which message. The decoding procedure occurs at a time
depending on the decoder's estimate of the number of active transmitters,
, thereby achieving a rate that varies with the number of active
transmitters. Single-bit feedback at each time , enables all
transmitters to determine the end of one coding epoch and the start of the
next. The central result of this work demonstrates the achievability on a RAC
of performance that is first-order optimal for the MAC in operation during each
coding epoch. While prior multiple access schemes for a fixed number of
transmitters require simultaneous threshold rules, the proposed
scheme uses a single threshold rule and achieves the same dispersion.Comment: Presented at ISIT18', submitted to IEEE Transactions on Information
Theor
Fixed-length lossy compression in the finite blocklength regime
This paper studies the minimum achievable source coding rate as a function of
blocklength and probability that the distortion exceeds a given
level . Tight general achievability and converse bounds are derived that
hold at arbitrary fixed blocklength. For stationary memoryless sources with
separable distortion, the minimum rate achievable is shown to be closely
approximated by , where
is the rate-distortion function, is the rate dispersion, a
characteristic of the source which measures its stochastic variability, and
is the inverse of the standard Gaussian complementary cdf
Channel Coding at Low Capacity
Low-capacity scenarios have become increasingly important in the technology
of the Internet of Things (IoT) and the next generation of mobile networks.
Such scenarios require efficient and reliable transmission of information over
channels with an extremely small capacity. Within these constraints, the
performance of state-of-the-art coding techniques is far from optimal in terms
of either rate or complexity. Moreover, the current non-asymptotic laws of
optimal channel coding provide inaccurate predictions for coding in the
low-capacity regime. In this paper, we provide the first comprehensive study of
channel coding in the low-capacity regime. We will investigate the fundamental
non-asymptotic limits for channel coding as well as challenges that must be
overcome for efficient code design in low-capacity scenarios.Comment: 39 pages, 5 figure
- …