885 research outputs found
Asymptotic Estimates in Information Theory with Non-Vanishing Error Probabilities
This monograph presents a unified treatment of single- and multi-user
problems in Shannon's information theory where we depart from the requirement
that the error probability decays asymptotically in the blocklength. Instead,
the error probabilities for various problems are bounded above by a
non-vanishing constant and the spotlight is shone on achievable coding rates as
functions of the growing blocklengths. This represents the study of asymptotic
estimates with non-vanishing error probabilities.
In Part I, after reviewing the fundamentals of information theory, we discuss
Strassen's seminal result for binary hypothesis testing where the type-I error
probability is non-vanishing and the rate of decay of the type-II error
probability with growing number of independent observations is characterized.
In Part II, we use this basic hypothesis testing result to develop second- and
sometimes, even third-order asymptotic expansions for point-to-point
communication. Finally in Part III, we consider network information theory
problems for which the second-order asymptotics are known. These problems
include some classes of channels with random state, the multiple-encoder
distributed lossless source coding (Slepian-Wolf) problem and special cases of
the Gaussian interference and multiple-access channels. Finally, we discuss
avenues for further research.Comment: Further comments welcom
Exponential Strong Converse for Successive Refinement with Causal Decoder Side Information
We consider the -user successive refinement problem with causal decoder
side information and derive an exponential strong converse theorem. The
rate-distortion region for the problem can be derived as a straightforward
extension of the two-user case by Maor and Merhav (2008). We show that for any
rate-distortion tuple outside the rate-distortion region of the -user
successive refinement problem with causal decoder side information, the joint
excess-distortion probability approaches one exponentially fast. Our proof
follows by judiciously adapting the recently proposed strong converse technique
by Oohama using the information spectrum method, the variational form of the
rate-distortion region and H\"older's inequality. The lossy source coding
problem with causal decoder side information considered by El Gamal and
Weissman is a special case () of the current problem. Therefore, the
exponential strong converse theorem for the El Gamal and Weissman problem
follows as a corollary of our result
Second-Order Coding Rates for Conditional Rate-Distortion
This paper characterizes the second-order coding rates for lossy source
coding with side information available at both the encoder and the decoder. We
first provide non-asymptotic bounds for this problem and then specialize the
non-asymptotic bounds for three different scenarios: discrete memoryless
sources, Gaussian sources, and Markov sources. We obtain the second-order
coding rates for these settings. It is interesting to observe that the
second-order coding rate for Gaussian source coding with Gaussian side
information available at both the encoder and the decoder is the same as that
for Gaussian source coding without side information. Furthermore, regardless of
the variance of the side information, the dispersion is nats squared per
source symbol.Comment: 20 pages, 2 figures, second-order coding rates, finite blocklength,
network information theor
Fixed-length lossy compression in the finite blocklength regime
This paper studies the minimum achievable source coding rate as a function of
blocklength and probability that the distortion exceeds a given
level . Tight general achievability and converse bounds are derived that
hold at arbitrary fixed blocklength. For stationary memoryless sources with
separable distortion, the minimum rate achievable is shown to be closely
approximated by , where
is the rate-distortion function, is the rate dispersion, a
characteristic of the source which measures its stochastic variability, and
is the inverse of the standard Gaussian complementary cdf
- …