247 research outputs found
Lossy joint source-channel coding in the finite blocklength regime
This paper finds new tight finite-blocklength bounds for the best achievable
lossy joint source-channel code rate, and demonstrates that joint
source-channel code design brings considerable performance advantage over a
separate one in the non-asymptotic regime. A joint source-channel code maps a
block of source symbols onto a length channel codeword, and the
fidelity of reproduction at the receiver end is measured by the probability
that the distortion exceeds a given threshold . For memoryless
sources and channels, it is demonstrated that the parameters of the best joint
source-channel code must satisfy , where and are the channel capacity and channel
dispersion, respectively; and are the source
rate-distortion and rate-dispersion functions; and is the standard Gaussian
complementary cdf. Symbol-by-symbol (uncoded) transmission is known to achieve
the Shannon limit when the source and channel satisfy a certain probabilistic
matching condition. In this paper we show that even when this condition is not
satisfied, symbol-by-symbol transmission is, in some cases, the best known
strategy in the non-asymptotic regime
Improved Finite Blocklength Converses for Slepian-Wolf Coding via Linear Programming
A new finite blocklength converse for the Slepian- Wolf coding problem is
presented which significantly improves on the best known converse for this
problem, due to Miyake and Kanaya [2]. To obtain this converse, an extension of
the linear programming (LP) based framework for finite blocklength point-
to-point coding problems from [3] is employed. However, a direct application of
this framework demands a complicated analysis for the Slepian-Wolf problem. An
analytically simpler approach is presented wherein LP-based finite blocklength
converses for this problem are synthesized from point-to-point lossless source
coding problems with perfect side-information at the decoder. New finite
blocklength metaconverses for these point-to-point problems are derived by
employing the LP-based framework, and the new converse for Slepian-Wolf coding
is obtained by an appropriate combination of these converses.Comment: under review with the IEEE Transactions on Information Theor
Joint source-channel coding with feedback
This paper quantifies the fundamental limits of variable-length transmission
of a general (possibly analog) source over a memoryless channel with noiseless
feedback, under a distortion constraint. We consider excess distortion, average
distortion and guaranteed distortion (-semifaithful codes). In contrast to
the asymptotic fundamental limit, a general conclusion is that allowing
variable-length codes and feedback leads to a sizable improvement in the
fundamental delay-distortion tradeoff. In addition, we investigate the minimum
energy required to reproduce source samples with a given fidelity after
transmission over a memoryless Gaussian channel, and we show that the required
minimum energy is reduced with feedback and an average (rather than maximal)
power constraint.Comment: To appear in IEEE Transactions on Information Theor
Asymptotic Estimates in Information Theory with Non-Vanishing Error Probabilities
This monograph presents a unified treatment of single- and multi-user
problems in Shannon's information theory where we depart from the requirement
that the error probability decays asymptotically in the blocklength. Instead,
the error probabilities for various problems are bounded above by a
non-vanishing constant and the spotlight is shone on achievable coding rates as
functions of the growing blocklengths. This represents the study of asymptotic
estimates with non-vanishing error probabilities.
In Part I, after reviewing the fundamentals of information theory, we discuss
Strassen's seminal result for binary hypothesis testing where the type-I error
probability is non-vanishing and the rate of decay of the type-II error
probability with growing number of independent observations is characterized.
In Part II, we use this basic hypothesis testing result to develop second- and
sometimes, even third-order asymptotic expansions for point-to-point
communication. Finally in Part III, we consider network information theory
problems for which the second-order asymptotics are known. These problems
include some classes of channels with random state, the multiple-encoder
distributed lossless source coding (Slepian-Wolf) problem and special cases of
the Gaussian interference and multiple-access channels. Finally, we discuss
avenues for further research.Comment: Further comments welcom
On privacy amplification, lossy compression, and their duality to channel coding
We examine the task of privacy amplification from information-theoretic and
coding-theoretic points of view. In the former, we give a one-shot
characterization of the optimal rate of privacy amplification against classical
adversaries in terms of the optimal type-II error in asymmetric hypothesis
testing. This formulation can be easily computed to give finite-blocklength
bounds and turns out to be equivalent to smooth min-entropy bounds by Renner
and Wolf [Asiacrypt 2005] and Watanabe and Hayashi [ISIT 2013], as well as a
bound in terms of the divergence by Yang, Schaefer, and Poor
[arXiv:1706.03866 [cs.IT]]. In the latter, we show that protocols for privacy
amplification based on linear codes can be easily repurposed for channel
simulation. Combined with known relations between channel simulation and lossy
source coding, this implies that privacy amplification can be understood as a
basic primitive for both channel simulation and lossy compression. Applied to
symmetric channels or lossy compression settings, our construction leads to
proto- cols of optimal rate in the asymptotic i.i.d. limit. Finally, appealing
to the notion of channel duality recently detailed by us in [IEEE Trans. Info.
Theory 64, 577 (2018)], we show that linear error-correcting codes for
symmetric channels with quantum output can be transformed into linear lossy
source coding schemes for classical variables arising from the dual channel.
This explains a "curious duality" in these problems for the (self-dual) erasure
channel observed by Martinian and Yedidia [Allerton 2003; arXiv:cs/0408008] and
partly anticipates recent results on optimal lossy compression by polar and
low-density generator matrix codes.Comment: v3: updated to include equivalence of the converse bound with smooth
entropy formulations. v2: updated to include comparison with the one-shot
bounds of arXiv:1706.03866. v1: 11 pages, 4 figure
The Reliability Function of Lossy Source-Channel Coding of Variable-Length Codes with Feedback
We consider transmission of discrete memoryless sources (DMSes) across
discrete memoryless channels (DMCs) using variable-length lossy source-channel
codes with feedback. The reliability function (optimum error exponent) is shown
to be equal to where is the rate-distortion
function of the source, is the maximum relative entropy between output
distributions of the DMC, and is the Shannon capacity of the channel. We
show that, in this setting and in this asymptotic regime, separate
source-channel coding is, in fact, optimal.Comment: Accepted to IEEE Transactions on Information Theory in Apr. 201
Fixed-length lossy compression in the finite blocklength regime
This paper studies the minimum achievable source coding rate as a function of
blocklength and probability that the distortion exceeds a given
level . Tight general achievability and converse bounds are derived that
hold at arbitrary fixed blocklength. For stationary memoryless sources with
separable distortion, the minimum rate achievable is shown to be closely
approximated by , where
is the rate-distortion function, is the rate dispersion, a
characteristic of the source which measures its stochastic variability, and
is the inverse of the standard Gaussian complementary cdf
- …