3,268 research outputs found
An identity of Chernoff bounds with an interpretation in statistical physics and applications in information theory
An identity between two versions of the Chernoff bound on the probability a
certain large deviations event, is established. This identity has an
interpretation in statistical physics, namely, an isothermal equilibrium of a
composite system that consists of multiple subsystems of particles. Several
information--theoretic application examples, where the analysis of this large
deviations probability naturally arises, are then described from the viewpoint
of this statistical mechanical interpretation. This results in several
relationships between information theory and statistical physics, which we
hope, the reader will find insightful.Comment: 29 pages, 1 figure. Submitted to IEEE Trans. on Information Theor
On the Reliability Function of Distributed Hypothesis Testing Under Optimal Detection
The distributed hypothesis testing problem with full side-information is
studied. The trade-off (reliability function) between the two types of error
exponents under limited rate is studied in the following way. First, the
problem is reduced to the problem of determining the reliability function of
channel codes designed for detection (in analogy to a similar result which
connects the reliability function of distributed lossless compression and
ordinary channel codes). Second, a single-letter random-coding bound based on a
hierarchical ensemble, as well as a single-letter expurgated bound, are derived
for the reliability of channel-detection codes. Both bounds are derived for a
system which employs the optimal detection rule. We conjecture that the
resulting random-coding bound is ensemble-tight, and consequently optimal
within the class of quantization-and-binning schemes
Asymptotic Estimates in Information Theory with Non-Vanishing Error Probabilities
This monograph presents a unified treatment of single- and multi-user
problems in Shannon's information theory where we depart from the requirement
that the error probability decays asymptotically in the blocklength. Instead,
the error probabilities for various problems are bounded above by a
non-vanishing constant and the spotlight is shone on achievable coding rates as
functions of the growing blocklengths. This represents the study of asymptotic
estimates with non-vanishing error probabilities.
In Part I, after reviewing the fundamentals of information theory, we discuss
Strassen's seminal result for binary hypothesis testing where the type-I error
probability is non-vanishing and the rate of decay of the type-II error
probability with growing number of independent observations is characterized.
In Part II, we use this basic hypothesis testing result to develop second- and
sometimes, even third-order asymptotic expansions for point-to-point
communication. Finally in Part III, we consider network information theory
problems for which the second-order asymptotics are known. These problems
include some classes of channels with random state, the multiple-encoder
distributed lossless source coding (Slepian-Wolf) problem and special cases of
the Gaussian interference and multiple-access channels. Finally, we discuss
avenues for further research.Comment: Further comments welcom
On optimum parameter modulation-estimation from a large deviations perspective
We consider the problem of jointly optimum modulation and estimation of a
real-valued random parameter, conveyed over an additive white Gaussian noise
(AWGN) channel, where the performance metric is the large deviations behavior
of the estimator, namely, the exponential decay rate (as a function of the
observation time) of the probability that the estimation error would exceed a
certain threshold. Our basic result is in providing an exact characterization
of the fastest achievable exponential decay rate, among all possible
modulator-estimator (transmitter-receiver) pairs, where the modulator is
limited only in the signal power, but not in bandwidth. This exponential rate
turns out to be given by the reliability function of the AWGN channel. We also
discuss several ways to achieve this optimum performance, and one of them is
based on quantization of the parameter, followed by optimum channel coding and
modulation, which gives rise to a separation-based transmitter, if one views
this setting from the perspective of joint source-channel coding. This is in
spite of the fact that, in general, when error exponents are considered, the
source-channel separation theorem does not hold true. We also discuss several
observations, modifications and extensions of this result in several
directions, including other channels, and the case of multidimensional
parameter vectors. One of our findings concerning the latter, is that there is
an abrupt threshold effect in the dimensionality of the parameter vector: below
a certain critical dimension, the probability of excess estimation error may
still decay exponentially, but beyond this value, it must converge to unity.Comment: 26 pages; Submitted to the IEEE Transactions on Information Theor
Guessing Revisited: A Large Deviations Approach
The problem of guessing a random string is revisited. A close relation
between guessing and compression is first established. Then it is shown that if
the sequence of distributions of the information spectrum satisfies the large
deviation property with a certain rate function, then the limiting guessing
exponent exists and is a scalar multiple of the Legendre-Fenchel dual of the
rate function. Other sufficient conditions related to certain continuity
properties of the information spectrum are briefly discussed. This approach
highlights the importance of the information spectrum in determining the
limiting guessing exponent. All known prior results are then re-derived as
example applications of our unifying approach.Comment: 16 pages, to appear in IEEE Transaction on Information Theor
Error Exponent in Asymmetric Quantum Hypothesis Testing and Its Application to Classical-Quantum Channel coding
In the simple quantum hypothesis testing problem, upper bound with asymmetric
setting is shown by using a quite useful inequality by Audenaert et al,
quant-ph/0610027, which was originally invented for symmetric setting. Using
this upper bound, we obtain the Hoeffding bound, which are identical with the
classical counter part if the hypotheses, composed of two density operators,
are mutually commutative. Our upper bound improves the bound by Ogawa-Hayashi,
and also provides a simpler proof of the direct part of the quantum Stein's
lemma. Further, using this bound, we obtain a better exponential upper bound of
the average error probability of classical-quantum channel coding
- …