4,383 research outputs found
Error Exponents for Variable-length Block Codes with Feedback and Cost Constraints
Variable-length block-coding schemes are investigated for discrete memoryless
channels with ideal feedback under cost constraints. Upper and lower bounds are
found for the minimum achievable probability of decoding error as
a function of constraints R, \AV, and on the transmission rate,
average cost, and average block length respectively. For given and \AV,
the lower and upper bounds to the exponent are
asymptotically equal as . The resulting reliability
function, , as a
function of and \AV, is concave in the pair (R, \AV) and generalizes
the linear reliability function of Burnashev to include cost constraints. The
results are generalized to a class of discrete-time memoryless channels with
arbitrary alphabets, including additive Gaussian noise channels with amplitude
and power constraints
Control-theoretic Approach to Communication with Feedback: Fundamental Limits and Code Design
Feedback communication is studied from a control-theoretic perspective,
mapping the communication problem to a control problem in which the control
signal is received through the same noisy channel as in the communication
problem, and the (nonlinear and time-varying) dynamics of the system determine
a subclass of encoders available at the transmitter. The MMSE capacity is
defined to be the supremum exponential decay rate of the mean square decoding
error. This is upper bounded by the information-theoretic feedback capacity,
which is the supremum of the achievable rates. A sufficient condition is
provided under which the upper bound holds with equality. For the special class
of stationary Gaussian channels, a simple application of Bode's integral
formula shows that the feedback capacity, recently characterized by Kim, is
equal to the maximum instability that can be tolerated by the controller under
a given power constraint. Finally, the control mapping is generalized to the
N-sender AWGN multiple access channel. It is shown that Kramer's code for this
channel, which is known to be sum rate optimal in the class of generalized
linear feedback codes, can be obtained by solving a linear quadratic Gaussian
control problem.Comment: Submitted to IEEE Transactions on Automatic Contro
On Channel Resolvability in Presence of Feedback
We study the problem of generating an approximately i.i.d. string at the
output of a discrete memoryless channel using a limited amount of randomness at
its input in presence of causal noiseless feedback. Feedback does not decrease
the channel resolution, the minimum entropy rate required to achieve an
accurate approximation of an i.i.d. output string. However, we show that, at
least over a binary symmetric channel, a significantly larger resolvability
exponent (the exponential decay rate of the divergence between the output
distribution and product measure), compared to the best known achievable
resolvability exponent in a system without feedback, is possible. We show that
by employing a variable-length resolvability scheme and using an average number
of coin-flips per channel use, the average divergence between the distribution
of the output sequence and product measure decays exponentially fast in the
average length of output sequence with an exponent equal to
where is the mutual information developed across the channel.Comment: 8 pages, 4 figures; to be presented at the 54th Annual Allerton
Conference on Communication, Control, and Computin
Unequal Error Protection Querying Policies for the Noisy 20 Questions Problem
In this paper, we propose an open-loop unequal-error-protection querying
policy based on superposition coding for the noisy 20 questions problem. In
this problem, a player wishes to successively refine an estimate of the value
of a continuous random variable by posing binary queries and receiving noisy
responses. When the queries are designed non-adaptively as a single block and
the noisy responses are modeled as the output of a binary symmetric channel the
20 questions problem can be mapped to an equivalent problem of channel coding
with unequal error protection (UEP). A new non-adaptive querying strategy based
on UEP superposition coding is introduced whose estimation error decreases with
an exponential rate of convergence that is significantly better than that of
the UEP repetition coding introduced by Variani et al. (2015). With the
proposed querying strategy, the rate of exponential decrease in the number of
queries matches the rate of a closed-loop adaptive scheme where queries are
sequentially designed with the benefit of feedback. Furthermore, the achievable
error exponent is significantly better than that of random block codes
employing equal error protection.Comment: To appear in IEEE Transactions on Information Theor
Asymptotic Estimates in Information Theory with Non-Vanishing Error Probabilities
This monograph presents a unified treatment of single- and multi-user
problems in Shannon's information theory where we depart from the requirement
that the error probability decays asymptotically in the blocklength. Instead,
the error probabilities for various problems are bounded above by a
non-vanishing constant and the spotlight is shone on achievable coding rates as
functions of the growing blocklengths. This represents the study of asymptotic
estimates with non-vanishing error probabilities.
In Part I, after reviewing the fundamentals of information theory, we discuss
Strassen's seminal result for binary hypothesis testing where the type-I error
probability is non-vanishing and the rate of decay of the type-II error
probability with growing number of independent observations is characterized.
In Part II, we use this basic hypothesis testing result to develop second- and
sometimes, even third-order asymptotic expansions for point-to-point
communication. Finally in Part III, we consider network information theory
problems for which the second-order asymptotics are known. These problems
include some classes of channels with random state, the multiple-encoder
distributed lossless source coding (Slepian-Wolf) problem and special cases of
the Gaussian interference and multiple-access channels. Finally, we discuss
avenues for further research.Comment: Further comments welcom
- β¦