30 research outputs found
Variations on a theme by Schalkwijk and Kailath
Schalkwijk and Kailath (1966) developed a class of block codes for Gaussian
channels with ideal feedback for which the probability of decoding error
decreases as a second-order exponent in block length for rates below capacity.
This well-known but surprising result is explained and simply derived here in
terms of a result by Elias (1956) concerning the minimum mean-square distortion
achievable in transmitting a single Gaussian random variable over multiple uses
of the same Gaussian channel. A simple modification of the Schalkwijk-Kailath
scheme is then shown to have an error probability that decreases with an
exponential order which is linearly increasing with block length. In the
infinite bandwidth limit, this scheme produces zero error probability using
bounded expected energy at all rates below capacity. A lower bound on error
probability for the finite bandwidth case is then derived in which the error
probability decreases with an exponential order which is linearly increasing in
block length at the same rate as the upper bound.Comment: 18 Pages, 4 figures (added reference
Optimal Feedback Communication via Posterior Matching
In this paper we introduce a fundamental principle for optimal communication
over general memoryless channels in the presence of noiseless feedback, termed
posterior matching. Using this principle, we devise a (simple, sequential)
generic feedback transmission scheme suitable for a large class of memoryless
channels and input distributions, achieving any rate below the corresponding
mutual information. This provides a unified framework for optimal feedback
communication in which the Horstein scheme (BSC) and the Schalkwijk-Kailath
scheme (AWGN channel) are special cases. Thus, as a corollary, we prove that
the Horstein scheme indeed attains the BSC capacity, settling a longstanding
conjecture. We further provide closed form expressions for the error
probability of the scheme over a range of rates, and derive the achievable
rates in a mismatch setting where the scheme is designed according to the wrong
channel model. Several illustrative examples of the posterior matching scheme
for specific channels are given, and the corresponding error probability
expressions are evaluated. The proof techniques employed utilize novel
relations between information rates and contraction properties of iterated
function systems.Comment: IEEE Transactions on Information Theor
Variable-Length Coding with Feedback: Finite-Length Codewords and Periodic Decoding
Theoretical analysis has long indicated that feedback improves the error
exponent but not the capacity of single-user memoryless channels. Recently
Polyanskiy et al. studied the benefit of variable-length feedback with
termination (VLFT) codes in the non-asymptotic regime. In that work,
achievability is based on an infinite length random code and decoding is
attempted at every symbol. The coding rate backoff from capacity due to channel
dispersion is greatly reduced with feedback, allowing capacity to be approached
with surprisingly small expected latency. This paper is mainly concerned with
VLFT codes based on finite-length codes and decoding attempts only at certain
specified decoding times. The penalties of using a finite block-length and
a sequence of specified decoding times are studied. This paper shows that
properly scaling with the expected latency can achieve the same performance
up to constant terms as with . The penalty introduced by periodic
decoding times is a linear term of the interval between decoding times and
hence the performance approaches capacity as the expected latency grows if the
interval between decoding times grows sub-linearly with the expected latency.Comment: 8 pages. A shorten version is submitted to ISIT 201