76 research outputs found
On the Evaluation of the Polyanskiy-Poor-Verdu Converse Bound for Finite Blocklength Coding in AWGN
A tight converse bound to channel coding rate in the finite block-length
regime and under AWGN conditions was recently proposed by Polyanskiy, Poor, and
Verdu (PPV). The bound is a generalization of a number of other classical
results, and it was also claimed to be equivalent to Shannon's 1959 cone
packing bound. Unfortunately, its numerical evaluation is troublesome even for
not too large values of the block-length n. In this paper we tackle the
numerical evaluation by compactly expressing the PPV converse bound in terms of
non-central chi-squared distributions, and by evaluating those through a an
integral expression and a corresponding series expansion which exploit a method
proposed by Temme. As a result, a robust evaluation method and new insights on
the bound's asymptotics, as well as new approximate expressions, are given.Comment: 13 pages, 10 figures. Matlab code available from
http://dgt.dei.unipd.it section Download->Finite Blocklength Regim
Beta-Beta Bounds: Finite-Blocklength Analog of the Golden Formula
It is well known that the mutual information between two random variables can
be expressed as the difference of two relative entropies that depend on an
auxiliary distribution, a relation sometimes referred to as the golden formula.
This paper is concerned with a finite-blocklength extension of this relation.
This extension consists of two elements: 1) a finite-blocklength channel-coding
converse bound by Polyanskiy and Verd\'{u} (2014), which involves the ratio of
two Neyman-Pearson functions (beta-beta converse bound); and 2) a novel
beta-beta channel-coding achievability bound, expressed again as the ratio of
two Neyman-Pearson functions.
To demonstrate the usefulness of this finite-blocklength extension of the
golden formula, the beta-beta achievability and converse bounds are used to
obtain a finite-blocklength extension of Verd\'{u}'s (2002) wideband-slope
approximation. The proof parallels the derivation of the latter, with the
beta-beta bounds used in place of the golden formula.
The beta-beta (achievability) bound is also shown to be useful in cases where
the capacity-achieving output distribution is not a product distribution due
to, e.g., a cost constraint or structural constraints on the codebook, such as
orthogonality or constant composition. As an example, the bound is used to
characterize the channel dispersion of the additive exponential-noise channel
and to obtain a finite-blocklength achievability bound (the tightest to date)
for multiple-input multiple-output Rayleigh-fading channels with perfect
channel state information at the receiver.Comment: to appear in IEEE Transactions on Information Theor
A Beta-Beta Achievability Bound with Applications
A channel coding achievability bound expressed in terms of the ratio between
two Neyman-Pearson functions is proposed. This bound is the dual of a
converse bound established earlier by Polyanskiy and Verd\'{u} (2014). The new
bound turns out to simplify considerably the analysis in situations where the
channel output distribution is not a product distribution, for example due to a
cost constraint or a structural constraint (such as orthogonality or constant
composition) on the channel inputs. Connections to existing bounds in the
literature are discussed. The bound is then used to derive 1) an achievability
bound on the channel dispersion of additive non-Gaussian noise channels with
random Gaussian codebooks, 2) the channel dispersion of the exponential-noise
channel, 3) a second-order expansion for the minimum energy per bit of an AWGN
channel, and 4) a lower bound on the maximum coding rate of a multiple-input
multiple-output Rayleigh-fading channel with perfect channel state information
at the receiver, which is the tightest known achievability result.Comment: extended version of a paper submitted to ISIT 201
Joint source-channel coding with feedback
This paper quantifies the fundamental limits of variable-length transmission
of a general (possibly analog) source over a memoryless channel with noiseless
feedback, under a distortion constraint. We consider excess distortion, average
distortion and guaranteed distortion (-semifaithful codes). In contrast to
the asymptotic fundamental limit, a general conclusion is that allowing
variable-length codes and feedback leads to a sizable improvement in the
fundamental delay-distortion tradeoff. In addition, we investigate the minimum
energy required to reproduce source samples with a given fidelity after
transmission over a memoryless Gaussian channel, and we show that the required
minimum energy is reduced with feedback and an average (rather than maximal)
power constraint.Comment: To appear in IEEE Transactions on Information Theor
Asymptotic Estimates in Information Theory with Non-Vanishing Error Probabilities
This monograph presents a unified treatment of single- and multi-user
problems in Shannon's information theory where we depart from the requirement
that the error probability decays asymptotically in the blocklength. Instead,
the error probabilities for various problems are bounded above by a
non-vanishing constant and the spotlight is shone on achievable coding rates as
functions of the growing blocklengths. This represents the study of asymptotic
estimates with non-vanishing error probabilities.
In Part I, after reviewing the fundamentals of information theory, we discuss
Strassen's seminal result for binary hypothesis testing where the type-I error
probability is non-vanishing and the rate of decay of the type-II error
probability with growing number of independent observations is characterized.
In Part II, we use this basic hypothesis testing result to develop second- and
sometimes, even third-order asymptotic expansions for point-to-point
communication. Finally in Part III, we consider network information theory
problems for which the second-order asymptotics are known. These problems
include some classes of channels with random state, the multiple-encoder
distributed lossless source coding (Slepian-Wolf) problem and special cases of
the Gaussian interference and multiple-access channels. Finally, we discuss
avenues for further research.Comment: Further comments welcom
Peak-to-average power ratio of good codes for Gaussian channel
Consider a problem of forward error-correction for the additive white
Gaussian noise (AWGN) channel. For finite blocklength codes the backoff from
the channel capacity is inversely proportional to the square root of the
blocklength. In this paper it is shown that codes achieving this tradeoff must
necessarily have peak-to-average power ratio (PAPR) proportional to logarithm
of the blocklength. This is extended to codes approaching capacity slower, and
to PAPR measured at the output of an OFDM modulator. As a by-product the
convergence of (Smith's) amplitude-constrained AWGN capacity to Shannon's
classical formula is characterized in the regime of large amplitudes. This
converse-type result builds upon recent contributions in the study of empirical
output distributions of good channel codes
Quasi-Static SIMO Fading Channels at Finite Blocklength
We investigate the maximal achievable rate for a given blocklength and error
probability over quasi-static single-input multiple-output (SIMO) fading
channels. Under mild conditions on the channel gains, it is shown that the
channel dispersion is zero regardless of whether the fading realizations are
available at the transmitter and/or the receiver. The result follows from
computationally and analytically tractable converse and achievability bounds.
Through numerical evaluation, we verify that, in some scenarios, zero
dispersion indeed entails fast convergence to outage capacity as the
blocklength increases. In the example of a particular 1*2 SIMO Rician channel,
the blocklength required to achieve 90% of capacity is about an order of
magnitude smaller compared to the blocklength required for an AWGN channel with
the same capacity.Comment: extended version of a paper submitted to ISIT 201
Joint source-channel coding with feedback
This paper quantifies the fundamental limits of variable-length transmission of a general (possibly analog) source over a memoryless channel with noiseless feedback, under a distortion constraint. We consider excess distortion, average distortion and guaranteed distortion (d-semifaithful codes). In contrast to the asymptotic fundamental limit, a general conclusion is that allowing variable-length codes and feedback leads to a sizable improvement in the fundamental delay-distortion tradeoff
Coding in the Finite-Blocklength Regime: Bounds based on Laplace Integrals and their Asymptotic Approximations
In this paper we provide new compact integral expressions and associated
simple asymptotic approximations for converse and achievability bounds in the
finite blocklength regime. The chosen converse and random coding union bounds
were taken from the recent work of Polyanskyi-Poor-Verdu, and are investigated
under parallel AWGN channels, the AWGN channels, the BI-AWGN channel, and the
BSC. The technique we use, which is a generalization of some recent results
available from the literature, is to map the probabilities of interest into a
Laplace integral, and then solve (or approximate) the integral by use of a
steepest descent technique. The proposed results are particularly useful for
short packet lengths, where the normal approximation may provide unreliable
results.Comment: 29 pages, 10 figures. Submitted to IEEE Trans. on Information Theory.
Matlab code available from http://dgt.dei.unipd.it section Download->Finite
Blocklength Regim
- …