30,918 research outputs found
Second-Order Asymptotics for the Classical Capacity of Image-Additive Quantum Channels
We study non-asymptotic fundamental limits for transmitting classical
information over memoryless quantum channels, i.e. we investigate the amount of
classical information that can be transmitted when a quantum channel is used a
finite number of times and a fixed, non-vanishing average error is permissible.
We consider the classical capacity of quantum channels that are image-additive,
including all classical to quantum channels, as well as the product state
capacity of arbitrary quantum channels. In both cases we show that the
non-asymptotic fundamental limit admits a second-order approximation that
illustrates the speed at which the rate of optimal codes converges to the
Holevo capacity as the blocklength tends to infinity. The behavior is governed
by a new channel parameter, called channel dispersion, for which we provide a
geometrical interpretation.Comment: v2: main results significantly generalized and improved; v3: extended
to image-additive channels, change of title, journal versio
Comparison of Channels: Criteria for Domination by a Symmetric Channel
This paper studies the basic question of whether a given channel can be
dominated (in the precise sense of being more noisy) by a -ary symmetric
channel. The concept of "less noisy" relation between channels originated in
network information theory (broadcast channels) and is defined in terms of
mutual information or Kullback-Leibler divergence. We provide an equivalent
characterization in terms of -divergence. Furthermore, we develop a
simple criterion for domination by a -ary symmetric channel in terms of the
minimum entry of the stochastic matrix defining the channel . The criterion
is strengthened for the special case of additive noise channels over finite
Abelian groups. Finally, it is shown that domination by a symmetric channel
implies (via comparison of Dirichlet forms) a logarithmic Sobolev inequality
for the original channel.Comment: 31 pages, 2 figures. Presented at 2017 IEEE International Symposium
on Information Theory (ISIT
The Divergence Index: A Decomposable Measure of Segregation and Inequality
Decomposition analysis is a critical tool for understanding the social and
spatial dimensions of inequality, segregation, and diversity. In this paper, I
propose a new measure - the Divergence Index - to address the need for a
decomposable measure of segregation. Although the Information Theory Index has
been used to decompose segregation within and between communities, I argue that
it measures relative diversity not segregation. I demonstrate the importance of
this conceptual distinction with two empirical analyses: I decompose
segregation and relative homogeneity in the Detroit metropolitan area, and I
analyze the relationship between the indexes in the 100 largest U.S. cities. I
show that it is problematic to interpret the Information Theory Index as a
measure of segregation, especially when analyzing local-level results or any
decomposition of overall results. Segregation and diversity are important
aspects of residential differentiation, and it is critical that we study each
concept as the structure and stratification of the U.S. population becomes more
complex
Dissipation of information in channels with input constraints
One of the basic tenets in information theory, the data processing inequality
states that output divergence does not exceed the input divergence for any
channel. For channels without input constraints, various estimates on the
amount of such contraction are known, Dobrushin's coefficient for the total
variation being perhaps the most well-known. This work investigates channels
with average input cost constraint. It is found that while the contraction
coefficient typically equals one (no contraction), the information nevertheless
dissipates. A certain non-linear function, the \emph{Dobrushin curve} of the
channel, is proposed to quantify the amount of dissipation. Tools for
evaluating the Dobrushin curve of additive-noise channels are developed based
on coupling arguments. Some basic applications in stochastic control,
uniqueness of Gibbs measures and fundamental limits of noisy circuits are
discussed.
As an application, it shown that in the chain of power-constrained relays
and Gaussian channels the end-to-end mutual information and maximal squared
correlation decay as , which is in stark
contrast with the exponential decay in chains of discrete channels. Similarly,
the behavior of noisy circuits (composed of gates with bounded fan-in) and
broadcasting of information on trees (of bounded degree) does not experience
threshold behavior in the signal-to-noise ratio (SNR). Namely, unlike the case
of discrete channels, the probability of bit error stays bounded away from
regardless of the SNR.Comment: revised; include appendix B on contraction coefficient for mutual
information on general alphabet
Convergence of Smoothed Empirical Measures with Applications to Entropy Estimation
This paper studies convergence of empirical measures smoothed by a Gaussian
kernel. Specifically, consider approximating , for
, by
, where is the empirical measure,
under different statistical distances. The convergence is examined in terms of
the Wasserstein distance, total variation (TV), Kullback-Leibler (KL)
divergence, and -divergence. We show that the approximation error under
the TV distance and 1-Wasserstein distance () converges at rate
in remarkable contrast to a typical
rate for unsmoothed (and ). For the
KL divergence, squared 2-Wasserstein distance (), and
-divergence, the convergence rate is , but only if
achieves finite input-output mutual information across the additive
white Gaussian noise channel. If the latter condition is not met, the rate
changes to for the KL divergence and , while
the -divergence becomes infinite - a curious dichotomy. As a main
application we consider estimating the differential entropy
in the high-dimensional regime. The distribution
is unknown but i.i.d samples from it are available. We first show that
any good estimator of must have sample complexity
that is exponential in . Using the empirical approximation results we then
show that the absolute-error risk of the plug-in estimator converges at the
parametric rate , thus establishing the minimax
rate-optimality of the plug-in. Numerical results that demonstrate a
significant empirical superiority of the plug-in approach to general-purpose
differential entropy estimators are provided.Comment: arXiv admin note: substantial text overlap with arXiv:1810.1158
A Simple Derivation of the Refined Sphere Packing Bound Under Certain Symmetry Hypotheses
A judicious application of the Berry-Esseen theorem via suitable Augustin
information measures is demonstrated to be sufficient for deriving the sphere
packing bound with a prefactor that is
for all codes on certain
families of channels -- including the Gaussian channels and the non-stationary
Renyi symmetric channels -- and for the constant composition codes on
stationary memoryless channels. The resulting non-asymptotic bounds have
definite approximation error terms. As a preliminary result that might be of
interest on its own, the trade-off between type I and type II error
probabilities in the hypothesis testing problem with (possibly non-stationary)
independent samples is determined up to some multiplicative constants, assuming
that the probabilities of both types of error are decaying exponentially with
the number of samples, using the Berry-Esseen theorem.Comment: 20 page
- β¦