6,201 research outputs found
A Characterization of the Shannon Ordering of Communication Channels
The ordering of communication channels was first introduced by Shannon. In
this paper, we aim to find a characterization of the Shannon ordering. We show
that contains if and only if is the skew-composition of with
a convex-product channel. This fact is used to derive a characterization of the
Shannon ordering that is similar to the Blackwell-Sherman-Stein theorem. Two
channels are said to be Shannon-equivalent if each one is contained in the
other. We investigate the topologies that can be constructed on the space of
Shannon-equivalent channels. We introduce the strong topology and the BRM
metric on this space. Finally, we study the continuity of a few channel
parameters and operations under the strong topology.Comment: 23 pages, presented in part at ISIT'17. arXiv admin note: text
overlap with arXiv:1702.0072
Communications-Inspired Projection Design with Application to Compressive Sensing
We consider the recovery of an underlying signal x \in C^m based on
projection measurements of the form y=Mx+w, where y \in C^l and w is
measurement noise; we are interested in the case l < m. It is assumed that the
signal model p(x) is known, and w CN(w;0,S_w), for known S_W. The objective is
to design a projection matrix M \in C^(l x m) to maximize key
information-theoretic quantities with operational significance, including the
mutual information between the signal and the projections I(x;y) or the Renyi
entropy of the projections h_a(y) (Shannon entropy is a special case). By
capitalizing on explicit characterizations of the gradients of the information
measures with respect to the projections matrix, where we also partially extend
the well-known results of Palomar and Verdu from the mutual information to the
Renyi entropy domain, we unveil the key operations carried out by the optimal
projections designs: mode exposure and mode alignment. Experiments are
considered for the case of compressive sensing (CS) applied to imagery. In this
context, we provide a demonstration of the performance improvement possible
through the application of the novel projection designs in relation to
conventional ones, as well as justification for a fast online projections
design method with which state-of-the-art adaptive CS signal recovery is
achieved.Comment: 25 pages, 7 figures, parts of material published in IEEE ICASSP 2012,
submitted to SIIM
A Proof of Entropy Minimization for Outputs in Deletion Channels via Hidden Word Statistics
From the output produced by a memoryless deletion channel from a uniformly
random input of known length , one obtains a posterior distribution on the
channel input. The difference between the Shannon entropy of this distribution
and that of the uniform prior measures the amount of information about the
channel input which is conveyed by the output of length , and it is natural
to ask for which outputs this is extremized. This question was posed in a
previous work, where it was conjectured on the basis of experimental data that
the entropy of the posterior is minimized and maximized by the constant strings
and and the alternating strings
and respectively. In the present
work we confirm the minimization conjecture in the asymptotic limit using
results from hidden word statistics. We show how the analytic-combinatorial
methods of Flajolet, Szpankowski and Vall\'ee for dealing with the hidden
pattern matching problem can be applied to resolve the case of fixed output
length and , by obtaining estimates for the entropy in
terms of the moments of the posterior distribution and establishing its
minimization via a measure of autocorrelation.Comment: 11 pages, 2 figure
Cores of Cooperative Games in Information Theory
Cores of cooperative games are ubiquitous in information theory, and arise
most frequently in the characterization of fundamental limits in various
scenarios involving multiple users. Examples include classical settings in
network information theory such as Slepian-Wolf source coding and multiple
access channels, classical settings in statistics such as robust hypothesis
testing, and new settings at the intersection of networking and statistics such
as distributed estimation problems for sensor networks. Cooperative game theory
allows one to understand aspects of all of these problems from a fresh and
unifying perspective that treats users as players in a game, sometimes leading
to new insights. At the heart of these analyses are fundamental dualities that
have been long studied in the context of cooperative games; for information
theoretic purposes, these are dualities between information inequalities on the
one hand and properties of rate, capacity or other resource allocation regions
on the other.Comment: 12 pages, published at
http://www.hindawi.com/GetArticle.aspx?doi=10.1155/2008/318704 in EURASIP
Journal on Wireless Communications and Networking, Special Issue on "Theory
and Applications in Multiuser/Multiterminal Communications", April 200
- …