11,709 research outputs found
Discrimination on the Grassmann Manifold: Fundamental Limits of Subspace Classifiers
We present fundamental limits on the reliable classification of linear and
affine subspaces from noisy, linear features. Drawing an analogy between
discrimination among subspaces and communication over vector wireless channels,
we propose two Shannon-inspired measures to characterize asymptotic classifier
performance. First, we define the classification capacity, which characterizes
necessary and sufficient conditions for the misclassification probability to
vanish as the signal dimension, the number of features, and the number of
subspaces to be discerned all approach infinity. Second, we define the
diversity-discrimination tradeoff which, by analogy with the
diversity-multiplexing tradeoff of fading vector channels, characterizes
relationships between the number of discernible subspaces and the
misclassification probability as the noise power approaches zero. We derive
upper and lower bounds on these measures which are tight in many regimes.
Numerical results, including a face recognition application, validate the
results in practice.Comment: 19 pages, 4 figures. Revised submission to IEEE Transactions on
Information Theor
A Characterization of the Shannon Ordering of Communication Channels
The ordering of communication channels was first introduced by Shannon. In
this paper, we aim to find a characterization of the Shannon ordering. We show
that contains if and only if is the skew-composition of with
a convex-product channel. This fact is used to derive a characterization of the
Shannon ordering that is similar to the Blackwell-Sherman-Stein theorem. Two
channels are said to be Shannon-equivalent if each one is contained in the
other. We investigate the topologies that can be constructed on the space of
Shannon-equivalent channels. We introduce the strong topology and the BRM
metric on this space. Finally, we study the continuity of a few channel
parameters and operations under the strong topology.Comment: 23 pages, presented in part at ISIT'17. arXiv admin note: text
overlap with arXiv:1702.0072
State Amplification
We consider the problem of transmitting data at rate R over a state dependent
channel p(y|x,s) with the state information available at the sender and at the
same time conveying the information about the channel state itself to the
receiver. The amount of state information that can be learned at the receiver
is captured by the mutual information I(S^n; Y^n) between the state sequence
S^n and the channel output Y^n. The optimal tradeoff is characterized between
the information transmission rate R and the state uncertainty reduction rate
\Delta, when the state information is either causally or noncausally available
at the sender. This result is closely related and in a sense dual to a recent
study by Merhav and Shamai, which solves the problem of masking the state
information from the receiver rather than conveying it.Comment: 9 pages, 4 figures, submitted to IEEE Trans. Inform. Theory, revise
Information-Theoretic Capacity and Error Exponents of Stationary Point Processes under Random Additive Displacements
This paper studies the Shannon regime for the random displacement of
stationary point processes. Let each point of some initial stationary point
process in give rise to one daughter point, the location of which is
obtained by adding a random vector to the coordinates of the mother point, with
all displacement vectors independently and identically distributed for all
points. The decoding problem is then the following one: the whole mother point
process is known as well as the coordinates of some daughter point; the
displacements are only known through their law; can one find the mother of this
daughter point? The Shannon regime is that where the dimension tends to
infinity and where the logarithm of the intensity of the point process is
proportional to . We show that this problem exhibits a sharp threshold: if
the sum of the proportionality factor and of the differential entropy rate of
the noise is positive, then the probability of finding the right mother point
tends to 0 with for all point processes and decoding strategies. If this
sum is negative, there exist mother point processes, for instance Poisson, and
decoding strategies, for instance maximum likelihood, for which the probability
of finding the right mother tends to 1 with . We then use large deviations
theory to show that in the latter case, if the entropy spectrum of the noise
satisfies a large deviation principle, then the error probability goes
exponentially fast to 0 with an exponent that is given in closed form in terms
of the rate function of the noise entropy spectrum. This is done for two
classes of mother point processes: Poisson and Mat\'ern. The practical interest
to information theory comes from the explicit connection that we also establish
between this problem and the estimation of error exponents in Shannon's
additive noise channel with power constraints on the codewords
- …