4,765 research outputs found
The Capacity of Channels with Feedback
We introduce a general framework for treating channels with memory and
feedback. First, we generalize Massey's concept of directed information and use
it to characterize the feedback capacity of general channels. Second, we
present coding results for Markov channels. This requires determining
appropriate sufficient statistics at the encoder and decoder. Third, a dynamic
programming framework for computing the capacity of Markov channels is
presented. Fourth, it is shown that the average cost optimality equation (ACOE)
can be viewed as an implicit single-letter characterization of the capacity.
Fifth, scenarios with simple sufficient statistics are described
Wiretap and Gelfand-Pinsker Channels Analogy and its Applications
An analogy framework between wiretap channels (WTCs) and state-dependent
point-to-point channels with non-causal encoder channel state information
(referred to as Gelfand-Pinker channels (GPCs)) is proposed. A good sequence of
stealth-wiretap codes is shown to induce a good sequence of codes for a
corresponding GPC. Consequently, the framework enables exploiting existing
results for GPCs to produce converse proofs for their wiretap analogs. The
analogy readily extends to multiuser broadcasting scenarios, encompassing
broadcast channels (BCs) with deterministic components, degradation ordering
between users, and BCs with cooperative receivers. Given a wiretap BC (WTBC)
with two receivers and one eavesdropper, an analogous Gelfand-Pinsker BC (GPBC)
is constructed by converting the eavesdropper's observation sequence into a
state sequence with an appropriate product distribution (induced by the
stealth-wiretap code for the WTBC), and non-causally revealing the states to
the encoder. The transition matrix of the state-dependent GPBC is extracted
from WTBC's transition law, with the eavesdropper's output playing the role of
the channel state. Past capacity results for the semi-deterministic (SD) GPBC
and the physically-degraded (PD) GPBC with an informed receiver are leveraged
to furnish analogy-based converse proofs for the analogous WTBC setups. This
characterizes the secrecy-capacity regions of the SD-WTBC and the PD-WTBC, in
which the stronger receiver also observes the eavesdropper's channel output.
These derivations exemplify how the wiretap-GP analogy enables translating
results on one problem into advances in the study of the other
Permutation Complexity and Coupling Measures in Hidden Markov Models
In [Haruna, T. and Nakajima, K., 2011. Physica D 240, 1370-1377], the authors
introduced the duality between values (words) and orderings (permutations) as a
basis to discuss the relationship between information theoretic measures for
finite-alphabet stationary stochastic processes and their permutation
analogues. It has been used to give a simple proof of the equality between the
entropy rate and the permutation entropy rate for any finite-alphabet
stationary stochastic process and show some results on the excess entropy and
the transfer entropy for finite-alphabet stationary ergodic Markov processes.
In this paper, we extend our previous results to hidden Markov models and show
the equalities between various information theoretic complexity and coupling
measures and their permutation analogues. In particular, we show the following
two results within the realm of hidden Markov models with ergodic internal
processes: the two permutation analogues of the transfer entropy, the symbolic
transfer entropy and the transfer entropy on rank vectors, are both equivalent
to the transfer entropy if they are considered as the rates, and the directed
information theory can be captured by the permutation entropy approach.Comment: 26 page
Optimal Feedback Communication via Posterior Matching
In this paper we introduce a fundamental principle for optimal communication
over general memoryless channels in the presence of noiseless feedback, termed
posterior matching. Using this principle, we devise a (simple, sequential)
generic feedback transmission scheme suitable for a large class of memoryless
channels and input distributions, achieving any rate below the corresponding
mutual information. This provides a unified framework for optimal feedback
communication in which the Horstein scheme (BSC) and the Schalkwijk-Kailath
scheme (AWGN channel) are special cases. Thus, as a corollary, we prove that
the Horstein scheme indeed attains the BSC capacity, settling a longstanding
conjecture. We further provide closed form expressions for the error
probability of the scheme over a range of rates, and derive the achievable
rates in a mismatch setting where the scheme is designed according to the wrong
channel model. Several illustrative examples of the posterior matching scheme
for specific channels are given, and the corresponding error probability
expressions are evaluated. The proof techniques employed utilize novel
relations between information rates and contraction properties of iterated
function systems.Comment: IEEE Transactions on Information Theor
Source Coding When the Side Information May Be Delayed
For memoryless sources, delayed side information at the decoder does not
improve the rate-distortion function. However, this is not the case for more
general sources with memory, as demonstrated by a number of works focusing on
the special case of (delayed) feedforward. In this paper, a setting is studied
in which the encoder is potentially uncertain about the delay with which
measurements of the side information are acquired at the decoder. Assuming a
hidden Markov model for the sources, at first, a single-letter characterization
is given for the set-up where the side information delay is arbitrary and known
at the encoder, and the reconstruction at the destination is required to be
(near) lossless. Then, with delay equal to zero or one source symbol, a
single-letter characterization is given of the rate-distortion region for the
case where side information may be delayed or not, unbeknownst to the encoder.
The characterization is further extended to allow for additional information to
be sent when the side information is not delayed. Finally, examples for binary
and Gaussian sources are provided.Comment: revised July 201
- …