4,623 research outputs found
Capacity Achieving Distributions & Information Lossless Randomized Strategies for Feedback Channels with Memory: The LQG Theory of Directed Information-Part II
A methodology is developed to realized optimal channel input conditional
distributions, which maximize the finite-time horizon directed information, for
channels with memory and feedback, by information lossless randomized
strategies. The methodology is applied to general Time-Varying Multiple Input
Multiple Output (MIMO) Gaussian Linear Channel Models (G-LCMs) with memory,
subject to average transmission cost constraints of quadratic form. The
realizations of optimal distributions by randomized strategies are shown to
exhibit a decomposion into a deterministic part and a random part. The
decomposition reveals the dual role of randomized strategies, to control the
channel output process and to transmit new information over the channels.
Moreover, a separation principle is shown between the computation of the
optimal deterministic part and the random part of the randomized strategies.
The dual role of randomized strategies generalizes the
Linear-Quadratic-Gaussian (LQG) stochastic optimal control theory to directed
information pay-offs. The characterizations of feedback capacity are obtained
from the per unit time limits of finite-time horizon directed information,
without imposing \'a priori assumptions, such as, stability of channel models
or ergodicity of channel input and output processes. For time-invariant MIMO
G-LCMs with memory, it is shown that whether feedback increases capacity, is
directly related to the channel parameters and the transmission cost function,
through the solutions of Riccati matrix equations, and moreover for unstable
channels, feedback capacity is non-zero, provided the power exceeds a critical
level.Comment: Submitted to the IEEE Transactions on Information Theory, Mar. 2016,
paper IT-16-016
Capacity Region of the Finite-State Multiple Access Channel with and without Feedback
The capacity region of the Finite-State Multiple Access Channel (FS-MAC) with
feedback that may be an arbitrary time-invariant function of the channel output
samples is considered. We characterize both an inner and an outer bound for
this region, using Masseys's directed information. These bounds are shown to
coincide, and hence yield the capacity region, of FS-MACs where the state
process is stationary and ergodic and not affected by the inputs.
Though `multi-letter' in general, our results yield explicit conclusions when
applied to specific scenarios of interest. E.g., our results allow us to:
- Identify a large class of FS-MACs, that includes the additive mod-2 noise
MAC where the noise may have memory, for which feedback does not enlarge the
capacity region.
- Deduce that, for a general FS-MAC with states that are not affected by the
input, if the capacity (region) without feedback is zero, then so is the
capacity (region) with feedback.
- Deduce that the capacity region of a MAC that can be decomposed into a
`multiplexer' concatenated by a point-to-point channel (with, without, or with
partial feedback), the capacity region is given by , where
C is the capacity of the point to point channel and m indexes the encoders.
Moreover, we show that for this family of channels source-channel coding
separation holds
Stationary and Ergodic Properties of Stochastic Non-Linear Systems Controlled over Communication Channels
This paper is concerned with the following problem: Given a stochastic
non-linear system controlled over a noisy channel, what is the largest class of
channels for which there exist coding and control policies so that the closed
loop system is stochastically stable? Stochastic stability notions considered
are stationarity, ergodicity or asymptotic mean stationarity. We do not
restrict the state space to be compact, for example systems considered can be
driven by unbounded noise. Necessary and sufficient conditions are obtained for
a large class of systems and channels. A generalization of Bode's Integral
Formula for a large class of non-linear systems and information channels is
obtained. The findings generalize existing results for linear systems.Comment: To appear in SIAM Journal on Control and Optimizatio
On the error exponent of variable-length block-coding schemes over finite-state Markov channels with feedback
The error exponent of Markov channels with feedback is studied in the
variable-length block-coding setting. Burnashev's classic result is extended
and a single letter characterization for the reliability function of
finite-state Markov channels is presented, under the assumption that the
channel state is causally observed both at the transmitter and at the receiver
side. Tools from stochastic control theory are used in order to treat channels
with intersymbol interference. In particular the convex analytical approach to
Markov decision processes is adopted to handle problems with stopping time
horizons arising from variable-length coding schemes
A Single-Letter Upper Bound on the Feedback Capacity of Unifilar Finite-State Channels
An upper bound on the feedback capacity of unifilar finite-state channels
(FSCs) is derived. A new technique, called the -contexts, is based on a
construction of a directed graph that is used to quantize recursively the
receiver's output sequences to a finite set of contexts. For any choice of
-graph, the feedback capacity is bounded by a single-letter expression,
, where the supremum is over and
the distribution of is their stationary distribution. It is shown that
the bound is tight for all unifilar FSCs where feedback capacity is known:
channels where the state is a function of the outputs, the trapdoor channel,
Ising channels, the no-consecutive-ones input-constrained erasure channel and
for the memoryless channel. Its efficiency is also demonstrated by deriving a
new capacity result for the dicode erasure channel (DEC); the upper bound is
obtained directly from the above expression and its tightness is concluded with
a general sufficient condition on the optimality of the upper bound. This
sufficient condition is based on a fixed point principle of the BCJR equation
and, indeed, formulated as a simple lower bound on feedback capacity of
unifilar FSCs for arbitrary -graphs. This upper bound indicates that a
single-letter expression might exist for the capacity of finite-state channels
with or without feedback based on a construction of auxiliary random variable
with specified structure, such as -graph, and not with i.i.d distribution.
The upper bound also serves as a non-trivial bound on the capacity of channels
without feedback, a problem that is still open
Capacity of the Trapdoor Channel with Feedback
We establish that the feedback capacity of the trapdoor channel is the
logarithm of the golden ratio and provide a simple communication scheme that
achieves capacity. As part of the analysis, we formulate a class of dynamic
programs that characterize capacities of unifilar finite-state channels. The
trapdoor channel is an instance that admits a simple analytic solution
Fundamental Limitations of Disturbance Attenuation in the Presence of Side Information
In this paper, we study fundamental limitations of disturbance attenuation of feedback systems, under the assumption that the controller has a finite horizon preview of the disturbance. In contrast with prior work, we extend Bode's integral equation for the case where the preview is made available to the controller via a general, finite capacity, communication system. Under asymptotic stationarity assumptions, our results show that the new fundamental limitation differs from Bode's only by a constant, which quantifies the information rate through the communication system. In the absence of asymptotic stationarity, we derive a universal lower bound which uses Shannon's entropy rate as a measure of performance. By means of a case-study, we show that our main bounds may be achieved
Random-Time, State-Dependent Stochastic Drift for Markov Chains and Application to Stochastic Stabilization Over Erasure Channels
It is known that state-dependent, multi-step Lyapunov bounds lead to greatly
simplified verification theorems for stability for large classes of Markov
chain models. This is one component of the "fluid model" approach to stability
of stochastic networks. In this paper we extend the general theory to
randomized multi-step Lyapunov theory to obtain criteria for stability and
steady-state performance bounds, such as finite moments.
These results are applied to a remote stabilization problem, in which a
controller receives measurements from an erasure channel with limited capacity.
Based on the general results in the paper it is shown that stability of the
closed loop system is assured provided that the channel capacity is greater
than the logarithm of the unstable eigenvalue, plus an additional correction
term. The existence of a finite second moment in steady-state is established
under additional conditions.Comment: To appear in IEEE Transactions on Automatic Contro
Infinite Horizon Optimal Transmission Power Control for Remote State Estimation over Fading Channels
Jointly optimal transmission power control and remote estimation over an
infinite horizon is studied. A sensor observes a dynamic process and sends its
observations to a remote estimator over a wireless fading channel characterized
by a time-homogeneous Markov chain. The successful transmission probability
depends on both the channel gains and the transmission power used by the
sensor. The transmission power control rule and the remote estimator should be
jointly designed, aiming to minimize an infinite-horizon cost consisting of the
power usage and the remote estimation error. A first question one may ask is:
Does this joint optimization problem have a solution? We formulate the joint
optimization problem as an average cost belief-state Markov decision process
and answer the question by proving that there exists an optimal deterministic
and stationary policy. We then show that when the monitored dynamic process is
scalar, the optimal remote estimates depend only on the most recently received
sensor observation, and the optimal transmission power is symmetric and
monotonically increasing with respect to the innovation error
A Nonstochastic Information Theory for Communication and State Estimation
In communications, unknown variables are usually modelled as random
variables, and concepts such as independence, entropy and information are
defined in terms of the underlying probability distributions. In contrast,
control theory often treats uncertainties and disturbances as bounded unknowns
having no statistical structure. The area of networked control combines both
fields, raising the question of whether it is possible to construct meaningful
analogues of stochastic concepts such as independence, Markovness, entropy and
information without assuming a probability space. This paper introduces a
framework for doing so, leading to the construction of a maximin information
functional for nonstochastic variables. It is shown that the largest maximin
information rate through a memoryless, error-prone channel in this framework
coincides with the block-coding zero-error capacity of the channel. Maximin
information is then used to derive tight conditions for uniformly estimating
the state of a linear time-invariant system over such a channel, paralleling
recent results of Matveev and Savkin
- …