16 research outputs found
Capacity Definitions for General Channels with Receiver Side Information
We consider three capacity definitions for general channels with channel side
information at the receiver, where the channel is modeled as a sequence of
finite dimensional conditional distributions not necessarily stationary,
ergodic, or information stable. The {\em Shannon capacity} is the highest rate
asymptotically achievable with arbitrarily small error probability. The {\em
capacity versus outage} is the highest rate asymptotically achievable with a
given probability of decoder-recognized outage. The {\em expected capacity} is
the highest average rate asymptotically achievable with a single encoder and
multiple decoders, where the channel side information determines the decoder in
use. As a special case of channel codes for expected rate, the code for
capacity versus outage has two decoders: one operates in the non-outage states
and decodes all transmitted information, and the other operates in the outage
states and decodes nothing. Expected capacity equals Shannon capacity for
channels governed by a stationary ergodic random process but is typically
greater for general channels. These alternative capacity definitions
essentially relax the constraint that all transmitted information must be
decoded at the receiver. We derive capacity theorems for these capacity
definitions through information density. Numerical examples are provided to
demonstrate their connections and differences. We also discuss the implication
of these alternative capacity definitions for end-to-end distortion,
source-channel coding and separation.Comment: Submitted to IEEE Trans. Inform. Theory, April 200
Characterization of Information Channels for Asymptotic Mean Stationarity and Stochastic Stability of Non-stationary/Unstable Linear Systems
Stabilization of non-stationary linear systems over noisy communication
channels is considered. Stochastically stable sources, and unstable but
noise-free or bounded-noise systems have been extensively studied in
information theory and control theory literature since 1970s, with a renewed
interest in the past decade. There have also been studies on non-causal and
causal coding of unstable/non-stationary linear Gaussian sources. In this
paper, tight necessary and sufficient conditions for stochastic stabilizability
of unstable (non-stationary) possibly multi-dimensional linear systems driven
by Gaussian noise over discrete channels (possibly with memory and feedback)
are presented. Stochastic stability notions include recurrence, asymptotic mean
stationarity and sample path ergodicity, and the existence of finite second
moments. Our constructive proof uses random-time state-dependent stochastic
drift criteria for stabilization of Markov chains. For asymptotic mean
stationarity (and thus sample path ergodicity), it is sufficient that the
capacity of a channel is (strictly) greater than the sum of the logarithms of
the unstable pole magnitudes for memoryless channels and a class of channels
with memory. This condition is also necessary under a mild technical condition.
Sufficient conditions for the existence of finite average second moments for
such systems driven by unbounded noise are provided.Comment: To appear in IEEE Transactions on Information Theor
Generalizing Capacity: New Definitions and Capacity Theorems for Composite Channels
We consider three capacity definitions for composite
channels with channel side information at the receiver. A composite
channel consists of a collection of different channels with a
distribution characterizing the probability that each channel is in
operation. The Shannon capacity of a channel is the highest rate
asymptotically achievable with arbitrarily small error probability.
Under this definition, the transmission strategy used to achieve
the capacity must achieve arbitrarily small error probability for
all channels in the collection comprising the composite channel.
The resulting capacity is dominated by the worst channel in its
collection, no matter how unlikely that channel is. We, therefore,
broaden the definition of capacity to allow for some outage.
The capacity versus outage is the highest rate asymptotically
achievable with a given probability of decoder-recognized outage.
The expected capacity is the highest average rate asymptotically
achievable with a single encoder and multiple decoders, where
channel side information determines the channel in use. The
expected capacity is a generalization of capacity versus outage
since codes designed for capacity versus outage decode at one of
two rates (rate zero when the channel is in outage and the target
rate otherwise) while codes designed for expected capacity can
decode at many rates. Expected capacity equals Shannon capacity
for channels governed by a stationary ergodic random process
but is typically greater for general channels. The capacity versus
outage and expected capacity definitions relax the constraint that
all transmitted information must be decoded at the receiver. We
derive channel coding theorems for these capacity definitions
through information density and provide numerical examples to
highlight their connections and differences. We also discuss the
implications of these alternative capacity definitions for end-to-end
distortion, source-channel coding, and separation
Tracking an Auto-Regressive Process with Limited Communication per Unit Time
Samples from a high-dimensional AR[1] process are observed by a sender which
can communicate only finitely many bits per unit time to a receiver. The
receiver seeks to form an estimate of the process value at every time instant
in real-time. We consider a time-slotted communication model in a slow-sampling
regime where multiple communication slots occur between two sampling instants.
We propose a successive update scheme which uses communication between sampling
instants to refine estimates of the latest sample and study the following
question: Is it better to collect communication of multiple slots to send
better refined estimates, making the receiver wait more for every refinement,
or to be fast but loose and send new information in every communication
opportunity? We show that the fast but loose successive update scheme with
ideal spherical codes is universally optimal asymptotically for a large
dimension. However, most practical quantization codes for fixed dimensions do
not meet the ideal performance required for this optimality, and they typically
will have a bias in the form of a fixed additive error. Interestingly, our
analysis shows that the fast but loose scheme is not an optimal choice in the
presence of such errors, and a judiciously chosen frequency of updates
outperforms it
Connecting the Speed-Accuracy Trade-Offs in Sensorimotor Control and Neurophysiology Reveals Diversity Sweet Spots in Layered Control Architectures
Nervous systems sense, communicate, compute, and actuate movement using distributed components with trade-offs in speed, accuracy, sparsity, noise, and saturation. Nevertheless, the resulting control can achieve remarkably fast, accurate, and robust performance due to a highly effective layered control architecture. However, this architecture has received little attention from the existing research. This is in part because of the lack of theory that connects speed-accuracy trade-offs (SATs) in the components neurophysiology with system-level sensorimotor control and characterizes the overall system performance when different layers (planning vs. reflex layer) act work jointly. In thesis, we present a theoretical framework that provides a synthetic perspective of both levels and layers. We then use this framework to clarify the properties of effective layered architectures and explain why there exists extreme diversity across layers (planning vs. reflex layers) and within levels (sensorimotor versus neural/muscle hardware levels). The framework characterizes how the sensorimotor SATs are constrained by the component SATs of neurons communicating with spikes and their sensory and muscle endpoints, in both stochastic and deterministic models. The theoretical predictions are also verified using driving experiments. Our results lead to a novel concept, termed ``diversity sweet spots (DSSs)'': the appropriate diversity in the properties of neurons and muscles across layers and within levels help create systems that are both fast and accurate despite being built from components that are individually slow or inaccurate. At the component level, this concept explains why there are extreme heterogeneities in the neural or muscle composition. At the system level, DSSs explain the benefits of layering to allow extreme heterogeneities in speed and accuracy in different sensorimotor loops. Similar issues and properties also extend down to the cellular level in biology and outward to our most advanced network technologies from smart grid to the Internet of Things. We present our initial step in expanding our framework to that area and widely-open area of research for future direction