125,735 research outputs found
On Zero-Error Source Coding with Feedback
We consider the problem of zero error source coding with limited feedback
when side information is present at the receiver. First, we derive an
achievable rate region for arbitrary joint distributions on the source and the
side information. When all source pairs of source and side information symbols
are observable with non-zero probability, we show that this characterization
gives the entire rate region. Next, we demonstrate a class of sources for which
asymptotically zero feedback suffices to achieve zero-error coding at the rate
promised by the Slepian-Wolf bound for asymptotically lossless coding. Finally,
we illustrate these results with the aid of three simple examples
An Upper Bound to Zero-Delay Rate Distortion via Kalman Filtering for Vector Gaussian Sources
We deal with zero-delay source coding of a vector Gaussian autoregressive
(AR) source subject to an average mean squared error (MSE) fidelity criterion.
Toward this end, we consider the nonanticipative rate distortion function
(NRDF) which is a lower bound to the causal and zero-delay rate distortion
function (RDF). We use the realization scheme with feedback proposed in [1] to
model the corresponding optimal "test-channel" of the NRDF, when considering
vector Gaussian AR(1) sources subject to an average MSE distortion. We give
conditions on the vector Gaussian AR(1) source to ensure asymptotic
stationarity of the realization scheme (bounded performance). Then, we encode
the vector innovations due to Kalman filtering via lattice quantization with
subtractive dither and memoryless entropy coding. This coding scheme provides a
tight upper bound to the zero-delay Gaussian RDF. We extend this result to
vector Gaussian AR sources of any finite order. Further, we show that for
infinite dimensional vector Gaussian AR sources of any finite order, the NRDF
coincides with the zero-delay RDF. Our theoretical framework is corroborated
with a simulation example.Comment: 7 pages, 6 figures, accepted for publication in IEEE Information
Theory Workshop (ITW
Coded Kalman Filtering Over Gaussian Channels with Feedback
This paper investigates the problem of zero-delay joint source-channel coding
of a vector Gauss-Markov source over a multiple-input multiple-output (MIMO)
additive white Gaussian noise (AWGN) channel with feedback. In contrast to the
classical problem of causal estimation using noisy observations, we examine a
system where the source can be encoded before transmission. An encoder,
equipped with feedback of past channel outputs, observes the source state and
encodes the information in a causal manner as inputs to the channel while
adhering to a power constraint. The objective of the code is to estimate the
source state with minimum mean square error at the infinite horizon. This work
shows a fundamental theorem for two scenarios: for the transmission of an
unstable vector Gauss-Markov source over either a multiple-input single-output
(MISO) or a single-input multiple-output (SIMO) AWGN channel, finite estimation
error is achievable if and only if the sum of logs of the unstable eigenvalues
of the state gain matrix is less than the Shannon channel capacity. We prove
these results by showing an optimal linear innovations encoder that can be
applied to sources and channels of any dimension and analyzing it together with
the corresponding Kalman filter decoder.Comment: Presented at 59th Allerton Conference on Communication, Control, and
Computin
Joint source-channel coding with feedback
This paper quantifies the fundamental limits of variable-length transmission
of a general (possibly analog) source over a memoryless channel with noiseless
feedback, under a distortion constraint. We consider excess distortion, average
distortion and guaranteed distortion (-semifaithful codes). In contrast to
the asymptotic fundamental limit, a general conclusion is that allowing
variable-length codes and feedback leads to a sizable improvement in the
fundamental delay-distortion tradeoff. In addition, we investigate the minimum
energy required to reproduce source samples with a given fidelity after
transmission over a memoryless Gaussian channel, and we show that the required
minimum energy is reduced with feedback and an average (rather than maximal)
power constraint.Comment: To appear in IEEE Transactions on Information Theor
Zero-Delay Rate Distortion via Filtering for Vector-Valued Gaussian Sources
We deal with zero-delay source coding of a vector-valued Gauss-Markov source
subject to a mean-squared error (MSE) fidelity criterion characterized by the
operational zero-delay vector-valued Gaussian rate distortion function (RDF).
We address this problem by considering the nonanticipative RDF (NRDF) which is
a lower bound to the causal optimal performance theoretically attainable (OPTA)
function and operational zero-delay RDF. We recall the realization that
corresponds to the optimal "test-channel" of the Gaussian NRDF, when
considering a vector Gauss-Markov source subject to a MSE distortion in the
finite time horizon. Then, we introduce sufficient conditions to show existence
of solution for this problem in the infinite time horizon. For the asymptotic
regime, we use the asymptotic characterization of the Gaussian NRDF to provide
a new equivalent realization scheme with feedback which is characterized by a
resource allocation (reverse-waterfilling) problem across the dimension of the
vector source. We leverage the new realization to derive a predictive coding
scheme via lattice quantization with subtractive dither and joint memoryless
entropy coding. This coding scheme offers an upper bound to the operational
zero-delay vector-valued Gaussian RDF. When we use scalar quantization, then
for "r" active dimensions of the vector Gauss-Markov source the gap between the
obtained lower and theoretical upper bounds is less than or equal to 0.254r + 1
bits/vector. We further show that it is possible when we use vector
quantization, and assume infinite dimensional Gauss-Markov sources to make the
previous gap to be negligible, i.e., Gaussian NRDF approximates the operational
zero-delay Gaussian RDF. We also extend our results to vector-valued Gaussian
sources of any finite memory under mild conditions. Our theoretical framework
is demonstrated with illustrative numerical experiments.Comment: 32 pages, 9 figures, published in IEEE Journal of Selected Topics in
Signal Processin
Distortion Exponent in MIMO Channels with Feedback
The transmission of a Gaussian source over a block-fading multiple antenna
channel in the presence of a feedback link is considered. The feedback link is
assumed to be an error and delay free link of capacity 1 bit per channel use.
Under the short-term power constraint, the optimal exponential behavior of the
end-to-end average distortion is characterized for all source-channel bandwidth
ratios. It is shown that the optimal transmission strategy is successive
refinement source coding followed by progressive transmission over the channel,
in which the channel block is allocated dynamically among the layers based on
the channel state using the feedback link as an instantaneous automatic repeat
request (ARQ) signal.Comment: Presented at the IEEE Information Theory Workshop (ITW), Taormina,
Italy, Oct. 200
Characterization of Information Channels for Asymptotic Mean Stationarity and Stochastic Stability of Non-stationary/Unstable Linear Systems
Stabilization of non-stationary linear systems over noisy communication
channels is considered. Stochastically stable sources, and unstable but
noise-free or bounded-noise systems have been extensively studied in
information theory and control theory literature since 1970s, with a renewed
interest in the past decade. There have also been studies on non-causal and
causal coding of unstable/non-stationary linear Gaussian sources. In this
paper, tight necessary and sufficient conditions for stochastic stabilizability
of unstable (non-stationary) possibly multi-dimensional linear systems driven
by Gaussian noise over discrete channels (possibly with memory and feedback)
are presented. Stochastic stability notions include recurrence, asymptotic mean
stationarity and sample path ergodicity, and the existence of finite second
moments. Our constructive proof uses random-time state-dependent stochastic
drift criteria for stabilization of Markov chains. For asymptotic mean
stationarity (and thus sample path ergodicity), it is sufficient that the
capacity of a channel is (strictly) greater than the sum of the logarithms of
the unstable pole magnitudes for memoryless channels and a class of channels
with memory. This condition is also necessary under a mild technical condition.
Sufficient conditions for the existence of finite average second moments for
such systems driven by unbounded noise are provided.Comment: To appear in IEEE Transactions on Information Theor
- …