37 research outputs found
Information Nonanticipative Rate Distortion Function and Its Applications
This paper investigates applications of nonanticipative Rate Distortion
Function (RDF) in a) zero-delay Joint Source-Channel Coding (JSCC) design based
on average and excess distortion probability, b) in bounding the Optimal
Performance Theoretically Attainable (OPTA) by noncausal and causal codes, and
computing the Rate Loss (RL) of zero-delay and causal codes with respect to
noncausal codes. These applications are described using two running examples,
the Binary Symmetric Markov Source with parameter p, (BSMS(p)) and the
multidimensional partially observed Gaussian-Markov source. For the
multidimensional Gaussian-Markov source with square error distortion, the
solution of the nonanticipative RDF is derived, its operational meaning using
JSCC design via a noisy coding theorem is shown by providing the optimal
encoding-decoding scheme over a vector Gaussian channel, and the RL of causal
and zero-delay codes with respect to noncausal codes is computed.
For the BSMS(p) with Hamming distortion, the solution of the nonanticipative
RDF is derived, the RL of causal codes with respect to noncausal codes is
computed, and an uncoded noisy coding theorem based on excess distortion
probability is shown. The information nonanticipative RDF is shown to be
equivalent to the nonanticipatory epsilon-entropy, which corresponds to the
classical RDF with an additional causality or nonanticipative condition imposed
on the optimal reproduction conditional distribution.Comment: 34 pages, 12 figures, part of this paper was accepted for publication
in IEEE International Symposium on Information Theory (ISIT), 2014 and in
book Coordination Control of Distributed Systems of series Lecture Notes in
Control and Information Sciences, 201
Applications of Information Nonanticipative Rate Distortion Function
The objective of this paper is to further investigate various applications of
information Nonanticipative Rate Distortion Function (NRDF) by discussing two
working examples, the Binary Symmetric Markov Source with parameter
(BSMS()) with Hamming distance distortion, and the multidimensional
partially observed Gaussian-Markov source. For the BSMS(), we give the
solution to the NRDF, and we use it to compute the Rate Loss (RL) of causal
codes with respect to noncausal codes. For the multidimensional Gaussian-Markov
source, we give the solution to the NRDF, we show its operational meaning via
joint source-channel matching over a vector of parallel Gaussian channels, and
we compute the RL of causal and zero-delay codes with respect to noncausal
codes.Comment: 5 pages, 3 figures, accepted for publication in IEEE International
Symposium on Information Theory (ISIT) proceedings, 201
Optimal Estimation via Nonanticipative Rate Distortion Function and Applications to Time-Varying Gauss-Markov Processes
In this paper, we develop {finite-time horizon} causal filters using the
nonanticipative rate distortion theory. We apply the {developed} theory to
{design optimal filters for} time-varying multidimensional Gauss-Markov
processes, subject to a mean square error fidelity constraint. We show that
such filters are equivalent to the design of an optimal \texttt{\{encoder,
channel, decoder\}}, which ensures that the error satisfies {a} fidelity
constraint. Moreover, we derive a universal lower bound on the mean square
error of any estimator of time-varying multidimensional Gauss-Markov processes
in terms of conditional mutual information. Unlike classical Kalman filters,
the filter developed is characterized by a reverse-waterfilling algorithm,
which ensures {that} the fidelity constraint is satisfied. The theoretical
results are demonstrated via illustrative examples.Comment: 35 pages, 6 figures, submitted for publication in SIAM Journal on
Control and Optimization (SICON
Capacity of Binary State Symmetric Channel with and without Feedback and Transmission Cost
We consider a unit memory channel, called Binary State Symmetric Channel
(BSSC), in which the channel state is the modulo2 addition of the current
channel input and the previous channel output. We derive closed form
expressions for the capacity and corresponding channel input distribution, of
this BSSC with and without feedback and transmission cost. We also show that
the capacity of the BSSC is not increased by feedback, and it is achieved by a
first order symmetric Markov process
Zero-Delay Rate Distortion via Filtering for Vector-Valued Gaussian Sources
We deal with zero-delay source coding of a vector-valued Gauss-Markov source
subject to a mean-squared error (MSE) fidelity criterion characterized by the
operational zero-delay vector-valued Gaussian rate distortion function (RDF).
We address this problem by considering the nonanticipative RDF (NRDF) which is
a lower bound to the causal optimal performance theoretically attainable (OPTA)
function and operational zero-delay RDF. We recall the realization that
corresponds to the optimal "test-channel" of the Gaussian NRDF, when
considering a vector Gauss-Markov source subject to a MSE distortion in the
finite time horizon. Then, we introduce sufficient conditions to show existence
of solution for this problem in the infinite time horizon. For the asymptotic
regime, we use the asymptotic characterization of the Gaussian NRDF to provide
a new equivalent realization scheme with feedback which is characterized by a
resource allocation (reverse-waterfilling) problem across the dimension of the
vector source. We leverage the new realization to derive a predictive coding
scheme via lattice quantization with subtractive dither and joint memoryless
entropy coding. This coding scheme offers an upper bound to the operational
zero-delay vector-valued Gaussian RDF. When we use scalar quantization, then
for "r" active dimensions of the vector Gauss-Markov source the gap between the
obtained lower and theoretical upper bounds is less than or equal to 0.254r + 1
bits/vector. We further show that it is possible when we use vector
quantization, and assume infinite dimensional Gauss-Markov sources to make the
previous gap to be negligible, i.e., Gaussian NRDF approximates the operational
zero-delay Gaussian RDF. We also extend our results to vector-valued Gaussian
sources of any finite memory under mild conditions. Our theoretical framework
is demonstrated with illustrative numerical experiments.Comment: 32 pages, 9 figures, published in IEEE Journal of Selected Topics in
Signal Processin