25 research outputs found
Optimal Estimation via Nonanticipative Rate Distortion Function and Applications to Time-Varying Gauss-Markov Processes
In this paper, we develop {finite-time horizon} causal filters using the
nonanticipative rate distortion theory. We apply the {developed} theory to
{design optimal filters for} time-varying multidimensional Gauss-Markov
processes, subject to a mean square error fidelity constraint. We show that
such filters are equivalent to the design of an optimal \texttt{\{encoder,
channel, decoder\}}, which ensures that the error satisfies {a} fidelity
constraint. Moreover, we derive a universal lower bound on the mean square
error of any estimator of time-varying multidimensional Gauss-Markov processes
in terms of conditional mutual information. Unlike classical Kalman filters,
the filter developed is characterized by a reverse-waterfilling algorithm,
which ensures {that} the fidelity constraint is satisfied. The theoretical
results are demonstrated via illustrative examples.Comment: 35 pages, 6 figures, submitted for publication in SIAM Journal on
Control and Optimization (SICON
Zero-Delay Rate Distortion via Filtering for Vector-Valued Gaussian Sources
We deal with zero-delay source coding of a vector-valued Gauss-Markov source
subject to a mean-squared error (MSE) fidelity criterion characterized by the
operational zero-delay vector-valued Gaussian rate distortion function (RDF).
We address this problem by considering the nonanticipative RDF (NRDF) which is
a lower bound to the causal optimal performance theoretically attainable (OPTA)
function and operational zero-delay RDF. We recall the realization that
corresponds to the optimal "test-channel" of the Gaussian NRDF, when
considering a vector Gauss-Markov source subject to a MSE distortion in the
finite time horizon. Then, we introduce sufficient conditions to show existence
of solution for this problem in the infinite time horizon. For the asymptotic
regime, we use the asymptotic characterization of the Gaussian NRDF to provide
a new equivalent realization scheme with feedback which is characterized by a
resource allocation (reverse-waterfilling) problem across the dimension of the
vector source. We leverage the new realization to derive a predictive coding
scheme via lattice quantization with subtractive dither and joint memoryless
entropy coding. This coding scheme offers an upper bound to the operational
zero-delay vector-valued Gaussian RDF. When we use scalar quantization, then
for "r" active dimensions of the vector Gauss-Markov source the gap between the
obtained lower and theoretical upper bounds is less than or equal to 0.254r + 1
bits/vector. We further show that it is possible when we use vector
quantization, and assume infinite dimensional Gauss-Markov sources to make the
previous gap to be negligible, i.e., Gaussian NRDF approximates the operational
zero-delay Gaussian RDF. We also extend our results to vector-valued Gaussian
sources of any finite memory under mild conditions. Our theoretical framework
is demonstrated with illustrative numerical experiments.Comment: 32 pages, 9 figures, published in IEEE Journal of Selected Topics in
Signal Processin
An Upper Bound to Zero-Delay Rate Distortion via Kalman Filtering for Vector Gaussian Sources
We deal with zero-delay source coding of a vector Gaussian autoregressive
(AR) source subject to an average mean squared error (MSE) fidelity criterion.
Toward this end, we consider the nonanticipative rate distortion function
(NRDF) which is a lower bound to the causal and zero-delay rate distortion
function (RDF). We use the realization scheme with feedback proposed in [1] to
model the corresponding optimal "test-channel" of the NRDF, when considering
vector Gaussian AR(1) sources subject to an average MSE distortion. We give
conditions on the vector Gaussian AR(1) source to ensure asymptotic
stationarity of the realization scheme (bounded performance). Then, we encode
the vector innovations due to Kalman filtering via lattice quantization with
subtractive dither and memoryless entropy coding. This coding scheme provides a
tight upper bound to the zero-delay Gaussian RDF. We extend this result to
vector Gaussian AR sources of any finite order. Further, we show that for
infinite dimensional vector Gaussian AR sources of any finite order, the NRDF
coincides with the zero-delay RDF. Our theoretical framework is corroborated
with a simulation example.Comment: 7 pages, 6 figures, accepted for publication in IEEE Information
Theory Workshop (ITW
Bounds on the Sum-Rate of MIMO Causal Source Coding Systems with Memory under Spatio-Temporal Distortion Constraints
In this paper, we derive lower and upper bounds on the OPTA of a two-user multi-input multi-output (MIMO) causal encoding and causal decoding problem. Each user’s source model is described by a multidimensional Markov source driven by additive i.i.d. noise process subject to three classes of spatio-temporal distortion constraints. To characterize the lower bounds, we use state augmentation techniques and a data processing theorem, which recovers a variant of rate distortion function as an information measure known in the literature as nonanticipatory ϵ-entropy, sequential or nonanticipative RDF. We derive lower bound characterizations for a system driven by an i.i.d. Gaussian noise process, which we solve using the SDP algorithm for all three classes of distortion constraints. We obtain closed form solutions when the system’s noise is possibly non-Gaussian for both users and when only one of the users is described by a source model driven by a Gaussian noise process. To obtain the upper bounds, we use the best linear forward test channel realization that corresponds to the optimal test channel realization when the system is driven by a Gaussian noise process and apply a sequential causal DPCM-based scheme with a feedback loop followed by a scaled ECDQ scheme that leads to upper bounds with certain performance guarantees. Then, we use the linear forward test channel as a benchmark to obtain upper bounds on the OPTA, when the system is driven by an additive i.i.d. non-Gaussian noise process. We support our framework with various simulation studies
Tracking an Auto-Regressive Process with Limited Communication per Unit Time
Samples from a high-dimensional AR[1] process are observed by a sender which
can communicate only finitely many bits per unit time to a receiver. The
receiver seeks to form an estimate of the process value at every time instant
in real-time. We consider a time-slotted communication model in a slow-sampling
regime where multiple communication slots occur between two sampling instants.
We propose a successive update scheme which uses communication between sampling
instants to refine estimates of the latest sample and study the following
question: Is it better to collect communication of multiple slots to send
better refined estimates, making the receiver wait more for every refinement,
or to be fast but loose and send new information in every communication
opportunity? We show that the fast but loose successive update scheme with
ideal spherical codes is universally optimal asymptotically for a large
dimension. However, most practical quantization codes for fixed dimensions do
not meet the ideal performance required for this optimality, and they typically
will have a bias in the form of a fixed additive error. Interestingly, our
analysis shows that the fast but loose scheme is not an optimal choice in the
presence of such errors, and a judiciously chosen frequency of updates
outperforms it