100,900 research outputs found
Distortion-Rate Function of Sub-Nyquist Sampled Gaussian Sources
The amount of information lost in sub-Nyquist sampling of a continuous-time
Gaussian stationary process is quantified. We consider a combined source coding
and sub-Nyquist reconstruction problem in which the input to the encoder is a
noisy sub-Nyquist sampled version of the analog source. We first derive an
expression for the mean squared error in the reconstruction of the process from
a noisy and information rate-limited version of its samples. This expression is
a function of the sampling frequency and the average number of bits describing
each sample. It is given as the sum of two terms: Minimum mean square error in
estimating the source from its noisy but otherwise fully observed sub-Nyquist
samples, and a second term obtained by reverse waterfilling over an average of
spectral densities associated with the polyphase components of the source. We
extend this result to multi-branch uniform sampling, where the samples are
available through a set of parallel channels with a uniform sampler and a
pre-sampling filter in each branch. Further optimization to reduce distortion
is then performed over the pre-sampling filters, and an optimal set of
pre-sampling filters associated with the statistics of the input signal and the
sampling frequency is found. This results in an expression for the minimal
possible distortion achievable under any analog to digital conversion scheme
involving uniform sampling and linear filtering. These results thus unify the
Shannon-Whittaker-Kotelnikov sampling theorem and Shannon rate-distortion
theory for Gaussian sources.Comment: Accepted for publication at the IEEE transactions on information
theor
Implementing rate-distortion optimization on a resource-limited H.264 encoder
Thesis (M. Eng.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2005.Includes bibliographical references (leaves 57-59).This thesis models the rate-distortion characteristics of an H.264 video compression encoder to improve its mode decision performance. First, it provides a background to the fundamentals of video compression. Then it describes the problem of estimating rate and distortion of a macroblock given limited computational resources. It derives the macroblock rate and distortion as a function of the residual SAD and H.264 quantization parameter QP. From the resulting equations, this thesis implements and verifies rate-distortion optimization on a resource-limited H.264 encoder. Finally, it explores other avenues of improvement.by Eric Syu.M.Eng
Bounds on inference
Lower bounds for the average probability of error of estimating a hidden
variable X given an observation of a correlated random variable Y, and Fano's
inequality in particular, play a central role in information theory. In this
paper, we present a lower bound for the average estimation error based on the
marginal distribution of X and the principal inertias of the joint distribution
matrix of X and Y. Furthermore, we discuss an information measure based on the
sum of the largest principal inertias, called k-correlation, which generalizes
maximal correlation. We show that k-correlation satisfies the Data Processing
Inequality and is convex in the conditional distribution of Y given X. Finally,
we investigate how to answer a fundamental question in inference and privacy:
given an observation Y, can we estimate a function f(X) of the hidden random
variable X with an average error below a certain threshold? We provide a
general method for answering this question using an approach based on
rate-distortion theory.Comment: Allerton 2013 with extended proof, 10 page
Estimation of the Rate-Distortion Function
Motivated by questions in lossy data compression and by theoretical
considerations, we examine the problem of estimating the rate-distortion
function of an unknown (not necessarily discrete-valued) source from empirical
data. Our focus is the behavior of the so-called "plug-in" estimator, which is
simply the rate-distortion function of the empirical distribution of the
observed data. Sufficient conditions are given for its consistency, and
examples are provided to demonstrate that in certain cases it fails to converge
to the true rate-distortion function. The analysis of its performance is
complicated by the fact that the rate-distortion function is not continuous in
the source distribution; the underlying mathematical problem is closely related
to the classical problem of establishing the consistency of maximum likelihood
estimators. General consistency results are given for the plug-in estimator
applied to a broad class of sources, including all stationary and ergodic ones.
A more general class of estimation problems is also considered, arising in the
context of lossy data compression when the allowed class of coding
distributions is restricted; analogous results are developed for the plug-in
estimator in that case. Finally, consistency theorems are formulated for
modified (e.g., penalized) versions of the plug-in, and for estimating the
optimal reproduction distribution.Comment: 18 pages, no figures [v2: removed an example with an error; corrected
typos; a shortened version will appear in IEEE Trans. Inform. Theory
- …