166 research outputs found
Results on the Redundancy of Universal Compression for Finite-Length Sequences
In this paper, we investigate the redundancy of universal coding schemes on
smooth parametric sources in the finite-length regime. We derive an upper bound
on the probability of the event that a sequence of length , chosen using
Jeffreys' prior from the family of parametric sources with unknown
parameters, is compressed with a redundancy smaller than
for any . Our results also confirm
that for large enough and , the average minimax redundancy provides a
good estimate for the redundancy of most sources. Our result may be used to
evaluate the performance of universal source coding schemes on finite-length
sequences. Additionally, we precisely characterize the minimax redundancy for
two--stage codes. We demonstrate that the two--stage assumption incurs a
negligible redundancy especially when the number of source parameters is large.
Finally, we show that the redundancy is significant in the compression of small
sequences.Comment: accepted in the 2011 IEEE International Symposium on Information
Theory (ISIT 2011
Mismatched Estimation in Large Linear Systems
We study the excess mean square error (EMSE) above the minimum mean square
error (MMSE) in large linear systems where the posterior mean estimator (PME)
is evaluated with a postulated prior that differs from the true prior of the
input signal. We focus on large linear systems where the measurements are
acquired via an independent and identically distributed random matrix, and are
corrupted by additive white Gaussian noise (AWGN). The relationship between the
EMSE in large linear systems and EMSE in scalar channels is derived, and closed
form approximations are provided. Our analysis is based on the decoupling
principle, which links scalar channels to large linear system analyses.
Numerical examples demonstrate that our closed form approximations are
accurate.Comment: 5 pages, 2 figure
- …