3,579 research outputs found
Efficient Universal Noiseless Source Codes
Although the existence of universal noiseless variable-rate codes for the class of discrete stationary ergodic sources has previously been established, very few practical universal encoding methods are available. Efficient implementable universal source coding techniques are discussed in this paper. Results are presented on source codes for which a small value of the maximum redundancy is achieved with a relatively short block length. A constructive proof of the existence of universal noiseless codes for discrete stationary sources is first presented. The proof is shown to provide a method for obtaining efficient universal noiseless variable-rate codes for various classes of sources. For memoryless sources, upper and lower bounds are obtained for the minimax redundancy as a function of the block length of the code. Several techniques for constructing universal noiseless source codes for memoryless sources are presented and their redundancies are compared with the bounds. Consideration is given to possible applications to data compression for certain nonstationary sources
Results on the Redundancy of Universal Compression for Finite-Length Sequences
In this paper, we investigate the redundancy of universal coding schemes on
smooth parametric sources in the finite-length regime. We derive an upper bound
on the probability of the event that a sequence of length , chosen using
Jeffreys' prior from the family of parametric sources with unknown
parameters, is compressed with a redundancy smaller than
for any . Our results also confirm
that for large enough and , the average minimax redundancy provides a
good estimate for the redundancy of most sources. Our result may be used to
evaluate the performance of universal source coding schemes on finite-length
sequences. Additionally, we precisely characterize the minimax redundancy for
two--stage codes. We demonstrate that the two--stage assumption incurs a
negligible redundancy especially when the number of source parameters is large.
Finally, we show that the redundancy is significant in the compression of small
sequences.Comment: accepted in the 2011 IEEE International Symposium on Information
Theory (ISIT 2011
- …