13 research outputs found
Universal Variable-to-Fixed Length Lossy Compression at Finite Blocklengths
We consider universal variable-to-fixed length compression of memoryless
sources with a fidelity criterion. We design a dictionary codebook over the
reproduction alphabet which is used to parse the source stream. Once a source
subsequence is within a specified distortion of a dictionary codeword, the
index of the codeword is emitted as the reproduced string. Our proposed
dictionary consists of coverings of type classes in the boundary of transition
from low to high empirical lossy rate. We derive the asymptotics of the
\epsilon-coding rate (up to the third-order term) of our coding scheme for
large enough dictionaries
Second-Order Coding Rates for Conditional Rate-Distortion
This paper characterizes the second-order coding rates for lossy source
coding with side information available at both the encoder and the decoder. We
first provide non-asymptotic bounds for this problem and then specialize the
non-asymptotic bounds for three different scenarios: discrete memoryless
sources, Gaussian sources, and Markov sources. We obtain the second-order
coding rates for these settings. It is interesting to observe that the
second-order coding rate for Gaussian source coding with Gaussian side
information available at both the encoder and the decoder is the same as that
for Gaussian source coding without side information. Furthermore, regardless of
the variance of the side information, the dispersion is nats squared per
source symbol.Comment: 20 pages, 2 figures, second-order coding rates, finite blocklength,
network information theor
Nonasymptotic noisy lossy source coding
This paper shows new general nonasymptotic achievability and converse bounds
and performs their dispersion analysis for the lossy compression problem in
which the compressor observes the source through a noisy channel. While this
problem is asymptotically equivalent to a noiseless lossy source coding problem
with a modified distortion function, nonasymptotically there is a noticeable
gap in how fast their minimum achievable coding rates approach the common
rate-distortion function, as evidenced both by the refined asymptotic analysis
(dispersion) and the numerical results. The size of the gap between the
dispersions of the noisy problem and the asymptotically equivalent noiseless
problem depends on the stochastic variability of the channel through which the
compressor observes the source.Comment: IEEE Transactions on Information Theory, 201
A vector quantization approach to universal noiseless coding and quantization
A two-stage code is a block code in which each block of data is coded in two stages: the first stage codes the identity of a block code among a collection of codes, and the second stage codes the data using the identified code. The collection of codes may be noiseless codes, fixed-rate quantizers, or variable-rate quantizers. We take a vector quantization approach to two-stage coding, in which the first stage code can be regarded as a vector quantizer that “quantizes” the input data of length n to one of a fixed collection of block codes. We apply the generalized Lloyd algorithm to the first-stage quantizer, using induced measures of rate and distortion, to design locally optimal two-stage codes. On a source of medical images, two-stage variable-rate vector quantizers designed in this way outperform standard (one-stage) fixed-rate vector quantizers by over 9 dB. The tail of the operational distortion-rate function of the first-stage quantizer determines the optimal rate of convergence of the redundancy of a universal sequence of two-stage codes. We show that there exist two-stage universal noiseless codes, fixed-rate quantizers, and variable-rate quantizers whose per-letter rate and distortion redundancies converge to zero as (k/2)n -1 log n, when the universe of sources has finite dimension k. This extends the achievability part of Rissanen's theorem from universal noiseless codes to universal quantizers. Further, we show that the redundancies converge as O(n-1) when the universe of sources is countable, and as O(n-1+ϵ) when the universe of sources is infinite-dimensional, under appropriate conditions
Variable-length compression allowing errors
This paper studies the fundamental limits of the minimum average length of
lossless and lossy variable-length compression, allowing a nonzero error
probability , for lossless compression. We give non-asymptotic bounds
on the minimum average length in terms of Erokhin's rate-distortion function
and we use those bounds to obtain a Gaussian approximation on the speed of
approach to the limit which is quite accurate for all but small blocklengths:
where is the functional
inverse of the standard Gaussian complementary cdf, and is the
source dispersion. A nonzero error probability thus not only reduces the
asymptotically achievable rate by a factor of , but this
asymptotic limit is approached from below, i.e. larger source dispersions and
shorter blocklengths are beneficial. Variable-length lossy compression under an
excess distortion constraint is shown to exhibit similar properties