This paper studies the fundamental limits of the minimum average length of
lossless and lossy variable-length compression, allowing a nonzero error
probability ϵ, for lossless compression. We give non-asymptotic bounds
on the minimum average length in terms of Erokhin's rate-distortion function
and we use those bounds to obtain a Gaussian approximation on the speed of
approach to the limit which is quite accurate for all but small blocklengths:
(1−ϵ)kH(S)−2πkV(S)e−2(Q−1(ϵ))2 where Q−1(⋅) is the functional
inverse of the standard Gaussian complementary cdf, and V(S) is the
source dispersion. A nonzero error probability thus not only reduces the
asymptotically achievable rate by a factor of 1−ϵ, but this
asymptotic limit is approached from below, i.e. larger source dispersions and
shorter blocklengths are beneficial. Variable-length lossy compression under an
excess distortion constraint is shown to exhibit similar properties