Critical Behavior in Lossy Source Coding

The following critical phenomenon was recently discovered. When a memoryless source is compressed using a variable-length fixed-distortion code, the fastest convergence rate of the (pointwise) compression ratio to the optimal $R(D)$ bits/symbol is either $O(\sqrt{n})$ or $O(\log n)$. We show it is always $O(\sqrt{n})$, except for discrete, uniformly distributed sources.Comment: 2 figure

Fixed-length lossy compression in the finite blocklength regime

This paper studies the minimum achievable source coding rate as a function of blocklength $n$ and probability $\epsilon$ that the distortion exceeds a given level $d$. Tight general achievability and converse bounds are derived that hold at arbitrary fixed blocklength. For stationary memoryless sources with separable distortion, the minimum rate achievable is shown to be closely approximated by $R(d) + \sqrt{\frac{V(d)}{n}} Q^{-1}(\epsilon)$, where $R(d)$ is the rate-distortion function, $V(d)$ is the rate dispersion, a characteristic of the source which measures its stochastic variability, and $Q^{-1}(\epsilon)$ is the inverse of the standard Gaussian complementary cdf

The Reliability Function of Lossy Source-Channel Coding of Variable-Length Codes with Feedback

We consider transmission of discrete memoryless sources (DMSes) across discrete memoryless channels (DMCs) using variable-length lossy source-channel codes with feedback. The reliability function (optimum error exponent) is shown to be equal to $\max\{0, B(1-R(D)/C)\},$ where $R(D)$ is the rate-distortion function of the source, $B$ is the maximum relative entropy between output distributions of the DMC, and $C$ is the Shannon capacity of the channel. We show that, in this setting and in this asymptotic regime, separate source-channel coding is, in fact, optimal.Comment: Accepted to IEEE Transactions on Information Theory in Apr. 201