A vector quantization approach to universal noiseless coding and quantization

A two-stage code is a block code in which each block of data is coded in two stages: the first stage codes the identity of a block code among a collection of codes, and the second stage codes the data using the identified code. The collection of codes may be noiseless codes, fixed-rate quantizers, or variable-rate quantizers. We take a vector quantization approach to two-stage coding, in which the first stage code can be regarded as a vector quantizer that “quantizes” the input data of length n to one of a fixed collection of block codes. We apply the generalized Lloyd algorithm to the first-stage quantizer, using induced measures of rate and distortion, to design locally optimal two-stage codes. On a source of medical images, two-stage variable-rate vector quantizers designed in this way outperform standard (one-stage) fixed-rate vector quantizers by over 9 dB. The tail of the operational distortion-rate function of the first-stage quantizer determines the optimal rate of convergence of the redundancy of a universal sequence of two-stage codes. We show that there exist two-stage universal noiseless codes, fixed-rate quantizers, and variable-rate quantizers whose per-letter rate and distortion redundancies converge to zero as (k/2)n -1 log n, when the universe of sources has finite dimension k. This extends the achievability part of Rissanen's theorem from universal noiseless codes to universal quantizers. Further, we show that the redundancies converge as O(n-1) when the universe of sources is countable, and as O(n-1+ϵ) when the universe of sources is infinite-dimensional, under appropriate conditions

Distributed Functional Scalar Quantization Simplified

Distributed functional scalar quantization (DFSQ) theory provides optimality conditions and predicts performance of data acquisition systems in which a computation on acquired data is desired. We address two limitations of previous works: prohibitively expensive decoder design and a restriction to sources with bounded distributions. We rigorously show that a much simpler decoder has equivalent asymptotic performance as the conditional expectation estimator previously explored, thus reducing decoder design complexity. The simpler decoder has the feature of decoupled communication and computation blocks. Moreover, we extend the DFSQ framework with the simpler decoder to acquire sources with infinite-support distributions such as Gaussian or exponential distributions. Finally, through simulation results we demonstrate that performance at moderate coding rates is well predicted by the asymptotic analysis, and we give new insight on the rate of convergence

Very Low-Rate Variable-Length Channel Quantization for Minimum Outage Probability

We identify a practical vector quantizer design problem where any fixed-length quantizer (FLQ) yields non-zero distortion at any finite rate, while there is a variable-length quantizer (VLQ) that can achieve zero distortion with arbitrarily low rate. The problem arises in a $t \times 1$ multiple-antenna fading channel where we would like to minimize the channel outage probability by employing beamforming via quantized channel state information at the transmitter (CSIT). It is well-known that in such a scenario, finite-rate FLQs cannot achieve the full-CSIT (zero distortion) outage performance. We construct VLQs that can achieve the full-CSIT performance with finite rate. In particular, with $P$ denoting the power constraint of the transmitter, we show that the necessary and sufficient VLQ rate that guarantees the full-CSIT performance is $\Theta(1/P)$. We also discuss several extensions (e.g. to precoding) of this result