A Rigorous Approach to High-Resolution Entropy-Constrained Vector Quantization

The nonnegativity of relative entropy implies that the differential entropy of a random vector X with probability density function (pdf) f is upper-bounded by -E[log g(X)]for any arbitrary pdf g. Using this inequality with a cleverly chosen g, we derive a lower bound on the asymptotic excess rate of entropy-constrained vector quantization for d-dimensional sources and rth-power distortion, where the asymptotic excess rate is defined as the difference between the smallest output entropy of a vector quantizer satisfying the distortion constraint and the rate-distortion function in the limit as the distortion tends to zero. Specialized to the one-dimensional case, this lower bound coincides with the asymptotic excess rate achieved by a uniform quantizer, thereby recovering the result by Gish and Pierce that uniform quantizers are asymptotically optimal as the allowed distortion tends to zero. Furthermore, in the one-dimensional case, the derivation of the lower bound reveals a necessary condition for a sequence of quantizers to be asymptotically optimal. This condition implies that any sequence of asymptotically optimal almost-regular quantizers must converge to a uniform quantizer as the distortion tends to zero. While the obtained lower bound itself is not novel, to the best of our knowledge, we present the first rigorous derivation that follows the direct approach by Gish and Pierce without resorting to heuristic high-resolution approximations commonly found in the quantization literature. Furthermore, our derivation holds for all d-dimensional sources having finite differential entropy and whose integer part has finite entropy. In contrast to Gish and Pierce, we do not require additional constraints on the continuity or decay of the source pdf.This work has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement number 714161), from the 7th European Union Framework Programme under Grant 333680, from the Ministerio de Economía y Competitividad of Spain under Grants TEC2013-41718-R, RYC-2014-16332, IJCI-2015-27020, TEC2015-69648-REDC, and TEC2016-78434-C3-3-R (AEI/FEDER, EU), and from the Comunidad de Madrid under Grant S2103/ICE-2845. The material in this paper was presented in part at the 2016 IEEE International Symposium on Information Theory, Barcelona, Spain, July 2016

A high-speed codebook design algorithm for ECVQ using angular constraint with search space partitioning

金沢大学大学院自然科学研究科情報システム金沢大学工学部In this paper, we propose a fast codebook generation algorithm for entropy-constrained vector quantization (ECVQ). The algorithm uses the angular constraint and employs a suitable hyperplane to partition the codebook and image data in order to reduce the search area and accelerate the search process in the codebook design. This algorithm allows significant acceleration in codebook design process. Experimental results are presented on image block data. These results show that our new algorithm performs better than the previously known methods

Conditional weighted universal source codes: second order statistics in universal coding

We consider the use of second order statistics in two-stage universal source coding. Examples of two-stage universal codes include the weighted universal vector quantization (WUVQ), weighted universal bit allocation (WUBA), and weighted universal transform coding (WUTC) algorithms. The second order statistics are incorporated in two-stage universal source codes in a manner analogous to the method by which second order statistics are incorporated in entropy constrained vector quantization (ECVQ) to yield conditional ECVQ (CECVQ). In this paper, we describe an optimal two-stage conditional entropy constrained universal source code along with its associated optimal design algorithm and a fast (but nonoptimal) variation of the original code. The design technique and coding algorithm here presented result in a new family of conditional entropy constrained universal codes including but not limited to the conditional entropy constrained WUVQ (CWUVQ), the conditional entropy constrained WUBA (CWUBA), and the conditional entropy constrained WUTC (CWUTC). The fast variation of the conditional entropy constrained universal codes allows the designer to trade off performance gains against storage and delay costs. We demonstrate the performance of the proposed codes on a collection of medical brain scans. On the given data set, the CWUVQ achieves up to 7.5 dB performance improvement over variable-rate WUVQ and up to 12 dB performance improvement over ECVQ. On the same data set, the fast variation of the CWUVQ achieves identical performance to that achieved by the original code at all but the lowest rates (less than 0.125 bits per pixel)

Multiresolution vector quantization

Multiresolution source codes are data compression algorithms yielding embedded source descriptions. The decoder of a multiresolution code can build a source reproduction by decoding the embedded bit stream in part or in whole. All decoding procedures start at the beginning of the binary source description and decode some fraction of that string. Decoding a small portion of the binary string gives a low-resolution reproduction; decoding more yields a higher resolution reproduction; and so on. Multiresolution vector quantizers are block multiresolution source codes. This paper introduces algorithms for designing fixed- and variable-rate multiresolution vector quantizers. Experiments on synthetic data demonstrate performance close to the theoretical performance limit. Experiments on natural images demonstrate performance improvements of up to 8 dB over tree-structured vector quantizers. Some of the lessons learned through multiresolution vector quantizer design lend insight into the design of more sophisticated multiresolution codes