A vector quantization approach to universal noiseless coding and quantization

A two-stage code is a block code in which each block of data is coded in two stages: the first stage codes the identity of a block code among a collection of codes, and the second stage codes the data using the identified code. The collection of codes may be noiseless codes, fixed-rate quantizers, or variable-rate quantizers. We take a vector quantization approach to two-stage coding, in which the first stage code can be regarded as a vector quantizer that “quantizes” the input data of length n to one of a fixed collection of block codes. We apply the generalized Lloyd algorithm to the first-stage quantizer, using induced measures of rate and distortion, to design locally optimal two-stage codes. On a source of medical images, two-stage variable-rate vector quantizers designed in this way outperform standard (one-stage) fixed-rate vector quantizers by over 9 dB. The tail of the operational distortion-rate function of the first-stage quantizer determines the optimal rate of convergence of the redundancy of a universal sequence of two-stage codes. We show that there exist two-stage universal noiseless codes, fixed-rate quantizers, and variable-rate quantizers whose per-letter rate and distortion redundancies converge to zero as (k/2)n -1 log n, when the universe of sources has finite dimension k. This extends the achievability part of Rissanen's theorem from universal noiseless codes to universal quantizers. Further, we show that the redundancies converge as O(n-1) when the universe of sources is countable, and as O(n-1+ϵ) when the universe of sources is infinite-dimensional, under appropriate conditions

Variable dimension weighted universal vector quantization and noiseless coding

A new algorithm for variable dimension weighted universal coding is introduced. Combining the multi-codebook system of weighted universal vector quantization (WUVQ), the partitioning technique of variable dimension vector quantization, and the optimal design strategy common to both, variable dimension WUVQ allows mixture sources to be effectively carved into their component subsources, each of which can then be encoded with the codebook best matched to that source. Application of variable dimension WUVQ to a sequence of medical images provides up to 4.8 dB improvement in signal to quantization noise ratio over WUVQ and up to 11 dB improvement over a standard full-search vector quantizer followed by an entropy code. The optimal partitioning technique can likewise be applied with a collection of noiseless codes, as found in weighted universal noiseless coding (WUNC). The resulting algorithm for variable dimension WUNC is also described

Quadratic optimal functional quantization of stochastic processes and numerical applications

In this paper, we present an overview of the recent developments of functional quantization of stochastic processes, with an emphasis on the quadratic case. Functional quantization is a way to approximate a process, viewed as a Hilbert-valued random variable, using a nearest neighbour projection on a finite codebook. A special emphasis is made on the computational aspects and the numerical applications, in particular the pricing of some path-dependent European options.Comment: 41 page

Vector quantization

During the past ten years Vector Quantization (VQ) has developed from a theoretical possibility promised by Shannon's source coding theorems into a powerful and competitive technique for speech and image coding and compression at medium to low bit rates. In this survey, the basic ideas behind the design of vector quantizers are sketched and some comments made on the state-of-the-art and current research efforts

Greedy vector quantization

We investigate the greedy version of the $L^p$-optimal vector quantization problem for an $\mathbb{R}^d$-valued random vector $X\!\in L^p$. We show the existence of a sequence $(a_N)_{N\ge 1}$ such that $a_N$ minimizes $a\mapsto\big \|\min_{1\le i\le N-1}|X-a_i|\wedge |X-a|\big\|_{L^p}$ ($L^p$-mean quantization error at level $N$ induced by $(a_1,\ldots,a_{N-1},a)$). We show that this sequence produces $L^p$-rate optimal $N$-tuples $a^{(N)}=(a_1,\ldots,a_{_N})$ ($i.e.$ the $L^p$-mean quantization error at level $N$ induced by $a^{(N)}$ goes to $0$ at rate $N^{-\frac 1d}$). Greedy optimal sequences also satisfy, under natural additional assumptions, the distortion mismatch property: the $N$-tuples $a^{(N)}$ remain rate optimal with respect to the $L^q$-norms, $p\le q <p+d$. Finally, we propose optimization methods to compute greedy sequences, adapted from usual Lloyd's I and Competitive Learning Vector Quantization procedures, either in their deterministic (implementable when $d=1$) or stochastic versions.Comment: 31 pages, 4 figures, few typos corrected (now an extended version of an eponym paper to appear in Journal of Approximation

Conditional hitting time estimation in a nonlinear filtering model by the Brownian bridge method

The model consists of a signal process $X$ which is a general Brownian diffusion process and an observation process $Y$, also a diffusion process, which is supposed to be correlated to the signal process. We suppose that the process $Y$ is observed from time 0 to $s>0$ at discrete times and aim to estimate, conditionally on these observations, the probability that the non-observed process $X$ crosses a fixed barrier after a given time $t>s$. We formulate this problem as a usual nonlinear filtering problem and use optimal quantization and Monte Carlo simulations techniques to estimate the involved quantities

Product Markovian quantization of an R^d -valued Euler scheme of a diffusion process with applications to finance

We introduce a new approach to quantize the Euler scheme of an $\mathbb{R}^d$-valued diffusion process. This method is based on a Markovian and componentwise product quantization and allows us, from a numerical point of view, to speak of {\em fast online quantization} in dimension greater than one since the product quantization of the Euler scheme of the diffusion process and its companion weights and transition probabilities may be computed quite instantaneously. We show that the resulting quantization process is a Markov chain, then, we compute the associated companion weights and transition probabilities from (semi-) closed formulas. From the analytical point of view, we show that the induced quantization errors at the $k$-th discretization step $t_k$ is a cumulative of the marginal quantization error up to time $t_k$. Numerical experiments are performed for the pricing of a Basket call option, for the pricing of a European call option in a Heston model and for the approximation of the solution of backward stochastic differential equations to show the performances of the method