The distributed Karhunen-Loeve transform

The Karhunen-Loeve transform is a key element of many signal processing tasks, including classification and compression. In this paper, we consider distributed signal processing scenarios with limited communication between correlated sources, and we investigate a distributed Karhunen-Loeve transform (KLT). In particular, a partial KLT (where only a subset of sources are observed) and a conditional KLT (where some sources act as side information) are posed and solved in a rate-distortion sense. The partial KLT leads to an original bit allocation problem, while the conditional KLT leads to a Wyner Ziv solution which is separable at the sources. These two cases can be seen as extreme cases of a distributed KLT

Suboptimality of the Karhunen-Loève transform for transform coding

We examine the performance of the Karhunen-Loeve transform (KLT) for transform coding applications. The KLT has long been viewed as the best available block transform for a system that orthogonally transforms a vector source, scalar quantizes the components of the transformed vector using optimal bit allocation, and then inverse transforms the vector. This paper treats fixed-rate and variable-rate transform codes of non-Gaussian sources. The fixed-rate approach uses an optimal fixed-rate scalar quantizer to describe the transform coefficients; the variable-rate approach uses a uniform scalar quantizer followed by an optimal entropy code, and each quantized component is encoded separately. Earlier work shows that for the variable-rate case there exist sources on which the KLT is not unique and the optimal quantization and coding stage matched to a "worst" KLT yields performance as much as 1.5 dB worse than the optimal quantization and coding stage matched to a "best" KLT. In this paper, we strengthen that result to show that in both the fixed-rate and the variable-rate coding frameworks there exist sources for which the performance penalty for using a "worst" KLT can be made arbitrarily large. Further, we demonstrate in both frameworks that there exist sources for which even a best KLT gives suboptimal performance. Finally, we show that even for vector sources where the KLT yields independent coefficients, the KLT can be suboptimal for fixed-rate coding

Reduced-dimension linear transform coding of distributed correlated signals with incomplete observations

We study the problem of optimal reduced-dimension linear transform coding and reconstruction of a signal based on distributed correlated observations of the signal. In the mean square estimation context this involves finding he optimal signal representation based on multiple incomplete or only partial observations that are correlated. In particular this leads to the study of finding the optimal Karhunen-Loeve basis based on the censored observations. The problem has been considered previously by Gestpar, Dragotti and Vitterli in the context of jointly Gaussian random variables based on using conditional covariances. In this paper, we derive the estimation results in the more general setting of second-order random variables with arbitrary distributions, using entirely different techniques based on the idea of innovations. We explicitly solve the single transform coder case, give a characterization of optimality in the multiple distributed transform coders scenario and provide additional insights into the structure of the problm