The Quadratic Gaussian Rate-Distortion Function for Source Uncorrelated Distortions

We characterize the rate-distortion function for zero-mean stationary Gaussian sources under the MSE fidelity criterion and subject to the additional constraint that the distortion is uncorrelated to the input. The solution is given by two equations coupled through a single scalar parameter. This has a structure similar to the well known water-filling solution obtained without the uncorrelated distortion restriction. Our results fully characterize the unique statistics of the optimal distortion. We also show that, for all positive distortions, the minimum achievable rate subject to the uncorrelation constraint is strictly larger than that given by the un-constrained rate-distortion function. This gap increases with the distortion and tends to infinity and zero, respectively, as the distortion tends to zero and infinity.Comment: Revised version, to be presented at the Data Compression Conference 200

Improved Upper Bounds to the Causal Quadratic Rate-Distortion Function for Gaussian Stationary Sources

We improve the existing achievable rate regions for causal and for zero-delay source coding of stationary Gaussian sources under an average mean squared error (MSE) distortion measure. To begin with, we find a closed-form expression for the information-theoretic causal rate-distortion function (RDF) under such distortion measure, denoted by $R_{c}^{it}(D)$, for first-order Gauss-Markov processes. Rc^{it}(D) is a lower bound to the optimal performance theoretically attainable (OPTA) by any causal source code, namely Rc^{op}(D). We show that, for Gaussian sources, the latter can also be upper bounded as Rc^{op}(D)\leq Rc^{it}(D) + 0.5 log_{2}(2\pi e) bits/sample. In order to analyze $R_{c}^{it}(D)$ for arbitrary zero-mean Gaussian stationary sources, we introduce \bar{Rc^{it}}(D), the information-theoretic causal RDF when the reconstruction error is jointly stationary with the source. Based upon \bar{Rc^{it}}(D), we derive three closed-form upper bounds to the additive rate loss defined as \bar{Rc^{it}}(D) - R(D), where R(D) denotes Shannon's RDF. Two of these bounds are strictly smaller than 0.5 bits/sample at all rates. These bounds differ from one another in their tightness and ease of evaluation; the tighter the bound, the more involved its evaluation. We then show that, for any source spectral density and any positive distortion D\leq \sigma_{x}^{2}, \bar{Rc^{it}}(D) can be realized by an AWGN channel surrounded by a unique set of causal pre-, post-, and feedback filters. We show that finding such filters constitutes a convex optimization problem. In order to solve the latter, we propose an iterative optimization procedure that yields the optimal filters and is guaranteed to converge to \bar{Rc^{it}}(D). Finally, by establishing a connection to feedback quantization we design a causal and a zero-delay coding scheme which, for Gaussian sources, achieves...Comment: 47 pages, revised version submitted to IEEE Trans. Information Theor

Capacity Bounds for Communication Systems with Quantization and Spectral Constraints

Low-resolution digital-to-analog and analog-to-digital converters (DACs and ADCs) have attracted considerable attention in efforts to reduce power consumption in millimeter wave (mmWave) and massive MIMO systems. This paper presents an information-theoretic analysis with capacity bounds for classes of linear transceivers with quantization. The transmitter modulates symbols via a unitary transform followed by a DAC and the receiver employs an ADC followed by the inverse unitary transform. If the unitary transform is set to an FFT matrix, the model naturally captures filtering and spectral constraints which are essential to model in any practical transceiver. In particular, this model allows studying the impact of quantization on out-of-band emission constraints. In the limit of a large random unitary transform, it is shown that the effect of quantization can be precisely described via an additive Gaussian noise model. This model in turn leads to simple and intuitive expressions for the power spectrum of the transmitted signal and a lower bound to the capacity with quantization. Comparison with non-quantized capacity and a capacity upper bound that does not make linearity assumptions suggests that while low resolution quantization has minimal impact on the achievable rate at typical parameters in 5G systems today, satisfying out-of-band emissions are potentially much more of a challenge.Comment: Appears in the Proceedings of IEEE International Symposium on Information Theory (ISIT) 202

Real-Time Perceptual Moving-Horizon Multiple-Description Audio Coding

A novel scheme for perceptual coding of audio for robust and real-time communication is designed and analyzed. As an alternative to PCM, DPCM, and more general noise-shaping converters, we propose to use psychoacoustically optimized noise-shaping quantizers based on the moving-horizon principle. In moving-horizon quantization, a few samples look-ahead is allowed at the encoder, which makes it possible to better shape the quantization noise and thereby reduce the resulting distortion over what is possible with conventional noise-shaping techniques. It is first shown that significant gains over linear PCM can be obtained without introducing a delay and without requiring postprocessing at the decoder, i.e., the encoded samples can be stored as, e.g., 16-bit linear PCM on CD-ROMs, and played out on standards-compliant CD players. We then show that multiple-description coding can be combined with moving-horizon quantization in order to combat possible erasures on the wireless link without introducing additional delays