Entropy-constrained scalar quantization with a lossy-compressed bit

We consider the compression of a continuous real-valued source X using scalar quantizers and average squared error distortion D. Using lossless compression of the quantizer's output, Gish and Pierce showed that uniform quantizing yields the smallest output entropy in the limit D -> 0, resulting in a rate penalty of 0.255 bits/sample above the Shannon Lower Bound (SLB). We present a scalar quantization scheme named lossy-bit entropy-constrained scalar quantization (Lb-ECSQ) that is able to reduce the D -> 0 gap to SLB to 0.251 bits/sample by combining both lossless and binary lossy compression of the quantizer's output. We also study the low-resolution regime and show that Lb-ECSQ significantly outperforms ECSQ in the case of 1-bit quantization.The authors wish to thank Tobias Koch and Gonzalo Vázquez Vilar for fruitful discussions and helpful comments to the manuscript. This work has been supported in part by the European Union 7th Framework Programme through the Marie Curie Initial Training Network “Machine Learning for Personalized Medicine” MLPM2012, Grant No. 316861, by the Spanish Ministry of Economy and Competitiveness and Ministry of Education under grants TEC2016-78434-C3-3-R (MINECO/FEDER, EU) and IJCI-2014-19150, and by Comunidad de Madrid (project ’CASI-CAM-CM’, id. S2013/ICE-2845).Publicad

Zero-Delay Joint Source-Channel Coding in the Presence of Interference Known at the Encoder

Zero-delay transmission of a Gaussian source over an additive white Gaussian noise (AWGN) channel is considered in the presence of an additive Gaussian interference signal. The mean squared error (MSE) distortion is minimized under an average power constraint assuming that the interference signal is known at the transmitter. Optimality of simple linear transmission does not hold in this setting due to the presence of the known interference signal. While the optimal encoder-decoder pair remains an open problem, various non-linear transmission schemes are proposed in this paper. In particular, interference concentration (ICO) and one-dimensional lattice (1DL) strategies, using both uniform and non-uniform quantization of the interference signal, are studied. It is shown that, in contrast to typical scalar quantization of Gaussian sources, a non-uniform quantizer, whose quantization intervals become smaller as we go further from zero, improves the performance. Given that the optimal decoder is the minimum MSE (MMSE) estimator, a necessary condition for the optimality of the encoder is derived, and the numerically optimized encoder (NOE) satisfying this condition is obtained. Based on the numerical results, it is shown that 1DL with nonuniform quantization performs closer (compared to the other schemes) to the numerically optimized encoder while requiring significantly lower complexity

Sparse Regression Codes for Multi-terminal Source and Channel Coding

We study a new class of codes for Gaussian multi-terminal source and channel coding. These codes are designed using the statistical framework of high-dimensional linear regression and are called Sparse Superposition or Sparse Regression codes. Codewords are linear combinations of subsets of columns of a design matrix. These codes were recently introduced by Barron and Joseph and shown to achieve the channel capacity of AWGN channels with computationally feasible decoding. They have also recently been shown to achieve the optimal rate-distortion function for Gaussian sources. In this paper, we demonstrate how to implement random binning and superposition coding using sparse regression codes. In particular, with minimum-distance encoding/decoding it is shown that sparse regression codes attain the optimal information-theoretic limits for a variety of multi-terminal source and channel coding problems.Comment: 9 pages, appeared in the Proceedings of the 50th Annual Allerton Conference on Communication, Control, and Computing - 201

Goal-Oriented Quantization: Analysis, Design, and Application to Resource Allocation

In this paper, the situation in which a receiver has to execute a task from a quantized version of the information source of interest is considered. The task is modeled by the minimization problem of a general goal function $f(x;g)$ for which the decision $x$ has to be taken from a quantized version of the parameters $g$. This problem is relevant in many applications e.g., for radio resource allocation (RA), high spectral efficiency communications, controlled systems, or data clustering in the smart grid. By resorting to high resolution (HR) analysis, it is shown how to design a quantizer that minimizes the gap between the minimum of $f$ (which would be reached by knowing $g$ perfectly) and what is effectively reached with a quantized $g$. The conducted formal analysis both provides quantization strategies in the HR regime and insights for the general regime and allows a practical algorithm to be designed. The analysis also allows one to provide some elements to the new and fundamental problem of the relationship between the goal function regularity properties and the hardness to quantize its parameters. The derived results are discussed and supported by a rich numerical performance analysis in which known RA goal functions are studied and allows one to exhibit very significant improvements by tailoring the quantization operation to the final task

Operational Rate-Distortion Performance of Single-source and Distributed Compressed Sensing

We consider correlated and distributed sources without cooperation at the encoder. For these sources, we derive the best achievable performance in the rate-distortion sense of any distributed compressed sensing scheme, under the constraint of high--rate quantization. Moreover, under this model we derive a closed--form expression of the rate gain achieved by taking into account the correlation of the sources at the receiver and a closed--form expression of the average performance of the oracle receiver for independent and joint reconstruction. Finally, we show experimentally that the exploitation of the correlation between the sources performs close to optimal and that the only penalty is due to the missing knowledge of the sparsity support as in (non distributed) compressed sensing. Even if the derivation is performed in the large system regime, where signal and system parameters tend to infinity, numerical results show that the equations match simulations for parameter values of practical interest.Comment: To appear in IEEE Transactions on Communication

A limited feedback scheme based on spatially correlated channels for coordinated multipoint systems

High spectral efficiency can be achieved in the downlink of multi-antenna coordinated multi-point systems provided that the multiuser interference is appropriately managed at the transmitter side. For this sake, downlink channel information needs to be sent back by the users, thus reducing the rate available at the uplink channel. The amount and type of feedback information required has been extensively studied and many limited feedback schemes have been proposed lately. A common pattern to all of them is that achieving low rates of feedback information is possible at the cost of increasing complexity at the user side and, sometimes, assuming that some statistics of the channel are known. In this article, we propose a simple and versatile limited feedback scheme that exploits the spatial correlation at each multi-antenna base station (BS) without requiring any previous statistical information of the channel and without adding significant computational complexity. It is based on the separate quantization of the channel impulse response modulus and phase and it shows better mean square error performance than the standard scheme based on quantization of real and imaginary parts. In order to evaluate the performance of the downlink regarding multiuser interference management, different precoding techniques at the BSs, such as zero-forcing (ZF), Tomlinson-Harashima precoding (THP) and lattice reduction Tomlinson- Harashima precoding (LRTHP), have been evaluated. Simulations results show that LRTHP and THP present a higher robustness than ZF precoding against channel quantization errors but at the cost of a higher complexity at the BS. Regarding sum-capacity and bit error rate performances, our versatile scheme achieves better results than the standard one in the medium and high SNR regime, that is, in the region where quantization errors are dominant against noise, for the same feedback cost measured in bits per user