Universal Sampling Rate Distortion

We examine the coordinated and universal rate-efficient sampling of a subset of correlated discrete memoryless sources followed by lossy compression of the sampled sources. The goal is to reconstruct a predesignated subset of sources within a specified level of distortion. The combined sampling mechanism and rate distortion code are universal in that they are devised to perform robustly without exact knowledge of the underlying joint probability distribution of the sources. In Bayesian as well as nonBayesian settings, single-letter characterizations are provided for the universal sampling rate distortion function for fixed-set sampling, independent random sampling and memoryless random sampling. It is illustrated how these sampling mechanisms are successively better. Our achievability proofs bring forth new schemes for joint source distribution-learning and lossy compression

Sampling Rate Distortion

Consider a memoryless multiple source with m components of which a (possibly randomized) subset of k ≤ m components are sampled at each time instant and jointly compressed with the objective of reconstructing a prespecified subset of the m components under a given distortion criterion. The combined sampling and lossy compression mechanisms are to be designed to perform robustly with or without exact knowledge of the underlying joint probability distribution of the source. In this dissertation, we introduce a new framework of sampling rate distortion to study the tradeoffs among sampling mechanism, encoder-decoder structure, compression rate and the desired level of accuracy in the reconstruction. We begin with a discrete memoryless multiple source whose joint probability mass function (pmf) is taken to be known. A notion of sampling rate distortion function is introduced to study the mentioned tradeoffs, and is characterized first for fixed-set sampling. Next, for independent random sampling performed without the knowledge of the source outputs, it is shown that the sampling rate distortion function is the same whether or not the decoder is informed of the sequence of sampled sets. For memoryless random sampling, with the sampling depending on the source outputs, it is shown that deterministic sampling, characterized by a conditional point-mass, is optimal and suffices to achieve the sampling rate distortion function. Building on this, we consider a universal setting where the joint pmf of a discrete memoryless multiple source is known only to belong to a {\it finite} family of pmfs. In Bayesian and nonBayesian settings, single-letter characterizations are provided for the universal sampling rate distortion function for the fixed-set sampling, independent random sampling and memoryless random sampling. We show that these sampling mechanisms successively improve upon each other: (i) in their ability to enable an associated encoder approximate the underlying joint pmf and (ii) in their ability to choose appropriate subsets of the multiple source for compression by the encoder. Lastly, we consider a jointly Gaussian multiple memoryless source, to be reconstructed under a mean-squared error distortion criterion, with joint probability distribution function known only to belong to an uncountable family of probability density functions (characterized by a convex compact subset in Euclidean space). For fixed-set sampling, we characterize the universal sampling rate distortion function in Bayesian and nonBayesian settings. We also provide optimal reconstruction algorithms, of reduced complexity, which compress and reconstruct the sampled source components first under a modified distortion criterion, and then form MMSE estimates for the unsampled components based on reconstructions of the former. The questions addressed in this dissertation are motivated by various applications, e.g., dynamic thermal management for multicore processors, in-network computation and satellite imaging

A General Framework for Transmission with Transceiver Distortion and Some Applications

A general theoretical framework is presented for analyzing information transmission over Gaussian channels with memoryless transceiver distortion, which encompasses various nonlinear distortion models including transmit-side clipping, receive-side analog-to-digital conversion, and others. The framework is based on the so-called generalized mutual information (GMI), and the analysis in particular benefits from the setup of Gaussian codebook ensemble and nearest-neighbor decoding, for which it is established that the GMI takes a general form analogous to the channel capacity of undistorted Gaussian channels, with a reduced "effective" signal-to-noise ratio (SNR) that depends on the nominal SNR and the distortion model. When applied to specific distortion models, an array of results of engineering relevance is obtained. For channels with transmit-side distortion only, it is shown that a conventional approach, which treats the distorted signal as the sum of the original signal part and a uncorrelated distortion part, achieves the GMI. For channels with output quantization, closed-form expressions are obtained for the effective SNR and the GMI, and related optimization problems are formulated and solved for quantizer design. Finally, super-Nyquist sampling is analyzed within the general framework, and it is shown that sampling beyond the Nyquist rate increases the GMI for all SNR. For example, with a binary symmetric output quantization, information rates exceeding one bit per channel use are achievable by sampling the output at four times the Nyquist rate.Comment: 32 pages (including 4 figures, 5 tables, and auxiliary materials); submitted to IEEE Transactions on Communication

Multiple-Description Coding by Dithered Delta-Sigma Quantization

We address the connection between the multiple-description (MD) problem and Delta-Sigma quantization. The inherent redundancy due to oversampling in Delta-Sigma quantization, and the simple linear-additive noise model resulting from dithered lattice quantization, allow us to construct a symmetric and time-invariant MD coding scheme. We show that the use of a noise shaping filter makes it possible to trade off central distortion for side distortion. Asymptotically as the dimension of the lattice vector quantizer and order of the noise shaping filter approach infinity, the entropy rate of the dithered Delta-Sigma quantization scheme approaches the symmetric two-channel MD rate-distortion function for a memoryless Gaussian source and MSE fidelity criterion, at any side-to-central distortion ratio and any resolution. In the optimal scheme, the infinite-order noise shaping filter must be minimum phase and have a piece-wise flat power spectrum with a single jump discontinuity. An important advantage of the proposed design is that it is symmetric in rate and distortion by construction, so the coding rates of the descriptions are identical and there is therefore no need for source splitting.Comment: Revised, restructured, significantly shortened and minor typos has been fixed. Accepted for publication in the IEEE Transactions on Information Theor

Demodulation and Detection Schemes for a Memoryless Optical WDM Channel

It is well known that matched filtering and sampling (MFS) demodulation together with minimum Euclidean distance (MD) detection constitute the optimal receiver for the additive white Gaussian noise channel. However, for a general nonlinear transmission medium, MFS does not provide sufficient statistics, and therefore is suboptimal. Nonetheless, this receiver is widely used in optical systems, where the Kerr nonlinearity is the dominant impairment at high powers. In this paper, we consider a suite of receivers for a two-user channel subject to a type of nonlinear interference that occurs in wavelength-division-multiplexed channels. The asymptotes of the symbol error rate (SER) of the considered receivers at high powers are derived or bounded analytically. Moreover, Monte-Carlo simulations are conducted to evaluate the SER for all the receivers. Our results show that receivers that are based on MFS cannot achieve arbitrary low SERs, whereas the SER goes to zero as the power grows for the optimal receiver. Furthermore, we devise a heuristic demodulator, which together with the MD detector yields a receiver that is simpler than the optimal one and can achieve arbitrary low SERs. The SER performance of the proposed receivers is also evaluated for some single-span fiber-optical channels via split-step Fourier simulations

Impact of Spatial Filtering on Distortion from Low-Noise Amplifiers in Massive MIMO Base Stations

In massive MIMO base stations, power consumption and cost of the low-noise amplifiers (LNAs) can be substantial because of the many antennas. We investigate the feasibility of inexpensive, power efficient LNAs, which inherently are less linear. A polynomial model is used to characterize the nonlinear LNAs and to derive the second-order statistics and spatial correlation of the distortion. We show that, with spatial matched filtering (maximum-ratio combining) at the receiver, some distortion terms combine coherently, and that the SINR of the symbol estimates therefore is limited by the linearity of the LNAs. Furthermore, it is studied how the power from a blocker in the adjacent frequency band leaks into the main band and creates distortion. The distortion term that scales cubically with the power received from the blocker has a spatial correlation that can be filtered out by spatial processing and only the coherent term that scales quadratically with the power remains. When the blocker is in free-space line-of-sight and the LNAs are identical, this quadratic term has the same spatial direction as the desired signal, and hence cannot be removed by linear receiver processing