Universal Sampling Rate Distortion

We examine the coordinated and universal rate-efficient sampling of a subset of correlated discrete memoryless sources followed by lossy compression of the sampled sources. The goal is to reconstruct a predesignated subset of sources within a specified level of distortion. The combined sampling mechanism and rate distortion code are universal in that they are devised to perform robustly without exact knowledge of the underlying joint probability distribution of the sources. In Bayesian as well as nonBayesian settings, single-letter characterizations are provided for the universal sampling rate distortion function for fixed-set sampling, independent random sampling and memoryless random sampling. It is illustrated how these sampling mechanisms are successively better. Our achievability proofs bring forth new schemes for joint source distribution-learning and lossy compression

Sampling Rate Distortion

Consider a memoryless multiple source with m components of which a (possibly randomized) subset of k ≤ m components are sampled at each time instant and jointly compressed with the objective of reconstructing a prespecified subset of the m components under a given distortion criterion. The combined sampling and lossy compression mechanisms are to be designed to perform robustly with or without exact knowledge of the underlying joint probability distribution of the source. In this dissertation, we introduce a new framework of sampling rate distortion to study the tradeoffs among sampling mechanism, encoder-decoder structure, compression rate and the desired level of accuracy in the reconstruction. We begin with a discrete memoryless multiple source whose joint probability mass function (pmf) is taken to be known. A notion of sampling rate distortion function is introduced to study the mentioned tradeoffs, and is characterized first for fixed-set sampling. Next, for independent random sampling performed without the knowledge of the source outputs, it is shown that the sampling rate distortion function is the same whether or not the decoder is informed of the sequence of sampled sets. For memoryless random sampling, with the sampling depending on the source outputs, it is shown that deterministic sampling, characterized by a conditional point-mass, is optimal and suffices to achieve the sampling rate distortion function. Building on this, we consider a universal setting where the joint pmf of a discrete memoryless multiple source is known only to belong to a {\it finite} family of pmfs. In Bayesian and nonBayesian settings, single-letter characterizations are provided for the universal sampling rate distortion function for the fixed-set sampling, independent random sampling and memoryless random sampling. We show that these sampling mechanisms successively improve upon each other: (i) in their ability to enable an associated encoder approximate the underlying joint pmf and (ii) in their ability to choose appropriate subsets of the multiple source for compression by the encoder. Lastly, we consider a jointly Gaussian multiple memoryless source, to be reconstructed under a mean-squared error distortion criterion, with joint probability distribution function known only to belong to an uncountable family of probability density functions (characterized by a convex compact subset in Euclidean space). For fixed-set sampling, we characterize the universal sampling rate distortion function in Bayesian and nonBayesian settings. We also provide optimal reconstruction algorithms, of reduced complexity, which compress and reconstruct the sampled source components first under a modified distortion criterion, and then form MMSE estimates for the unsampled components based on reconstructions of the former. The questions addressed in this dissertation are motivated by various applications, e.g., dynamic thermal management for multicore processors, in-network computation and satellite imaging

A General Framework for Transmission with Transceiver Distortion and Some Applications

A general theoretical framework is presented for analyzing information transmission over Gaussian channels with memoryless transceiver distortion, which encompasses various nonlinear distortion models including transmit-side clipping, receive-side analog-to-digital conversion, and others. The framework is based on the so-called generalized mutual information (GMI), and the analysis in particular benefits from the setup of Gaussian codebook ensemble and nearest-neighbor decoding, for which it is established that the GMI takes a general form analogous to the channel capacity of undistorted Gaussian channels, with a reduced "effective" signal-to-noise ratio (SNR) that depends on the nominal SNR and the distortion model. When applied to specific distortion models, an array of results of engineering relevance is obtained. For channels with transmit-side distortion only, it is shown that a conventional approach, which treats the distorted signal as the sum of the original signal part and a uncorrelated distortion part, achieves the GMI. For channels with output quantization, closed-form expressions are obtained for the effective SNR and the GMI, and related optimization problems are formulated and solved for quantizer design. Finally, super-Nyquist sampling is analyzed within the general framework, and it is shown that sampling beyond the Nyquist rate increases the GMI for all SNR. For example, with a binary symmetric output quantization, information rates exceeding one bit per channel use are achievable by sampling the output at four times the Nyquist rate.Comment: 32 pages (including 4 figures, 5 tables, and auxiliary materials); submitted to IEEE Transactions on Communication

Multiple-Description Coding by Dithered Delta-Sigma Quantization

We address the connection between the multiple-description (MD) problem and Delta-Sigma quantization. The inherent redundancy due to oversampling in Delta-Sigma quantization, and the simple linear-additive noise model resulting from dithered lattice quantization, allow us to construct a symmetric and time-invariant MD coding scheme. We show that the use of a noise shaping filter makes it possible to trade off central distortion for side distortion. Asymptotically as the dimension of the lattice vector quantizer and order of the noise shaping filter approach infinity, the entropy rate of the dithered Delta-Sigma quantization scheme approaches the symmetric two-channel MD rate-distortion function for a memoryless Gaussian source and MSE fidelity criterion, at any side-to-central distortion ratio and any resolution. In the optimal scheme, the infinite-order noise shaping filter must be minimum phase and have a piece-wise flat power spectrum with a single jump discontinuity. An important advantage of the proposed design is that it is symmetric in rate and distortion by construction, so the coding rates of the descriptions are identical and there is therefore no need for source splitting.Comment: Revised, restructured, significantly shortened and minor typos has been fixed. Accepted for publication in the IEEE Transactions on Information Theor

Experimental demonstration of digital predistortion for orthogonal frequency-division multiplexing-radio over fibre links near laser resonance

Radio over fibre (RoF), an enabling technology for distribution of wireless broadband service signals through analogue optical links, suffers from non-linear distortion. Digital predistortion has been demonstrated as an effective approach to overcome the RoF non-linearity. However, questions remain as to how the approach performs close to laser resonance, a region of significant dynamic non-linearity, and how resilient the approach is to changes in input signal and link operating conditions. In this work, the performance of a digital predistortion approach is studied for directly modulated orthogonal frequency-division multiplexing RoF links operating from 2.47 to 3.7 GHz. It extends previous works to higher frequencies, and to higher quadrature amplitude modulation (QAM) levels. In addition, the resilience of the predistortion approach to changes in modulation level of QAM schemes, and average power levels are investigated, and a novel predistortion training approach is proposed and demonstrated. Both memoryless and memory polynomial predistorter models, and a simple off-line least-squares-based identification method, are used, with excellent performance improvements demonstrated up to 3.0 GHz

Estimation of the Rate-Distortion Function

Motivated by questions in lossy data compression and by theoretical considerations, we examine the problem of estimating the rate-distortion function of an unknown (not necessarily discrete-valued) source from empirical data. Our focus is the behavior of the so-called "plug-in" estimator, which is simply the rate-distortion function of the empirical distribution of the observed data. Sufficient conditions are given for its consistency, and examples are provided to demonstrate that in certain cases it fails to converge to the true rate-distortion function. The analysis of its performance is complicated by the fact that the rate-distortion function is not continuous in the source distribution; the underlying mathematical problem is closely related to the classical problem of establishing the consistency of maximum likelihood estimators. General consistency results are given for the plug-in estimator applied to a broad class of sources, including all stationary and ergodic ones. A more general class of estimation problems is also considered, arising in the context of lossy data compression when the allowed class of coding distributions is restricted; analogous results are developed for the plug-in estimator in that case. Finally, consistency theorems are formulated for modified (e.g., penalized) versions of the plug-in, and for estimating the optimal reproduction distribution.Comment: 18 pages, no figures [v2: removed an example with an error; corrected typos; a shortened version will appear in IEEE Trans. Inform. Theory