Information Masking and Amplification: The Source Coding Setting

The complementary problems of masking and amplifying channel state information in the Gel'fand-Pinsker channel have recently been solved by Merhav and Shamai, and Kim et al., respectively. In this paper, we study a related source coding problem. Specifically, we consider the two-encoder source coding setting where one source is to be amplified, while the other source is to be masked. In general, there is a tension between these two objectives which is characterized by the amplification-masking tradeoff. In this paper, we give a single-letter description of this tradeoff. We apply this result, together with a recent theorem by Courtade and Weissman on multiterminal source coding, to solve a fundamental entropy characterization problem.Comment: 6 pages, 1 figure, to appear at the IEEE 2012 International Symposium on Information Theory (ISIT 2012

A Universal Scheme for Wyner–Ziv Coding of Discrete Sources

We consider the Wyner–Ziv (WZ) problem of lossy compression where the decompressor observes a noisy version of the source, whose statistics are unknown. A new family of WZ coding algorithms is proposed and their universal optimality is proven. Compression consists of sliding-window processing followed by Lempel–Ziv (LZ) compression, while the decompressor is based on a modification of the discrete universal denoiser (DUDE) algorithm to take advantage of side information. The new algorithms not only universally attain the fundamental limits, but also suggest a paradigm for practical WZ coding. The effectiveness of our approach is illustrated with experiments on binary images, and English text using a low complexity algorithm motivated by our class of universally optimal WZ codes

Distributed Binary Detection with Lossy Data Compression

Consider the problem where a statistician in a two-node system receives rate-limited information from a transmitter about marginal observations of a memoryless process generated from two possible distributions. Using its own observations, this receiver is required to first identify the legitimacy of its sender by declaring the joint distribution of the process, and then depending on such authentication it generates the adequate reconstruction of the observations satisfying an average per-letter distortion. The performance of this setup is investigated through the corresponding rate-error-distortion region describing the trade-off between: the communication rate, the error exponent induced by the detection and the distortion incurred by the source reconstruction. In the special case of testing against independence, where the alternative hypothesis implies that the sources are independent, the optimal rate-error-distortion region is characterized. An application example to binary symmetric sources is given subsequently and the explicit expression for the rate-error-distortion region is provided as well. The case of "general hypotheses" is also investigated. A new achievable rate-error-distortion region is derived based on the use of non-asymptotic binning, improving the quality of communicated descriptions. Further improvement of performance in the general case is shown to be possible when the requirement of source reconstruction is relaxed, which stands in contrast to the case of general hypotheses.Comment: to appear on IEEE Trans. Information Theor

Wyner-Ziv coding based on TCQ and LDPC codes and extensions to multiterminal source coding

Driven by a host of emerging applications (e.g., sensor networks and wireless video), distributed source coding (i.e., Slepian-Wolf coding, Wyner-Ziv coding and various other forms of multiterminal source coding), has recently become a very active research area. In this thesis, we first design a practical coding scheme for the quadratic Gaussian Wyner-Ziv problem, because in this special case, no rate loss is suffered due to the unavailability of the side information at the encoder. In order to approach the Wyner-Ziv distortion limit D??W Z(R), the trellis coded quantization (TCQ) technique is employed to quantize the source X, and irregular LDPC code is used to implement Slepian-Wolf coding of the quantized source input Q(X) given the side information Y at the decoder. An optimal non-linear estimator is devised at the joint decoder to compute the conditional mean of the source X given the dequantized version of Q(X) and the side information Y . Assuming ideal Slepian-Wolf coding, our scheme performs only 0.2 dB away from the Wyner-Ziv limit D??W Z(R) at high rate, which mirrors the performance of entropy-coded TCQ in classic source coding. Practical designs perform 0.83 dB away from D??W Z(R) at medium rates. With 2-D trellis-coded vector quantization, the performance gap to D??W Z(R) is only 0.66 dB at 1.0 b/s and 0.47 dB at 3.3 b/s. We then extend the proposed Wyner-Ziv coding scheme to the quadratic Gaussian multiterminal source coding problem with two encoders. Both direct and indirect settings of multiterminal source coding are considered. An asymmetric code design containing one classical source coding component and one Wyner-Ziv coding component is first introduced and shown to be able to approach the corner points on the theoretically achievable limits in both settings. To approach any point on the theoretically achievable limits, a second approach based on source splitting is then described. One classical source coding component, two Wyner-Ziv coding components, and a linear estimator are employed in this design. Proofs are provided to show the achievability of any point on the theoretical limits in both settings by assuming that both the source coding and the Wyner-Ziv coding components are optimal. The performance of practical schemes is only 0.15 b/s away from the theoretical limits for the asymmetric approach, and up to 0.30 b/s away from the limits for the source splitting approach

Integer-forcing in multiterminal coding: uplink-downlink duality and source-channel duality

Interference is considered to be a major obstacle to wireless communication. Popular approaches, such as the zero-forcing receiver in MIMO (multiple-input and multiple-output) multiple-access channel (MAC) and zero-forcing (ZF) beamforming in MIMO broadcast channel (BC), eliminate the interference first and decode each codeword separately using a conventional single-user decoder. Recently, a transceiver architecture called integer-forcing (IF) has been proposed in the context of the MIMO Gaussian multiple-access channel to exploit integer-linear combinations of the codewords. Instead of treating other codewords as interference, the integer-forcing approach decodes linear combinations of the codewords from different users and solves for desired codewords. Integer-forcing can closely approach the performance of the optimal joint maximum likelihood decoder. An advanced version called successive integer-forcing can achieve the sum capacity of the MIMO MAC channel. Several extensions of integer-forcing have been developed in various scenarios, such as integer-forcing for the Gaussian MIMO broadcast channel, integer-forcing for Gaussian distributed source coding and integer-forcing interference alignment for the Gaussian interference channel. This dissertation demonstrates duality relationships for integer-forcing among three different channel models. We explore in detail two distinct duality types in this thesis: uplink-downlink duality and source-channel duality. Uplink-downlink duality is established for integer-forcing between the Gaussian MIMO multiple-access channel and its dual Gaussian MIMO broadcast channel. We show that under a total power constraint, integer-forcing can achieve the same sum rate in both cases. We further develop a dirty-paper integer-forcing scheme for the Gaussian MIMO BC and show an uplink-downlink duality with successive integer-forcing for the Gaussian MIMO MAC. The source-channel duality is established for integer-forcing between the Gaussian MIMO multiple-access channel and its dual Gaussian distributed source coding problem. We extend previous results for integer-forcing source coding to allow for successive cancellation. For integer-forcing without successive cancellation in both channel coding and source coding, we show the rates in two scenarios lie within a constant gap of one another. We further show that there exists a successive cancellation scheme such that both integer-forcing channel coding and integer-forcing source coding achieve the same rate tuple