Blahut-Arimoto Algorithm and Code Design for Action-Dependent Source Coding Problems

The source coding problem with action-dependent side information at the decoder has recently been introduced to model data acquisition in resource-constrained systems. In this paper, an efficient algorithm for numerical computation of the rate-distortion-cost function for this problem is proposed, and a convergence proof is provided. Moreover, a two-stage code design based on multiplexing is put forth, whereby the first stage encodes the actions and the second stage is composed of an array of classical Wyner-Ziv codes, one for each action. Specific coding/decoding strategies are designed based on LDGM codes and message passing. Through numerical examples, the proposed code design is shown to achieve performance close to the lower bound dictated by the rate-distortion-cost function.Comment: Extended version of a paper submitted to ISI

Source coding with in-block memory and controllable causal side information

Abstract-The recently proposed set-up of source coding with a side information "vending machine" allows the decoder to select actions in order to control the quality of the side information. The actions can depend on the message received from the encoder and on the previously measured samples of the side information, and are cost constrained. Moreover, the final estimate of the source by the decoder is a function of the encoder's message and depends causally on the side information sequence. Previous work by Permuter and Weissman has characterized the rate-distortioncost function in the special case in which the source and the "vending machine" are memoryless. In this work, motivated by the related channel coding model introduced by Kramer, the ratedistortion-cost function characterization is extended to a model with in-block memory. Various special cases are studied including block-feedforward and side information repeat request models

Adaptive data acquisition for communication networks

In an increasing number of communication systems, such as sensor networks or local area networks within medical, financial or military institutions, nodes communicate information sources (e.g., video, audio) over multiple hops. Moreover, nodes have, or can acquire, correlated information sources from the environment, e.g., from data bases or from measurements. Among the new design problems raised by the outlined scenarios, two key issues are addressed in this dissertation: 1) How to preserve the consistency of sensitive information across multiple hops; 2) How to incorporate the design of actuation in the form of data acquisition and network probing in the optimization of the communication network. These aspects are investigated by using information-theoretic (source and channel coding) models, obtaining fundamental insights that have been corroborated by various illustrative examples. To address point 1), the problem of cascade source coding with side information is investigated. The motivating observation is that, in this class of problems, the estimate of the source obtained at the decoder cannot be generally reproduced at the encoder if it depends directly on the side information. In some applications, such as the one mentioned above, this lack of consistency may be undesirable, and a so called Common Reconstruction (CR) requirement, whereby one imposes that the encoder be able to agree on the decoder’s estimate, may be instead in order. The rate-distortion region is here derived for some special cases of the cascade source coding problem and of the related Heegard-Berger (HB) problem under the CR constraint. As for point 2), the work is motivated by the fact that, in order to enable, or to facilitate, the exchange of information, nodes of a communication network routinely take various types of actions, such as data acquisition or network probing. For instance, sensor nodes schedule the operation of their sensing devices to measure given physical quantities of interest, and wireless nodes probe the state of the channel via training. The problem of optimal data acquisition is studied for a cascade source coding problem, a distributed source coding problem and a two-way source coding problem assuming that the side information sequences can be controlled via the selection of cost-constrained actions. It is shown that a joint design of the description of the source and of the control signals used to guide the selection of the actions at downstream nodes is generally necessary for an efficient use of the available communication links. Instead, the problem of optimal channel probing is studied for a broadcast channel and a point-to-point link in which the decoder is interested in estimating not only the message, but also the state sequence. Finally, the problem of embedding information on the actions is studied for both the source and the channel coding set-ups described above

Systematic hybrid analog/digital signal coding

Thesis (Ph.D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2000.Includes bibliographical references (p. 201-206).This thesis develops low-latency, low-complexity signal processing solutions for systematic source coding, or source coding with side information at the decoder. We consider an analog source signal transmitted through a hybrid channel that is the composition of two channels: a noisy analog channel through which the source is sent unprocessed and a secondary rate-constrained digital channel; the source is processed prior to transmission through the digital channel. The challenge is to design a digital encoder and decoder that provide a minimum-distortion reconstruction of the source at the decoder, which has observations of analog and digital channel outputs. The methods described in this thesis have importance to a wide array of applications. For example, in the case of in-band on-channel (IBOC) digital audio broadcast (DAB), an existing noisy analog communications infrastructure may be augmented by a low-bandwidth digital side channel for improved fidelity, while compatibility with existing analog receivers is preserved. Another application is a source coding scheme which devotes a fraction of available bandwidth to the analog source and the rest of the bandwidth to a digital representation. This scheme is applicable in a wireless communications environment (or any environment with unknown SNR), where analog transmission has the advantage of a gentle roll-off of fidelity with SNR. A very general paradigm for low-latency, low-complexity source coding is composed of three basic cascaded elements: 1) a space rotation, or transformation, 2) quantization, and 3) lossless bitstream coding. The paradigm has been applied with great success to conventional source coding, and it applies equally well to systematic source coding. Focusing on the case involving a Gaussian source, Gaussian channel and mean-squared distortion, we determine optimal or near-optimal components for each of the three elements, each of which has analogous components in conventional source coding. The space rotation can take many forms such as linear block transforms, lapped transforms, or subband decomposition, all for which we derive conditions of optimality. For a very general case we develop algorithms for the design of locally optimal quantizers. For the Gaussian case, we describe a low-complexity scalar quantizer, the nested lattice scalar quantizer, that has performance very near that of the optimal systematic scalar quantizer. Analogous to entropy coding for conventional source coding, Slepian-Wolf coding is shown to be an effective lossless bitstream coding stage for systematic source coding.by Richard J. Barron.Ph.D