Rate-Distortion Function for a Heegard-Berger Problem with Two Sources and Degraded Reconstruction sets

In this work, we investigate an instance of the Heegard-Berger problem with two sources and arbitrarily correlated side information sequences at two decoders, in which the reconstruction sets at the decoders are degraded. Specifically, two sources are to be encoded in a manner that one of the two is reproduced losslessly by both decoders, and the other is reproduced to within some prescribed distortion level at one of the two decoders. We establish a single-letter characterization of the rate-distortion function for this model. The investigation of this result in some special cases also sheds light on the utility of joint compression of the two sources. Furthermore, we also generalize our result to the setting in which the source component that is to be recovered by both users is reconstructed in a lossy fashion, under the requirement that all terminals (i.e., the encoder and both decoders) can share an exact copy of the compressed version of this source component, i.e., a common encoder-decoders reconstruction constraint. For this model as well, we establish a single-letter characterization of the associated rate-distortion function.Comment: Submitted to IEEE Trans. on Information Theor

Rate-Distortion Region of a Gray–Wyner Model with Side Information

In this work, we establish a full single-letter characterization of the rate-distortion region of an instance of the Gray–Wyner model with side information at the decoders. Specifically, in this model, an encoder observes a pair of memoryless, arbitrarily correlated, sources (Sn1,Sn2) and communicates with two receivers over an error-free rate-limited link of capacity R0 , as well as error-free rate-limited individual links of capacities R1 to the first receiver and R2 to the second receiver. Both receivers reproduce the source component Sn2 losslessly; and Receiver 1 also reproduces the source component Sn1 lossily, to within some prescribed fidelity level D1 . In addition, Receiver 1 and Receiver 2 are equipped, respectively, with memoryless side information sequences Yn1 and Yn2 . Important in this setup, the side information sequences are arbitrarily correlated among them, and with the source pair (Sn1,Sn2) ; and are not assumed to exhibit any particular ordering. Furthermore, by specializing the main result to two Heegard–Berger models with successive refinement and scalable coding, we shed light on the roles of the common and private descriptions that the encoder should produce and the role of each of the common and private links. We develop intuitions by analyzing the developed single-letter rate-distortion regions of these models, and discuss some insightful binary examples

Heegard-Berger and Cascade Source Coding Problems with Common Reconstruction Constraints

For the HB problem with the CR constraint, the rate-distortion function is derived under the assumption that the side information sequences are (stochastically) degraded. The rate-distortion function is also calculated explicitly for three examples, namely Gaussian source and side information with quadratic distortion metric, and binary source and side information with erasure and Hamming distortion metrics. The rate-distortion function is then characterized for the HB problem with cooperating decoders and (physically) degraded side information. For the cascade problem with the CR constraint, the rate-distortion region is obtained under the assumption that side information at the final node is physically degraded with respect to that at the intermediate node. For the latter two cases, it is worth emphasizing that the corresponding problem without the CR constraint is still open. Outer and inner bounds on the rate-distortion region are also obtained for the cascade problem under the assumption that the side information at the intermediate node is physically degraded with respect to that at the final node. For the three examples mentioned above, the bounds are shown to coincide. Finally, for the HB problem, the rate-distortion function is obtained under the more general requirement of constrained reconstruction, whereby the decoder's estimate must be recovered at the encoder only within some distortion.Comment: to appear in IEEE Trans. Inform. Theor

Adaptive data acquisition for communication networks

In an increasing number of communication systems, such as sensor networks or local area networks within medical, financial or military institutions, nodes communicate information sources (e.g., video, audio) over multiple hops. Moreover, nodes have, or can acquire, correlated information sources from the environment, e.g., from data bases or from measurements. Among the new design problems raised by the outlined scenarios, two key issues are addressed in this dissertation: 1) How to preserve the consistency of sensitive information across multiple hops; 2) How to incorporate the design of actuation in the form of data acquisition and network probing in the optimization of the communication network. These aspects are investigated by using information-theoretic (source and channel coding) models, obtaining fundamental insights that have been corroborated by various illustrative examples. To address point 1), the problem of cascade source coding with side information is investigated. The motivating observation is that, in this class of problems, the estimate of the source obtained at the decoder cannot be generally reproduced at the encoder if it depends directly on the side information. In some applications, such as the one mentioned above, this lack of consistency may be undesirable, and a so called Common Reconstruction (CR) requirement, whereby one imposes that the encoder be able to agree on the decoder’s estimate, may be instead in order. The rate-distortion region is here derived for some special cases of the cascade source coding problem and of the related Heegard-Berger (HB) problem under the CR constraint. As for point 2), the work is motivated by the fact that, in order to enable, or to facilitate, the exchange of information, nodes of a communication network routinely take various types of actions, such as data acquisition or network probing. For instance, sensor nodes schedule the operation of their sensing devices to measure given physical quantities of interest, and wireless nodes probe the state of the channel via training. The problem of optimal data acquisition is studied for a cascade source coding problem, a distributed source coding problem and a two-way source coding problem assuming that the side information sequences can be controlled via the selection of cost-constrained actions. It is shown that a joint design of the description of the source and of the control signals used to guide the selection of the actions at downstream nodes is generally necessary for an efficient use of the available communication links. Instead, the problem of optimal channel probing is studied for a broadcast channel and a point-to-point link in which the decoder is interested in estimating not only the message, but also the state sequence. Finally, the problem of embedding information on the actions is studied for both the source and the channel coding set-ups described above

MAC with Action-Dependent State Information at One Encoder

Problems dealing with the ability to take an action that affects the states of state-dependent communication channels are of timely interest and importance. Therefore, we extend the study of action-dependent channels, which until now focused on point-to-point models, to multiple-access channels (MAC). In this paper, we consider a two-user, state-dependent MAC, in which one of the encoders, called the informed encoder, is allowed to take an action that affects the formation of the channel states. Two independent messages are to be sent through the channel: a common message known to both encoders and a private message known only to the informed encoder. In addition, the informed encoder has access to the sequence of channel states in a non-causal manner. Our framework generalizes previously evaluated settings of state dependent point-to-point channels with actions and MACs with common messages. We derive a single letter characterization of the capacity region for this setting. Using this general result, we obtain and compute the capacity region for the Gaussian action-dependent MAC. The unique methods used in solving the Gaussian case are then applied to obtain the capacity of the Gaussian action-dependent point-to-point channel; a problem was left open until this work. Finally, we establish some dualities between action-dependent channel coding and source coding problems. Specifically, we obtain a duality between the considered MAC setting and the rate distortion model known as "Successive Refinement with Actions". This is done by developing a set of simple duality principles that enable us to successfully evaluate the outcome of one problem given the other.Comment: 1. Parts of this paper appeared in the IEEE International Symposium on Information Theory (ISIT 2012),Cambridge, MA, US, July 2012 and at the IEEE 27th Convention of Electrical and Electronics Engineers in Israel (IEEEI 2012), Nov. 2012. 2. This work has been supported by the CORNET Consortium Israel Ministry for Industry and Commerc

Distributed Binary Detection with Lossy Data Compression

Consider the problem where a statistician in a two-node system receives rate-limited information from a transmitter about marginal observations of a memoryless process generated from two possible distributions. Using its own observations, this receiver is required to first identify the legitimacy of its sender by declaring the joint distribution of the process, and then depending on such authentication it generates the adequate reconstruction of the observations satisfying an average per-letter distortion. The performance of this setup is investigated through the corresponding rate-error-distortion region describing the trade-off between: the communication rate, the error exponent induced by the detection and the distortion incurred by the source reconstruction. In the special case of testing against independence, where the alternative hypothesis implies that the sources are independent, the optimal rate-error-distortion region is characterized. An application example to binary symmetric sources is given subsequently and the explicit expression for the rate-error-distortion region is provided as well. The case of "general hypotheses" is also investigated. A new achievable rate-error-distortion region is derived based on the use of non-asymptotic binning, improving the quality of communicated descriptions. Further improvement of performance in the general case is shown to be possible when the requirement of source reconstruction is relaxed, which stands in contrast to the case of general hypotheses.Comment: to appear on IEEE Trans. Information Theor

Rate-Distortion with Side-Information at Many Decoders

We present a new inner bound for the rate region of the $t$-stage successive-refinement problem with side-information. We also present a new upper bound for the rate-distortion function for lossy-source coding with multiple decoders and side-information. Characterising this rate-distortion function is a long-standing open problem, and it is widely believed that the tightest upper bound is provided by Theorem 2 of Heegard and Berger's paper "Rate Distortion when Side Information may be Absent", \emph{IEEE Trans. Inform. Theory}, 1985. We give a counterexample to Heegard and Berger's result.Comment: 36 pages. Submitted to IEEE Transactions on Information Theory. In proc. ISIT 2010