Source Coding Problems with Conditionally Less Noisy Side Information

A computable expression for the rate-distortion (RD) function proposed by Heegard and Berger has eluded information theory for nearly three decades. Heegard and Berger's single-letter achievability bound is well known to be optimal for \emph{physically degraded} side information; however, it is not known whether the bound is optimal for arbitrarily correlated side information (general discrete memoryless sources). In this paper, we consider a new setup in which the side information at one receiver is \emph{conditionally less noisy} than the side information at the other. The new setup includes degraded side information as a special case, and it is motivated by the literature on degraded and less noisy broadcast channels. Our key contribution is a converse proving the optimality of Heegard and Berger's achievability bound in a new setting. The converse rests upon a certain \emph{single-letterization} lemma, which we prove using an information theoretic telescoping identity {recently presented by Kramer}. We also generalise the above ideas to two different successive-refinement problems

Rate-Distortion Function for a Heegard-Berger Problem with Two Sources and Degraded Reconstruction sets

In this work, we investigate an instance of the Heegard-Berger problem with two sources and arbitrarily correlated side information sequences at two decoders, in which the reconstruction sets at the decoders are degraded. Specifically, two sources are to be encoded in a manner that one of the two is reproduced losslessly by both decoders, and the other is reproduced to within some prescribed distortion level at one of the two decoders. We establish a single-letter characterization of the rate-distortion function for this model. The investigation of this result in some special cases also sheds light on the utility of joint compression of the two sources. Furthermore, we also generalize our result to the setting in which the source component that is to be recovered by both users is reconstructed in a lossy fashion, under the requirement that all terminals (i.e., the encoder and both decoders) can share an exact copy of the compressed version of this source component, i.e., a common encoder-decoders reconstruction constraint. For this model as well, we establish a single-letter characterization of the associated rate-distortion function.Comment: Submitted to IEEE Trans. on Information Theor

Successive refinement with conditionally less noisy side information

We consider the successive refinement of information problem with decoder side information. The rate-distortion region is unknown in general; Steinberg &amp; Merhav and Tian &amp; Diggavi solved it in the special case of degraded side information. We extend this special case to a new setup, conditionally less noisy side information, and we give a single-letter solution when one distortion function is deterministic.QC 20140108</p

MAC with Action-Dependent State Information at One Encoder

Problems dealing with the ability to take an action that affects the states of state-dependent communication channels are of timely interest and importance. Therefore, we extend the study of action-dependent channels, which until now focused on point-to-point models, to multiple-access channels (MAC). In this paper, we consider a two-user, state-dependent MAC, in which one of the encoders, called the informed encoder, is allowed to take an action that affects the formation of the channel states. Two independent messages are to be sent through the channel: a common message known to both encoders and a private message known only to the informed encoder. In addition, the informed encoder has access to the sequence of channel states in a non-causal manner. Our framework generalizes previously evaluated settings of state dependent point-to-point channels with actions and MACs with common messages. We derive a single letter characterization of the capacity region for this setting. Using this general result, we obtain and compute the capacity region for the Gaussian action-dependent MAC. The unique methods used in solving the Gaussian case are then applied to obtain the capacity of the Gaussian action-dependent point-to-point channel; a problem was left open until this work. Finally, we establish some dualities between action-dependent channel coding and source coding problems. Specifically, we obtain a duality between the considered MAC setting and the rate distortion model known as "Successive Refinement with Actions". This is done by developing a set of simple duality principles that enable us to successfully evaluate the outcome of one problem given the other.Comment: 1. Parts of this paper appeared in the IEEE International Symposium on Information Theory (ISIT 2012),Cambridge, MA, US, July 2012 and at the IEEE 27th Convention of Electrical and Electronics Engineers in Israel (IEEEI 2012), Nov. 2012. 2. This work has been supported by the CORNET Consortium Israel Ministry for Industry and Commerc

A Rate-Distortion Approach to Index Coding

We approach index coding as a special case of rate-distortion with multiple receivers, each with some side information about the source. Specifically, using techniques developed for the rate-distortion problem, we provide two upper bounds and one lower bound on the optimal index coding rate. The upper bounds involve specific choices of the auxiliary random variables in the best existing scheme for the rate-distortion problem. The lower bound is based on a new lower bound for the general rate-distortion problem. The bounds are shown to coincide for a number of (groupcast) index coding instances, including all instances for which the number of decoders does not exceed three.Comment: Substantially extended version. Submitted to IEEE Transactions on Information Theor

Biometric and Physical Identifiers with Correlated Noise for Controllable Private Authentication

The problem of secret-key based authentication under privacy and storage constraints on the source sequence is considered. The identifier measurement channels during authentication are assumed to be controllable via a cost-constrained action sequence. Single-letter inner and outer bounds for the key-leakage-storage-cost regions are derived for a generalization of a classic two-terminal key agreement model with an eavesdropper that observes a sequence that is correlated with the sequences observed by the legitimate terminals. The additions to the model are that the encoder observes a noisy version of a remote source, and the noisy output and the remote source output together with an action sequence are given as inputs to the measurement channel at the decoder. Thus, correlation is introduced between the noise components on the encoder and decoder measurements. The model with a secret key generated by an encoder is extended to the randomized models, where a secret-key is embedded to the encoder. The results are relevant for several user and device authentication scenarios including physical and biometric identifiers with multiple measurements that provide diversity and multiplexing gains. To illustrate the behavior of the rate region, achievable (secret-key rate, storage-rate, cost) tuples are given for binary identifiers and measurement channels that can be represented as a mixture of binary symmetric subchannels. The gains from using an action sequence such as a large secret-key rate at a significantly small hardware cost, are illustrated to motivate the use of low-complexity transform-coding algorithms with cost-constrained actions.Comment: Shorter version to appear in the IEEE International Symposium on Information Theory 202

Source Coding Problems With Conditionally Less Noisy Side Information

A computable expression for Heegard and Berger's rate-distortion function has eluded information theory for nearly three decades. Heegard and Berger's single-letter achievability bound is well known to be optimal for physically degraded side information; however, it is not known whether the bound is optimal for arbitrarily correlated side information (general discrete memoryless sources). In this paper, we consider a new setup where the side information at one receiver is conditionally less noisy than that at the other. The new setup includes degraded side information as a special case, and it is motivated by the literature on degraded and less noisy broadcast channels. Our key contribution is a converse proving the optimality of Heegard and Berger's achievability bound in a new setting, where the side information is conditionally less noisy and one distortion function is deterministic. The less noisy setup is also generalized to two different successive-refinement problems.QC 20141104</p

Function Computation over Networks:Efficient Information Processing for Cache and Sensor Applications

This thesis looks at efficient information processing for two network applications: content delivery with caching and collecting summary statistics in wireless sensor networks. Both applications are studied under the same paradigm: function computation over networks, where distributed source nodes cooperatively communicate some functions of individual observations to one or multiple destinations. One approach that always works is to convey all observations and then let the destinations compute the desired functions by themselves. However, if the available communication resources are limited, then revealing less unwanted information becomes critical. Centered on this goal, this thesis develops new coding schemes using information-theoretic tools. The first part of this thesis focuses on content delivery with caching. Caching is a technique that facilitates reallocation of communication resources in order to avoid network congestion during peak-traffic times. An information-theoretic model, termed sequential coding for computing, is proposed to analyze the potential gains offered by the caching technique. For the single-user case, the proposed framework succeeds in verifying the optimality of some simple caching strategies and in providing guidance towards optimal caching strategies. For the two-user case, five representative subproblems are considered, which draw connections with classic source coding problems including the Gray-Wyner system, successive refinement, and the Kaspi/Heegard-Berger problem. Afterwards, the problem of distributed computing with successive refinement is considered. It is shown that if full data recovery is required in the second stage of successive refinement, then any information acquired in the first stage will be useful later in the second stage. The second part of this thesis looks at the collection of summary statistics in wireless sensor networks. Summary statistics include arithmetic mean, median, standard deviation, etc, and they belong to the class of symmetric functions. This thesis develops arithmetic computation coding in order to efficiently perform in-network computation for weighted arithmetic sums and symmetric functions. The developed arithmetic computation coding increases the achievable computation rate from $\Theta((\log L)/L)$ to $\Theta(1/\log L)$, where $L$ is the number of sensors. Finally, this thesis demonstrates that interaction among sensors is beneficial for computation of type-threshold functions, e.g., the maximum and the indicator function, and that a non-vanishing computation rate is achievable