A new sufficient condition for sum-rate tightness in quadratic Gaussian multiterminal source coding

This work considers the quadratic Gaussian multiterminal (MT) source coding problem and provides a new sufficient condition for the Berger-Tung sum-rate bound to be tight. The converse proof utilizes a set of virtual remote sources given which the MT sources are block independent with a maximum block size of two. The given MT source coding problem is then related to a set of two-terminal problems with matrix-distortion constraints, for which a new lower bound on the sum-rate is given. Finally, a convex optimization problem is formulated and a sufficient condition derived for the optimal BT scheme to satisfy the subgradient based Karush-Kuhn-Tucker condition. The set of sum-rate tightness problems defined by our new sufficient condition subsumes all previously known tight cases, and opens new direction for a more general partial solution

The Non-Regular CEO Problem

We consider the CEO problem for non-regular source distributions (such as uniform or truncated Gaussian). A group of agents observe independently corrupted versions of data and transmit coded versions over rate-limited links to a CEO. The CEO then estimates the underlying data based on the received coded observations. Agents are not allowed to convene before transmitting their observations. This formulation is motivated by the practical problem of a firm's CEO estimating (non-regular) beliefs about a sequence of events, before acting on them. Agents' observations are modeled as jointly distributed with the underlying data through a given conditional probability density function. We study the asymptotic behavior of the minimum achievable mean squared error distortion at the CEO in the limit when the number of agents $L$ and the sum rate $R$ tend to infinity. We establish a $1/R^2$ convergence of the distortion, an intermediate regime of performance between the exponential behavior in discrete CEO problems [Berger, Zhang, and Viswanathan (1996)], and the $1/R$ behavior in Gaussian CEO problems [Viswanathan and Berger (1997)]. Achievability is proved by a layered architecture with scalar quantization, distributed entropy coding, and midrange estimation. The converse is proved using the Bayesian Chazan-Zakai-Ziv bound.Comment: 18 pages, 1 figur

Distributed Source Coding of Correlated Gaussian Sources

We consider the distributed source coding system of $L$ correlated Gaussian sources $Y_i,i=1,2,...,L$ which are noisy observations of correlated Gaussian remote sources $X_k, k=1,2,...,K$. We assume that $Y^{L}={}^{\rm t}(Y_1,Y_2,$ $..., Y_L)$ is an observation of the source vector $X^K={}^{\rm t}(X_1,X_2,..., X_K)$, having the form $Y^L=AX^K+N^L$, where $A$ is a $L\times K$ matrix and $N^L={}^{\rm t}(N_1,N_2,...,N_L)$ is a vector of $L$ independent Gaussian random variables also independent of $X^K$. In this system $L$ correlated Gaussian observations are separately compressed by $L$ encoders and sent to the information processing center. We study the remote source coding problem where the decoder at the center attempts to reconstruct the remote source $X^K$. We consider three distortion criteria based on the covariance matrix of the estimation error on $X^K$. For each of those three criteria we derive explicit inner and outer bounds of the rate distortion region. Next, in the case of $K=L$ and $A=I_L$, we study the multiterminal source coding problem where the decoder wishes to reconstruct the observation $Y^L=X^L+N^L$. To investigate this problem we shall establish a result which provides a strong connection between the remote source coding problem and the multiterminal source coding problem. Using this result, we drive several new partial solutions to the multiterminal source coding problem.Comment: 30 pages 4 figure

Vector Gaussian CEO Problem Under Logarithmic Loss and Applications

We study the vector Gaussian Chief Executive Officer (CEO) problem under logarithmic loss distortion measure. Specifically, $K \geq 2$ agents observe independently corrupted Gaussian noisy versions of a remote vector Gaussian source, and communicate independently with a decoder or CEO over rate-constrained noise-free links. The CEO also has its own Gaussian noisy observation of the source and wants to reconstruct the remote source to within some prescribed distortion level where the incurred distortion is measured under the logarithmic loss penalty criterion. We find an explicit characterization of the rate-distortion region of this model. The result can be seen as the counterpart to the vector Gaussian setting of that by Courtade-Weissman which provides the rate-distortion region of the model in the discrete memoryless setting. For the proof of this result, we obtain an outer bound by means of a technique that relies on the de Bruijn identity and the properties of Fisher information. The approach is similar to Ekrem-Ulukus outer bounding technique for the vector Gaussian CEO problem under quadratic distortion measure, for which it was there found generally non-tight; but it is shown here to yield a complete characterization of the region for the case of logarithmic loss measure. Also, we show that Gaussian test channels with time-sharing exhaust the Berger-Tung inner bound, which is optimal. Furthermore, application of our results allows us to find the complete solutions of two related problems: a quadratic vector Gaussian CEO problem with determinant constraint and the vector Gaussian distributed Information Bottleneck problem. Finally, we develop Blahut-Arimoto type algorithms that allow to compute numerically the regions provided in this paper, for both discrete and Gaussian models. We illustrate the efficiency of our algorithms through some numerical examples.Comment: accepted for publication in IEEE Transactions on Information Theor

Rate Region of the Quadratic Gaussian Two-Encoder Source-Coding Problem

We determine the rate region of the quadratic Gaussian two-encoder source-coding problem. This rate region is achieved by a simple architecture that separates the analog and digital aspects of the compression. Furthermore, this architecture requires higher rates to send a Gaussian source than it does to send any other source with the same covariance. Our techniques can also be used to determine the sum rate of some generalizations of this classical problem. Our approach involves coupling the problem to a quadratic Gaussian ``CEO problem.''Comment: Contains additional results. To appear in IEEE Trans. Inf. Theor

Robust Distributed Source Coding

We consider a distributed source coding system in which several observations are communicated to the decoder using limited transmission rate. The observations must be separately coded. We introduce a robust distributed coding scheme which flexibly trades off between system robustness and compression efficiency. The optimality of this coding scheme is proved for various special cases.Comment: 40 pages, submitted to the IEEE Transactions on Information Theor

An Infeasibility Result for the Multiterminal Source-Coding Problem

We prove a new outer bound on the rate-distortion region for the multiterminal source-coding problem. This bound subsumes the best outer bound in the literature and improves upon it strictly in some cases. The improved bound enables us to obtain a new, conclusive result for the binary erasure version of the "CEO problem." The bound recovers many of the converse results that have been established for special cases of the problem, including the recent one for the Gaussian version of the CEO problem.Comment: 42 pages; submitted to IEEE Trans. Inf. Theor

On the Vacationing CEO Problem: Achievable Rates and Outer Bounds

This paper studies a class of source coding problems that combines elements of the CEO problem with the multiple description problem. In this setting, noisy versions of one remote source are observed by two nodes with encoders (which is similar to the CEO problem). However, it differs from the CEO problem in that each node must generate multiple descriptions of the source. This problem is of interest in multiple scenarios in efficient communication over networks. In this paper, an achievable region and an outer bound are presented for this problem, which is shown to be sum rate optimal for a class of distortion constraints.Comment: 19 pages, 1 figure, ISIT 201

The CEO Problem with rrth Power of Difference and Logarithmic Distortions

The CEO problem has received much attention since first introduced by Berger et al., but there are limited results on non-Gaussian models with non-quadratic distortion measures. In this work, we extend the quadratic Gaussian CEO problem to two continuous alphabet settings with general $r$th power of difference distortion, and study asymptotics of distortion decay as the number of agents and sum rate, $R_{\textsf{sum}}$, grow without bound, while individual rates vanish. The first setting is a regular source-observation model, such as jointly Gaussian, with difference distortion and we establish that the distortion decays at $\mathcal{O}(R_{\textsf{sum}}^{-r/2})$ when $r \ge 2$. We use sample median estimation after the Berger-Tung scheme for achievability. The other setting is a non-regular source-observation model, such as copula or uniform additive noise models, with difference distortion for which estimation-theoretic regularity conditions do not hold. The decay $\mathcal{O}(R_{\textsf{sum}}^{-r})$ when $r \ge 1$ is obtained for the non-regular model by midrange estimator following the Berger-Tung scheme. We also provide converses based on the Shannon lower bound for the regular model, and the Chazan-Zakai-Ziv bound for the non-regular model. Interestingly, the regular model converse when $r=2$ recovers the Viswanathan-Berger converse and is thus tight. Lastly, we provide a sufficient condition for the regular model, under which quadratic and logarithmic distortions are asymptotically equivalent by an entropy power relationship as the number of agents grows. This proof relies on the Bernstein-von Mises theorem

Distributed Source Coding of Correlated Gaussian Remote Sources

We consider the distributed source coding system for $L$ correlated Gaussian observations $Y_i, i=1,2, ..., L$. Let $X_i,i=1,2, ..., L$ be $L$ correlated Gaussian random variables and $N_i,$ $i=1,2,... L$ be independent additive Gaussian noises also independent of $X_i, i=1,2,..., L$. We consider the case where for each $i=1,2,..., L$, $Y_i$ is a noisy observation of $X_i$, that is, $Y_i=X_i+N_i$. On this coding system the determination problem of the rate distortion region remains open. In this paper, we derive explicit outer and inner bounds of the rate distortion region. We further find an explicit sufficient condition for those two to match. We also study the sum rate part of the rate distortion region when the correlation has some symmetrical property and derive a new lower bound of the sum rate part. We derive a sufficient condition for this lower bound to be tight. The derived sufficient condition depends only on the correlation property of the sources and their observations.Comment: 20 pages,3 figre