On the Continuity of Achievable Rate Regions for Source Coding over Networks

The continuity property of achievable rate regions for source coding over networks is considered. We show rate- distortion regions are continuous with respect to distortion vectors. Then we focus on the continuity of lossless rate regions with respect to source distribution: First, the proof of continuity for general networks with independent sources is given; then, for the case of dependent sources, continuity is proven both in examples where one-letter characterizations are known and in examples where one-letter characterizations are not known; the proofs in the latter case rely on the concavity of the rate regions for those networks

On Approximating the Rate Region for Source Coding with Coded Side Information

The achievable rate region for the problem of lossless source coding with coded side information was derived by Ahlswede and Körner in 1975. While the Ahlswede-Körner bound completely characterizes the achievable rate region when the source and side information are memoryless, calculating this bound for a given memoryless joint probability mass function on the source and side information requires an optimization over all possible auxiliary random variables meeting a given Markov condition and alphabet size constraint. This optimization turns out to be surprisingly difficult even for very simple distributions on the source and side information. We here propose a (1 + ε)-approximation algorithm for the given rate region. The proposed technique involves quantization of a space of conditional distributions followed by linear programming. The resulting algorithm guarantees performance within a multiplicative factor (1 + ε) of the optimal performance - even when that optimal performance is unknown

A Continuity Theory for Lossless Source Coding over Networks

A continuity theory of lossless source coding over networks is established and its implications are investigated. In the given model, source and side-information random variables X and Y have finite alphabets, and the input sequences are drawn i.i.d. according to a generic distribution P_(X,Y) on (X,Y). We consider traditional source coding, where all demands equal source random variables. We define a family of lossless source coding problems that includes prior example network source coding problems as special cases. We show that the lossless rate region R_L(P_(X,Y)) is inner semi-continuous in P_(X,Y). We further show that for a special type of networks called super-source networks, where there is a super source node v* that has access to (X,Y) and any other node with access to some source random variable X_i is directly connected to v*, R_L(P_(X,Y)) is also outer semi-continuous in P_(X,Y). Based on the continuity of super-source networks with respect to P_(X,Y), we conjecture that R_L(P_(X,Y)) is also outer semi-continuous and therefore continuous in P_(X,Y) for general networks

On Network Coding of Independent and Dependent Sources in Line Networks

We investigate the network coding capacity for line networks. For independent sources and a special class of dependent sources, we fully characterize the capacity region of line networks for all possible demand structures (e.g., multiple unicast, mixtures of unicasts and multicasts, etc.) Our achievability bound is derived by first decomposing a line network into single-demand components and then adding the component rate regions to get rates for the parent network. For general dependent sources, we give an achievability result and provide examples where the result is and is not tight

On Source Coding with Coded Side Information for a Binary Source with Binary Side Information

The lossless rate region for the coded side information problem is "solved," but its solution is expressed in terms of an auxiliary random variable. As a result, finding the rate region for any fixed example requires an optimization over a family of allowed auxiliary random variables. While intuitive constructions are easy to come by and optimal solutions are known under some special conditions, proving the optimal solution is surprisingly difficult even for examples as basic as a binary source with binary side information. We derive the optimal auxiliary random variables and corresponding achievable rate regions for a family of problems where both the source and side information are binary. Our solution involves first tightening known bounds on the alphabet size of the auxiliary random variable and then optimizing the auxiliary random variable subject to this constraint. The technique used to tighten the bound on the alphabet size applies to a variety of problems beyond the one studied here

A strong converse for a collection of network source coding problems

We prove a strong converse for particular source coding problems: the Ahlswede-Korner (coded side information) problem, lossless source coding for multicast networks with side-information at the end nodes, and the Gray-Wyner problem. Source and side-information sequences are drawn i.i.d. according to a given distribution on a finite alphabet. The strong converse discussed here states that when a given rate vector R is not D-achievable, the probability of observing distortion D for any sequence of block codes at rate R must decrease exponentially to 0 as the block length grows without bound. This strong converse implies the prior strong converses for the point-to-point network, Slepian-Wolf problem, and Ahlswede-Korner (coded side information) problem

On Achievable Rate Regions for Source Coding Over Networks

In the field of source coding over networks, a central goal is to understand the best possible performance for compressing and transmitting dependent data distributed over a network. The achievable rate region for such a network describes all link capacities that suffice to satisfy the reproduction demands. Here all the links in the networks are error-free, the data dependency is given by a joint distribution of the source random variables, and the source sequences are drawn i.i.d. according to the given source distribution. In this thesis, I study the achievable rate regions for general networks, deriving new properties for the rate regions of general network source coding problems, developing approximation algorithms to calculate these regions for particular examples, and deriving bounds on the regions for basic multi-hop and multi-path examples. In the first part, I define a family of network source coding problems. That family contains all of the example networks in the literature as special cases. For the given family, I investigate abstract properties of the achievable rate regions for general networks. These properties include (1) continuity of the achievable rate regions with respect to both the source distribution and the distortion constraint vector and (2) a strong converse that implies the traditional strong converse. Those properties might be useful for studying a long-standing open question: whether a single-letter characterization of a given achievable rate region always exists. In the second part, I develop a family of algorithms to approximate the achievable rate regions for some example network source coding problems based on their single-letter characterizations by using linear programming tools. Those examples contain (1) the lossless coded side information problem by Ahlswede and Korner, (2) the Wyner-Ziv rate-distortion function, and (3) the Berger et al. bound for the lossy coded side information problem. The algorithms may apply more widely to other examples. In the third part, I study two basic networks of different types: the two-hop and the diamond networks. The two-hop network is a basic example of line networks with single relay node on the path from the source to the destination, and the diamond network is a basic example of multi-path networks that has two paths from the source to the destination, where each of the paths contains a relay node. I derive performance bounds for the achievable rate regions for these two networks.</p