Network Information Flow with Correlated Sources

In this paper, we consider a network communications problem in which multiple correlated sources must be delivered to a single data collector node, over a network of noisy independent point-to-point channels. We prove that perfect reconstruction of all the sources at the sink is possible if and only if, for all partitions of the network nodes into two subsets S and S^c such that the sink is always in S^c, we have that H(U_S|U_{S^c}) < \sum_{i\in S,j\in S^c} C_{ij}. Our main finding is that in this setup a general source/channel separation theorem holds, and that Shannon information behaves as a classical network flow, identical in nature to the flow of water in pipes. At first glance, it might seem surprising that separation holds in a fairly general network situation like the one we study. A closer look, however, reveals that the reason for this is that our model allows only for independent point-to-point channels between pairs of nodes, and not multiple-access and/or broadcast channels, for which separation is well known not to hold. This ``information as flow'' view provides an algorithmic interpretation for our results, among which perhaps the most important one is the optimality of implementing codes using a layered protocol stack.Comment: Final version, to appear in the IEEE Transactions on Information Theory -- contains (very) minor changes based on the last round of review

Zero-error Slepian-Wolf Coding of Confined Correlated Sources with Deviation Symmetry

In this paper, we use linear codes to study zero-error Slepian-Wolf coding of a set of sources with deviation symmetry, where the sources are generalization of the Hamming sources over an arbitrary field. We extend our previous codes, Generalized Hamming Codes for Multiple Sources, to Matrix Partition Codes and use the latter to efficiently compress the target sources. We further show that every perfect or linear-optimal code is a Matrix Partition Code. We also present some conditions when Matrix Partition Codes are perfect and/or linear-optimal. Detail discussions of Matrix Partition Codes on Hamming sources are given at last as examples.Comment: submitted to IEEE Trans Information Theor

Distributed Structure: Joint Expurgation for the Multiple-Access Channel

In this work we show how an improved lower bound to the error exponent of the memoryless multiple-access (MAC) channel is attained via the use of linear codes, thus demonstrating that structure can be beneficial even in cases where there is no capacity gain. We show that if the MAC channel is modulo-additive, then any error probability, and hence any error exponent, achievable by a linear code for the corresponding single-user channel, is also achievable for the MAC channel. Specifically, for an alphabet of prime cardinality, where linear codes achieve the best known exponents in the single-user setting and the optimal exponent above the critical rate, this performance carries over to the MAC setting. At least at low rates, where expurgation is needed, our approach strictly improves performance over previous results, where expurgation was used at most for one of the users. Even when the MAC channel is not additive, it may be transformed into such a channel. While the transformation is lossy, we show that the distributed structure gain in some "nearly additive" cases outweighs the loss, and thus the error exponent can improve upon the best known error exponent for these cases as well. Finally we apply a similar approach to the Gaussian MAC channel. We obtain an improvement over the best known achievable exponent, given by Gallager, for certain rate pairs, using lattice codes which satisfy a nesting condition.Comment: Submitted to the IEEE Trans. Info. Theor

Side-information Scalable Source Coding

The problem of side-information scalable (SI-scalable) source coding is considered in this work, where the encoder constructs a progressive description, such that the receiver with high quality side information will be able to truncate the bitstream and reconstruct in the rate distortion sense, while the receiver with low quality side information will have to receive further data in order to decode. We provide inner and outer bounds for general discrete memoryless sources. The achievable region is shown to be tight for the case that either of the decoders requires a lossless reconstruction, as well as the case with degraded deterministic distortion measures. Furthermore we show that the gap between the achievable region and the outer bounds can be bounded by a constant when square error distortion measure is used. The notion of perfectly scalable coding is introduced as both the stages operate on the Wyner-Ziv bound, and necessary and sufficient conditions are given for sources satisfying a mild support condition. Using SI-scalable coding and successive refinement Wyner-Ziv coding as basic building blocks, a complete characterization is provided for the important quadratic Gaussian source with multiple jointly Gaussian side-informations, where the side information quality does not have to be monotonic along the scalable coding order. Partial result is provided for the doubly symmetric binary source with Hamming distortion when the worse side information is a constant, for which one of the outer bound is strictly tighter than the other one.Comment: 35 pages, submitted to IEEE Transaction on Information Theor

Security for correlated sources across wiretap network

A thesis submitted in ful llment of the requirements for the degree of Doctor of Philosophy in the School of Electrical and Information Engineering Faculty of Engineering University of the Witwatersrand July 2015This thesis presents research conducted for the security aspects of correlated sources across a wiretap network. Correlated sources are present in communication systems where protocols ensure that there is some predetermined information for sources to transmit. Systems that contain correlated sources are for example broadcast channels, smart grid systems, wireless sensor networks and social media networks. In these systems there exist common information between the nodes in a network, which gives rise to security risks as common information can be determined about more than one source. In this work the security aspects of correlated sources are investigated. Correlated source coding in terms of the Slepian-Wolf theorem is investigated to determine the amount of information leakage for various correlated source models. The perfect secrecy approach developed by Shannon has also been incorporated as a security approach. In order to explore these security aspects the techniques employed range from typical sequences used to prove Slepian-Wolf's theorem to coding methods incorporating matrix partitions for correlated sources. A generalized correlated source model is presented and the procedure to determine the information leakage is initially illustrated using this model. A novel scenario for two correlated sources across a channel with eavesdroppers is also investigated. It is a basic model catering for the correlated source applications that have been detailed. The information leakage quanti cation is provided, where bounds specify the quantity of information leaked for various cases of eavesdropped channel information. The required transmission rates for perfect secrecy when some channel information has been wiretapped is further determined, followed by a method to reduce the key length required for perfect secrecy. The implementation thereafter provided shows how the information leakage is determined practically. In the same way using the information leakage quanti cation, Shannon's cipher system approach and practical implementation a novel two correlated source model where channel information and some source data symbols (predetermined information) are wiretapped is investigated. The adversary in this situation has access to more information than if a link is wiretapped only and can thus determine more about a particular source. This scenario caters for an application where the eavesdropper has access to some predetermined information. The security aspects and coding implementation have further been developed for a novel correlated source model with a heterogeneous encoding method. The model caters for situations where a wiretapper is able to easily access a particular source. iii The interesting link between information theory and coding theory is explored for the novel models presented in this research. A matrix partition method is utilized and the information leakage for various cases of wiretapped syndromes are presented. The research explores the security for correlated sources in the presence of wiretappers. Both the information leakage and Shannon's cipher system approach are used to achieve these security aspects. The implementation shows the practicality of using these security aspects in communications systems. The research contained herein is signi cant as evident from the various applications it may be used for and to the author's knowledge is novel