Broadcast Caching Networks with Two Receivers and Multiple Correlated Sources

The correlation among the content distributed across a cache-aided broadcast network can be exploited to reduce the delivery load on the shared wireless link. This paper considers a two-user three-file network with correlated content, and studies its fundamental limits for the worst-case demand. A class of achievable schemes based on a two-step source coding approach is proposed. Library files are first compressed using Gray-Wyner source coding, and then cached and delivered using a combination of correlation-unaware cache-aided coded multicast schemes. The second step is interesting in its own right and considers a multiple-request caching problem, whose solution requires coding in the placement phase. A lower bound on the optimal peak rate-memory trade-off is derived, which is used to evaluate the performance of the proposed scheme. It is shown that for symmetric sources the two-step strategy achieves the lower bound for large cache capacities, and it is within half of the joint entropy of two of the sources conditioned on the third source for all other cache sizes.Comment: in Proceedings of Asilomar Conference on Signals, Systems and Computers, Pacific Grove, California, November 201

Secure Cascade Channel Synthesis

We consider the problem of generating correlated random variables in a distributed fashion, where communication is constrained to a cascade network. The first node in the cascade observes an i.i.d. sequence $X^n$ locally before initiating communication along the cascade. All nodes share bits of common randomness that are independent of $X^n$. We consider secure synthesis - random variables produced by the system appear to be appropriately correlated and i.i.d. even to an eavesdropper who is cognizant of the communication transmissions. We characterize the optimal tradeoff between the amount of common randomness used and the required rates of communication. We find that not only does common randomness help, its usage exceeds the communication rate requirements. The most efficient scheme is based on a superposition codebook, with the first node selecting messages for all downstream nodes. We also provide a fleeting view of related problems, demonstrating how the optimal rate region may shrink or expand.Comment: Submitted to IEEE Transactions on Information Theor

Multi-User Privacy: The Gray-Wyner System and Generalized Common Information

The problem of preserving privacy when a multivariate source is required to be revealed partially to multiple users is modeled as a Gray-Wyner source coding problem with K correlated sources at the encoder and K decoders in which the kth decoder, k = 1, 2, ...,K, losslessly reconstructs the kth source via a common link and a private link. The privacy requirement of keeping each decoder oblivious of all sources other than the one intended for it is introduced via an equivocation constraint at each decoder such that the total equivocation summed over all decoders is E. The set of achievable rates-equivocation tuples is completely characterized. Using this characterization, two different definitions of common information are presented and are shown to be equivalent.Comment: accepted for publication and presentation at ISIT 201

On the Optimality of Secret Key Agreement via Omniscience

For the multiterminal secret key agreement problem under a private source model, it is known that the maximum key rate, i.e., the secrecy capacity, can be achieved through communication for omniscience, but the omniscience strategy can be strictly suboptimal in terms of minimizing the public discussion rate. While a single-letter characterization is not known for the minimum discussion rate needed for achieving the secrecy capacity, we derive single-letter lower and upper bounds that yield some simple conditions for omniscience to be discussion-rate optimal. These conditions turn out to be enough to deduce the optimality of omniscience for a large class of sources including the hypergraphical sources. Through conjectures and examples, we explore other source models to which our methods do not easily extend

Quantifying synergistic mutual information

Quantifying cooperation or synergy among random variables in predicting a single target random variable is an important problem in many complex systems. We review three prior information-theoretic measures of synergy and introduce a novel synergy measure defined as the difference between the whole and the union of its parts. We apply all four measures against a suite of binary circuits to demonstrate that our measure alone quantifies the intuitive concept of synergy across all examples. We show that for our measure of synergy that independent predictors can have positive redundant information.Comment: 15 pages; 12 page appendix. Lots of figures. Guided Self Organization: Inception. Ed: Mikhail Prokopenko. (2014); ISBN 978-3-642-53734-

Information Theoretic Study of Gaussian Graphical Models and Their Applications

In many problems we are dealing with characterizing a behavior of a complex stochastic system or its response to a set of particular inputs. Such problems span over several topics such as machine learning, complex networks, e.g., social or communication networks; biology, etc. Probabilistic graphical models (PGMs) are powerful tools that offer a compact modeling of complex systems. They are designed to capture the random behavior, i.e., the joint distribution of the system to the best possible accuracy. Our goal is to study certain algebraic and topological properties of a special class of graphical models, known as Gaussian graphs. First, we show that how Gaussian trees can be used to determine a particular complex system\u27s random behavior, i.e., determining a security robustness of a public communication channel characterized by a Gaussian tree. We show that in such public channels the secrecy capacity of the legitimate users Alice and Bob, in the presence of a passive adversary Eve, is strongly dependent on the underlying structure of the channel. This is done by defining a relevant privacy metric to capture the secrecy capacity of a communication and studying topological and algebraic features of a given Gaussian tree to quantify its security robustness. Next, we examine on how one can effectively produce random samples from such Gaussian tree. The primary concern in synthesis problems is about efficiency in terms of the amount of random bits required for synthesis, as well as the modeling complexity of the given stochastic system through which the Gaussian vector is synthesized. This is done through an optimization problem to propose an efficient algorithm by which we can effectively generate such random vectors. We further generalize the optimization formulation from Gaussian trees to Gaussian vectors with arbitrary structures. This is done by introducing a new latent factor model obtained by solving a constrained minimum determinant factor analysis (CMDFA) problem. We discuss the benefits of factor models in machine learning applications and in particular 3D image reconstruction problems, where our newly proposed CMDFA problem may be beneficial

The Common Information for N Dependent Random Variables

This paper generalizes Wyner’s definition of common information of a pair or random variables to that of N random variables. We prove coding theorems that show the same operational meanings for the common information of two random variables generalize to that of N random variables. As a byproduct of our proof, we show that the Gray-Wyner source coding network can be generalized to N source sequences with N decoders. We also establish a monotone property of Wyner’s common information which is in contrast to other notions of the common information, specifically Shannon’s mutual information and Gács and Körner’s common randomness. Examples about the computation of Wyner’s common information of N random variables are also given