Synthesis of Gaussian Trees with Correlation Sign Ambiguity: An Information Theoretic Approach

In latent Gaussian trees the pairwise correlation signs between the variables are intrinsically unrecoverable. Such information is vital since it completely determines the direction in which two variables are associated. In this work, we resort to information theoretical approaches to achieve two fundamental goals: First, we quantify the amount of information loss due to unrecoverable sign information. Second, we show the importance of such information in determining the maximum achievable rate region, in which the observed output vector can be synthesized, given its probability density function. In particular, we model the graphical model as a communication channel and propose a new layered encoding framework to synthesize observed data using upper layer Gaussian inputs and independent Bernoulli correlation sign inputs from each layer. We find the achievable rate region for the rate tuples of multi-layer latent Gaussian messages to synthesize the desired observables.Comment: 14 pages, 9 figures, part of this work is submitted to Allerton 2016 conference, UIUC, IL, US

Information Theoretic Study of Gaussian Graphical Models and Their Applications

In many problems we are dealing with characterizing a behavior of a complex stochastic system or its response to a set of particular inputs. Such problems span over several topics such as machine learning, complex networks, e.g., social or communication networks; biology, etc. Probabilistic graphical models (PGMs) are powerful tools that offer a compact modeling of complex systems. They are designed to capture the random behavior, i.e., the joint distribution of the system to the best possible accuracy. Our goal is to study certain algebraic and topological properties of a special class of graphical models, known as Gaussian graphs. First, we show that how Gaussian trees can be used to determine a particular complex system\u27s random behavior, i.e., determining a security robustness of a public communication channel characterized by a Gaussian tree. We show that in such public channels the secrecy capacity of the legitimate users Alice and Bob, in the presence of a passive adversary Eve, is strongly dependent on the underlying structure of the channel. This is done by defining a relevant privacy metric to capture the secrecy capacity of a communication and studying topological and algebraic features of a given Gaussian tree to quantify its security robustness. Next, we examine on how one can effectively produce random samples from such Gaussian tree. The primary concern in synthesis problems is about efficiency in terms of the amount of random bits required for synthesis, as well as the modeling complexity of the given stochastic system through which the Gaussian vector is synthesized. This is done through an optimization problem to propose an efficient algorithm by which we can effectively generate such random vectors. We further generalize the optimization formulation from Gaussian trees to Gaussian vectors with arbitrary structures. This is done by introducing a new latent factor model obtained by solving a constrained minimum determinant factor analysis (CMDFA) problem. We discuss the benefits of factor models in machine learning applications and in particular 3D image reconstruction problems, where our newly proposed CMDFA problem may be beneficial

Topological and algebraic properties for classifying unrooted Gaussian trees under privacy constraints

In this paper, our objective is to find out how topological and algebraic properties of unrooted Gaussian tree models determine their security robustness, which is measured by our proposed max-min information (MaMI) metric. Such metric quantifies the amount of common randomness extractable through public discussion between two legitimate nodes under an eavesdropper attack. We show some general topological properties that the desired max-min solutions shall satisfy. Under such properties, we develop conditions under which comparable trees are put together to form partially ordered sets (posets). Each poset contains the most favorable structure as the poset leader, and the least favorable structure. Then, we compute the Tutte-like polynomial for each tree in a poset in order to assign a polynomial to any tree in a poset. Moreover, we propose a novel method, based on restricted integer partitions, to effectively enumerate all poset leaders. The results not only help us understand the security strength of different Gaussian trees, which is critical when we evaluate the information leakage issues for various jointly Gaussian distributed measurements in networks, but also provide us both an algebraic and a topological perspective in grasping some fundamental properties of such models