Information Theoretic Study of Gaussian Graphical Models and Their Applications

In many problems we are dealing with characterizing a behavior of a complex stochastic system or its response to a set of particular inputs. Such problems span over several topics such as machine learning, complex networks, e.g., social or communication networks; biology, etc. Probabilistic graphical models (PGMs) are powerful tools that offer a compact modeling of complex systems. They are designed to capture the random behavior, i.e., the joint distribution of the system to the best possible accuracy. Our goal is to study certain algebraic and topological properties of a special class of graphical models, known as Gaussian graphs. First, we show that how Gaussian trees can be used to determine a particular complex system\u27s random behavior, i.e., determining a security robustness of a public communication channel characterized by a Gaussian tree. We show that in such public channels the secrecy capacity of the legitimate users Alice and Bob, in the presence of a passive adversary Eve, is strongly dependent on the underlying structure of the channel. This is done by defining a relevant privacy metric to capture the secrecy capacity of a communication and studying topological and algebraic features of a given Gaussian tree to quantify its security robustness. Next, we examine on how one can effectively produce random samples from such Gaussian tree. The primary concern in synthesis problems is about efficiency in terms of the amount of random bits required for synthesis, as well as the modeling complexity of the given stochastic system through which the Gaussian vector is synthesized. This is done through an optimization problem to propose an efficient algorithm by which we can effectively generate such random vectors. We further generalize the optimization formulation from Gaussian trees to Gaussian vectors with arbitrary structures. This is done by introducing a new latent factor model obtained by solving a constrained minimum determinant factor analysis (CMDFA) problem. We discuss the benefits of factor models in machine learning applications and in particular 3D image reconstruction problems, where our newly proposed CMDFA problem may be beneficial

Evaluation of security robustness against information leakage in Gaussian polytree graphical models

Extensive works have been undertaken to develop efficient statistical inference algorithms based on graphical models. However, there still lacks sufficient understanding about how topological properties affect certain information related metrics for certain graphs. In this paper, we are particularly interested in finding out how topological properties of rooted polytrees for Gaussian random variables determine its security robustness, which is measured by our proposed max-min information (MaMI) metric. MaMI is defined as the maximin value of the conditional mutual information between any two random variables (nodes) in a given DAG, conditioned on the value of a third random variable, which is at full disposal of an eavesdropper, under a constraint of a given fixed joint entropy. We show some general topological properties which the desired max-min solutions satisfy. Under such properties, we prove the superior max-min feature of the linear topology for a simple but non-trivial case. The results not only help us understand the security strength of different rooted polytree type DAGs, which is critical when we evaluate the information leakage issues for various jointly Gaussian distributed measurements in networks, but also provide us another algebraic and analysis perspective in grasping some fundamental properties of such DAGs