Maximum a Posteriori Estimation in Graphical Models Using Local Linear Approximation

Sparse structure learning in high-dimensional Gaussian graphical models is an important problem in multivariate statistical signal processing; since the sparsity pattern naturally encodes the conditional independence relationship among variables. However, maximum a posteriori (MAP) estimation is challenging under hierarchical prior models, and traditional numerical optimization routines or expectation--maximization algorithms are difficult to implement. To this end, our contribution is a novel local linear approximation scheme that circumvents this issue using a very simple computational algorithm. Most importantly, the condition under which our algorithm is guaranteed to converge to the MAP estimate is explicitly stated and is shown to cover a broad class of completely monotone priors, including the graphical horseshoe. Further, the resulting MAP estimate is shown to be sparse and consistent in the $\ell_2$-norm. Numerical results validate the speed, scalability, and statistical performance of the proposed method

MODEL-FORM UNCERTAINTY QUANTIFICATION FOR PREDICTIVE PROBABILISTIC GRAPHICAL MODELS

In this thesis, we focus on Uncertainty Quantification and Sensitivity Analysis, which can provide performance guarantees for predictive models built with both aleatoric and epistemic uncertainties, as well as data, and identify which components in a model have the most influence on predictions of our quantities of interest. In the first part (Chapter 2), we propose non-parametric methods for both local and global sensitivity analysis of chemical reaction models with correlated parameter dependencies. The developed mathematical and statistical tools are applied to a benchmark Langmuir competitive adsorption model on a close packed platinum surface, whose parameters, estimated from quantum-scale computations, are correlated and are limited in size (small data). The proposed mathematical methodology employs gradient-based methods to compute sensitivity indices. We observe that ranking influential parameters depend critically on whether or not correlations between parameters are taken into account. The impact of uncertainty in the correlation and the necessity of the proposed non-parametric perspective are demonstrated. In the second part (Chapter 3-4), we develop new information-based uncertainty quantification and sensitivity analysis methods for Probabilistic Graphical Models. Probabilistic graphical models are an important class of methods for probabilistic modeling and inference, probabilistic machine learning, and probabilistic artificial intelligence. Its hierarchical structure allows us to bring together in a systematic way statistical and multi-scale physical modeling, different types of data, incorporating expert knowledge, correlations, and causal relationships. However, due to multi-scale modeling, learning from sparse data, and mechanisms without full knowledge, many predictive models will necessarily have diverse sources of uncertainty at different scales. The new model-form uncertainty quantification indices we developed can handle both parametric and non-parametric probabilistic graphical models, as well as small and large model/parameter perturbations in a single, unified mathematical framework and provide an envelope of model predictions for our quantities of interest. Moreover, we propose a model-form Sensitivity Index, which allows us to rank the impact of each component of the probabilistic graphical model, and provide a systematic methodology to close the experiment - model - simulation - prediction loop and improve the computational model iteratively based on our new uncertainty quantification and sensitivity analysis methods. To illustrate our ideas, we explore a physicochemical application on the Oxygen Reduction Reaction (ORR) in Chapter 4, whose optimization was identified as a key to the performance of fuel cells. In the last part (Chapter 5), we complete our discussion for the uncertainty quantification and sensitivity analysis methods on probabilistic graphical models by introducing a new sensitivity analysis method for the case where we know the real model sits in a certain parametric family. Note that the uncertainty indices above may be too pessimistic (as they are inherently non-parametric) when studying uncertainty/sensitivity questions for models confined within a given parametric family. Therefore, we develop a method using likelihood ratio and fisher information matrix, which can capture correlations and causal dependencies in the graphical models, and we show it can provide us more accurate results for the parametric probabilistic graphical models

Learning loopy graphical models with latent variables: Efficient methods and guarantees

The problem of structure estimation in graphical models with latent variables is considered. We characterize conditions for tractable graph estimation and develop efficient methods with provable guarantees. We consider models where the underlying Markov graph is locally tree-like, and the model is in the regime of correlation decay. For the special case of the Ising model, the number of samples $n$ required for structural consistency of our method scales as $n=\Omega(\theta_{\min}^{-\delta\eta(\eta+1)-2}\log p)$, where p is the number of variables, $\theta_{\min}$ is the minimum edge potential, $\delta$ is the depth (i.e., distance from a hidden node to the nearest observed nodes), and $\eta$ is a parameter which depends on the bounds on node and edge potentials in the Ising model. Necessary conditions for structural consistency under any algorithm are derived and our method nearly matches the lower bound on sample requirements. Further, the proposed method is practical to implement and provides flexibility to control the number of latent variables and the cycle lengths in the output graph.Comment: Published in at http://dx.doi.org/10.1214/12-AOS1070 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org