480 research outputs found
Exact Hybrid Covariance Thresholding for Joint Graphical Lasso
This paper considers the problem of estimating multiple related Gaussian
graphical models from a -dimensional dataset consisting of different
classes. Our work is based upon the formulation of this problem as group
graphical lasso. This paper proposes a novel hybrid covariance thresholding
algorithm that can effectively identify zero entries in the precision matrices
and split a large joint graphical lasso problem into small subproblems. Our
hybrid covariance thresholding method is superior to existing uniform
thresholding methods in that our method can split the precision matrix of each
individual class using different partition schemes and thus split group
graphical lasso into much smaller subproblems, each of which can be solved very
fast. In addition, this paper establishes necessary and sufficient conditions
for our hybrid covariance thresholding algorithm. The superior performance of
our thresholding method is thoroughly analyzed and illustrated by a few
experiments on simulated data and real gene expression data
TIGER: A Tuning-Insensitive Approach for Optimally Estimating Gaussian Graphical Models
We propose a new procedure for estimating high dimensional Gaussian graphical
models. Our approach is asymptotically tuning-free and non-asymptotically
tuning-insensitive: it requires very few efforts to choose the tuning parameter
in finite sample settings. Computationally, our procedure is significantly
faster than existing methods due to its tuning-insensitive property.
Theoretically, the obtained estimator is simultaneously minimax optimal for
precision matrix estimation under different norms. Empirically, we illustrate
the advantages of our method using thorough simulated and real examples. The R
package bigmatrix implementing the proposed methods is available on the
Comprehensive R Archive Network: http://cran.r-project.org/
Recommended from our members
Sparse Recovery Problems in High Dimensions: Statistical Inference and Learning Theory
The statistical analysis of high dimensional data requires new techniques, extending results from nonparametric statistics, analysis, probability, approximation theory, and theoretical computer science. The main problem is how to unveil, (or to mimic performance of) sparse models for the data. Sparsity is generally meant in terms of the number of variables included, but may also be described in terms of smoothness, entropy, or geometric structures. A key objective is to adapt to unknown sparsity, yet keeping computational feasibility
Maximum a Posteriori Estimation in Graphical Models Using Local Linear Approximation
Sparse structure learning in high-dimensional Gaussian graphical models is an
important problem in multivariate statistical signal processing; since the
sparsity pattern naturally encodes the conditional independence relationship
among variables. However, maximum a posteriori (MAP) estimation is challenging
under hierarchical prior models, and traditional numerical optimization
routines or expectation--maximization algorithms are difficult to implement. To
this end, our contribution is a novel local linear approximation scheme that
circumvents this issue using a very simple computational algorithm. Most
importantly, the condition under which our algorithm is guaranteed to converge
to the MAP estimate is explicitly stated and is shown to cover a broad class of
completely monotone priors, including the graphical horseshoe. Further, the
resulting MAP estimate is shown to be sparse and consistent in the
-norm. Numerical results validate the speed, scalability, and
statistical performance of the proposed method
High-Dimensional Gaussian Graphical Model Selection: Walk Summability and Local Separation Criterion
We consider the problem of high-dimensional Gaussian graphical model
selection. We identify a set of graphs for which an efficient estimation
algorithm exists, and this algorithm is based on thresholding of empirical
conditional covariances. Under a set of transparent conditions, we establish
structural consistency (or sparsistency) for the proposed algorithm, when the
number of samples n=omega(J_{min}^{-2} log p), where p is the number of
variables and J_{min} is the minimum (absolute) edge potential of the graphical
model. The sufficient conditions for sparsistency are based on the notion of
walk-summability of the model and the presence of sparse local vertex
separators in the underlying graph. We also derive novel non-asymptotic
necessary conditions on the number of samples required for sparsistency
Hybrid approximate message passing
Gaussian and quadratic approximations of message passing algorithms on graphs have attracted considerable recent attention due to their computational simplicity, analytic tractability, and wide applicability in optimization and statistical inference problems. This paper presents a systematic framework for incorporating such approximate message passing (AMP) methods in general graphical models. The key concept is a partition of dependencies of a general graphical model into strong and weak edges, with the weak edges representing interactions through aggregates of small, linearizable couplings of variables. AMP approximations based on the Central Limit Theorem can be readily applied to aggregates of many weak edges and integrated with standard message passing updates on the strong edges. The resulting algorithm, which we call hybrid generalized approximate message passing (HyGAMP), can yield significantly simpler implementations of sum-product and max-sum loopy belief propagation. By varying the partition of strong and weak edges, a performance--complexity trade-off can be achieved. Group sparsity and multinomial logistic regression problems are studied as examples of the proposed methodology.The work of S. Rangan was supported in part by the National Science Foundation under Grants 1116589, 1302336, and 1547332, and in part by the industrial affiliates of NYU WIRELESS. The work of A. K. Fletcher was supported in part by the National Science Foundation under Grants 1254204 and 1738286 and in part by the Office of Naval Research under Grant N00014-15-1-2677. The work of V. K. Goyal was supported in part by the National Science Foundation under Grant 1422034. The work of E. Byrne and P. Schniter was supported in part by the National Science Foundation under Grant CCF-1527162. (1116589 - National Science Foundation; 1302336 - National Science Foundation; 1547332 - National Science Foundation; 1254204 - National Science Foundation; 1738286 - National Science Foundation; 1422034 - National Science Foundation; CCF-1527162 - National Science Foundation; NYU WIRELESS; N00014-15-1-2677 - Office of Naval Research
- …