2,118 research outputs found
The Graphical Lasso: New Insights and Alternatives
The graphical lasso \citep{FHT2007a} is an algorithm for learning the
structure in an undirected Gaussian graphical model, using
regularization to control the number of zeros in the precision matrix
{\B\Theta}={\B\Sigma}^{-1} \citep{BGA2008,yuan_lin_07}. The {\texttt R}
package \GL\ \citep{FHT2007a} is popular, fast, and allows one to efficiently
build a path of models for different values of the tuning parameter.
Convergence of \GL\ can be tricky; the converged precision matrix might not be
the inverse of the estimated covariance, and occasionally it fails to converge
with warm starts. In this paper we explain this behavior, and propose new
algorithms that appear to outperform \GL.
By studying the "normal equations" we see that, \GL\ is solving the {\em
dual} of the graphical lasso penalized likelihood, by block coordinate ascent;
a result which can also be found in \cite{BGA2008}.
In this dual, the target of estimation is \B\Sigma, the covariance matrix,
rather than the precision matrix \B\Theta. We propose similar primal
algorithms \PGL\ and \DPGL, that also operate by block-coordinate descent,
where \B\Theta is the optimization target. We study all of these algorithms,
and in particular different approaches to solving their coordinate
sub-problems. We conclude that \DPGL\ is superior from several points of view.Comment: This is a revised version of our previous manuscript with the same
name ArXiv id: http://arxiv.org/abs/1111.547
Sparse inverse covariance estimation with the lasso
We consider the problem of estimating sparse graphs by a lasso penalty
applied to the inverse covariance matrix. Using a coordinate descent procedure
for the lasso, we develop a simple algorithm that is remarkably fast: in the
worst cases, it solves a 1000 node problem (~500,000 parameters) in about a
minute, and is 50 to 2000 times faster than competing methods. It also provides
a conceptual link between the exact problem and the approximation suggested by
Meinhausen and Buhlmann (2006). We illustrate the method on some cell-signaling
data from proteomics.Comment: submitte
A convex pseudo-likelihood framework for high dimensional partial correlation estimation with convergence guarantees
Sparse high dimensional graphical model selection is a topic of much interest
in modern day statistics. A popular approach is to apply l1-penalties to either
(1) parametric likelihoods, or, (2) regularized regression/pseudo-likelihoods,
with the latter having the distinct advantage that they do not explicitly
assume Gaussianity. As none of the popular methods proposed for solving
pseudo-likelihood based objective functions have provable convergence
guarantees, it is not clear if corresponding estimators exist or are even
computable, or if they actually yield correct partial correlation graphs. This
paper proposes a new pseudo-likelihood based graphical model selection method
that aims to overcome some of the shortcomings of current methods, but at the
same time retain all their respective strengths. In particular, we introduce a
novel framework that leads to a convex formulation of the partial covariance
regression graph problem, resulting in an objective function comprised of
quadratic forms. The objective is then optimized via a coordinate-wise
approach. The specific functional form of the objective function facilitates
rigorous convergence analysis leading to convergence guarantees; an important
property that cannot be established using standard results, when the dimension
is larger than the sample size, as is often the case in high dimensional
applications. These convergence guarantees ensure that estimators are
well-defined under very general conditions, and are always computable. In
addition, the approach yields estimators that have good large sample properties
and also respect symmetry. Furthermore, application to simulated/real data,
timing comparisons and numerical convergence is demonstrated. We also present a
novel unifying framework that places all graphical pseudo-likelihood methods as
special cases of a more general formulation, leading to important insights
- β¦