22 research outputs found
Tensor Graphical Lasso (TeraLasso)
This paper introduces a multi-way tensor generalization of the Bigraphical
Lasso (BiGLasso), which uses a two-way sparse Kronecker-sum multivariate-normal
model for the precision matrix to parsimoniously model conditional dependence
relationships of matrix-variate data based on the Cartesian product of graphs.
We call this generalization the {\bf Te}nsor g{\bf ra}phical Lasso (TeraLasso).
We demonstrate using theory and examples that the TeraLasso model can be
accurately and scalably estimated from very limited data samples of high
dimensional variables with multiway coordinates such as space, time and
replicates. Statistical consistency and statistical rates of convergence are
established for both the BiGLasso and TeraLasso estimators of the precision
matrix and estimators of its support (non-sparsity) set, respectively. We
propose a scalable composite gradient descent algorithm and analyze the
computational convergence rate, showing that the composite gradient descent
algorithm is guaranteed to converge at a geometric rate to the global minimizer
of the TeraLasso objective function. Finally, we illustrate the TeraLasso using
both simulation and experimental data from a meteorological dataset, showing
that we can accurately estimate precision matrices and recover meaningful
conditional dependency graphs from high dimensional complex datasets.Comment: accepted to JRSS-
Network inference in matrix-variate Gaussian models with non-independent noise
Inferring a graphical model or network from observational data from a large
number of variables is a well studied problem in machine learning and
computational statistics. In this paper we consider a version of this problem
that is relevant to the analysis of multiple phenotypes collected in genetic
studies. In such datasets we expect correlations between phenotypes and between
individuals. We model observations as a sum of two matrix normal variates such
that the joint covariance function is a sum of Kronecker products. This model,
which generalizes the Graphical Lasso, assumes observations are correlated due
to known genetic relationships and corrupted with non-independent noise. We
have developed a computationally efficient EM algorithm to fit this model. On
simulated datasets we illustrate substantially improved performance in network
reconstruction by allowing for a general noise distribution
antGLasso: An Efficient Tensor Graphical Lasso Algorithm
The class of bigraphical lasso algorithms (and, more broadly,
'tensor'-graphical lasso algorithms) has been used to estimate dependency
structures within matrix and tensor data. However, all current methods to do so
take prohibitively long on modestly sized datasets. We present a novel
tensor-graphical lasso algorithm that analytically estimates the dependency
structure, unlike its iterative predecessors. This provides a speedup of
multiple orders of magnitude, allowing this class of algorithms to be used on
large, real-world datasets.Comment: 9 pages (21 including supplementary material), 8 figures, submitted
to the GLFrontiers workshop at NeurIPS 202
Sharper rates of convergence for the tensor graphical Lasso estimator
Many modern datasets exhibit dependencies among observations as well as
variables. This gives rise to the challenging problem of analyzing
high-dimensional matrix-variate data with unknown dependence structures. To
address this challenge, Kalaitzis et. al. (2013) proposed the Bigraphical Lasso
(BiGLasso), an estimator for precision matrices of matrix-normals based on the
Cartesian product of graphs. Subsequently, Greenewald, Zhou and Hero (GZH 2019)
introduced a multiway tensor generalization of the BiGLasso estimator, known as
the TeraLasso estimator. In this paper, we provide sharper rates of convergence
in the Frobenius and operator norm for both BiGLasso and TeraLasso estimators
for estimating inverse covariance matrices. This improves upon the rates
presented in GZH 2019. In particular, (a) we strengthen the bounds for the
relative errors in the operator and Frobenius norm by a factor of approximately
; (b) Crucially, this improvement allows for finite-sample estimation
errors in both norms to be derived for the two-way Kronecker sum model. The
two-way regime is important because it is the setting that is the most
theoretically challenging, and simultaneously the most common in applications.
Normality is not needed in our proofs; instead, we consider sub-gaussian
ensembles and derive tight concentration of measure bounds, using tensor
unfolding techniques. The proof techniques may be of independent interest