911 research outputs found
Tensor Graphical Lasso (TeraLasso)
This paper introduces a multi-way tensor generalization of the Bigraphical
Lasso (BiGLasso), which uses a two-way sparse Kronecker-sum multivariate-normal
model for the precision matrix to parsimoniously model conditional dependence
relationships of matrix-variate data based on the Cartesian product of graphs.
We call this generalization the {\bf Te}nsor g{\bf ra}phical Lasso (TeraLasso).
We demonstrate using theory and examples that the TeraLasso model can be
accurately and scalably estimated from very limited data samples of high
dimensional variables with multiway coordinates such as space, time and
replicates. Statistical consistency and statistical rates of convergence are
established for both the BiGLasso and TeraLasso estimators of the precision
matrix and estimators of its support (non-sparsity) set, respectively. We
propose a scalable composite gradient descent algorithm and analyze the
computational convergence rate, showing that the composite gradient descent
algorithm is guaranteed to converge at a geometric rate to the global minimizer
of the TeraLasso objective function. Finally, we illustrate the TeraLasso using
both simulation and experimental data from a meteorological dataset, showing
that we can accurately estimate precision matrices and recover meaningful
conditional dependency graphs from high dimensional complex datasets.Comment: accepted to JRSS-
Group Symmetry and non-Gaussian Covariance Estimation
We consider robust covariance estimation with group symmetry constraints.
Non-Gaussian covariance estimation, e.g., Tyler scatter estimator and
Multivariate Generalized Gaussian distribution methods, usually involve
non-convex minimization problems. Recently, it was shown that the underlying
principle behind their success is an extended form of convexity over the
geodesics in the manifold of positive definite matrices. A modern approach to
improve estimation accuracy is to exploit prior knowledge via additional
constraints, e.g., restricting the attention to specific classes of covariances
which adhere to prior symmetry structures. In this paper, we prove that such
group symmetry constraints are also geodesically convex and can therefore be
incorporated into various non-Gaussian covariance estimators. Practical
examples of such sets include: circulant, persymmetric and complex/quaternion
proper structures. We provide a simple numerical technique for finding maximum
likelihood estimates under such constraints, and demonstrate their performance
advantage using synthetic experiments
Sparse Identification and Estimation of Large-Scale Vector AutoRegressive Moving Averages
The Vector AutoRegressive Moving Average (VARMA) model is fundamental to the
theory of multivariate time series; however, in practice, identifiability
issues have led many authors to abandon VARMA modeling in favor of the simpler
Vector AutoRegressive (VAR) model. Such a practice is unfortunate since even
very simple VARMA models can have quite complicated VAR representations. We
narrow this gap with a new optimization-based approach to VARMA identification
that is built upon the principle of parsimony. Among all equivalent
data-generating models, we seek the parameterization that is "simplest" in a
certain sense. A user-specified strongly convex penalty is used to measure
model simplicity, and that same penalty is then used to define an estimator
that can be efficiently computed. We show that our estimator converges to a
parsimonious element in the set of all equivalent data-generating models, in a
double asymptotic regime where the number of component time series is allowed
to grow with sample size. Further, we derive non-asymptotic upper bounds on the
estimation error of our method relative to our specially identified target.
Novel theoretical machinery includes non-asymptotic analysis of infinite-order
VAR, elastic net estimation under a singular covariance structure of
regressors, and new concentration inequalities for quadratic forms of random
variables from Gaussian time series. We illustrate the competitive performance
of our methods in simulation and several application domains, including
macro-economic forecasting, demand forecasting, and volatility forecasting
- …