4 research outputs found
Interpolating Convex and Non-Convex Tensor Decompositions via the Subspace Norm
We consider the problem of recovering a low-rank tensor from its noisy
observation. Previous work has shown a recovery guarantee with signal to noise
ratio for recovering a th order rank one
tensor of size by recursive unfolding. In this paper,
we first improve this bound to by a much simpler approach, but
with a more careful analysis. Then we propose a new norm called the subspace
norm, which is based on the Kronecker products of factors obtained by the
proposed simple estimator. The imposed Kronecker structure allows us to show a
nearly ideal bound, in which the parameter
controls the blend from the non-convex estimator to mode-wise nuclear norm
minimization. Furthermore, we empirically demonstrate that the subspace norm
achieves the nearly ideal denoising performance even with
The Sup-norm Perturbation of HOSVD and Low Rank Tensor Denoising
The higher order singular value decomposition (HOSVD) of tensors is a
generalization of matrix SVD. The perturbation analysis of HOSVD under random
noise is more delicate than its matrix counterpart. Recently, polynomial time
algorithms have been proposed where statistically optimal estimates of the
singular subspaces and the low rank tensors are attainable in the Euclidean
norm. In this article, we analyze the sup-norm perturbation bounds of HOSVD and
introduce estimators of the singular subspaces with sharp deviation bounds in
the sup-norm. We also investigate a low rank tensor denoising estimator and
demonstrate its fast convergence rate with respect to the entry-wise errors.
The sup-norm perturbation bounds reveal unconventional phase transitions for
statistical learning applications such as the exact clustering in high
dimensional Gaussian mixture model and the exact support recovery in sub-tensor
localizations. In addition, the bounds established for HOSVD also elaborate the
one-sided sup-norm perturbation bounds for the singular subspaces of unbalanced
(or fat) matrices
A Sharp Blockwise Tensor Perturbation Bound for Orthogonal Iteration
In this paper, we develop novel perturbation bounds for the high-order
orthogonal iteration (HOOI) [DLDMV00b]. Under mild regularity conditions, we
establish blockwise tensor perturbation bounds for HOOI with guarantees for
both tensor reconstruction in Hilbert-Schmidt norm \|\widehat{\bcT} - \bcT
\|_{\tHS} and mode- singular subspace estimation in Schatten- norm \|
\sin \Theta (\widehat{\U}_k, \U_k) \|_q for any . We show the upper
bounds of mode- singular subspace estimation are unilateral and converge
linearly to a quantity characterized by blockwise errors of the perturbation
and signal strength. For the tensor reconstruction error bound, we express the
bound through a simple quantity , which depends only on perturbation and
the multilinear rank of the underlying signal. Rate matching deterministic
lower bound for tensor reconstruction, which demonstrates the optimality of
HOOI, is also provided. Furthermore, we prove that one-step HOOI (i.e., HOOI
with only a single iteration) is also optimal in terms of tensor reconstruction
and can be used to lower the computational cost. The perturbation results are
also extended to the case that only partial modes of \bcT have low-rank
structure. We support our theoretical results by extensive numerical studies.
Finally, we apply the novel perturbation bounds of HOOI on two applications,
tensor denoising and tensor co-clustering, from machine learning and
statistics, which demonstrates the superiority of the new perturbation results
Inference for Low-rank Tensors -- No Need to Debias
In this paper, we consider the statistical inference for several low-rank
tensor models. Specifically, in the Tucker low-rank tensor PCA or regression
model, provided with any estimates achieving some attainable error rate, we
develop the data-driven confidence regions for the singular subspace of the
parameter tensor based on the asymptotic distribution of an updated estimate by
two-iteration alternating minimization. The asymptotic distributions are
established under some essential conditions on the signal-to-noise ratio (in
PCA model) or sample size (in regression model). If the parameter tensor is
further orthogonally decomposable, we develop the methods and non-asymptotic
theory for inference on each individual singular vector. For the rank-one
tensor PCA model, we establish the asymptotic distribution for general linear
forms of principal components and confidence interval for each entry of the
parameter tensor. Finally, numerical simulations are presented to corroborate
our theoretical discoveries.
In all these models, we observe that different from many matrix/vector
settings in existing work, debiasing is not required to establish the
asymptotic distribution of estimates or to make statistical inference on
low-rank tensors. In fact, due to the widely observed
statistical-computational-gap for low-rank tensor estimation, one usually
requires stronger conditions than the statistical (or information-theoretic)
limit to ensure the computationally feasible estimation is achievable.
Surprisingly, such conditions ``incidentally" render a feasible low-rank tensor
inference without debiasing.Comment: to appear at the Annals of Statistic