30 research outputs found
Recommended from our members
Structured Tensor Recovery and Decomposition
Tensors, a.k.a. multi-dimensional arrays, arise naturally when modeling higher-order objects and relations. Among ubiquitous applications including image processing, collaborative filtering, demand forecasting and higher-order statistics, there are two recurring themes in general: tensor recovery and tensor decomposition. The first one aims to recover the underlying tensor from incomplete information; the second one is to study a variety of tensor decompositions to represent the array more concisely and moreover to capture the salient characteristics of the underlying data. Both topics are respectively addressed in this thesis.
Chapter 2 and Chapter 3 focus on low-rank tensor recovery (LRTR) from both theoretical and algorithmic perspectives. In Chapter 2, we first provide a negative result to the sum of nuclear norms (SNN) model---an existing convex model widely used for LRTR; then we propose a novel convex model and prove this new model is better than the SNN model in terms of the number of measurements required to recover the underlying low-rank tensor. In Chapter 3, we first build up the connection between robust low-rank tensor recovery and the compressive principle component pursuit (CPCP), a convex model for robust low-rank matrix recovery. Then we focus on developing convergent and scalable optimization methods to solve the CPCP problem. In specific, our convergent method, proposed by combining classical ideas from Frank-Wolfe and proximal methods, achieves scalability with linear per-iteration cost.
Chapter 4 generalizes the successive rank-one approximation (SROA) scheme for matrix eigen-decomposition to a special class of tensors called symmetric and orthogonally decomposable (SOD) tensor. We prove that the SROA scheme can robustly recover the symmetric canonical decomposition of the underlying SOD tensor even in the presence of noise. Perturbation bounds, which can be regarded as a higher-order generalization of the Davis-Kahan theorem, are provided in terms of the noise magnitude
Online and Differentially-Private Tensor Decomposition
In this paper, we resolve many of the key algorithmic questions regarding
robustness, memory efficiency, and differential privacy of tensor
decomposition. We propose simple variants of the tensor power method which
enjoy these strong properties. We present the first guarantees for online
tensor power method which has a linear memory requirement. Moreover, we present
a noise calibrated tensor power method with efficient privacy guarantees. At
the heart of all these guarantees lies a careful perturbation analysis derived
in this paper which improves up on the existing results significantly.Comment: 19 pages, 9 figures. To appear at the 30th Annual Conference on
Advances in Neural Information Processing Systems (NIPS 2016), to be held at
Barcelona, Spain. Fix small typos in proofs of Lemmas C.5 and C.
Online and Differentially-Private Tensor Decomposition
Tensor decomposition is positioned to be a pervasive tool in the era of big data. In this paper, we resolve many of the key algorithmic questions regarding robustness, memory efficiency, and differential privacy of tensor decomposition. We propose simple variants of the tensor power method which enjoy these strong properties. We propose the first streaming method with a linear memory requirement. Moreover, we present a noise calibrated tensor power method with efficient privacy guarantees. At the heart of all these guarantees lies a careful perturbation analysis derived in this paper which improves up on the existing results significantly
Multivariate Analysis for Multiple Network Data via Semi-Symmetric Tensor PCA
Network data are commonly collected in a variety of applications,
representing either directly measured or statistically inferred connections
between features of interest. In an increasing number of domains, these
networks are collected over time, such as interactions between users of a
social media platform on different days, or across multiple subjects, such as
in multi-subject studies of brain connectivity. When analyzing multiple large
networks, dimensionality reduction techniques are often used to embed networks
in a more tractable low-dimensional space. To this end, we develop a framework
for principal components analysis (PCA) on collections of networks via a
specialized tensor decomposition we term Semi-Symmetric Tensor PCA or SS-TPCA.
We derive computationally efficient algorithms for computing our proposed
SS-TPCA decomposition and establish statistical efficiency of our approach
under a standard low-rank signal plus noise model. Remarkably, we show that
SS-TPCA achieves the same estimation accuracy as classical matrix PCA, with
error proportional to the square root of the number of vertices in the network
and not the number of edges as might be expected. Our framework inherits many
of the strengths of classical PCA and is suitable for a wide range of
unsupervised learning tasks, including identifying principal networks,
isolating meaningful changepoints or outlying observations, and for
characterizing the "variability network" of the most varying edges. Finally, we
demonstrate the effectiveness of our proposal on simulated data and on an
example from empirical legal studies. The techniques used to establish our main
consistency results are surprisingly straightforward and may find use in a
variety of other network analysis problems