7 research outputs found
A dual framework for low-rank tensor completion
One of the popular approaches for low-rank tensor completion is to use the
latent trace norm regularization. However, most existing works in this
direction learn a sparse combination of tensors. In this work, we fill this gap
by proposing a variant of the latent trace norm that helps in learning a
non-sparse combination of tensors. We develop a dual framework for solving the
low-rank tensor completion problem. We first show a novel characterization of
the dual solution space with an interesting factorization of the optimal
solution. Overall, the optimal solution is shown to lie on a Cartesian product
of Riemannian manifolds. Furthermore, we exploit the versatile Riemannian
optimization framework for proposing computationally efficient trust region
algorithm. The experiments illustrate the efficacy of the proposed algorithm on
several real-world datasets across applications.Comment: Aceepted to appear in Advances of Nueral Information Processing
Systems (NIPS), 2018. A shorter version appeared in the NIPS workshop on
Synergies in Geometric Data Analysis 201
SVDinsTN: An Integrated Method for Tensor Network Representation with Efficient Structure Search
Tensor network (TN) representation is a powerful technique for data analysis
and machine learning. It practically involves a challenging TN structure search
(TN-SS) problem, which aims to search for the optimal structure to achieve a
compact representation. Existing TN-SS methods mainly adopt a bi-level
optimization method that leads to excessive computational costs due to repeated
structure evaluations. To address this issue, we propose an efficient
integrated (single-level) method named SVD-inspired TN decomposition
(SVDinsTN), eliminating the need for repeated tedious structure evaluation. By
inserting a diagonal factor for each edge of the fully-connected TN, we
calculate TN cores and diagonal factors simultaneously, with factor sparsity
revealing the most compact TN structure. Experimental results on real-world
data demonstrate that SVDinsTN achieves approximately times
acceleration in runtime compared to the existing TN-SS methods while
maintaining a comparable level of representation ability
Optimization and Learning over Riemannian Manifolds
Learning over smooth nonlinear spaces has found wide applications. A principled approach for addressing such problems is to endow the search space with a Riemannian manifold geometry and numerical optimization can be performed intrinsically. Recent years have seen a surge of interest in leveraging Riemannian optimization for nonlinearly-constrained problems. This thesis investigates and improves on the existing algorithms for Riemannian optimization, with a focus on unified analysis frameworks and generic strategies. To this end, the first chapter systematically studies the choice of Riemannian geometries and their impacts on algorithmic convergence, on the manifold of positive definite matrices. The second chapter considers stochastic optimization on manifolds and proposes a unified framework for analyzing and improving the convergence of Riemannian variance reduction methods for nonconvex functions. The third chapter introduces a generic acceleration scheme based on the idea of extrapolation, which achieves optimal convergence rate asymptotically while being empirically efficient