1 research outputs found
Learning Efficient Tensor Representations with Ring Structure Networks
Tensor train (TT) decomposition is a powerful representation for high-order
tensors, which has been successfully applied to various machine learning tasks
in recent years. However, since the tensor product is not commutative,
permutation of data dimensions makes solutions and TT-ranks of TT decomposition
inconsistent. To alleviate this problem, we propose a permutation symmetric
network structure by employing circular multilinear products over a sequence of
low-order core tensors. This network structure can be graphically interpreted
as a cyclic interconnection of tensors, and thus we call it tensor ring (TR)
representation. We develop several efficient algorithms to learn TR
representation with adaptive TR-ranks by employing low-rank approximations.
Furthermore, mathematical properties are investigated, which enables us to
perform basic operations in a computationally efficiently way by using TR
representations. Experimental results on synthetic signals and real-world
datasets demonstrate that the proposed TR network is more expressive and
consistently informative than existing TT networks.Comment: arXiv admin note: substantial text overlap with arXiv:1606.0553