217 research outputs found
Tensorizing Neural Networks
Deep neural networks currently demonstrate state-of-the-art performance in
several domains. At the same time, models of this class are very demanding in
terms of computational resources. In particular, a large amount of memory is
required by commonly used fully-connected layers, making it hard to use the
models on low-end devices and stopping the further increase of the model size.
In this paper we convert the dense weight matrices of the fully-connected
layers to the Tensor Train format such that the number of parameters is reduced
by a huge factor and at the same time the expressive power of the layer is
preserved. In particular, for the Very Deep VGG networks we report the
compression factor of the dense weight matrix of a fully-connected layer up to
200000 times leading to the compression factor of the whole network up to 7
times
Compact Neural Networks based on the Multiscale Entanglement Renormalization Ansatz
This paper demonstrates a method for tensorizing neural networks based upon
an efficient way of approximating scale invariant quantum states, the
Multi-scale Entanglement Renormalization Ansatz (MERA). We employ MERA as a
replacement for the fully connected layers in a convolutional neural network
and test this implementation on the CIFAR-10 and CIFAR-100 datasets. The
proposed method outperforms factorization using tensor trains, providing
greater compression for the same level of accuracy and greater accuracy for the
same level of compression. We demonstrate MERA layers with 14000 times fewer
parameters and a reduction in accuracy of less than 1% compared to the
equivalent fully connected layers, scaling like O(N).Comment: 8 pages, 2 figure
Compact neural networks based on the multiscale entanglement renormalization Ansatz
This paper demonstrates a method for tensorizing neural networks based upon an
efficient way of approximating scale invariant quantum states, the Multi-scale Entanglement Renormalization Ansatz (MERA). We employ MERA as a replacement for the
fully connected layers in a convolutional neural network and test this implementation on
the CIFAR-10 and CIFAR-100 datasets. The proposed method outperforms factorization
using tensor trains, providing greater compression for the same level of accuracy and
greater accuracy for the same level of compression. We demonstrate MERA layers with
14000 times fewer parameters and a reduction in accuracy of less than 1% compared to
the equivalent fully connected layers, scaling like O(N)
- …