8,578 research outputs found
Discrete Denoising Diffusion Approach to Integer Factorization
Integer factorization is a famous computational problem unknown whether being
solvable in the polynomial time. With the rise of deep neural networks, it is
interesting whether they can facilitate faster factorization. We present an
approach to factorization utilizing deep neural networks and discrete denoising
diffusion that works by iteratively correcting errors in a partially-correct
solution. To this end, we develop a new seq2seq neural network architecture,
employ relaxed categorical distribution and adapt the reverse diffusion process
to cope better with inaccuracies in the denoising step. The approach is able to
find factors for integers of up to 56 bits long. Our analysis indicates that
investment in training leads to an exponential decrease of sampling steps
required at inference to achieve a given success rate, thus counteracting an
exponential run-time increase depending on the bit-length.Comment: International Conference on Artificial Neural Networks ICANN 202
Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference
The rising popularity of intelligent mobile devices and the daunting
computational cost of deep learning-based models call for efficient and
accurate on-device inference schemes. We propose a quantization scheme that
allows inference to be carried out using integer-only arithmetic, which can be
implemented more efficiently than floating point inference on commonly
available integer-only hardware. We also co-design a training procedure to
preserve end-to-end model accuracy post quantization. As a result, the proposed
quantization scheme improves the tradeoff between accuracy and on-device
latency. The improvements are significant even on MobileNets, a model family
known for run-time efficiency, and are demonstrated in ImageNet classification
and COCO detection on popular CPUs.Comment: 14 pages, 12 figure
On the efficient representation and execution of deep acoustic models
In this paper we present a simple and computationally efficient quantization
scheme that enables us to reduce the resolution of the parameters of a neural
network from 32-bit floating point values to 8-bit integer values. The proposed
quantization scheme leads to significant memory savings and enables the use of
optimized hardware instructions for integer arithmetic, thus significantly
reducing the cost of inference. Finally, we propose a "quantization aware"
training process that applies the proposed scheme during network training and
find that it allows us to recover most of the loss in accuracy introduced by
quantization. We validate the proposed techniques by applying them to a long
short-term memory-based acoustic model on an open-ended large vocabulary speech
recognition task.Comment: Accepted conference paper: "The Annual Conference of the
International Speech Communication Association (Interspeech), 2016
- …