6,997 research outputs found
DenseShift: Towards Accurate and Efficient Low-Bit Power-of-Two Quantization
Efficiently deploying deep neural networks on low-resource edge devices is
challenging due to their ever-increasing resource requirements. To address this
issue, researchers have proposed multiplication-free neural networks, such as
Power-of-Two quantization, or also known as Shift networks, which aim to reduce
memory usage and simplify computation. However, existing low-bit Shift networks
are not as accurate as their full-precision counterparts, typically suffering
from limited weight range encoding schemes and quantization loss. In this
paper, we propose the DenseShift network, which significantly improves the
accuracy of Shift networks, achieving competitive performance to full-precision
networks for vision and speech applications. In addition, we introduce a method
to deploy an efficient DenseShift network using non-quantized floating-point
activations, while obtaining 1.6X speed-up over existing methods. To achieve
this, we demonstrate that zero-weight values in low-bit Shift networks do not
contribute to model capacity and negatively impact inference computation. To
address this issue, we propose a zero-free shifting mechanism that simplifies
inference and increases model capacity. We further propose a sign-scale
decomposition design to enhance training efficiency and a low-variance random
initialization strategy to improve the model's transfer learning performance.
Our extensive experiments on various computer vision and speech tasks
demonstrate that DenseShift outperforms existing low-bit multiplication-free
networks and achieves competitive performance compared to full-precision
networks. Furthermore, our proposed approach exhibits strong transfer learning
performance without a drop in accuracy. Our code was released on GitHub
PalQuant: Accelerating High-precision Networks on Low-precision Accelerators
Recently low-precision deep learning accelerators (DLAs) have become popular
due to their advantages in chip area and energy consumption, yet the
low-precision quantized models on these DLAs bring in severe accuracy
degradation. One way to achieve both high accuracy and efficient inference is
to deploy high-precision neural networks on low-precision DLAs, which is rarely
studied. In this paper, we propose the PArallel Low-precision Quantization
(PalQuant) method that approximates high-precision computations via learning
parallel low-precision representations from scratch. In addition, we present a
novel cyclic shuffle module to boost the cross-group information communication
between parallel low-precision groups. Extensive experiments demonstrate that
PalQuant has superior performance to state-of-the-art quantization methods in
both accuracy and inference speed, e.g., for ResNet-18 network quantization,
PalQuant can obtain 0.52\% higher accuracy and 1.78 speedup
simultaneously over their 4-bit counter-part on a state-of-the-art 2-bit
accelerator. Code is available at \url{https://github.com/huqinghao/PalQuant}.Comment: accepted by ECCV202
CoNLoCNN: Exploiting Correlation and Non-Uniform Quantization for Energy-Efficient Low-precision Deep Convolutional Neural Networks
In today's era of smart cyber-physical systems, Deep Neural Networks (DNNs) have become ubiquitous due to their state-of-the-art performance in complex real-world applications. The high computational complexity of these networks, which translates to increased energy consumption, is the foremost obstacle towards deploying large DNNs in resource-constrained systems. Fixed-Point (FP) implementations achieved through post-training quantization are commonly used to curtail the energy consumption of these networks. However, the uniform quantization intervals in FP restrict the bit-width of data structures to large values due to the need to represent most of the numbers with sufficient resolution and avoid high quantization errors. In this paper, we leverage the key insight that (in most of the scenarios) DNN weights and activations are mostly concentrated near zero and only a few of them have large magnitudes. We propose CoNLoCNN, a framework to enable energy-efficient low-precision deep convolutional neural network inference by exploiting: (1) non-uniform quantization of weights enabling simplification of complex multiplication operations; and (2) correlation between activation values enabling partial compensation of quantization errors at low cost without any run-time overheads. To significantly benefit from non-uniform quantization, we also propose a novel data representation format, Encoded Low-Precision Binary Signed Digit, to compress the bit-width of weights while ensuring direct use of the encoded weight for processing using a novel multiply-and-accumulate (MAC) unit design
- …