8 research outputs found
SQuantizer: Simultaneous Learning for Both Sparse and Low-precision Neural Networks
Deep neural networks have achieved state-of-the-art accuracies in a wide
range of computer vision, speech recognition, and machine translation tasks.
However the limits of memory bandwidth and computational power constrain the
range of devices capable of deploying these modern networks. To address this
problem, we propose SQuantizer, a new training method that jointly optimizes
for both sparse and low-precision neural networks while maintaining high
accuracy and providing a high compression rate. This approach brings
sparsification and low-bit quantization into a single training pass, employing
these techniques in an order demonstrated to be optimal. Our method achieves
state-of-the-art accuracies using 4-bit and 2-bit precision for ResNet18,
MobileNet-v2 and ResNet50, even with high degree of sparsity. The compression
rates of 18x for ResNet18 and 17x for ResNet50, and 9x for MobileNet-v2 are
obtained when SQuantizing both weights and activations within 1% and 2% loss in
accuracy for ResNets and MobileNet-v2 respectively. An extension of these
techniques to object detection also demonstrates high accuracy on YOLO-v3.
Additionally, our method allows for fast single pass training, which is
important for rapid prototyping and neural architecture search techniques.
Finally extensive results from this simultaneous training approach allows us to
draw some useful insights into the relative merits of sparsity and
quantization
Optimality Assessment of Memory-Bounded ConvNets Deployed on Resource-Constrained RISC Cores
A cost-effective implementation of Convolutional Neural Nets on the mobile edge of the Internet-of-Things (IoT) requires smart optimizations to fit large models into memory-constrained cores. Reduction methods that use a joint combination of filter pruning and weight quantization have proven efficient in searching the compression that ensures minimum model size without accuracy loss. However, there exist other optimal configurations that stem from the memory constraint. The objective of this work is to make an assessment of such memory-bounded implementations and to show that most of them are centred on specific parameter settings that are found difficult to be implemented on a low-power RISC. Hence, the focus is on quantifying the distance to optimality of the closest implementations that instead can be actually deployed on hardware. The analysis is powered by a two-stage framework that efficiently explores the memory-accuracy space using a lightweight, hardware-conscious heuristic optimization. Results are collected from three realistic IoT tasks (Image Classification on CIFAR-10, Keyword Spotting on the Speech Commands Dataset, Facial Expression Recognition on Fer2013) run on RISC cores (Cortex-M by ARM) with few hundreds KB of on-chip RAM