7,959 research outputs found
Learning Sparse & Ternary Neural Networks with Entropy-Constrained Trained Ternarization (EC2T)
Deep neural networks (DNN) have shown remarkable success in a variety of
machine learning applications. The capacity of these models (i.e., number of
parameters), endows them with expressive power and allows them to reach the
desired performance. In recent years, there is an increasing interest in
deploying DNNs to resource-constrained devices (i.e., mobile devices) with
limited energy, memory, and computational budget. To address this problem, we
propose Entropy-Constrained Trained Ternarization (EC2T), a general framework
to create sparse and ternary neural networks which are efficient in terms of
storage (e.g., at most two binary-masks and two full-precision values are
required to save a weight matrix) and computation (e.g., MAC operations are
reduced to a few accumulations plus two multiplications). This approach
consists of two steps. First, a super-network is created by scaling the
dimensions of a pre-trained model (i.e., its width and depth). Subsequently,
this super-network is simultaneously pruned (using an entropy constraint) and
quantized (that is, ternary values are assigned layer-wise) in a training
process, resulting in a sparse and ternary network representation. We validate
the proposed approach in CIFAR-10, CIFAR-100, and ImageNet datasets, showing
its effectiveness in image classification tasks.Comment: Proceedings of the CVPR'20 Joint Workshop on Efficient Deep Learning
in Computer Vision. Code is available at
https://github.com/d-becking/efficientCNN
Reduced Memory Region Based Deep Convolutional Neural Network Detection
Accurate pedestrian detection has a primary role in automotive safety: for
example, by issuing warnings to the driver or acting actively on car's brakes,
it helps decreasing the probability of injuries and human fatalities. In order
to achieve very high accuracy, recent pedestrian detectors have been based on
Convolutional Neural Networks (CNN). Unfortunately, such approaches require
vast amounts of computational power and memory, preventing efficient
implementations on embedded systems. This work proposes a CNN-based detector,
adapting a general-purpose convolutional network to the task at hand. By
thoroughly analyzing and optimizing each step of the detection pipeline, we
develop an architecture that outperforms methods based on traditional image
features and achieves an accuracy close to the state-of-the-art while having
low computational complexity. Furthermore, the model is compressed in order to
fit the tight constrains of low power devices with a limited amount of embedded
memory available. This paper makes two main contributions: (1) it proves that a
region based deep neural network can be finely tuned to achieve adequate
accuracy for pedestrian detection (2) it achieves a very low memory usage
without reducing detection accuracy on the Caltech Pedestrian dataset.Comment: IEEE 2016 ICCE-Berli
- …