14,465 research outputs found
A "network pruning network" Approach to deep model compression
We present a filter pruning approach for deep model compression, using a
multitask network. Our approach is based on learning a a pruner network to
prune a pre-trained target network. The pruner is essentially a multitask deep
neural network with binary outputs that help identify the filters from each
layer of the original network that do not have any significant contribution to
the model and can therefore be pruned. The pruner network has the same
architecture as the original network except that it has a
multitask/multi-output last layer containing binary-valued outputs (one per
filter), which indicate which filters have to be pruned. The pruner's goal is
to minimize the number of filters from the original network by assigning zero
weights to the corresponding output feature-maps. In contrast to most of the
existing methods, instead of relying on iterative pruning, our approach can
prune the network (original network) in one go and, moreover, does not require
specifying the degree of pruning for each layer (and can learn it instead). The
compressed model produced by our approach is generic and does not need any
special hardware/software support. Moreover, augmenting with other methods such
as knowledge distillation, quantization, and connection pruning can increase
the degree of compression for the proposed approach. We show the efficacy of
our proposed approach for classification and object detection tasks.Comment: Accepted in WACV'2
Data-Free Backbone Fine-Tuning for Pruned Neural Networks
Model compression techniques reduce the computational load and memory
consumption of deep neural networks. After the compression operation, e.g.
parameter pruning, the model is normally fine-tuned on the original training
dataset to recover from the performance drop caused by compression. However,
the training data is not always available due to privacy issues or other
factors. In this work, we present a data-free fine-tuning approach for pruning
the backbone of deep neural networks. In particular, the pruned network
backbone is trained with synthetically generated images, and our proposed
intermediate supervision to mimic the unpruned backbone's output feature map.
Afterwards, the pruned backbone can be combined with the original network head
to make predictions. We generate synthetic images by back-propagating gradients
to noise images while relying on L1-pruning for the backbone pruning. In our
experiments, we show that our approach is task-independent due to pruning only
the backbone. By evaluating our approach on 2D human pose estimation, object
detection, and image classification, we demonstrate promising performance
compared to the unpruned model. Our code is available at
https://github.com/holzbock/dfbf.Comment: Accpeted for presentation at the 31st European Signal Processing
Conference (EUSIPCO) 2023, September 4-8, 2023, Helsinki, Finlan
Reduced Memory Region Based Deep Convolutional Neural Network Detection
Accurate pedestrian detection has a primary role in automotive safety: for
example, by issuing warnings to the driver or acting actively on car's brakes,
it helps decreasing the probability of injuries and human fatalities. In order
to achieve very high accuracy, recent pedestrian detectors have been based on
Convolutional Neural Networks (CNN). Unfortunately, such approaches require
vast amounts of computational power and memory, preventing efficient
implementations on embedded systems. This work proposes a CNN-based detector,
adapting a general-purpose convolutional network to the task at hand. By
thoroughly analyzing and optimizing each step of the detection pipeline, we
develop an architecture that outperforms methods based on traditional image
features and achieves an accuracy close to the state-of-the-art while having
low computational complexity. Furthermore, the model is compressed in order to
fit the tight constrains of low power devices with a limited amount of embedded
memory available. This paper makes two main contributions: (1) it proves that a
region based deep neural network can be finely tuned to achieve adequate
accuracy for pedestrian detection (2) it achieves a very low memory usage
without reducing detection accuracy on the Caltech Pedestrian dataset.Comment: IEEE 2016 ICCE-Berli
Priming Neural Networks
Visual priming is known to affect the human visual system to allow detection
of scene elements, even those that may have been near unnoticeable before, such
as the presence of camouflaged animals. This process has been shown to be an
effect of top-down signaling in the visual system triggered by the said cue. In
this paper, we propose a mechanism to mimic the process of priming in the
context of object detection and segmentation. We view priming as having a
modulatory, cue dependent effect on layers of features within a network. Our
results show how such a process can be complementary to, and at times more
effective than simple post-processing applied to the output of the network,
notably so in cases where the object is hard to detect such as in severe noise.
Moreover, we find the effects of priming are sometimes stronger when early
visual layers are affected. Overall, our experiments confirm that top-down
signals can go a long way in improving object detection and segmentation.Comment: fixed error in author nam
Automated Pruning for Deep Neural Network Compression
In this work we present a method to improve the pruning step of the current
state-of-the-art methodology to compress neural networks. The novelty of the
proposed pruning technique is in its differentiability, which allows pruning to
be performed during the backpropagation phase of the network training. This
enables an end-to-end learning and strongly reduces the training time. The
technique is based on a family of differentiable pruning functions and a new
regularizer specifically designed to enforce pruning. The experimental results
show that the joint optimization of both the thresholds and the network weights
permits to reach a higher compression rate, reducing the number of weights of
the pruned network by a further 14% to 33% compared to the current
state-of-the-art. Furthermore, we believe that this is the first study where
the generalization capabilities in transfer learning tasks of the features
extracted by a pruned network are analyzed. To achieve this goal, we show that
the representations learned using the proposed pruning methodology maintain the
same effectiveness and generality of those learned by the corresponding
non-compressed network on a set of different recognition tasks.Comment: 8 pages, 5 figures. Published as a conference paper at ICPR 201
Optimisation of the PointPillars network for 3D object detection in point clouds
In this paper we present our research on the optimisation of a deep neural
network for 3D object detection in a point cloud. Techniques like quantisation
and pruning available in the Brevitas and PyTorch tools were used. We performed
the experiments for the PointPillars network, which offers a reasonable
compromise between detection accuracy and calculation complexity. The aim of
this work was to propose a variant of the network which we will ultimately
implement in an FPGA device. This will allow for real-time LiDAR data
processing with low energy consumption. The obtained results indicate that even
a significant quantisation from 32-bit floating point to 2-bit integer in the
main part of the algorithm, results in 5%-9% decrease of the detection
accuracy, while allowing for almost a 16-fold reduction in size of the model.Comment: 7 pages, 2 figures, submitted to SPA 2020 conferenc
- …