51 research outputs found
A "network pruning network" Approach to deep model compression
We present a filter pruning approach for deep model compression, using a
multitask network. Our approach is based on learning a a pruner network to
prune a pre-trained target network. The pruner is essentially a multitask deep
neural network with binary outputs that help identify the filters from each
layer of the original network that do not have any significant contribution to
the model and can therefore be pruned. The pruner network has the same
architecture as the original network except that it has a
multitask/multi-output last layer containing binary-valued outputs (one per
filter), which indicate which filters have to be pruned. The pruner's goal is
to minimize the number of filters from the original network by assigning zero
weights to the corresponding output feature-maps. In contrast to most of the
existing methods, instead of relying on iterative pruning, our approach can
prune the network (original network) in one go and, moreover, does not require
specifying the degree of pruning for each layer (and can learn it instead). The
compressed model produced by our approach is generic and does not need any
special hardware/software support. Moreover, augmenting with other methods such
as knowledge distillation, quantization, and connection pruning can increase
the degree of compression for the proposed approach. We show the efficacy of
our proposed approach for classification and object detection tasks.Comment: Accepted in WACV'2
Fine-Pruning: Joint Fine-Tuning and Compression of a Convolutional Network with Bayesian Optimization
When approaching a novel visual recognition problem in a specialized image
domain, a common strategy is to start with a pre-trained deep neural network
and fine-tune it to the specialized domain. If the target domain covers a
smaller visual space than the source domain used for pre-training (e.g.
ImageNet), the fine-tuned network is likely to be over-parameterized. However,
applying network pruning as a post-processing step to reduce the memory
requirements has drawbacks: fine-tuning and pruning are performed
independently; pruning parameters are set once and cannot adapt over time; and
the highly parameterized nature of state-of-the-art pruning methods make it
prohibitive to manually search the pruning parameter space for deep networks,
leading to coarse approximations. We propose a principled method for jointly
fine-tuning and compressing a pre-trained convolutional network that overcomes
these limitations. Experiments on two specialized image domains (remote sensing
images and describable textures) demonstrate the validity of the proposed
approach.Comment: BMVC 2017 ora
- …