37,355 research outputs found
Neural Dataset Generality
Often the filters learned by Convolutional Neural Networks (CNNs) from
different datasets appear similar. This is prominent in the first few layers.
This similarity of filters is being exploited for the purposes of transfer
learning and some studies have been made to analyse such transferability of
features. This is also being used as an initialization technique for different
tasks in the same dataset or for the same task in similar datasets.
Off-the-shelf CNN features have capitalized on this idea to promote their
networks as best transferable and most general and are used in a cavalier
manner in day-to-day computer vision tasks.
It is curious that while the filters learned by these CNNs are related to the
atomic structures of the images from which they are learnt, all datasets learn
similar looking low-level filters. With the understanding that a dataset that
contains many such atomic structures learn general filters and are therefore
useful to initialize other networks with, we propose a way to analyse and
quantify generality among datasets from their accuracies on transferred
filters. We applied this metric on several popular character recognition,
natural image and a medical image dataset, and arrived at some interesting
conclusions. On further experimentation we also discovered that particular
classes in a dataset themselves are more general than others.Comment: Long version of the paper accepted at IEEE International Conference
on Image Processing 201
Revisiting knowledge transfer for training object class detectors
We propose to revisit knowledge transfer for training object detectors on
target classes from weakly supervised training images, helped by a set of
source classes with bounding-box annotations. We present a unified knowledge
transfer framework based on training a single neural network multi-class object
detector over all source classes, organized in a semantic hierarchy. This
generates proposals with scores at multiple levels in the hierarchy, which we
use to explore knowledge transfer over a broad range of generality, ranging
from class-specific (bicycle to motorbike) to class-generic (objectness to any
class). Experiments on the 200 object classes in the ILSVRC 2013 detection
dataset show that our technique: (1) leads to much better performance on the
target classes (70.3% CorLoc, 36.9% mAP) than a weakly supervised baseline
which uses manually engineered objectness [11] (50.5% CorLoc, 25.4% mAP). (2)
delivers target object detectors reaching 80% of the mAP of their fully
supervised counterparts. (3) outperforms the best reported transfer learning
results on this dataset (+41% CorLoc and +3% mAP over [18, 46], +16.2% mAP over
[32]). Moreover, we also carry out several across-dataset knowledge transfer
experiments [27, 24, 35] and find that (4) our technique outperforms the weakly
supervised baseline in all dataset pairs by 1.5x-1.9x, establishing its general
applicability.Comment: CVPR 1
Automated Pruning for Deep Neural Network Compression
In this work we present a method to improve the pruning step of the current
state-of-the-art methodology to compress neural networks. The novelty of the
proposed pruning technique is in its differentiability, which allows pruning to
be performed during the backpropagation phase of the network training. This
enables an end-to-end learning and strongly reduces the training time. The
technique is based on a family of differentiable pruning functions and a new
regularizer specifically designed to enforce pruning. The experimental results
show that the joint optimization of both the thresholds and the network weights
permits to reach a higher compression rate, reducing the number of weights of
the pruned network by a further 14% to 33% compared to the current
state-of-the-art. Furthermore, we believe that this is the first study where
the generalization capabilities in transfer learning tasks of the features
extracted by a pruned network are analyzed. To achieve this goal, we show that
the representations learned using the proposed pruning methodology maintain the
same effectiveness and generality of those learned by the corresponding
non-compressed network on a set of different recognition tasks.Comment: 8 pages, 5 figures. Published as a conference paper at ICPR 201
- …