332,180 research outputs found
Benchmark Analysis of Representative Deep Neural Network Architectures
This work presents an in-depth analysis of the majority of the deep neural
networks (DNNs) proposed in the state of the art for image recognition. For
each DNN multiple performance indices are observed, such as recognition
accuracy, model complexity, computational complexity, memory usage, and
inference time. The behavior of such performance indices and some combinations
of them are analyzed and discussed. To measure the indices we experiment the
use of DNNs on two different computer architectures, a workstation equipped
with a NVIDIA Titan X Pascal and an embedded system based on a NVIDIA Jetson
TX1 board. This experimentation allows a direct comparison between DNNs running
on machines with very different computational capacity. This study is useful
for researchers to have a complete view of what solutions have been explored so
far and in which research directions are worth exploring in the future; and for
practitioners to select the DNN architecture(s) that better fit the resource
constraints of practical deployments and applications. To complete this work,
all the DNNs, as well as the software used for the analysis, are available
online.Comment: Will appear in IEEE Acces
Instance-based Deep Transfer Learning
Deep transfer learning recently has acquired significant research interest.
It makes use of pre-trained models that are learned from a source domain, and
utilizes these models for the tasks in a target domain. Model-based deep
transfer learning is probably the most frequently used method. However, very
little research work has been devoted to enhancing deep transfer learning by
focusing on the influence of data. In this paper, we propose an instance-based
approach to improve deep transfer learning in a target domain. Specifically, we
choose a pre-trained model from a source domain and apply this model to
estimate the influence of training samples in a target domain. Then we optimize
the training data of the target domain by removing the training samples that
will lower the performance of the pre-trained model. We later either fine-tune
the pre-trained model with the optimized training data in the target domain, or
build a new model which is initialized partially based on the pre-trained
model, and fine-tune it with the optimized training data in the target domain.
Using this approach, transfer learning can help deep learning models to capture
more useful features. Extensive experiments demonstrate the effectiveness of
our approach on boosting the quality of deep learning models for some common
computer vision tasks, such as image classification.Comment: Accepted to WACV 2019. This is a preprint versio
Practical Block-wise Neural Network Architecture Generation
Convolutional neural networks have gained a remarkable success in computer
vision. However, most usable network architectures are hand-crafted and usually
require expertise and elaborate design. In this paper, we provide a block-wise
network generation pipeline called BlockQNN which automatically builds
high-performance networks using the Q-Learning paradigm with epsilon-greedy
exploration strategy. The optimal network block is constructed by the learning
agent which is trained sequentially to choose component layers. We stack the
block to construct the whole auto-generated network. To accelerate the
generation process, we also propose a distributed asynchronous framework and an
early stop strategy. The block-wise generation brings unique advantages: (1) it
performs competitive results in comparison to the hand-crafted state-of-the-art
networks on image classification, additionally, the best network generated by
BlockQNN achieves 3.54% top-1 error rate on CIFAR-10 which beats all existing
auto-generate networks. (2) in the meanwhile, it offers tremendous reduction of
the search space in designing networks which only spends 3 days with 32 GPUs,
and (3) moreover, it has strong generalizability that the network built on
CIFAR also performs well on a larger-scale ImageNet dataset.Comment: Accepted to CVPR 201
- …