110,068 research outputs found
Deep Co-Space: Sample Mining Across Feature Transformation for Semi-Supervised Learning
Aiming at improving performance of visual classification in a cost-effective
manner, this paper proposes an incremental semi-supervised learning paradigm
called Deep Co-Space (DCS). Unlike many conventional semi-supervised learning
methods usually performing within a fixed feature space, our DCS gradually
propagates information from labeled samples to unlabeled ones along with deep
feature learning. We regard deep feature learning as a series of steps pursuing
feature transformation, i.e., projecting the samples from a previous space into
a new one, which tends to select the reliable unlabeled samples with respect to
this setting. Specifically, for each unlabeled image instance, we measure its
reliability by calculating the category variations of feature transformation
from two different neighborhood variation perspectives, and merged them into an
unified sample mining criterion deriving from Hellinger distance. Then, those
samples keeping stable correlation to their neighboring samples (i.e., having
small category variation in distribution) across the successive feature space
transformation, are automatically received labels and incorporated into the
model for incrementally training in terms of classification. Our extensive
experiments on standard image classification benchmarks (e.g., Caltech-256 and
SUN-397) demonstrate that the proposed framework is capable of effectively
mining from large-scale unlabeled images, which boosts image classification
performance and achieves promising results compared to other semi-supervised
learning methods.Comment: To appear in IEEE Transactions on Circuits and Systems for Video
Technology (T-CSVT), 201
Incremental Learning Using a Grow-and-Prune Paradigm with Efficient Neural Networks
Deep neural networks (DNNs) have become a widely deployed model for numerous
machine learning applications. However, their fixed architecture, substantial
training cost, and significant model redundancy make it difficult to
efficiently update them to accommodate previously unseen data. To solve these
problems, we propose an incremental learning framework based on a
grow-and-prune neural network synthesis paradigm. When new data arrive, the
neural network first grows new connections based on the gradients to increase
the network capacity to accommodate new data. Then, the framework iteratively
prunes away connections based on the magnitude of weights to enhance network
compactness, and hence recover efficiency. Finally, the model rests at a
lightweight DNN that is both ready for inference and suitable for future
grow-and-prune updates. The proposed framework improves accuracy, shrinks
network size, and significantly reduces the additional training cost for
incoming data compared to conventional approaches, such as training from
scratch and network fine-tuning. For the LeNet-300-100 and LeNet-5 neural
network architectures derived for the MNIST dataset, the framework reduces
training cost by up to 64% (63%) and 67% (63%) compared to training from
scratch (network fine-tuning), respectively. For the ResNet-18 architecture
derived for the ImageNet dataset and DeepSpeech2 for the AN4 dataset, the
corresponding training cost reductions against training from scratch (network
fine-tunning) are 64% (60%) and 67% (62%), respectively. Our derived models
contain fewer network parameters but achieve higher accuracy relative to
conventional baselines
iCaRL: Incremental Classifier and Representation Learning
A major open problem on the road to artificial intelligence is the
development of incrementally learning systems that learn about more and more
concepts over time from a stream of data. In this work, we introduce a new
training strategy, iCaRL, that allows learning in such a class-incremental way:
only the training data for a small number of classes has to be present at the
same time and new classes can be added progressively. iCaRL learns strong
classifiers and a data representation simultaneously. This distinguishes it
from earlier works that were fundamentally limited to fixed data
representations and therefore incompatible with deep learning architectures. We
show by experiments on CIFAR-100 and ImageNet ILSVRC 2012 data that iCaRL can
learn many classes incrementally over a long period of time where other
strategies quickly fail.Comment: Accepted paper at CVPR 201
- …