35 research outputs found
Further advantages of data augmentation on convolutional neural networks
Data augmentation is a popular technique largely used to enhance the training
of convolutional neural networks. Although many of its benefits are well known
by deep learning researchers and practitioners, its implicit regularization
effects, as compared to popular explicit regularization techniques, such as
weight decay and dropout, remain largely unstudied. As a matter of fact,
convolutional neural networks for image object classification are typically
trained with both data augmentation and explicit regularization, assuming the
benefits of all techniques are complementary. In this paper, we systematically
analyze these techniques through ablation studies of different network
architectures trained with different amounts of training data. Our results
unveil a largely ignored advantage of data augmentation: networks trained with
just data augmentation more easily adapt to different architectures and amount
of training data, as opposed to weight decay and dropout, which require
specific fine-tuning of their hyperparameters.Comment: Preprint of the manuscript accepted for presentation at the
International Conference on Artificial Neural Networks (ICANN) 2018. Best
Paper Awar
Regression Networks for Meta-Learning Few-Shot Classification
We propose regression networks for the problem of few-shot classification,
where a classifier must generalize to new classes not seen in the training set,
given only a small number of examples of each class. In high dimensional
embedding spaces the direction of data generally contains richer information
than magnitude. Next to this, state-of-the-art few-shot metric methods that
compare distances with aggregated class representations, have shown superior
performance. Combining these two insights, we propose to meta-learn
classification of embedded points by regressing the closest approximation in
every class subspace while using the regression error as a distance metric.
Similarly to recent approaches for few-shot learning, regression networks
reflect a simple inductive bias that is beneficial in this limited-data regime
and they achieve excellent results, especially when more aggregate class
representations can be formed with multiple shots.Comment: 7th ICML Workshop on Automated Machine Learning (2020
Attributes-Guided and Pure-Visual Attention Alignment for Few-Shot Recognition
The purpose of few-shot recognition is to recognize novel categories with a
limited number of labeled examples in each class. To encourage learning from a
supplementary view, recent approaches have introduced auxiliary semantic
modalities into effective metric-learning frameworks that aim to learn a
feature similarity between training samples (support set) and test samples
(query set). However, these approaches only augment the representations of
samples with available semantics while ignoring the query set, which loses the
potential for the improvement and may lead to a shift between the modalities
combination and the pure-visual representation. In this paper, we devise an
attributes-guided attention module (AGAM) to utilize human-annotated attributes
and learn more discriminative features. This plug-and-play module enables
visual contents and corresponding attributes to collectively focus on important
channels and regions for the support set. And the feature selection is also
achieved for query set with only visual information while the attributes are
not available. Therefore, representations from both sets are improved in a
fine-grained manner. Moreover, an attention alignment mechanism is proposed to
distill knowledge from the guidance of attributes to the pure-visual branch for
samples without attributes. Extensive experiments and analysis show that our
proposed module can significantly improve simple metric-based approaches to
achieve state-of-the-art performance on different datasets and settings.Comment: An expanded version of the same-name paper accepted by AAAI-202