15 research outputs found
A Kernel Perspective for Regularizing Deep Neural Networks
We propose a new point of view for regularizing deep neural networks by using
the norm of a reproducing kernel Hilbert space (RKHS). Even though this norm
cannot be computed, it admits upper and lower approximations leading to various
practical strategies. Specifically, this perspective (i) provides a common
umbrella for many existing regularization principles, including spectral norm
and gradient penalties, or adversarial training, (ii) leads to new effective
regularization penalties, and (iii) suggests hybrid strategies combining lower
and upper bounds to get better approximations of the RKHS norm. We
experimentally show this approach to be effective when learning on small
datasets, or to obtain adversarially robust models.Comment: ICM
A Kernel Perspective for Regularizing Deep Neural Networks
International audienc
Diversity with Cooperation: Ensemble Methods for Few-Shot Classification
Added experiments with different network architectures and input image resolutionsInternational audienc
Selecting Relevant Features from a Multi-domain Representation for Few-shot Classification
Popular approaches for few-shot classification consist of first learning a
generic data representation based on a large annotated dataset, before adapting
the representation to new classes given only a few labeled samples. In this
work, we propose a new strategy based on feature selection, which is both
simpler and more effective than previous feature adaptation approaches. First,
we obtain a multi-domain representation by training a set of semantically
different feature extractors. Then, given a few-shot learning task, we use our
multi-domain feature bank to automatically select the most relevant
representations. We show that a simple non-parametric classifier built on top
of such features produces high accuracy and generalizes to domains never seen
during training, which leads to state-of-the-art results on MetaDataset and
improved accuracy on mini-ImageNet.Comment: ECCV'2