4 research outputs found
Contextualizing Multiple Tasks via Learning to Decompose
One single instance could possess multiple portraits and reveal diverse
relationships with others according to different contexts. Those ambiguities
increase the difficulty of learning a generalizable model when there exists one
concept or mixed concepts in a task. We propose a general approach Learning to
Decompose Network (LeadNet) for both two cases, which contextualizes a model
through meta-learning multiple maps for concepts discovery -- the
representations of instances are decomposed and adapted conditioned on the
contexts. Through taking a holistic view over multiple latent components over
instances in a sampled pseudo task, LeadNet learns to automatically select the
right concept via incorporating those rich semantics inside and between
objects. LeadNet demonstrates its superiority in various applications,
including exploring multiple views of confusing tasks, out-of-distribution
recognition, and few-shot image classification
How to Train Your MAML to Excel in Few-Shot Classification
Model-agnostic meta-learning (MAML) is arguably the most popular
meta-learning algorithm nowadays, given its flexibility to incorporate various
model architectures and to be applied to different problems. Nevertheless, its
performance on few-shot classification is far behind many recent algorithms
dedicated to the problem. In this paper, we point out several key facets of how
to train MAML to excel in few-shot classification. First, we find that a large
number of gradient steps are needed for the inner loop update, which
contradicts the common usage of MAML for few-shot classification. Second, we
find that MAML is sensitive to the permutation of class assignments in
meta-testing: for a few-shot task of classes, there are exponentially many
ways to assign the learned initialization of the -way classifier to the
classes, leading to an unavoidably huge variance. Third, we investigate several
ways for permutation invariance and find that learning a shared classifier
initialization for all the classes performs the best. On benchmark datasets
such as MiniImageNet and TieredImageNet, our approach, which we name
UNICORN-MAML, performs on a par with or even outperforms state-of-the-art
algorithms, while keeping the simplicity of MAML without adding any extra
sub-networks
Few-Shot Learning with a Strong Teacher
Few-shot learning (FSL) aims to train a strong classifier using limited
labeled examples. Many existing works take the meta-learning approach, sampling
few-shot tasks in turn and optimizing the few-shot learner's performance on
classifying the query examples. In this paper, we point out two potential
weaknesses of this approach. First, the sampled query examples may not provide
sufficient supervision for the few-shot learner. Second, the effectiveness of
meta-learning diminishes sharply with increasing shots (i.e., the number of
training examples per class). To resolve these issues, we propose a novel
objective to directly train the few-shot learner to perform like a strong
classifier. Concretely, we associate each sampled few-shot task with a strong
classifier, which is learned with ample labeled examples. The strong classifier
has a better generalization ability and we use it to supervise the few-shot
learner. We present an efficient way to construct the strong classifier, making
our proposed objective an easily plug-and-play term to existing meta-learning
based FSL methods. We validate our approach in combinations with many
representative meta-learning methods. On several benchmark datasets including
miniImageNet and tiredImageNet, our approach leads to a notable improvement
across a variety of tasks. More importantly, with our approach, meta-learning
based FSL methods can consistently outperform non-meta-learning based ones,
even in a many-shot setting, greatly strengthening their applicability