8,465 research outputs found
Fast Context Adaptation via Meta-Learning
We propose CAVIA for meta-learning, a simple extension to MAML that is less
prone to meta-overfitting, easier to parallelise, and more interpretable. CAVIA
partitions the model parameters into two parts: context parameters that serve
as additional input to the model and are adapted on individual tasks, and
shared parameters that are meta-trained and shared across tasks. At test time,
only the context parameters are updated, leading to a low-dimensional task
representation. We show empirically that CAVIA outperforms MAML for regression,
classification, and reinforcement learning. Our experiments also highlight
weaknesses in current benchmarks, in that the amount of adaptation needed in
some cases is small.Comment: Published at the International Conference on Machine Learning (ICML)
201
Hypernetwork approach to Bayesian MAML
The main goal of Few-Shot learning algorithms is to enable learning from
small amounts of data. One of the most popular and elegant Few-Shot learning
approaches is Model-Agnostic Meta-Learning (MAML). The main idea behind this
method is to learn the shared universal weights of a meta-model, which are then
adapted for specific tasks. However, the method suffers from over-fitting and
poorly quantifies uncertainty due to limited data size. Bayesian approaches
could, in principle, alleviate these shortcomings by learning weight
distributions in place of point-wise weights. Unfortunately, previous
modifications of MAML are limited due to the simplicity of Gaussian posteriors,
MAML-like gradient-based weight updates, or by the same structure enforced for
universal and adapted weights.
In this paper, we propose a novel framework for Bayesian MAML called
BayesianHMAML, which employs Hypernetworks for weight updates. It learns the
universal weights point-wise, but a probabilistic structure is added when
adapted for specific tasks. In such a framework, we can use simple Gaussian
distributions or more complicated posteriors induced by Continuous Normalizing
Flows.Comment: arXiv admin note: text overlap with arXiv:2205.1574
- …