4,506 research outputs found
EMO: Episodic Memory Optimization for Few-Shot Meta-Learning
Few-shot meta-learning presents a challenge for gradient descent optimization
due to the limited number of training samples per task. To address this issue,
we propose an episodic memory optimization for meta-learning, we call
\emph{EMO}, which is inspired by the human ability to recall past learning
experiences from the brain's memory. EMO retains the gradient history of past
experienced tasks in external memory, enabling few-shot learning in a
memory-augmented way. By learning to retain and recall the learning process of
past training tasks, EMO nudges parameter updates in the right direction, even
when the gradients provided by a limited number of examples are uninformative.
We prove theoretically that our algorithm converges for smooth, strongly convex
objectives. EMO is generic, flexible, and model-agnostic, making it a simple
plug-and-play optimizer that can be seamlessly embedded into existing
optimization-based few-shot meta-learning approaches. Empirical results show
that EMO scales well with most few-shot classification benchmarks and improves
the performance of optimization-based meta-learning methods, resulting in
accelerated convergence.Comment: Accepted by CoLLAs 202
Graph Neural Network Expressivity and Meta-Learning for Molecular Property Regression
We demonstrate the applicability of model-agnostic algorithms for
meta-learning, specifically Reptile, to GNN models in molecular regression
tasks. Using meta-learning we are able to learn new chemical prediction tasks
with only a few model updates, as compared to using randomly initialized GNNs
which require learning each regression task from scratch. We experimentally
show that GNN layer expressivity is correlated to improved meta-learning.
Additionally, we also experiment with GNN emsembles which yield best
performance and rapid convergence for k-shot learning
- …