2 research outputs found
Provable Guarantees for Gradient-Based Meta-Learning
We study the problem of meta-learning through the lens of online convex
optimization, developing a meta-algorithm bridging the gap between popular
gradient-based meta-learning and classical regularization-based multi-task
transfer methods. Our method is the first to simultaneously satisfy good sample
efficiency guarantees in the convex setting, with generalization bounds that
improve with task-similarity, while also being computationally scalable to
modern deep learning architectures and the many-task setting. Despite its
simplicity, the algorithm matches, up to a constant factor, a lower bound on
the performance of any such parameter-transfer method under natural task
similarity assumptions. We use experiments in both convex and deep learning
settings to verify and demonstrate the applicability of our theory.Comment: ICML 201
Adaptive Gradient-Based Meta-Learning Methods
We build a theoretical framework for designing and understanding practical
meta-learning methods that integrates sophisticated formalizations of
task-similarity with the extensive literature on online convex optimization and
sequential prediction algorithms. Our approach enables the task-similarity to
be learned adaptively, provides sharper transfer-risk bounds in the setting of
statistical learning-to-learn, and leads to straightforward derivations of
average-case regret bounds for efficient algorithms in settings where the
task-environment changes dynamically or the tasks share a certain geometric
structure. We use our theory to modify several popular meta-learning algorithms
and improve their meta-test-time performance on standard problems in few-shot
learning and federated learning.Comment: NeurIPS 201