2 research outputs found
Online Continual Adaptation with Active Self-Training
Models trained with offline data often suffer from continual distribution
shifts and expensive labeling in changing environments. This calls for a new
online learning paradigm where the learner can continually adapt to changing
environments with limited labels. In this paper, we propose a new online
setting -- Online Active Continual Adaptation, where the learner aims to
continually adapt to changing distributions using both unlabeled samples and
active queries of limited labels. To this end, we propose Online Self-Adaptive
Mirror Descent (OSAMD), which adopts an online teacher-student structure to
enable online self-training from unlabeled data, and a margin-based criterion
that decides whether to query the labels to track changing distributions.
Theoretically, we show that, in the separable case, OSAMD has an
dynamic regret bound under mild assumptions, which is even tighter than the
lower bound of traditional online learning with full labels.
In the general case, we show a regret bound of , where denotes the separability of domains and is
usually small. Our theoretical results show that OSAMD can fast adapt to
changing environments with active queries. Empirically, we demonstrate that
OSAMD achieves favorable regrets under changing environments with limited
labels on both simulated and real-world data, which corroborates our
theoretical findings
Understanding Self-Training for Gradual Domain Adaptation
Machine learning systems must adapt to data distributions that evolve over
time, in applications ranging from sensor networks and self-driving car
perception modules to brain-machine interfaces. We consider gradual domain
adaptation, where the goal is to adapt an initial classifier trained on a
source domain given only unlabeled data that shifts gradually in distribution
towards a target domain. We prove the first non-vacuous upper bound on the
error of self-training with gradual shifts, under settings where directly
adapting to the target domain can result in unbounded error. The theoretical
analysis leads to algorithmic insights, highlighting that regularization and
label sharpening are essential even when we have infinite data, and suggesting
that self-training works particularly well for shifts with small
Wasserstein-infinity distance. Leveraging the gradual shift structure leads to
higher accuracies on a rotating MNIST dataset and a realistic Portraits
dataset