To improve instance-level detection/segmentation performance, existing
self-supervised and semi-supervised methods extract either task-unrelated or
task-specific training signals from unlabeled data. We show that these two
approaches, at the two extreme ends of the task-specificity spectrum, are
suboptimal for the task performance. Utilizing too little task-specific
training signals causes underfitting to the ground-truth labels of downstream
tasks, while the opposite causes overfitting to the ground-truth labels. To
this end, we propose a novel Class-Agnostic Semi-Supervised Learning (CA-SSL)
framework to achieve a more favorable task-specificity balance in extracting
training signals from unlabeled data. CA-SSL has three training stages that act
on either ground-truth labels (labeled data) or pseudo labels (unlabeled data).
This decoupling strategy avoids the complicated scheme in traditional SSL
methods that balances the contributions from both data types. Especially, we
introduce a warmup training stage to achieve a more optimal balance in task
specificity by ignoring class information in the pseudo labels, while
preserving localization training signals. As a result, our warmup model can
better avoid underfitting/overfitting when fine-tuned on the ground-truth
labels in detection and segmentation tasks. Using 3.6M unlabeled data, we
achieve a significant performance gain of 4.7% over ImageNet-pretrained
baseline on FCOS object detection. In addition, our warmup model demonstrates
excellent transferability to other detection and segmentation frameworks.Comment: Appeared in ECCV202