1 research outputs found
Robust Semi-Supervised Learning with Out of Distribution Data
Recent Semi-supervised learning (SSL) works show significant improvement in
SSL algorithms' performance using better-unlabeled data representations.
However, recent work [Oliver et al., 2018] shows that the SSL algorithm's
performance could degrade when the unlabeled set has out-of-distribution
examples (OODs). In this work, we first study the critical causes of OOD's
negative impact on SSL algorithms. We found that (1) the OOD's effect on the
SSL algorithm's performance increases as its distance to the decision boundary
decreases, and (2) Batch Normalization (BN), a popular module, could degrade
the performance instead of improving the performance when the unlabeled set
contains OODs. To address the above causes, we proposed a novel unified-robust
SSL approach that can be easily extended to many existing SSL algorithms, and
improve their robustness against OODs. In particular, we propose a simple
modification of batch normalization, called weighted batch normalization, that
improves BN's robustness against OODs. We also developed two efficient
hyper-parameter optimization algorithms that have different tradeoffs in
computational efficiency and accuracy. Extensive experiments on synthetic and
real-world datasets prove that our proposed approaches significantly improves
the robustness of four representative SSL algorithms against OODs compared with
four state-of-the-art robust SSL approaches.Comment: Preprin