In semi-supervised learning, the paradigm of self-training refers to the idea
of learning from pseudo-labels suggested by the learner itself. Across various
domains, corresponding methods have proven effective and achieve
state-of-the-art performance. However, pseudo-labels typically stem from ad-hoc
heuristics, relying on the quality of the predictions though without
guaranteeing their validity. One such method, so-called credal self-supervised
learning, maintains pseudo-supervision in the form of sets of (instead of
single) probability distributions over labels, thereby allowing for a flexible
yet uncertainty-aware labeling. Again, however, there is no justification
beyond empirical effectiveness. To address this deficiency, we make use of
conformal prediction, an approach that comes with guarantees on the validity of
set-valued predictions. As a result, the construction of credal sets of labels
is supported by a rigorous theoretical foundation, leading to better calibrated
and less error-prone supervision for unlabeled data. Along with this, we
present effective algorithms for learning from credal self-supervision. An
empirical study demonstrates excellent calibration properties of the
pseudo-supervision, as well as the competitiveness of our method on several
benchmark datasets.Comment: 26 pages, 5 figures, 10 tables, to be published at the 12th Symposium
on Conformal and Probabilistic Prediction with Applications (COPA 2023