Spurious correlations can cause strong biases in deep neural networks,
impairing generalization ability. While most existing debiasing methods require
full supervision on either spurious attributes or target labels, training a
debiased model from a limited amount of both annotations is still an open
question. To address this issue, we investigate an interesting phenomenon using
the spectral analysis of latent representations: spuriously correlated
attributes make neural networks inductively biased towards encoding lower
effective rank representations. We also show that a rank regularization can
amplify this bias in a way that encourages highly correlated features.
Leveraging these findings, we propose a self-supervised debiasing framework
potentially compatible with unlabeled samples. Specifically, we first pretrain
a biased encoder in a self-supervised manner with the rank regularization,
serving as a semantic bottleneck to enforce the encoder to learn the spuriously
correlated attributes. This biased encoder is then used to discover and
upweight bias-conflicting samples in a downstream task, serving as a boosting
to effectively debias the main model. Remarkably, the proposed debiasing
framework significantly improves the generalization performance of
self-supervised learning baselines and, in some cases, even outperforms
state-of-the-art supervised debiasing approaches