Thanks to their ability to learn flexible data-driven losses, Generative
Adversarial Networks (GANs) are an integral part of many semi- and
weakly-supervised methods for medical image segmentation. GANs jointly optimise
a generator and an adversarial discriminator on a set of training data. After
training is complete, the discriminator is usually discarded, and only the
generator is used for inference. But should we discard discriminators? In this
work, we argue that training stable discriminators produces expressive loss
functions that we can re-use at inference to detect and \textit{correct}
segmentation mistakes. First, we identify key challenges and suggest possible
solutions to make discriminators re-usable at inference. Then, we show that we
can combine discriminators with image reconstruction costs (via decoders) to
endow a causal perspective to test-time training and further improve the model.
Our method is simple and improves the test-time performance of pre-trained
GANs. Moreover, we show that it is compatible with standard post-processing
techniques and it has the potential to be used for Online Continual Learning.
With our work, we open new research avenues for re-using adversarial
discriminators at inference. Our code is available at
https://vios-s.github.io/adversarial-test-time-training.Comment: Accepted for publication at the Journal of Machine Learning for
Biomedical Imaging (MELBA) https://www.melba-journal.org/papers/2022:014.htm