Unsupervised domain adaptation (UDA) aims to learn a well-performed model in
an unlabeled target domain by leveraging labeled data from one or multiple
related source domains. It remains a great challenge due to 1) the lack of
annotations in the target domain and 2) the rich discrepancy between the
distributions of source and target data. We propose Spectral UDA (SUDA), an
efficient yet effective UDA technique that works in the spectral space and is
generic across different visual recognition tasks in detection, classification
and segmentation. SUDA addresses UDA challenges from two perspectives. First,
it mitigates inter-domain discrepancies by a spectrum transformer (ST) that
maps source and target images into spectral space and learns to enhance
domain-invariant spectra while suppressing domain-variant spectra
simultaneously. To this end, we design novel adversarial multi-head spectrum
attention that leverages contextual information to identify domain-variant and
domain-invariant spectra effectively. Second, it mitigates the lack of
annotations in target domain by introducing multi-view spectral learning which
aims to learn comprehensive yet confident target representations by maximizing
the mutual information among multiple ST augmentations capturing different
spectral views of each target sample. Extensive experiments over different
visual tasks (e.g., detection, classification and segmentation) show that SUDA
achieves superior accuracy and it is also complementary with state-of-the-art
UDA methods with consistent performance boosts but little extra computation