In this study, we delve into the task of few-shot Generative Domain
Adaptation (GDA), which involves transferring a pre-trained generator from one
domain to a new domain using only a few reference images. Inspired by the way
human brains acquire knowledge in new domains, we present an innovative
generator structure called Domain Re-Modulation (DoRM). DoRM not only meets the
criteria of high quality, large synthesis diversity, and cross-domain
consistency, which were achieved by previous research in GDA, but also
incorporates memory and domain association, akin to how human brains operate.
Specifically, DoRM freezes the source generator and introduces new mapping and
affine modules (M&A modules) to capture the attributes of the target domain
during GDA. This process resembles the formation of new synapses in human
brains. Consequently, a linearly combinable domain shift occurs in the style
space. By incorporating multiple new M&A modules, the generator gains the
capability to perform high-fidelity multi-domain and hybrid-domain generation.
Moreover, to maintain cross-domain consistency more effectively, we introduce a
similarity-based structure loss. This loss aligns the auto-correlation map of
the target image with its corresponding auto-correlation map of the source
image during training. Through extensive experiments, we demonstrate the
superior performance of our DoRM and similarity-based structure loss in
few-shot GDA, both quantitatively and qualitatively. The code will be available
at https://github.com/wuyi2020/DoRM.Comment: Under Revie