113 research outputs found
One-for-All: Towards Universal Domain Translation with a Single StyleGAN
In this paper, we propose a novel translation model, UniTranslator, for
transforming representations between visually distinct domains under conditions
of limited training data and significant visual differences. The main idea
behind our approach is leveraging the domain-neutral capabilities of CLIP as a
bridging mechanism, while utilizing a separate module to extract abstract,
domain-agnostic semantics from the embeddings of both the source and target
realms. Fusing these abstract semantics with target-specific semantics results
in a transformed embedding within the CLIP space. To bridge the gap between the
disparate worlds of CLIP and StyleGAN, we introduce a new non-linear mapper,
the CLIP2P mapper. Utilizing CLIP embeddings, this module is tailored to
approximate the latent distribution in the P space, effectively acting as a
connector between these two spaces. The proposed UniTranslator is versatile and
capable of performing various tasks, including style mixing, stylization, and
translations, even in visually challenging scenarios across different visual
domains. Notably, UniTranslator generates high-quality translations that
showcase domain relevance, diversity, and improved image quality. UniTranslator
surpasses the performance of existing general-purpose models and performs well
against specialized models in representative tasks. The source code and trained
models will be released to the public
Adaptive Feature Interpolation for Low-Shot Image Generation
Training of generative models especially Generative Adversarial Networks can
easily diverge in low-data setting. To mitigate this issue, we propose a novel
implicit data augmentation approach which facilitates stable training and
synthesize high-quality samples without need of label information.
Specifically, we view the discriminator as a metric embedding of the real data
manifold, which offers proper distances between real data points. We then
utilize information in the feature space to develop a fully unsupervised and
data-driven augmentation method. Experiments on few-shot generation tasks show
the proposed method significantly improve results from strong baselines with
hundreds of training samples.Comment: ECCV'22. Code available at
https://github.com/dzld00/Adaptive-Feature-Interpolation-for-Low-Shot-Image-Generatio
- …