4,586 research outputs found
Embedded CycleGAN for Shape-Agnostic Image-to-Image Translation
Image-to-Image translation is the task of translating images between domains while maintaining the identity of the images. The task can be used for entertainment purposes and applications, data augmentation, semantic image segmentation, and more. Generative Adversarial Networks (GANs), and in particular Conditional GANs have recently shown incredible success in image-to-image translation and semantic manipulation. However, such methods require paired data, meaning that an image must have ground-truth translations across domains. Cycle-consistent GANs solve this problem by using unpaired data. Such methods work well for translations that involve color and texture changes but fail when shape changes are required. This research analyzes the trade-offs between the cycle-consistency importance and the necessary shape changes required for natural looking imagery. The research proposes simple architectural and loss changes to maintain cycle-consistency strength while allowing the model to perform shape changes as required. The results demonstrate improved translations between domains that require shape changes while preserving performance between domains that don’t require shape changes
AlignFlow: Cycle Consistent Learning from Multiple Domains via Normalizing Flows
Given datasets from multiple domains, a key challenge is to efficiently
exploit these data sources for modeling a target domain. Variants of this
problem have been studied in many contexts, such as cross-domain translation
and domain adaptation. We propose AlignFlow, a generative modeling framework
that models each domain via a normalizing flow. The use of normalizing flows
allows for a) flexibility in specifying learning objectives via adversarial
training, maximum likelihood estimation, or a hybrid of the two methods; and b)
learning and exact inference of a shared representation in the latent space of
the generative model. We derive a uniform set of conditions under which
AlignFlow is marginally-consistent for the different learning objectives.
Furthermore, we show that AlignFlow guarantees exact cycle consistency in
mapping datapoints from a source domain to target and back to the source
domain. Empirically, AlignFlow outperforms relevant baselines on image-to-image
translation and unsupervised domain adaptation and can be used to
simultaneously interpolate across the various domains using the learned
representation.Comment: AAAI 202
Recycle-GAN: Unsupervised Video Retargeting
We introduce a data-driven approach for unsupervised video retargeting that
translates content from one domain to another while preserving the style native
to a domain, i.e., if contents of John Oliver's speech were to be transferred
to Stephen Colbert, then the generated content/speech should be in Stephen
Colbert's style. Our approach combines both spatial and temporal information
along with adversarial losses for content translation and style preservation.
In this work, we first study the advantages of using spatiotemporal constraints
over spatial constraints for effective retargeting. We then demonstrate the
proposed approach for the problems where information in both space and time
matters such as face-to-face translation, flower-to-flower, wind and cloud
synthesis, sunrise and sunset.Comment: ECCV 2018; Please refer to project webpage for videos -
http://www.cs.cmu.edu/~aayushb/Recycle-GA
- …