75 research outputs found
A Unified Generative Adversarial Network Training via Self-Labeling and Self-Attention
We propose a novel GAN training scheme that can handle any level of labeling
in a unified manner. Our scheme introduces a form of artificial labeling that
can incorporate manually defined labels, when available, and induce an
alignment between them. To define the artificial labels, we exploit the
assumption that neural network generators can be trained more easily to map
nearby latent vectors to data with semantic similarities, than across separate
categories. We use generated data samples and their corresponding artificial
conditioning labels to train a classifier. The classifier is then used to
self-label real data. To boost the accuracy of the self-labeling, we also use
the exponential moving average of the classifier. However, because the
classifier might still make mistakes, especially at the beginning of the
training, we also refine the labels through self-attention, by using the
labeling of real data samples only when the classifier outputs a high
classification probability score. We evaluate our approach on CIFAR-10, STL-10
and SVHN, and show that both self-labeling and self-attention consistently
improve the quality of generated data. More surprisingly, we find that the
proposed scheme can even outperform class-conditional GANs
SEAN: Image Synthesis with Semantic Region-Adaptive Normalization
We propose semantic region-adaptive normalization (SEAN), a simple but
effective building block for Generative Adversarial Networks conditioned on
segmentation masks that describe the semantic regions in the desired output
image. Using SEAN normalization, we can build a network architecture that can
control the style of each semantic region individually, e.g., we can specify
one style reference image per region. SEAN is better suited to encode,
transfer, and synthesize style than the best previous method in terms of
reconstruction quality, variability, and visual quality. We evaluate SEAN on
multiple datasets and report better quantitative metrics (e.g. FID, PSNR) than
the current state of the art. SEAN also pushes the frontier of interactive
image editing. We can interactively edit images by changing segmentation masks
or the style for any given region. We can also interpolate styles from two
reference images per region.Comment: Accepted as a CVPR 2020 oral paper. The interactive demo is available
at https://youtu.be/0Vbj9xFgoU
- …