19,194 research outputs found
Quality Aware Generative Adversarial Networks
Generative Adversarial Networks (GANs) have become a very popular tool for
implicitly learning high-dimensional probability distributions. Several
improvements have been made to the original GAN formulation to address some of
its shortcomings like mode collapse, convergence issues, entanglement, poor
visual quality etc. While a significant effort has been directed towards
improving the visual quality of images generated by GANs, it is rather
surprising that objective image quality metrics have neither been employed as
cost functions nor as regularizers in GAN objective functions. In this work, we
show how a distance metric that is a variant of the Structural SIMilarity
(SSIM) index (a popular full-reference image quality assessment algorithm), and
a novel quality aware discriminator gradient penalty function that is inspired
by the Natural Image Quality Evaluator (NIQE, a popular no-reference image
quality assessment algorithm) can each be used as excellent regularizers for
GAN objective functions. Specifically, we demonstrate state-of-the-art
performance using the Wasserstein GAN gradient penalty (WGAN-GP) framework over
CIFAR-10, STL10 and CelebA datasets.Comment: 10 pages, NeurIPS 201
GAGAN: Geometry-Aware Generative Adversarial Networks
Deep generative models learned through adversarial training have become
increasingly popular for their ability to generate naturalistic image textures.
However, aside from their texture, the visual appearance of objects is
significantly influenced by their shape geometry; information which is not
taken into account by existing generative models. This paper introduces the
Geometry-Aware Generative Adversarial Networks (GAGAN) for incorporating
geometric information into the image generation process. Specifically, in GAGAN
the generator samples latent variables from the probability space of a
statistical shape model. By mapping the output of the generator to a canonical
coordinate frame through a differentiable geometric transformation, we enforce
the geometry of the objects and add an implicit connection from the prior to
the generated object. Experimental results on face generation indicate that the
GAGAN can generate realistic images of faces with arbitrary facial attributes
such as facial expression, pose, and morphology, that are of better quality
than current GAN-based methods. Our method can be used to augment any existing
GAN architecture and improve the quality of the images generated
NeRF-GAN Distillation for Efficient 3D-Aware Generation with Convolutions
Pose-conditioned convolutional generative models struggle with high-quality
3D-consistent image generation from single-view datasets, due to their lack of
sufficient 3D priors. Recently, the integration of Neural Radiance Fields
(NeRFs) and generative models, such as Generative Adversarial Networks (GANs),
has transformed 3D-aware generation from single-view images. NeRF-GANs exploit
the strong inductive bias of neural 3D representations and volumetric rendering
at the cost of higher computational complexity. This study aims at revisiting
pose-conditioned 2D GANs for efficient 3D-aware generation at inference time by
distilling 3D knowledge from pretrained NeRF-GANs. We propose a simple and
effective method, based on re-using the well-disentangled latent space of a
pre-trained NeRF-GAN in a pose-conditioned convolutional network to directly
generate 3D-consistent images corresponding to the underlying 3D
representations. Experiments on several datasets demonstrate that the proposed
method obtains results comparable with volumetric rendering in terms of quality
and 3D consistency while benefiting from the computational advantage of
convolutional networks. The code will be available at:
https://github.com/mshahbazi72/NeRF-GAN-Distillatio
- …