45,064 research outputs found
ST-GAN: Spatial Transformer Generative Adversarial Networks for Image Compositing
We address the problem of finding realistic geometric corrections to a
foreground object such that it appears natural when composited into a
background image. To achieve this, we propose a novel Generative Adversarial
Network (GAN) architecture that utilizes Spatial Transformer Networks (STNs) as
the generator, which we call Spatial Transformer GANs (ST-GANs). ST-GANs seek
image realism by operating in the geometric warp parameter space. In
particular, we exploit an iterative STN warping scheme and propose a sequential
training strategy that achieves better results compared to naive training of a
single generator. One of the key advantages of ST-GAN is its applicability to
high-resolution images indirectly since the predicted warp parameters are
transferable between reference frames. We demonstrate our approach in two
applications: (1) visualizing how indoor furniture (e.g. from product images)
might be perceived in a room, (2) hallucinating how accessories like glasses
would look when matched with real portraits.Comment: Accepted to CVPR 2018 (website & code:
https://chenhsuanlin.bitbucket.io/spatial-transformer-GAN/
GAGAN: Geometry-Aware Generative Adversarial Networks
Deep generative models learned through adversarial training have become
increasingly popular for their ability to generate naturalistic image textures.
However, aside from their texture, the visual appearance of objects is
significantly influenced by their shape geometry; information which is not
taken into account by existing generative models. This paper introduces the
Geometry-Aware Generative Adversarial Networks (GAGAN) for incorporating
geometric information into the image generation process. Specifically, in GAGAN
the generator samples latent variables from the probability space of a
statistical shape model. By mapping the output of the generator to a canonical
coordinate frame through a differentiable geometric transformation, we enforce
the geometry of the objects and add an implicit connection from the prior to
the generated object. Experimental results on face generation indicate that the
GAGAN can generate realistic images of faces with arbitrary facial attributes
such as facial expression, pose, and morphology, that are of better quality
than current GAN-based methods. Our method can be used to augment any existing
GAN architecture and improve the quality of the images generated
Global versus Localized Generative Adversarial Nets
In this paper, we present a novel localized Generative Adversarial Net (GAN)
to learn on the manifold of real data. Compared with the classic GAN that {\em
globally} parameterizes a manifold, the Localized GAN (LGAN) uses local
coordinate charts to parameterize distinct local geometry of how data points
can transform at different locations on the manifold. Specifically, around each
point there exists a {\em local} generator that can produce data following
diverse patterns of transformations on the manifold. The locality nature of
LGAN enables local generators to adapt to and directly access the local
geometry without need to invert the generator in a global GAN. Furthermore, it
can prevent the manifold from being locally collapsed to a dimensionally
deficient tangent subspace by imposing an orthonormality prior between
tangents. This provides a geometric approach to alleviating mode collapse at
least locally on the manifold by imposing independence between data
transformations in different tangent directions. We will also demonstrate the
LGAN can be applied to train a robust classifier that prefers locally
consistent classification decisions on the manifold, and the resultant
regularizer is closely related with the Laplace-Beltrami operator. Our
experiments show that the proposed LGANs can not only produce diverse image
transformations, but also deliver superior classification performances
- …