480 research outputs found
MonoFlow: Rethinking Divergence GANs via the Perspective of Wasserstein Gradient Flows
The conventional understanding of adversarial training in generative
adversarial networks (GANs) is that the discriminator is trained to estimate a
divergence, and the generator learns to minimize this divergence. We argue that
despite the fact that many variants of GANs were developed following this
paradigm, the current theoretical understanding of GANs and their practical
algorithms are inconsistent. In this paper, we leverage Wasserstein gradient
flows which characterize the evolution of particles in the sample space, to
gain theoretical insights and algorithmic inspiration of GANs. We introduce a
unified generative modeling framework - MonoFlow: the particle evolution is
rescaled via a monotonically increasing mapping of the log density ratio. Under
our framework, adversarial training can be viewed as a procedure first
obtaining MonoFlow's vector field via training the discriminator and the
generator learns to draw the particle flow defined by the corresponding vector
field. We also reveal the fundamental difference between variational divergence
minimization and adversarial training. This analysis helps us to identify what
types of generator loss functions can lead to the successful training of GANs
and suggest that GANs may have more loss designs beyond the literature (e.g.,
non-saturated loss), as long as they realize MonoFlow. Consistent empirical
studies are included to validate the effectiveness of our framework
- …