4,823 research outputs found
Max-Sliced Wasserstein Distance and its use for GANs
Generative adversarial nets (GANs) and variational auto-encoders have
significantly improved our distribution modeling capabilities, showing promise
for dataset augmentation, image-to-image translation and feature learning.
However, to model high-dimensional distributions, sequential training and
stacked architectures are common, increasing the number of tunable
hyper-parameters as well as the training time. Nonetheless, the sample
complexity of the distance metrics remains one of the factors affecting GAN
training. We first show that the recently proposed sliced Wasserstein distance
has compelling sample complexity properties when compared to the Wasserstein
distance. To further improve the sliced Wasserstein distance we then analyze
its `projection complexity' and develop the max-sliced Wasserstein distance
which enjoys compelling sample complexity while reducing projection complexity,
albeit necessitating a max estimation. We finally illustrate that the proposed
distance trains GANs on high-dimensional images up to a resolution of 256x256
easily.Comment: Accepted to CVPR 201
Demystifying MMD GANs
We investigate the training and performance of generative adversarial networks using the Maximum Mean Discrepancy (MMD) as critic, termed MMD GANs. As our main theoretical contribution, we clarify the situation with bias in GAN loss functions raised by recent work: we show that gradient estimators used in the optimization process for both MMD GANs and Wasserstein GANs are unbiased, but learning a discriminator based on samples leads to biased gradients for the generator parameters. We also discuss the issue of kernel choice for the MMD critic, and characterize the kernel corresponding to the energy distance used for the Cramer GAN critic. Being an integral probability metric, the MMD benefits from training strategies recently developed for Wasserstein GANs. In experiments, the MMD GAN is able to employ a smaller critic network than the Wasserstein GAN, resulting in a simpler and faster-training algorithm with matching performance. We also propose an improved measure of GAN convergence, the Kernel Inception Distance, and show how to use it to dynamically adapt learning rates during GAN training
Wasserstein Divergence for GANs
In many domains of computer vision, generative adversarial networks (GANs)
have achieved great success, among which the family of Wasserstein GANs (WGANs)
is considered to be state-of-the-art due to the theoretical contributions and
competitive qualitative performance. However, it is very challenging to
approximate the -Lipschitz constraint required by the Wasserstein-1
metric~(W-met). In this paper, we propose a novel Wasserstein
divergence~(W-div), which is a relaxed version of W-met and does not require
the -Lipschitz constraint. As a concrete application, we introduce a
Wasserstein divergence objective for GANs~(WGAN-div), which can faithfully
approximate W-div through optimization. Under various settings, including
progressive growing training, we demonstrate the stability of the proposed
WGAN-div owing to its theoretical and practical advantages over WGANs. Also, we
study the quantitative and visual performance of WGAN-div on standard image
synthesis benchmarks of computer vision, showing the superior performance of
WGAN-div compared to the state-of-the-art methods.Comment: accepted by eccv_2018, correct minor error
- …