30,565 research outputs found
Generative Ratio Matching Networks
Deep generative models can learn to generate realistic-looking images, but
many of the most effective methods are adversarial and involve a saddlepoint
optimization, which requires a careful balancing of training between a
generator network and a critic network. Maximum mean discrepancy networks
(MMD-nets) avoid this issue by using kernel as a fixed adversary, but
unfortunately, they have not on their own been able to match the generative
quality of adversarial training. In this work, we take their insight of using
kernels as fixed adversaries further and present a novel method for training
deep generative models that does not involve saddlepoint optimization. We call
our method generative ratio matching or GRAM for short. In GRAM, the generator
and the critic networks do not play a zero-sum game against each other,
instead, they do so against a fixed kernel. Thus GRAM networks are not only
stable to train like MMD-nets but they also match and beat the generative
quality of adversarially trained generative networks.Comment: ICLR 2020; Code: https://github.com/GRAM-net
Generative Adversarial Networks (GANs): Challenges, Solutions, and Future Directions
Generative Adversarial Networks (GANs) is a novel class of deep generative
models which has recently gained significant attention. GANs learns complex and
high-dimensional distributions implicitly over images, audio, and data.
However, there exists major challenges in training of GANs, i.e., mode
collapse, non-convergence and instability, due to inappropriate design of
network architecture, use of objective function and selection of optimization
algorithm. Recently, to address these challenges, several solutions for better
design and optimization of GANs have been investigated based on techniques of
re-engineered network architectures, new objective functions and alternative
optimization algorithms. To the best of our knowledge, there is no existing
survey that has particularly focused on broad and systematic developments of
these solutions. In this study, we perform a comprehensive survey of the
advancements in GANs design and optimization solutions proposed to handle GANs
challenges. We first identify key research issues within each design and
optimization technique and then propose a new taxonomy to structure solutions
by key research issues. In accordance with the taxonomy, we provide a detailed
discussion on different GANs variants proposed within each solution and their
relationships. Finally, based on the insights gained, we present the promising
research directions in this rapidly growing field.Comment: 42 pages, Figure 13, Table
- …