5,576 research outputs found
Sparsity Aware Normalization for GANs
Generative adversarial networks (GANs) are known to benefit from
regularization or normalization of their critic (discriminator) network during
training. In this paper, we analyze the popular spectral normalization scheme,
find a significant drawback and introduce sparsity aware normalization (SAN), a
new alternative approach for stabilizing GAN training. As opposed to other
normalization methods, our approach explicitly accounts for the sparse nature
of the feature maps in convolutional networks with ReLU activations. We
illustrate the effectiveness of our method through extensive experiments with a
variety of network architectures. As we show, sparsity is particularly dominant
in critics used for image-to-image translation settings. In these cases our
approach improves upon existing methods, in less training epochs and with
smaller capacity networks, while requiring practically no computational
overhead.Comment: AAAI Conference on Artificial Intelligence (AAAI-21
SALSA-TEXT : self attentive latent space based adversarial text generation
Inspired by the success of self attention mechanism and Transformer
architecture in sequence transduction and image generation applications, we
propose novel self attention-based architectures to improve the performance of
adversarial latent code- based schemes in text generation. Adversarial latent
code-based text generation has recently gained a lot of attention due to their
promising results. In this paper, we take a step to fortify the architectures
used in these setups, specifically AAE and ARAE. We benchmark two latent
code-based methods (AAE and ARAE) designed based on adversarial setups. In our
experiments, the Google sentence compression dataset is utilized to compare our
method with these methods using various objective and subjective measures. The
experiments demonstrate the proposed (self) attention-based models outperform
the state-of-the-art in adversarial code-based text generation.Comment: 10 pages, 3 figures, under review at ICLR 201
- …