Generative Adversarial Mapping Networks

Generative Adversarial Networks (GANs) have shown impressive performance in generating photo-realistic images. They fit generative models by minimizing certain distance measure between the real image distribution and the generated data distribution. Several distance measures have been used, such as Jensen-Shannon divergence, $f$-divergence, and Wasserstein distance, and choosing an appropriate distance measure is very important for training the generative network. In this paper, we choose to use the maximum mean discrepancy (MMD) as the distance metric, which has several nice theoretical guarantees. In fact, generative moment matching network (GMMN) (Li, Swersky, and Zemel 2015) is such a generative model which contains only one generator network $G$ trained by directly minimizing MMD between the real and generated distributions. However, it fails to generate meaningful samples on challenging benchmark datasets, such as CIFAR-10 and LSUN. To improve on GMMN, we propose to add an extra network $F$, called mapper. $F$ maps both real data distribution and generated data distribution from the original data space to a feature representation space $\mathcal{R}$, and it is trained to maximize MMD between the two mapped distributions in $\mathcal{R}$, while the generator $G$ tries to minimize the MMD. We call the new model generative adversarial mapping networks (GAMNs). We demonstrate that the adversarial mapper $F$ can help $G$ to better capture the underlying data distribution. We also show that GAMN significantly outperforms GMMN, and is also superior to or comparable with other state-of-the-art GAN based methods on MNIST, CIFAR-10 and LSUN-Bedrooms datasets.Comment: 9 pages, 7 figure

Theoretical Insights into the Use of Structural Similarity Index In Generative Models and Inferential Autoencoders

Generative models and inferential autoencoders mostly make use of $\ell_2$ norm in their optimization objectives. In order to generate perceptually better images, this short paper theoretically discusses how to use Structural Similarity Index (SSIM) in generative models and inferential autoencoders. We first review SSIM, SSIM distance metrics, and SSIM kernel. We show that the SSIM kernel is a universal kernel and thus can be used in unconditional and conditional generated moment matching networks. Then, we explain how to use SSIM distance in variational and adversarial autoencoders and unconditional and conditional Generative Adversarial Networks (GANs). Finally, we propose to use SSIM distance rather than $\ell_2$ norm in least squares GAN.Comment: Accepted (to appear) in International Conference on Image Analysis and Recognition (ICIAR) 2020, Springe

Regularization of Conditional Generative Adversarial Networks by Moment Matching for Multimodal Generation

학위논문(석사)--서울대학교 대학원 :공과대학 컴퓨터공학부,2019. 8. 김건희.최근 조건부 GAN(conditional generative adversarial networks)의 등장으로 이미지 변환(image-to-image translation), 이미지 채우기(image inpainting)와 같은 조건부 이미지 생성 기술이 발달하게 되었다. 조건부 GAN은 거의 모든 경우 GAN 손실 함수와 재건 손실 함수를 함께 사용하여 트레이닝 되는데, 우리는 이 일반적인 트레이닝 방법론이 생성물의 다양성을 크게 훼손한다는 것을 밝힌다. 우리는 트레이닝의 안정성과 생성 다양성을 모두 달성하기 위해 새로운 손실 함수와 트레이닝 방식을 제안한다. 우리의 손실 함수는 재건 손실 함수만을 간단히 대체하기 때문에 사실상 모든 조건부 생성 문제에 적용할 수 있다. 우리는 Cityscapes와 CelebA 데이터셋을 대상으로 이미지 변환, 이미지 채우기, 초해상(super-resolution) 실험을 진행하여 우리의 방법론이 일반적으로 적용될 수 있음을 보이고, 정량적 평가를 통해서도 우리의 방법론이 이미지의 품질을 해치지 않으면서 높은 생성 다양성을 달성하는 것을 확인한다.Recent advances in conditional image generation tasks, such as image-to-image translation and image inpainting, can largely be accounted to the success of conditional GAN models, which are often optimized by the joint use of the GAN loss with the reconstruction loss. However, we show that this training recipe shared by almost all existing methods is problematic and has one critical side effect: lack of diversity in output samples. In order to accomplish both training stability and multimodal output generation, we propose novel training schemes with a new set of losses that simply replace the reconstruction loss, and thus are applicable to any conditional generation task. We show this by performing thorough experiments on image-to-image translation, super-resolution, and image inpainting tasks using Cityscapes, and CelebA dataset. Quantitative evaluation also confirms that our methods achieve a great diversity in outputs while retaining or even improving the quality of images.Chapter 1 Introduction 1 Chapter 2 Related Works 3 Chapter 3 Loss Mismatch of Conditional GANs 5 3.1 Preliminary: The Objective of Conditional GANs 5 3.2 Loss of Modality by the Reconstruction Loss 7 Chapter 4 Approach 10 4.1 The MLE for Mean and Variance 12 4.2 The Moment Reconstruction Loss 12 4.3 The Proxy Moment Reconstruction Loss 13 4.4 Analyses 14 Chapter 5 Experiments 17 5.1 Qualitative Evaluation 17 5.2 Quantitative Evaluation 18 Chapter 6 Conclusion 21 Appendix A Algorithms 28 Appendix B TrainingDetails 33 B.1 Common Configurations 33 B.2 Pix2Pix 34 B.3 SRGAN 34 B.4 GLCIC 35 Appendix C Preventive Effects on the Mode Collapse 37 Appendix D Generated Samples 39 D.1 Image-to-Image Translation (Pix2Pix) 39 D.2 Super-Resolution (SRGAN) 39 D.3 Image Inpainting (GLCIC) 39 Appendix E Mismatch between L1 Loss and GAN Loss 44 Appendix F Experiments on More Combinations of Loss Functions 46 요약 49 Acknowledgements 50Maste

Scenario Generation for Cooling, Heating, and Power Loads Using Generative Moment Matching Networks

Scenario generations of cooling, heating, and power loads are of great significance for the economic operation and stability analysis of integrated energy systems. In this paper, a novel deep generative network is proposed to model cooling, heating, and power load curves based on a generative moment matching networks (GMMN) where an auto-encoder transforms high-dimensional load curves into low-dimensional latent variables and the maximum mean discrepancy represents the similarity metrics between the generated samples and the real samples. After training the model, the new scenarios are generated by feeding Gaussian noises to the scenario generator of the GMMN. Unlike the explicit density models, the proposed GMMN does not need to artificially assume the probability distribution of the load curves, which leads to stronger universality. The simulation results show that the GMMN not only fits the probability distribution of multi-class load curves well, but also accurately captures the shape (e.g., large peaks, fast ramps, and fluctuation), frequency-domain characteristics, and temporal-spatial correlations of cooling, heating, and power loads. Furthermore, the energy consumption of generated samples closely resembles that of real samples.Comment: This paper has been accepted by CSEE Journal of Power and Energy System