Generative Temporal Models with Spatial Memory for Partially Observed Environments

In model-based reinforcement learning, generative and temporal models of environments can be leveraged to boost agent performance, either by tuning the agent's representations during training or via use as part of an explicit planning mechanism. However, their application in practice has been limited to simplistic environments, due to the difficulty of training such models in larger, potentially partially-observed and 3D environments. In this work we introduce a novel action-conditioned generative model of such challenging environments. The model features a non-parametric spatial memory system in which we store learned, disentangled representations of the environment. Low-dimensional spatial updates are computed using a state-space model that makes use of knowledge on the prior dynamics of the moving agent, and high-dimensional visual observations are modelled with a Variational Auto-Encoder. The result is a scalable architecture capable of performing coherent predictions over hundreds of time steps across a range of partially observed 2D and 3D environments.Comment: ICML 201

Leveraging the Exact Likelihood of Deep Latent Variable Models

Deep latent variable models (DLVMs) combine the approximation abilities of deep neural networks and the statistical foundations of generative models. Variational methods are commonly used for inference; however, the exact likelihood of these models has been largely overlooked. The purpose of this work is to study the general properties of this quantity and to show how they can be leveraged in practice. We focus on important inferential problems that rely on the likelihood: estimation and missing data imputation. First, we investigate maximum likelihood estimation for DLVMs: in particular, we show that most unconstrained models used for continuous data have an unbounded likelihood function. This problematic behaviour is demonstrated to be a source of mode collapse. We also show how to ensure the existence of maximum likelihood estimates, and draw useful connections with nonparametric mixture models. Finally, we describe an algorithm for missing data imputation using the exact conditional likelihood of a deep latent variable model. On several data sets, our algorithm consistently and significantly outperforms the usual imputation scheme used for DLVMs

Memory-Based Learning of Latent Structures for Generative Adversarial Networks

학위논문 (석사)-- 서울대학교 대학원 : 공과대학 컴퓨터공학부, 2019. 2. 김건희.본 연구는 Generative Adversarial Network (GAN) 모델의 학습 과정에서 발생하는 두가지 문제점을 해결하는 방안을 제시하였다. 먼저, 일반적인 GAN 모델은 사진과 같은 복잡한 확률변수의 분포를 모델링할 때 잠재변수의 사전확률분포로 표준정규분포를 사용한다. 그러나 이런 연속적인 잠재변수를 사용할 경우 서로 다른 데이터 샘플간의 구조적 불연속성을 반영하기 어렵다. 또 다른 문제점으로, GAN 모델에서 판별자는 학습 과정에서 과거에 생성자 모델이 생성했던 데이터 샘플에 대한 정보를 망각하며, 이로인해 학습 과정이 불안정해진다. 이 두가지 문제점은 생성자가 판별자가 공유하는 memory network를 동시에 학습함으로써 크게 완화할 수 있다. 생성자가 학습 데이터에 내재된 군집의 분포를 학습한다면 이를 통해 구조적 불연속성으로 인한 성능 하락을 피할 수 있으며, 판별자가 주어진 입력 데이터에 대한 판별을 할 때 학습 전 과정에 걸쳐 생성자가 생성했던 데이터 샘플들로부터 학습된 군집 분포를 참조한다면 망각 문제로 인한 영향을 덜 받게 된다. 본 연구에서 제시한 memoryGAN 모델은 비지도학습을 통해 데이터에 내재된 군집의 분포를 학습하여 구조적 불연속성 문제와 망각 문제를 완화하며, 대부분의 GAN 모델에 적용할 수 있다. Fashion-MNIST, CelebA, CIFAR10, 그리고 Chairs 데이터셋에 대한 성능 평가 및 시각화 실험을 통해 memoryGAN이 확률론적으로 해석 가능한 모델이며, 높은 수준의 사진 샘플을 생성한다는 것을 보였다. 특히 memoryGAN은 개선된 최적화 방법이나 Weaker divergence를 도입하지 않고도 CIFAR10 데이터셋에서 Inception Score를 기준으로 비지도학습 방식의 GAN 모델 중 높은 성능을 달성했다.We propose an approach to address two issues that commonly occur during training of unsupervised GANs. First, since GANs use only a continuous latent distribution to embed multiple classes or clusters of data, they often do not correctly handle the structural discontinuity between disparate classes in a latent space. Second, discriminators of GANs easily forget about past generated samples by generators, incurring instability during adversarial training. We argue that these two infamous problems of unsupervised GAN training can be largely alleviated by a learnable memory network to which both generators and discriminators can access. Generators can effectively learn representation of training samples to understand underlying cluster distributions of data, which ease the structure discontinuity problem. At the same time, discriminators can better memorize clusters of previously generated samples, which mitigate the forgetting problem. We propose a novel end-to-end GAN model named memoryGAN, which involves a memory network that is unsupervisedly trainable and integrable to many existing GAN models. With evaluations on multiple datasets such as Fashion-MNIST, CelebA, CIFAR10, and Chairs, we show that our model is probabilistically interpretable, and generates realistic image samples of high visual fidelity. The memoryGAN also achieves the state-of-the-art inception scores over unsupervised GAN models on the CIFAR10 dataset, without any optimization tricks and weaker divergences.Introduction Related Works The MemoryGAN Experiments ConclusionMaste

Learning to Learn Variational Semantic Memory

In this paper, we introduce variational semantic memory into meta-learning to acquire long-term knowledge for few-shot learning. The variational semantic memory accrues and stores semantic information for the probabilistic inference of class prototypes in a hierarchical Bayesian framework. The semantic memory is grown from scratch and gradually consolidated by absorbing information from tasks it experiences. By doing so, it is able to accumulate long-term, general knowledge that enables it to learn new concepts of objects. We formulate memory recall as the variational inference of a latent memory variable from addressed contents, which offers a principled way to adapt the knowledge to individual tasks. Our variational semantic memory, as a new long-term memory module, confers principled recall and update mechanisms that enable semantic information to be efficiently accrued and adapted for few-shot learning. Experiments demonstrate that the probabilistic modelling of prototypes achieves a more informative representation of object classes compared to deterministic vectors. The consistent new state-of-the-art performance on four benchmarks shows the benefit of variational semantic memory in boosting few-shot recognition.Comment: accepted to NeurIPS 2020; code is available in https://github.com/YDU-uva/VS