1,827 research outputs found
Variational Walkback: Learning a Transition Operator as a Stochastic Recurrent Net
We propose a novel method to directly learn a stochastic transition operator
whose repeated application provides generated samples. Traditional undirected
graphical models approach this problem indirectly by learning a Markov chain
model whose stationary distribution obeys detailed balance with respect to a
parameterized energy function. The energy function is then modified so the
model and data distributions match, with no guarantee on the number of steps
required for the Markov chain to converge. Moreover, the detailed balance
condition is highly restrictive: energy based models corresponding to neural
networks must have symmetric weights, unlike biological neural circuits. In
contrast, we develop a method for directly learning arbitrarily parameterized
transition operators capable of expressing non-equilibrium stationary
distributions that violate detailed balance, thereby enabling us to learn more
biologically plausible asymmetric neural networks and more general non-energy
based dynamical systems. The proposed training objective, which we derive via
principled variational methods, encourages the transition operator to "walk
back" in multi-step trajectories that start at data-points, as quickly as
possible back to the original data points. We present a series of experimental
results illustrating the soundness of the proposed approach, Variational
Walkback (VW), on the MNIST, CIFAR-10, SVHN and CelebA datasets, demonstrating
superior samples compared to earlier attempts to learn a transition operator.
We also show that although each rapid training trajectory is limited to a
finite but variable number of steps, our transition operator continues to
generate good samples well past the length of such trajectories, thereby
demonstrating the match of its non-equilibrium stationary distribution to the
data distribution. Source Code: http://github.com/anirudh9119/walkback_nips17Comment: To appear at NIPS 201
One-Shot Learning using Mixture of Variational Autoencoders: a Generalization Learning approach
Deep learning, even if it is very successful nowadays, traditionally needs
very large amounts of labeled data to perform excellent on the classification
task. In an attempt to solve this problem, the one-shot learning paradigm,
which makes use of just one labeled sample per class and prior knowledge,
becomes increasingly important. In this paper, we propose a new one-shot
learning method, dubbed MoVAE (Mixture of Variational AutoEncoders), to perform
classification. Complementary to prior studies, MoVAE represents a shift of
paradigm in comparison with the usual one-shot learning methods, as it does not
use any prior knowledge. Instead, it starts from zero knowledge and one labeled
sample per class. Afterward, by using unlabeled data and the generalization
learning concept (in a way, more as humans do), it is capable to gradually
improve by itself its performance. Even more, if there are no unlabeled data
available MoVAE can still perform well in one-shot learning classification. We
demonstrate empirically the efficiency of our proposed approach on three
datasets, i.e. the handwritten digits (MNIST), fashion products
(Fashion-MNIST), and handwritten characters (Omniglot), showing that MoVAE
outperforms state-of-the-art one-shot learning algorithms
- …