162,735 research outputs found
Neural Discrete Representation Learning
Learning useful representations without supervision remains a key challenge
in machine learning. In this paper, we propose a simple yet powerful generative
model that learns such discrete representations. Our model, the Vector
Quantised-Variational AutoEncoder (VQ-VAE), differs from VAEs in two key ways:
the encoder network outputs discrete, rather than continuous, codes; and the
prior is learnt rather than static. In order to learn a discrete latent
representation, we incorporate ideas from vector quantisation (VQ). Using the
VQ method allows the model to circumvent issues of "posterior collapse" --
where the latents are ignored when they are paired with a powerful
autoregressive decoder -- typically observed in the VAE framework. Pairing
these representations with an autoregressive prior, the model can generate high
quality images, videos, and speech as well as doing high quality speaker
conversion and unsupervised learning of phonemes, providing further evidence of
the utility of the learnt representations
Autonomous Reinforcement of Behavioral Sequences in Neural Dynamics
We introduce a dynamic neural algorithm called Dynamic Neural (DN)
SARSA(\lambda) for learning a behavioral sequence from delayed reward.
DN-SARSA(\lambda) combines Dynamic Field Theory models of behavioral sequence
representation, classical reinforcement learning, and a computational
neuroscience model of working memory, called Item and Order working memory,
which serves as an eligibility trace. DN-SARSA(\lambda) is implemented on both
a simulated and real robot that must learn a specific rewarding sequence of
elementary behaviors from exploration. Results show DN-SARSA(\lambda) performs
on the level of the discrete SARSA(\lambda), validating the feasibility of
general reinforcement learning without compromising neural dynamics.Comment: Sohrob Kazerounian, Matthew Luciw are Joint first author
Discrete and fuzzy dynamical genetic programming in the XCSF learning classifier system
A number of representation schemes have been presented for use within
learning classifier systems, ranging from binary encodings to neural networks.
This paper presents results from an investigation into using discrete and fuzzy
dynamical system representations within the XCSF learning classifier system. In
particular, asynchronous random Boolean networks are used to represent the
traditional condition-action production system rules in the discrete case and
asynchronous fuzzy logic networks in the continuous-valued case. It is shown
possible to use self-adaptive, open-ended evolution to design an ensemble of
such dynamical systems within XCSF to solve a number of well-known test
problems
Topological Neural Discrete Representation Learning \`a la Kohonen
Unsupervised learning of discrete representations from continuous ones in
neural networks (NNs) is the cornerstone of several applications today. Vector
Quantisation (VQ) has become a popular method to achieve such representations,
in particular in the context of generative models such as Variational
Auto-Encoders (VAEs). For example, the exponential moving average-based VQ
(EMA-VQ) algorithm is often used. Here we study an alternative VQ algorithm
based on the learning rule of Kohonen Self-Organising Maps (KSOMs; 1982) of
which EMA-VQ is a special case. In fact, KSOM is a classic VQ algorithm which
is known to offer two potential benefits over the latter: empirically, KSOM is
known to perform faster VQ, and discrete representations learned by KSOM form a
topological structure on the grid whose nodes are the discrete symbols,
resulting in an artificial version of the topographic map in the brain. We
revisit these properties by using KSOM in VQ-VAEs for image processing. In
particular, our experiments show that, while the speed-up compared to
well-configured EMA-VQ is only observable at the beginning of training, KSOM is
generally much more robust than EMA-VQ, e.g., w.r.t. the choice of
initialisation schemes. Our code is public.Comment: Two first author
- …