21,169 research outputs found
SuperSpike: Supervised learning in multi-layer spiking neural networks
A vast majority of computation in the brain is performed by spiking neural
networks. Despite the ubiquity of such spiking, we currently lack an
understanding of how biological spiking neural circuits learn and compute
in-vivo, as well as how we can instantiate such capabilities in artificial
spiking circuits in-silico. Here we revisit the problem of supervised learning
in temporally coding multi-layer spiking neural networks. First, by using a
surrogate gradient approach, we derive SuperSpike, a nonlinear voltage-based
three factor learning rule capable of training multi-layer networks of
deterministic integrate-and-fire neurons to perform nonlinear computations on
spatiotemporal spike patterns. Second, inspired by recent results on feedback
alignment, we compare the performance of our learning rule under different
credit assignment strategies for propagating output errors to hidden units.
Specifically, we test uniform, symmetric and random feedback, finding that
simpler tasks can be solved with any type of feedback, while more complex tasks
require symmetric feedback. In summary, our results open the door to obtaining
a better scientific understanding of learning and computation in spiking neural
networks by advancing our ability to train them to solve nonlinear problems
involving transformations between different spatiotemporal spike-time patterns
Diffusion of Context and Credit Information in Markovian Models
This paper studies the problem of ergodicity of transition probability
matrices in Markovian models, such as hidden Markov models (HMMs), and how it
makes very difficult the task of learning to represent long-term context for
sequential data. This phenomenon hurts the forward propagation of long-term
context information, as well as learning a hidden state representation to
represent long-term context, which depends on propagating credit information
backwards in time. Using results from Markov chain theory, we show that this
problem of diffusion of context and credit is reduced when the transition
probabilities approach 0 or 1, i.e., the transition probability matrices are
sparse and the model essentially deterministic. The results found in this paper
apply to learning approaches based on continuous optimization, such as gradient
descent and the Baum-Welch algorithm.Comment: See http://www.jair.org/ for any accompanying file
Learning scalable and transferable multi-robot/machine sequential assignment planning via graph embedding
Can the success of reinforcement learning methods for simple combinatorial
optimization problems be extended to multi-robot sequential assignment
planning? In addition to the challenge of achieving near-optimal performance in
large problems, transferability to an unseen number of robots and tasks is
another key challenge for real-world applications. In this paper, we suggest a
method that achieves the first success in both challenges for robot/machine
scheduling problems.
Our method comprises of three components. First, we show a robot scheduling
problem can be expressed as a random probabilistic graphical model (PGM). We
develop a mean-field inference method for random PGM and use it for Q-function
inference. Second, we show that transferability can be achieved by carefully
designing two-step sequential encoding of problem state. Third, we resolve the
computational scalability issue of fitted Q-iteration by suggesting a heuristic
auction-based Q-iteration fitting method enabled by transferability we
achieved.
We apply our method to discrete-time, discrete space problems (Multi-Robot
Reward Collection (MRRC)) and scalably achieve 97% optimality with
transferability. This optimality is maintained under stochastic contexts. By
extending our method to continuous time, continuous space formulation, we claim
to be the first learning-based method with scalable performance among
multi-machine scheduling problems; our method scalability achieves comparable
performance to popular metaheuristics in Identical parallel machine scheduling
(IPMS) problems
- …