6,302 research outputs found
Spiking Neural Networks for Inference and Learning: A Memristor-based Design Perspective
On metrics of density and power efficiency, neuromorphic technologies have
the potential to surpass mainstream computing technologies in tasks where
real-time functionality, adaptability, and autonomy are essential. While
algorithmic advances in neuromorphic computing are proceeding successfully, the
potential of memristors to improve neuromorphic computing have not yet born
fruit, primarily because they are often used as a drop-in replacement to
conventional memory. However, interdisciplinary approaches anchored in machine
learning theory suggest that multifactor plasticity rules matching neural and
synaptic dynamics to the device capabilities can take better advantage of
memristor dynamics and its stochasticity. Furthermore, such plasticity rules
generally show much higher performance than that of classical Spike Time
Dependent Plasticity (STDP) rules. This chapter reviews the recent development
in learning with spiking neural network models and their possible
implementation with memristor-based hardware
On-chip Few-shot Learning with Surrogate Gradient Descent on a Neuromorphic Processor
Recent work suggests that synaptic plasticity dynamics in biological models
of neurons and neuromorphic hardware are compatible with gradient-based
learning (Neftci et al., 2019). Gradient-based learning requires iterating
several times over a dataset, which is both time-consuming and constrains the
training samples to be independently and identically distributed. This is
incompatible with learning systems that do not have boundaries between training
and inference, such as in neuromorphic hardware. One approach to overcome these
constraints is transfer learning, where a portion of the network is pre-trained
and mapped into hardware and the remaining portion is trained online. Transfer
learning has the advantage that pre-training can be accelerated offline if the
task domain is known, and few samples of each class are sufficient for learning
the target task at reasonable accuracies. Here, we demonstrate on-line
surrogate gradient few-shot learning on Intel's Loihi neuromorphic research
processor using features pre-trained with spike-based gradient
backpropagation-through-time. Our experimental results show that the Loihi chip
can learn gestures online using a small number of shots and achieve results
that are comparable to the models simulated on a conventional processor
Correlation-based model of artificially induced plasticity in motor cortex by a bidirectional brain-computer interface
Experiments show that spike-triggered stimulation performed with
Bidirectional Brain-Computer-Interfaces (BBCI) can artificially strengthen
connections between separate neural sites in motor cortex (MC). What are the
neuronal mechanisms responsible for these changes and how does targeted
stimulation by a BBCI shape population-level synaptic connectivity? The present
work describes a recurrent neural network model with probabilistic spiking
mechanisms and plastic synapses capable of capturing both neural and synaptic
activity statistics relevant to BBCI conditioning protocols. When spikes from a
neuron recorded at one MC site trigger stimuli at a second target site after a
fixed delay, the connections between sites are strengthened for spike-stimulus
delays consistent with experimentally derived spike time dependent plasticity
(STDP) rules. However, the relationship between STDP mechanisms at the level of
networks, and their modification with neural implants remains poorly
understood. Using our model, we successfully reproduces key experimental
results and use analytical derivations, along with novel experimental data. We
then derive optimal operational regimes for BBCIs, and formulate predictions
concerning the efficacy of spike-triggered stimulation in different regimes of
cortical activity.Comment: 35 pages, 9 figure
- …