8,237 research outputs found
Learning Spiking Neural Systems with the Event-Driven Forward-Forward Process
We develop a novel credit assignment algorithm for information processing
with spiking neurons without requiring feedback synapses. Specifically, we
propose an event-driven generalization of the forward-forward and the
predictive forward-forward learning processes for a spiking neural system that
iteratively processes sensory input over a stimulus window. As a result, the
recurrent circuit computes the membrane potential of each neuron in each layer
as a function of local bottom-up, top-down, and lateral signals, facilitating a
dynamic, layer-wise parallel form of neural computation. Unlike spiking neural
coding, which relies on feedback synapses to adjust neural electrical activity,
our model operates purely online and forward in time, offering a promising way
to learn distributed representations of sensory data patterns with temporal
spike signals. Notably, our experimental results on several pattern datasets
demonstrate that the even-driven forward-forward (ED-FF) framework works well
for training a dynamic recurrent spiking system capable of both classification
and reconstruction
The Predictive Forward-Forward Algorithm
We propose the predictive forward-forward (PFF) algorithm for conducting
credit assignment in neural systems. Specifically, we design a novel, dynamic
recurrent neural system that learns a directed generative circuit jointly and
simultaneously with a representation circuit. Notably, the system integrates
learnable lateral competition, noise injection, and elements of predictive
coding, an emerging and viable neurobiological process theory of cortical
function, with the forward-forward (FF) adaptation scheme. Furthermore, PFF
efficiently learns to propagate learning signals and updates synapses with
forward passes only, eliminating key structural and computational constraints
imposed by backpropagation-based schemes. Besides computational advantages, the
PFF process could prove useful for understanding the learning mechanisms behind
biological neurons that use local signals despite missing feedback connections.
We run experiments on image data and demonstrate that the PFF procedure works
as well as backpropagation, offering a promising brain-inspired algorithm for
classifying, reconstructing, and synthesizing data patterns.Comment: More revisions/edits, update to key diagram depicting PFF process,
link to algorithm / simulation code (repo) now include
Convolutional Neural Generative Coding: Scaling Predictive Coding to Natural Images
In this work, we develop convolutional neural generative coding (Conv-NGC), a
generalization of predictive coding to the case of
convolution/deconvolution-based computation. Specifically, we concretely
implement a flexible neurobiologically-motivated algorithm that progressively
refines latent state maps in order to dynamically form a more accurate internal
representation/reconstruction model of natural images. The performance of the
resulting sensory processing system is evaluated on several benchmark datasets
such as Color-MNIST, CIFAR-10, and Street House View Numbers (SVHN). We study
the effectiveness of our brain-inspired neural system on the tasks of
reconstruction and image denoising and find that it is competitive with
convolutional auto-encoding systems trained by backpropagation of errors and
notably outperforms them with respect to out-of-distribution reconstruction
(including on the full 90k CINIC-10 test set)
- …