5 research outputs found
Beyond spiking networks: the computational advantages of dendritic amplification and input segregation
The brain can efficiently learn a wide range of tasks, motivating the search
for biologically inspired learning rules for improving current artificial
intelligence technology. Most biological models are composed of point neurons,
and cannot achieve the state-of-the-art performances in machine learning.
Recent works have proposed that segregation of dendritic input (neurons receive
sensory information and higher-order feedback in segregated compartments) and
generation of high-frequency bursts of spikes would support error
backpropagation in biological neurons. However, these approaches require
propagating errors with a fine spatio-temporal structure to the neurons, which
is unlikely to be feasible in a biological network.
To relax this assumption, we suggest that bursts and dendritic input
segregation provide a natural support for biologically plausible target-based
learning, which does not require error propagation. We propose a pyramidal
neuron model composed of three separated compartments. A coincidence mechanism
between the basal and the apical compartments allows for generating
high-frequency bursts of spikes. This architecture allows for a burst-dependent
learning rule, based on the comparison between the target bursting activity
triggered by the teaching signal and the one caused by the recurrent
connections, providing the support for target-based learning. We show that this
framework can be used to efficiently solve spatio-temporal tasks, such as the
store and recall of 3D trajectories.
Finally, we suggest that this neuronal architecture naturally allows for
orchestrating ``hierarchical imitation learning'', enabling the decomposition
of challenging long-horizon decision-making tasks into simpler subtasks. This
can be implemented in a two-level network, where the high-network acts as a
``manager'' and produces the contextual signal for the low-network, the
``worker''.Comment: arXiv admin note: substantial text overlap with arXiv:2201.1171
Active Predicting Coding: Brain-Inspired Reinforcement Learning for Sparse Reward Robotic Control Problems
In this article, we propose a backpropagation-free approach to robotic
control through the neuro-cognitive computational framework of neural
generative coding (NGC), designing an agent built completely from powerful
predictive coding/processing circuits that facilitate dynamic, online learning
from sparse rewards, embodying the principles of planning-as-inference.
Concretely, we craft an adaptive agent system, which we call active predictive
coding (ActPC), that balances an internally-generated epistemic signal (meant
to encourage intelligent exploration) with an internally-generated instrumental
signal (meant to encourage goal-seeking behavior) to ultimately learn how to
control various simulated robotic systems as well as a complex robotic arm
using a realistic robotics simulator, i.e., the Surreal Robotics Suite, for the
block lifting task and can pick-and-place problems. Notably, our experimental
results demonstrate that our proposed ActPC agent performs well in the face of
sparse (extrinsic) reward signals and is competitive with or outperforms
several powerful backprop-based RL approaches.Comment: Contains appendix with pseudocode and additional detail
Brain-Inspired Computational Intelligence via Predictive Coding
Artificial intelligence (AI) is rapidly becoming one of the key technologies
of this century. The majority of results in AI thus far have been achieved
using deep neural networks trained with the error backpropagation learning
algorithm. However, the ubiquitous adoption of this approach has highlighted
some important limitations such as substantial computational cost, difficulty
in quantifying uncertainty, lack of robustness, unreliability, and biological
implausibility. It is possible that addressing these limitations may require
schemes that are inspired and guided by neuroscience theories. One such theory,
called predictive coding (PC), has shown promising performance in machine
intelligence tasks, exhibiting exciting properties that make it potentially
valuable for the machine learning community: PC can model information
processing in different brain areas, can be used in cognitive control and
robotics, and has a solid mathematical grounding in variational inference,
offering a powerful inversion scheme for a specific class of continuous-state
generative models. With the hope of foregrounding research in this direction,
we survey the literature that has contributed to this perspective, highlighting
the many ways that PC might play a role in the future of machine learning and
computational intelligence at large.Comment: 37 Pages, 9 Figure
A Theoretical Framework for Target Propagation
The success of deep learning, a brain-inspired form of AI, has sparked
interest in understanding how the brain could similarly learn across multiple
layers of neurons. However, the majority of biologically-plausible learning
algorithms have not yet reached the performance of backpropagation (BP), nor
are they built on strong theoretical foundations. Here, we analyze target
propagation (TP), a popular but not yet fully understood alternative to BP,
from the standpoint of mathematical optimization. Our theory shows that TP is
closely related to Gauss-Newton optimization and thus substantially differs
from BP. Furthermore, our analysis reveals a fundamental limitation of
difference target propagation (DTP), a well-known variant of TP, in the
realistic scenario of non-invertible neural networks. We provide a first
solution to this problem through a novel reconstruction loss that improves
feedback weight training, while simultaneously introducing architectural
flexibility by allowing for direct feedback connections from the output to each
hidden layer. Our theory is corroborated by experimental results that show
significant improvements in performance and in the alignment of forward weight
updates with loss gradients, compared to DTP.Comment: 13 pages and 4 figures in main manuscript; 41 pages and 8 figures in
supplementary materia