412 research outputs found
Deep Reinforcement Learning with Modulated Hebbian plus Q Network Architecture
This paper presents a new neural architecture that combines a modulated
Hebbian network (MOHN) with DQN, which we call modulated Hebbian plus Q network
architecture (MOHQA). The hypothesis is that such a combination allows MOHQA to
solve difficult partially observable Markov decision process (POMDP) problems
which impair temporal difference (TD)-based RL algorithms such as DQN, as the
TD error cannot be easily derived from observations. The key idea is to use a
Hebbian network with bio-inspired neural traces in order to bridge temporal
delays between actions and rewards when confounding observations and sparse
rewards result in inaccurate TD errors. In MOHQA, DQN learns low level features
and control, while the MOHN contributes to the high-level decisions by
associating rewards with past states and actions. Thus the proposed
architecture combines two modules with significantly different learning
algorithms, a Hebbian associative network and a classical DQN pipeline,
exploiting the advantages of both. Simulations on a set of POMDPs and on the
MALMO environment show that the proposed algorithm improved DQN's results and
even outperformed control tests with A2C, QRDQN+LSTM and REINFORCE algorithms
on some POMDPs with confounding stimuli and sparse rewards
The Cognitive Architecture of Spatial Navigation: Hippocampal and Striatal Contributions
Spatial navigation can serve as a model system in cognitive neuroscience, in which specific neural representations, learning rules, and control strategies can be inferred from the vast experimental literature that exists across many species, including humans. Here, we review this literature, focusing on the contributions of hippocampal and striatal systems, and attempt to outline a minimal cognitive architecture that is consistent with the experimental literature and that synthesizes previous related computational modeling. The resulting architecture includes striatal reinforcement learning based on egocentric representations of sensory states and actions, incidental Hebbian association of sensory information with allocentric state representations in the hippocampus, and arbitration of the outputs of both systems based on confidence/uncertainty in medial prefrontal cortex. We discuss the relationship between this architecture and learning in model-free and model-based systems, episodic memory, imagery, and planning, including some open questions and directions for further experiments
Neuro-Modulated Hebbian Learning for Fully Test-Time Adaptation
Fully test-time adaptation aims to adapt the network model based on
sequential analysis of input samples during the inference stage to address the
cross-domain performance degradation problem of deep neural networks. We take
inspiration from the biological plausibility learning where the neuron
responses are tuned based on a local synapse-change procedure and activated by
competitive lateral inhibition rules. Based on these feed-forward learning
rules, we design a soft Hebbian learning process which provides an unsupervised
and effective mechanism for online adaptation. We observe that the performance
of this feed-forward Hebbian learning for fully test-time adaptation can be
significantly improved by incorporating a feedback neuro-modulation layer. It
is able to fine-tune the neuron responses based on the external feedback
generated by the error back-propagation from the top inference layers. This
leads to our proposed neuro-modulated Hebbian learning (NHL) method for fully
test-time adaptation. With the unsupervised feed-forward soft Hebbian learning
being combined with a learned neuro-modulator to capture feedback from external
responses, the source model can be effectively adapted during the testing
process. Experimental results on benchmark datasets demonstrate that our
proposed method can significantly improve the adaptation performance of network
models and outperforms existing state-of-the-art methods.Comment: CVPR2023 accepte
A self-adaptive hardware with resistive switching synapses for experience-based neurocomputing
: Neurobiological systems continually interact with the surrounding environment to refine their behaviour toward the best possible reward. Achieving such learning by experience is one of the main challenges of artificial intelligence, but currently it is hindered by the lack of hardware capable of plastic adaptation. Here, we propose a bio-inspired recurrent neural network, mastered by a digital system on chip with resistive-switching synaptic arrays of memory devices, which exploits homeostatic Hebbian learning for improved efficiency. All the results are discussed experimentally and theoretically, proposing a conceptual framework for benchmarking the main outcomes in terms of accuracy and resilience. To test the proposed architecture for reinforcement learning tasks, we study the autonomous exploration of continually evolving environments and verify the results for the Mars rover navigation. We also show that, compared to conventional deep learning techniques, our in-memory hardware has the potential to achieve a significant boost in speed and power-saving
Learning flexible sensori-motor mappings in a complex network
Given the complex structure of the brain, how can synaptic plasticity explain the learning and forgetting of associations when these are continuously changing? We address this question by studying different reinforcement learning rules in a multilayer network in order to reproduce monkey behavior in a visuomotor association task. Our model can only reproduce the learning performance of the monkey if the synaptic modifications depend on the pre- and postsynaptic activity, and if the intrinsic level of stochasticity is low. This favored learning rule is based on reward modulated Hebbian synaptic plasticity and shows the interesting feature that the learning performance does not substantially degrade when adding layers to the network, even for a complex problem
- âŠ