Comparison of network performance contrasting Hebbian and Error Driven learning rules.

<p>Network performance plotted across various network sizes and training set sizes. <i>A)</i> Surface plots of the average Name Error across the full training set plotted on the left for both the theta-phase (i.e. error-driven learning in both MSP and TSP) and the Hebbian network, and the difference between theta-phase and Hebbian surfaces plotted on the right with an asterisk showing values significantly(p<0.005) different from 0. Cyan dots in the surface plots are data points where performance was measured <i>B)</i> Same Name Error surface plots with the left panel (labeled TSP ErrDrv) showing performance from a network with error-driven learning in the TSP and Hebbian learning the MSP. Middle panel (labeled MSP ErrDrv) shows performance from a network with Hebbian learning in the TSP and error-driven learning in the MSP. Difference between these two shown on the right. <i>C)</i> Plot of network performance taken from A and B for a single network size of 80 CA3 units and 400 DG units. Line color is shown in A and B where these data were extracted from the surface plots, and the magenta and purple lines(labeled TSP+PT and MSP+PT respectively) come from networks with the same error-driven configuration as in B, however these networks were pretrained on the input patters for 15 epochs within the MSP. Full surface plots for these two networks are not shown.</p

Hippocampal connectivity.

<p><i>A)</i> Schematic of hippocampal connections with Entorhinal Cortex(EC) <i>B)</i> Image of neural network model used in this work on the right. Two pathways are highlighted: the Mono-Synaptic Pathway (MSP) in green, and the Tri-Synaptic Pathway (TSP) in blue. An individual EC <i>slot</i> is highlighted in orange within the neural network on the right.</p

Deep reinforcement learning with modulated Hebbian plus Q-network architecture

In this article, we consider a subclass of partially observable Markov decision process (POMDP) problems which we termed confounding POMDPs. In these types of POMDPs, temporal difference (TD)-based reinforcement learning (RL) algorithms struggle, as TD error cannot be easily derived from observations. We solve these types of problems using a new bio-inspired neural architecture that combines a modulated Hebbian network (MOHN) with deep Q-network (DQN), which we call modulated Hebbian plus Q-network architecture (MOHQA). The key idea is to use a Hebbian network with rarely correlated bio-inspired neural traces to bridge temporal delays between actions and rewards when confounding observations and sparse rewards result in inaccurate TD errors. In MOHQA, DQN learns low-level features and control, while the MOHN contributes to high-level decisions by associating rewards with past states and actions. Thus, the proposed architecture combines two modules with significantly different learning algorithms, a Hebbian associative network and a classical DQN pipeline, exploiting the advantages of both. Simulations on a set of POMDPs and on the Malmo environment show that the proposed algorithm improved DQN's results and even outperformed control tests with advantage-actor critic (A2C), quantile regression DQN with long short-term memory (QRDQN + LSTM), Monte Carlo policy gradient (REINFORCE), and aggregated memory for reinforcement learning (AMRL) algorithms on most difficult POMDPs with confounding stimuli and sparse rewards

Biological underpinnings for lifelong learning machines

Biological organisms learn from interactions with their environment throughout their lifetime. For artificial systems to successfully act and adapt in the real world, it is desirable to similarly be able to learn on a continual basis. This challenge is known as lifelong learning, and remains to a large extent unsolved. In this Perspective article, we identify a set of key capabilities that artificial systems will need to achieve lifelong learning. We describe a number of biological mechanisms, both neuronal and non-neuronal, that help explain how organisms solve these challenges, and present examples of biologically inspired models and biologically plausible mechanisms that have been applied to artificial systems in the quest towards development of lifelong learning machines. We discuss opportunities to further our understanding and advance the state of the art in lifelong learning, aiming to bridge the gap between natural and artificial intelligence

A domain-agnostic approach for characterization of lifelong learning systems

Despite the advancement of machine learning techniques in recent years, state-of-the-art systems lack robustness to “real world” events, where the input distributions and tasks encountered by the deployed systems will not be limited to the original training context, and systems will instead need to adapt to novel distributions and tasks while deployed. This critical gap may be addressed through the development of “Lifelong Learning” systems that are capable of (1) Continuous Learning, (2) Transfer and Adaptation, and (3) Scalability. Unfortunately, efforts to improve these capabilities are typically treated as distinct areas of research that are assessed independently, without regard to the impact of each separate capability on other aspects of the system. We instead propose a holistic approach, using a suite of metrics and an evaluation framework to assess Lifelong Learning in a principled way that is agnostic to specific domains or system techniques. Through five case studies, we show that this suite of metrics can inform the development of varied and complex Lifelong Learning systems. We highlight how the proposed suite of metrics quantifies performance trade-offs present during Lifelong Learning system development — both the widely discussed Stability-Plasticity dilemma and the newly proposed relationship between Sample Efficient and Robust Learning. Further, we make recommendations for the formulation and use of metrics to guide the continuing development of Lifelong Learning systems and assess their progress in the future