A Neural Network Model of Continual Learning with Cognitive Control

Neural networks struggle in continual learning settings from catastrophic forgetting: when trials are blocked, new learning can overwrite the learning from previous blocks. Humans learn effectively in these settings, in some cases even showing an advantage of blocking, suggesting the brain contains mechanisms to overcome this problem. Here, we build on previous work and show that neural networks equipped with a mechanism for cognitive control do not exhibit catastrophic forgetting when trials are blocked. We further show an advantage of blocking over interleaving when there is a bias for active maintenance in the control signal, implying a tradeoff between maintenance and the strength of control. Analyses of map-like representations learned by the networks provided additional insights into these mechanisms. Our work highlights the potential of cognitive control to aid continual learning in neural networks, and offers an explanation for the advantage of blocking that has been observed in humans.Comment: 7 pages, 5 figures, paper accepted as a talk to CogSci 2022 (https://escholarship.org/uc/item/3gn3w58z

A Vector-Integration-to-Endpoint Model for Performance of Viapoint Movements

Viapoint (VP) movements are movements to a desired point that are constrained to pass through an intermediate point. Studies have shown that VP movements possess properties, such as smooth curvature around the VP, that are not explicable by treating VP movements as strict concatenations of simpler point-to-point (PTP) movements. Such properties have led some theorists to propose whole-trajectory optimization models, which imply that the entire trajectory is pre-computed before movement initiation. This paper reports new experiments conducted to systematically compare VP with PTP trajectories. Analyses revealed a statistically significant early directional deviation in VP movements but no associated curvature change. An explanation of this effect is offered by extending the Vector-Integration-To-Endpoint (VITE) model (Bullock and Grossberg, 1988), which postulates that voluntary movement trajectories emerge as internal gating signals control the integration of continuously computed vector commands based on the evolving, perceptible difference between desired and actual position variables. The model explains the observed trajectories of VP and PTP movements as emergent properties of a dynamical system that does not precompute entire trajectories before movement initiation. The new model includes a working memory and a stage sensitive to time-to-contact information. These cooperate to control serial performance. The structural and functional relationships proposed in the model are consistent with available data on forebrain physiology and anatomy.Office of Naval Research (N00014-92-J-1309, N00014-93-1-1364, N0014-95-1-0409

Flexible Working Memory Through Selective Gating and Attentional Tagging

Working memory is essential: it serves to guide intelligent behavior of humans and nonhuman primates when task-relevant stimuli are no longer present to the senses. Moreover, complex tasks often require that multiple working memory representations can be flexibly and independently maintained, prioritized, and updated according to changing task demands. Thus far, neural network models of working memory have been unable to offer an integrative account of how such control mechanisms can be acquired in a biologically plausible manner. Here, we present WorkMATe, a neural network architecture that models cognitive control over working memory content and learns the appropriate control operations needed to solve complex working memory tasks. Key components of the model include a gated memory circuit that is controlled by internal actions, encoding sensory information through untrained connections, and a neural circuit that matches sensory inputs to memory content. The network is trained by means of a biologically plausible reinforcement learning rule that relies on attentional feedback and reward prediction errors to guide synaptic updates. We demonstrate that the model successfully acquires policies to solve classical working memory tasks, such as delayed recognition and delayed pro-saccade/anti-saccade tasks. In addition, the model solves much more complex tasks, including the hierarchical 12-AX task or the ABAB ordered recognition task, both of which demand an agent to independently store and updated multiple items separately in memory. Furthermore, the control strategies that the model acquires for these tasks subsequently generalize to new task contexts with novel stimuli, thus bringing symbolic production rule qualities to a neural network architecture. As such, WorkMATe provides a new solution for the neural implementation of flexible memory control

A Neurocomputational Model of the Functional Role of Dopamine in Stimulus-Response Task Learning and Performance

Thesis (Ph.D.) - Indiana University, Psychology, 2009The neuromodulatory neurotransmitter dopamine (DA) plays a complex, but central role in the learning and performance of stimulus-response (S-R) behaviors. Studies have implicated DA's role in reward-driven learning and also its role in setting the overall level of vigor or frequency of response. Here, a neurocomputational model is developed which models DA's influence on a set of brain regions believed to be involved in the learning and execution of S-R tasks, including frontal cortex, basal ganglia, and cingulate cortex. An `actor' component of the model is trained, using `babble' (random behavior selection) and `critic' (rewarding and punishing) components of the model, to perform acceptance/rejection responses upon presentation of color stimuli in the context of recently presented auditory tones. The model behaves like an autonomous organism learning (and relearning) through `trial-and-error'. The focus of the study, the impact of hypo- and hyper-normal DA activity on this model, is investigated by three different dopaminergic pathways--two striatal and one prefrontal cortical--being manipulated independently during the learning and performance of the color response task. Hypo-DA conditions, analogous to Parkinsonism, cause slowing and reduction of frequency of learned responses, and, at extremes, degrade the learning (either initial or reversal) of the task. Hyper-DA conditions, analogous to psychostimulant effects, cause more rapid response times, but also can lead to perseveration of incorrect learning of response on the task. The presence of these effects often depends on which DA-ergic pathway is manipulated, however, which has implications for interpretation of the pharmacological experimental data. The proposed model embodies an integrative theory of dopamine function which suggests that the base rate of DA cell activity encodes the overall `activity-oriented motivation' of the organism, with hunger and/or expectation of reward driving both response vigor and tendency to generate an explorative `babble' response. This more `tonic' feature of DA functionality coexists naturally with the more extensively-studied `phasic' reward-learning features. The model may provide better insights on the role of DA system dysfunction in the cognitive and motivational symptoms of disorders such as Parkinsonism, psychostimulant abuse, ADHD, OCD, and schizophrenia, accounting for deficits in both learning and performance of tasks