Modeling Interval Timing By Recurrent Neural Nets

The purpose of this study was to take a new approach in showing how the central nervous system might encode time at the supra-second level using recurrent neural nets (RNNs). This approach utilizes units with a delayed feedback, whose feedback weight determines the temporal properties of specific neurons in the network architecture. When these feedback neurons are coupled, they form a multilayered dynamical system that can be used to model temporal responses to steps of input in multidimensional systems. The timing network was implemented using separate recurrent “Go” and “NoGo” neural processing units to process an individual stimulus indicating the time of reward availability. Outputs from these distinct units on each time step are converted to a pulse reflecting a weighted sum of the separate Go and No-Go signals. This output pulse then drives an integrator unit, whose feedback weight and input weights shape the pulse distribution. This system was used to model empirical data from rodents performing in an instrumental “peak interval timing” task for two stimuli, Tone and Flash. For each of these stimuli, reward availability was signaled after different times from stimulus onset during training. Rodent performance was assessed on non-rewarded trials, following training, with each stimulus tested individually and simultaneously in a stimulus compound. The associated weights in the Go/No-Go network were trained using experimental data showing the mean distribution of bar press rates across an 80 s period in which a tone stimulus signaled reward after 5 s and a flash stimulus after 30 s from stimulus onset. Different Go/No-Go systems were used for each stimulus, but the weighted output of each fed into a final recurrent integrator unit, whose weights were unmodifiable. The recurrent neural net (RNN) model was implemented using Matlab and Matlab’s machine learning tools were utilized to train the network using the data from non-rewarded trials. The neural net output accurately fit the temporal distribution of tone and flash-initiated bar press data. Furthermore, a “Temporal Averaging” effect was also obtained when the flash and tone stimuli were combined. These results indicated that the system combining tone and flash responses were not superposed as in a linear system, but that there was a non-linearity, which interacted between tone and flash. In order to achieve an accurate fit to the empirical averaging data it was necessary to implement non-linear “saliency functions” that limited the output signal of each stimulus to the final integrator when the other was co-present. The model suggests that the central nervous system encodes timing generation as a dynamical system whose timing properties are embedded in the connection weights of the system. In this way, event timing is coded similar to the way other sensory-motor systems, such as the vestibuloocular and optokinetic systems, which combine sensory inputs from the vestibular and visual systems to generate the temporal aspects of compensatory eye movements

Extinction of cue-evoked drug-seeking relies on degrading hierarchical instrumental expectancies

There has long been need for a behavioural intervention that attenuates cue-evoked drug-seeking, but the optimal method remains obscure. To address this, we report three approaches to extinguish cue-evoked drug-seeking measured in a Pavlovian to instrumental transfer design, in non-treatment seeking adult smokers and alcohol drinkers. The results showed that the ability of a drug stimulus to transfer control over a separately trained drug-seeking response was not affected by the stimulus undergoing Pavlovian extinction training in experiment 1, but was abolished by the stimulus undergoing discriminative extinction training in experiment 2, and was abolished by explicit verbal instructions stating that the stimulus did not signal a more effective response-drug contingency in experiment 3. These data suggest that cue-evoked drug-seeking is mediated by a propositional hierarchical instrumental expectancy that the drug-seeking response is more likely to be rewarded in that stimulus. Methods which degraded this hierarchical expectancy were effective in the laboratory, and so may have therapeutic potential

Delamater, Tu, & Huang (2021) Data

Raw data from preference test

At the Interface of Learning and Cognition:An Associative Learning Perspective

This paper reviews some of the literature on Pavlovian and instrumental conditioning as they relate to“cognitive” factors in behavior. Studies of Pavlovian learning have centered around the notion that a representation of the unconditioned stimulus plays a critical role in performance. However, much work will need to go into characterizing the nature of the representations that mediate learning. In particular, current research illustrates that “images” and “expectancies” of reward may differ in fundamental ways, and also that learning about temporal, motivational, and sensory properties of reward might involve different systems. The study of instrumental learning also poses challenges for addressing the question of what representations, i.e., associative structures, underlie such learning.Current work reveals a host of associative structures that may participate in learning and performance though how these different structures participate in a unified approach is currently unknown. The associative approach can be contrasted with inferential reasoning approaches to instrumental action, and there are two key findings that seem outside the scope of a reasoning approach. Nevertheless, future work will be required to determine just how far purely associative models will be able to go in order to account for complex behavior

Location as a feature in pigeons' recognition of visual objects

Stimulus files, raw data, statistical analysis files (data, syntax and results) and report of a pilot experiment for a paper entitled "Location as an feature in pigeons' recognition of visual objects

Temporal Averaging Across Stimuli Signaling the Same or Different Reinforcing Outcomes in the Peak Procedure

The present study examined factors that affect temporal averaging in rats when discriminative stimuli are compounded following separate training indicating the availability of reward after different fixed intervals (FI) on a peak procedure. One group of rats, Group Differential, learned that a flashing light stimulus signaled that one type of food pellet reward could be earned for lever pressing after an FI 5 s interval and that a second type of food pellet reward could be earned after an FI 20 s interval in the presence of a tone stimulus. A second group of rats, Group Non-Differential, was similarly trained except that both types of rewards were scheduled across flash and tone trials. When given non-reinforced flash + tone compound test trials the interval containing the maximal response rate was no different than on flash alone test trials, although some responding also appeared near the long FI time. After these FI contingencies were reversed (flash signaled FI 20 s and tone signaled FI 5 s), however, further compound test trials more clearly revealed a temporal averaging pattern in both groups. The peak interval was shifted to the right of the FI 5 stimulus. Moreover, Group Differential rats acquired the reversed discrimination somewhat more rapidly than Group Non-Differential rats, and in a final selective satiation test Group Differential rats responded less in later intervals after they had been sated on the FI 20 s reward. These data suggest that temporal averaging in stimulus compound tests occurs even when the stimuli being combined signal qualitatively different rewards, but that decreasing the value of one of those rewards can shift responding away from the relevant time interval in a selective satiation test. However, when an especially salient stimulus (e.g., flashing light) signals a short FI, rats tend to process the compound stimulus more in terms of its individual elements