Representing uncertainty in the Rescorla-Wagner model: Blocking, the redundancy effect, and outcome base rate

It is generally assumed that the Rescorla and Wagner (1972) model adequately accommodates the full results of simple cue competition experiments in humans (e.g. Dickinson et al., 1984), while the Bush and Mosteller (1951) model cannot. We present simulations that demonstrate this assumption is wrong in at least some circumstances. The Rescorla-Wagner model, as usually applied, fits the full results of a simple forward cue-competition experiment no better than the Bush-Mosteller model. Additionally, we present a novel finding, where letting the associative strength of all cues start at an intermediate value (rather than zero), allows this modified model to provide a better account of the experimental data than the (equivalently modified) Bush-Mosteller model. This modification also allows the Rescorla-Wagner model to account for a redundancy effect experiment (Uengoer et al., 2013); something that the unmodified model is not able to do. Furthermore, the modified Rescorla-Wagner model can accommodate the effect of varying the proportion of trials on which the outcome occurs (i.e. the base rate) on the redundancy effect (Jones et al., 2019). Interestingly, the initial associative strength of cues varies in line with the outcome base rate. We propose that this modification provides a simple way of mathematically representing uncertainty about the causal status of novel cues within the confines of the Rescorla-Wagner model. The theoretical implications of this modification are discussed. We also briefly introduce free and open resources to support formal modelling in associative learning. Keywords: associative learning, prediction error, uncertainty, modelling, blocking, redundancy effect, open science.</jats:p

Elaboration of a model of Pavlovian learning and performance: HeiDI

The model elaborated here adapts the influential pooled error term, first described by Allan R. Wagner and his colleague Robert A. Rescorla, to govern the formation of reciprocal associations between any pair of stimuli that are presented on a given trial. In the context of Pavlovian conditioning, these stimuli include various conditioned and unconditioned stimuli. This elaboration enables the model to deal with cue competition phenomena, including the relative validity effect, and evidence implicating separate error terms and attentional processes in association formation. The model also includes a performance rule, which provides a natural basis for (individual) variation in the strength and nature of conditioned behaviors that are observed in Pavlovian conditioning procedures. The new model thereby begins to address theoretical and empirical issues that were apparent when the Rescorla-Wagner model was first described, together with research inspired by the model over ensuing 50 years

A new functional role for lateral inhibition in the striatum: Pavlovian conditioning

The striatum has long been implicated in reinforcement learning and has been suggested by several neurophysiological studies as the substrate for encoding the reward value of stimuli. Reward prediction error (RPE) has been used in several basal ganglia models as the underlying learning signal, which leads to Pavlovian conditioning abilities that can be simulated by the Rescorla-Wagner model.&#xd;&#xa;&#xd;&#xa;Lateral inhibition between striatal projection neurons was once thought to have a winner-take-all function, useful in selecting between possible actions. However, it has been noted that the necessary reciprocal connections for this interpretation are too few, and the relative strength of these synaptic connections is weak. Still, modeling studies show that lateral inhibition does have an overall suppression effect on striatal activity and may play an important role in striatal processing. &#xd;&#xa;&#xd;&#xa;Neurophysiological recordings show task-relevant ensembles of responsive neurons at specific points in a behavioral paradigm (Barnes et al., 2005), which appear to be induced by lateral inhibition (see Ponzi and Wickens, 2010). We have developed a similarly responding, RPE-based model of the striatum by incorporating lateral inhibition. Model neurons are assigned to either the direct or the indirect pathway but lateral connections occur within and between these groups, leading to competition between both the individual neurons and their pathways. We successfully applied this model to the simulation of Pavlovian phenomena beyond those of the Rescorla-Wagner model, including negative patterning, unovershadowing, and external inhibition

On the modeling of neural cognition for social network applications

In this paper, we study neural cognition in social network. A stochastic model is introduced and shown to incorporate two well-known models in Pavlovian conditioning and social networks as special case, namely Rescorla-Wagner model and Friedkin-Johnsen model. The interpretation and comparison of these model are discussed. We consider two cases when the disturbance is independent identical distributed for all time and when the distribution of the random variable evolves according to a markov chain. We show that the systems for both cases are mean square stable and the expectation of the states converges to consensus.Comment: submitted to IEEE CCAT 201

Elemental Representations of Stimuli in Associative Learning

This paper reviews evidence and theories concerning the nature of stimulus representations in Pavlovian conditioning. It focuses on the elemental approach developed in Stimulus Sampling Theory (Atkinson & Estes, 1963; Bush & Mosteller, 1951b) and extended by McLaren and Mackintosh (2000; 2002), and contrasts this with models that that invoke notions of configural representations that uniquely code for different patterns of stimulus inputs (e.g., Pearce, 1987, 1994; Rescorla & Wagner, 1972; Wagner & Brandon, 2001). The paper then presents a new elemental model that emphasizes interactions between stimulus elements. This model is shown to explain a range of behavioral findings, including those (e.g., negative patterning and biconditional discriminations) traditionally thought beyond the explanatory capabilities of elemental models. Moreover, the model offers a ready explanation for recent findings reported by Rescorla (2000; 2001; 2002b) concerning the way that stimuli with different conditioning histories acquire associative strength when conditioned in compoun

Retrospective revaluation as simple associative learning

Backward blocking, unovershadowing and backward conditioned inhibition are examples of "retrospective revaluation" phenomena, that have been suggested to involve more than simple associative learning. Models of these phenomena have thus employed additional concepts, e.g. appealing to attentional effects or more elaborate learning mechanisms. I show that a suitable representation of stimuli, paired with a careful analysis of the discriminations faced by animals, leads to an account of these and other phenomena in terms of a simple "elemental" model of associative learning, with essentially the same learning mechanism as the Rescorla and Wagner (1972) model. I conclude with a discussion of some implications for theories of learning

Associative change to cues conditioned in compound

Models of associative learning that rely on a common error term to determine associative change, such as the Rescorla-Wagner model, assert that all cues on a conditioning trial undergo equivalent amounts of change. Recent evidence from Rescorla (2000, 2001) himself, as well as Spicer and colleagues (2020) have been interpreted to be incompatible with this fundamental assertion. This thesis evaluates both sources of evidence and, thereby, the challenge they present to the Rescorla-Wagner model. Chapter 2 provides an assessment of Rescorla's (2000, 2001) compound test data, his interpretation of unequal change to cues conditioned in compound, and an alternative explanation for the results that preserves the key insight of the Rescorla-Wagner model by appealing to non-linearity in the function that maps (or translates) learning into observable behaviour. Chapter 3 evaluates Spicer et al.'s (2020) findings which have been taken to indicate that prediction certainty rather than prediction error is the greater determinant of associative change in people. These results too can be accounted for under the mapping function introduced in Chapter 2. Finally, Chapter 4 presents a series of experiments that examine the effect of certainty/uncertainty on associative change in human causal learning using designs that address the interpretive issues raised in Chapter 3. These experiments use a compound test procedure in a continuous outcome allergist task and provide evidence for greater change to more uncertain cues in the case of associative gains, but not associative losses. The results are discussed with respect to the Rescorla-Wagner model, Rescorla’s proposed modification to the model, and the Spicer et al. notion of “theory protection” which is beyond the explanatory powers of the model

A Rescorla-Wagner Drift-Diffusion Model of Conditioning and Timing

Computational models of classical conditioning have made significant contributions to the theoretic understanding of associative learning, yet they still struggle when the temporal aspects of conditioning are taken into account. Interval timing models have contributed a rich variety of time representations and provided accurate predictions for the timing of responses, but they usually have little to say about associative learning. In this article we present a unified model of conditioning and timing that is based on the influential Rescorla-Wagner conditioning model and the more recently developed Timing Drift-Diffusion model. We test the model by simulating 10 experimental phenomena and show that it can provide an adequate account for 8, and a partial account for the other 2. We argue that the model can account for more phenomena in the chosen set than these other similar in scope models: CSC-TD, MS-TD, Learning to Time and Modular Theory. A comparison and analysis of the mechanisms in these models is provided, with a focus on the types of time representation and associative learning rule used

The Rescorla-Wagner Drift-Diffusion Model

Computational models of classical conditioning have made significant contributions to the theoretic understanding of associative learning, yet they still struggle when the temporal aspects of conditioning are taken into account. Interval timing models have contributed a rich variety of time representations and provided accurate predictions for the timing of responses, but they usually have little to say about associative learning. In this thesis we present a unified model of conditioning and timing that is based on the influential Rescorla-Wagner conditioning model and the more recently developed Timing Drift-Diffusion model. We test the model by simulating 11 experimental phenomena and show that it can provide an adequate account for 9, and a partial account for the other 2. We argue that the model can account for more phenomena in the chosen set than these other similar in scope models: CSCTD, MS-TD, Learning to Time and Modular Theory. A comparison and analysis of the mechanisms in these models is provided, with a focus on the types of time representation and associative learning rule used