Behavioral Momentum Theory: Equations and Applications

Behavioral momentum theory provides a quantitative account of how reinforcers experienced within a discriminative stimulus context govern the persistence of behavior that occurs in that context. The theory suggests that all reinforcers obtained in the presence of a discriminative stimulus increase resistance to change, regardless of whether those reinforcers are contingent on the target behavior, are noncontingent, or are even contingent on an alternative behavior. In this paper, we describe the equations that constitute the theory and address their application to issues of particular importance in applied settings. The theory provides a framework within which to consider the effects of interventions such as extinction, noncontingent reinforcement, differential reinforcement of alternative behavior, and other phenomena (e.g., resurgence). Finally, the theory predicts some counterintuitive and potentially counterproductive effects of alternative reinforcement, and can serve as an integrative guide for intervention when its terms are identified with the relevant conditions of applied settings

Experience with Dynamic Reinforcement Rates Decreases Resistance to Extinction

The ability of organisms to detect reinforcer-rate changes in choice preparations is positively related to two factors: the magnitude of the change in rate and the frequency with which rates change. Gallistel (2012) suggested similar rate-detection processes are responsible for decreases in responding during operant extinction. Although effects of magnitude of change in reinforcer rate on resistance to extinction are well known (e.g., the partial-reinforcement-extinction effect), effects of frequency of changes in rate prior to extinction are unknown. Thus, the present experiments examined whether frequency of changes in baseline reinforcer rates impacts resistance to extinction. Pigeons pecked keys for variable-interval food under conditions where reinforcer rates were stable and where they changed within and between sessions. Overall reinforcer rates between conditions were controlled. In Experiment 1, resistance to extinction was lower following exposure to dynamic reinforcement schedules than to static schedules. Experiment 2 showed that resistance to presession feeding, a disruptor that should not involve change-detection processes, was unaffected by baseline-schedule dynamics. These findings are consistent with the suggestion that change detection contributes to extinction. We discuss implications of change-detection processes for extinction of simple and discriminated operant behavior and relate these processes to the behavioral-momentum based approach to understanding extinction

Concurrent Chains Schedules as a Method to Study Choice Between Alcohol Associated Conditioned Reinforcers

An extensive body of research using concurrent-chains schedules of reinforcement has shown that choice for one of two differentially valued food-associated stimuli is dependent upon the overall temporal context in which those stimuli are embedded. The present experiments examined whether the concurrent chains procedure was useful for the study of behavior maintained by alcohol and alcohol-associated stimuli. In Experiment 1, rats responded on concurrent-chains schedules with equal variable-interval (VI) 10-s schedules in the initial links. Across conditions, fixed-interval schedules in the terminal links were varied to yield 1:1, 9:1, and 1:9 ratios of alcohol delivery. Initial-link response rates reflected changes in terminal-link schedules, with greater relative responding in the rich terminal link. In Experiment 2, terminal-link schedules remained constant with a 9:1 ratio of alcohol delivery rates while the length of two equal duration initial-link schedules was varied. Preference for the rich terminal link was less extreme when initial links were longer (i.e., the initial-link effect), as has been previously reported with food reinforcers. This result suggests that the conditioned reinforcing value of an alcohol associated stimulus depends on the temporal context in which it is embedded. The concurrent-chains procedure and quantitative models of concurrent chains performance may provide a useful framework within which to study how contextual variables modulate preference for drug-associated conditioned reinforcers

Temporal contingency

Contingency, and more particularly temporal contingency, has often figured in thinking about the nature of learning. However, it has never been formally defined in such a way as to make it a measure that can be applied to most animal learning protocols. We use elementary information theory to define contingency in such a way as to make it a measurable property of almost any conditioning protocol. We discuss how making it a measurable construct enables the exploration of the role of different contingencies in the acquisition and performance of classically and operantly conditioned behavior

Delayed Matching to Sample: Reinforcement has Opposite Effects on Resistance to Change in Two Related Procedures

The effects of reinforcement on delayed matching to sample (DMTS) have been studied in two within-subjects procedures. In one, reinforcer magnitudes or probabilities vary from trial to trial and are signaled within trials (designated signaled DMTS trials). In the other, reinforcer probabilities are consistent for a series of trials produced by responding on variable-interval (VI) schedules within multiple-schedule components (designated multiple VI DMTS). In both procedures, forgetting functions in rich trials or components are higher than and roughly parallel to those in lean trials or components. However, during disruption, accuracy has been found to decrease more in rich than in lean signaled DMTS trials and, conversely, to decrease more in lean than in rich multiple VI DMTS components. In the present study, we compared these procedures in two groups of pigeons. In baseline, forgetting functions in rich trials or components were higher than and roughly parallel to those in lean trials or components, and were similar between the procedures. During disruption by prefeeding or extinction, accuracy decreased more in rich signaled DMTS trials, whereas accuracy decreased more in lean multiple VI DMTS components. These results replicate earlier studies and are predicted by a model of DMTS from Nevin, Davison, Odum, and Shahan (2007)

Stimuli Produced by Observing Responses Make Rats’ Ethanol Self-Administration More Resistant to Price Increases

Rationale: Observing responses bring sensory receptors into contact with environmental stimuli. In the observing-response procedure, periods in which an operant response (e.g. pressing a lever) is reinforced by drug deliveries alternate with periods in which this response is never reinforced (i.e. extinction). These alternating periods of drug availability versus extinction are not signaled. Observing responses (i.e. presses on a second lever) produce brief stimuli signaling whether drug is available or not for responses on the first lever. Little is known about how parameters of the drug reinforcer affect drug-stimulus observing. Objectives: The effects of changes in the unit price (responses/reinforcer magnitude) of self-administered ethanol on rats\u27 observing were examined. Also, the effects of an observing-response-produced ethanol stimulus on ethanol consumption were examined by comparing consumption during signaled and unsignaled periods of ethanol availability. Methods: Rats self-administered oral ethanol in the observing-response procedure. The unit price of ethanol in the observing-response procedure was increased by increasing the response requirement for ethanol across conditions. Results: Observing and response rates on the ethanol lever increased and then decreased with increases in the unit price of ethanol. However, ethanol-lever responding and ethanol consumption during periods when ethanol was available were less sensitive to increases in price when the observing-response-produced ethanol stimulus was present. Conclusions: Observing varies as an orderly function of unit price of a drug reinforcer, and drug stimuli produced by observing responses can make drug consumption less sensitive to increases in price. This procedure may provide an animal model of both attending to drug stimuli and the resultant effects of these stimuli on drug taking

Ethanol-Maintained Responding of Rats is More Resistant to Change in a Context with Added Non-Drug Reinforcement

Alternative non-drug reinforcers reliably decrease drug-maintained responding in self-administration procedures. Studies of the resistance to change of food-maintained behavior, however, have found that responding in the presence of a stimulus associated with an alternative reinforcer is more resistant to disruption. This increase in persistence occurs despite lower response rates when the alternative reinforcer is present. The present experiment examined if, in addition to decreasing response rates, an alternative non-drug reinforcer also increases the persistence of drug-maintained responding. Rats self-administered oral ethanol in a multiple schedule of reinforcement in which responding was reinforced in two components signaled by different stimuli. In one component, response-independent food was delivered in addition to the earned ethanol. The effects of the alternative food reinforcer on response rates and resistance to extinction in the two components were examined. As in previous experiments on the resistance to change of food-maintained operant behavior, response rates were lower, but more resistant to extinction in the presence of the stimulus associated with the alternative reinforcer. These findings suggest that all the reinforcers obtained in a context in which drugs are consumed may contribute to the persistence of drug seeking in that context. This increase in persistence may occur even if the alternative reinforcers interfere with drug seeking