Molar and Molecular Models of Performance for Rewarding Brain Stimulation

This dissertation reports analyses of performance for rewarding brain stimu- lation in a three-part sequential task. A session of self-stimulation was composed of three trial types, during which the strength and opportunity cost of electrical stim- ulation were kept constant. Within a trial, a lever allowed animals to harvest brain stimulation rewards as soon as the lever had been held for a set, cumulative amount of time. When the time spent holding reached this criterion, the lever retracted and a burst of rewarding electrical stimulation was delivered. A flashing house light and 10s inter-trial interval signalled the start of a new trial. Rats were presented with strong/inexpensive/certain stimulation on one trial, a randomly selected strength, cost and risk on the next trial, and weak/inexpensive/certain stimulation on the third trial of a sequence. The sequence then repeated. Rewards during the second trial of this sequence were delivered with cued probabilities ranging from 0.5 to 1.0. The current thesis evaluates the ability of a previously published (molar) model of performance during a trial to accurately detect the effect of risk on payoff but not reward intensity. Although animals were less willing to work for stimulation trains that may not be delivered than those delivered with certainty, risk did not change the relative reward produced by stimulation. We also present evidence on a fine time scale that self-stimulating rats develop a model of their world. The first pause made as a trial began was a function of the payoff the animal had yet to receive, indicat- ing that rats had a model of the triad sequence. Analysis of the conditions under which pauses were uncharacteristic also provides evidence of what this model might be. Analysis of the fine scale of performance provides evidence that animals had a model of the stability of trial conditions. Finally, we present a (molecular) model of performance for brain stimulation rewards in real-time. Our results demonstrate that rats develop a model of the testing paradigm and can adjust to changes in reward contingencies with as few as one exemplar

Rattus Psychologicus: Construction of preferences by self-stimulating rats

Behavioral economists have proposed that human preferences are constructed during their elicitation and are thus influenced by the elicitation procedure. For example, different preferences are expressed when options are encountered one at a time or concurrently. This phenomenon has been attributed to differences in the “evaluability” of a particular attribute when comparison to an option with a different value of this attribute is or is not available. Research on the preferences of laboratory animals has often been carried out by means of operant conditioning methods. Formal treatments of operant behavior relate preferences to variables such as the strength and cost of reward but do not address the evaluability of these variables. Two experiments assessed the impact of procedural factors likely to alter the evaluability of an opportunity cost (“price”): the work time required for a rat to earn a train of rewarding electrical brain stimulation. The results support the notion that comparison between recently encountered prices is necessary to render the price variable highly evaluable. When price is held constant over many trials and test sessions, the evaluability of this variable appears to decline. Implications are discussed for the design of procedures for estimating subjective reward strengths and costs in operant conditioning experiments aimed at characterizing, identifying and understanding neural circuitry underlying evaluation and choice

Optimal indolence: a normative microscopic approach to work and leisure

Dividing limited time between work and leisure when both have their attractions is a common everyday decision. We provide a normative control theoretic treatment of this decision that bridges economic and psychological accounts. We show how our framework applies to free-operant behavioural experiments in which subjects are required to work (depressing a lever) for sufficient total time (called the price) to receive a reward. When the microscopic benefit-of-leisure increases nonlinearly with duration, the model generates behaviour that qualitatively matches various microfeatures of subjects’ choices, including the distribution of leisure bout durations as a function of the payoff. We relate our model to traditional accounts by deriving macroscopic, molar, quantities from microscopic choices

At What Stage of Neural Processing Does Cocaine Act to Boost Pursuit of Rewards?

Dopamine-containing neurons have been implicated in reward and decision making. One element of the supporting evidence is that cocaine, like other drugs that increase dopaminergic neurotransmission, powerfully potentiates reward seeking. We analyze this phenomenon from a novel perspective, introducing a new conceptual framework and new methodology for determining the stage(s) of neural processing at which drugs, lesions and physiological manipulations act to influence reward-seeking behavior. Cocaine strongly boosts the proclivity of rats to work for rewarding electrical brain stimulation. We show that the conventional conceptual framework and methods do not distinguish between three conflicting accounts of how the drug produces this effect: increased sensitivity of brain reward circuitry, increased gain, or decreased subjective reward costs. Sensitivity determines the stimulation strength required to produce a reward of a given intensity (a measure analogous to the KM of an enzyme) whereas gain determines the maximum intensity attainable (a measure analogous to the vmax of an enzyme-catalyzed reaction). To distinguish sensitivity changes from the other determinants, we measured and modeled reward seeking as a function of both stimulation strength and opportunity cost. The principal effect of cocaine was a two-fourfold increase in willingness to pay for the electrical reward, an effect consistent with increased gain or decreased subjective cost. This finding challenges the long-standing view that cocaine increases the sensitivity of brain reward circuitry. We discuss the implications of the results and the analytic approach for theories of how dopaminergic neurons and other diffuse modulatory brain systems contribute to reward pursuit, and we explore the implications of the conceptual framework for the study of natural rewards, drug reward, and mood