A general theory of time discounting: The reference-time theory of intertemporal choice

We develop a general theory of intertemporal choice: the reference-time theory, RT. RT is a synthesis of ideas from the hyperbolic model and subadditivity of time discounting. These models are extended to allow for a reference point for time as well as wealth. RT is able to account for all the 6 main anomalies of time discounting: gain-loss asymmetry, magnitude effect, common difference effect, delay-speedup asymmetry, apparent intransitivity of time preferences, and non-additivity of time discounting. We provide a class of utility functions compatible with RT. We show how RT can be extended to incorporate uncertainty and attribute models of intertemporal choice.Anomalies of the discounted utility model; Hyperbolic discounting; Prospect theory; gamma-delay; alpha-subadditivity; SIE value functions

A general theory of time discounting: The reference-time theory of intertemporal choice

We develop a general theory of intertemporal choice: the reference-time theory, RT. RT is a synthesis of ideas from the generalized hyperbolic model (Loewenstein and Prelec 1992), the quasi-hyperbolic model (Phelps and Pollak 1968, Laibson 1997) and subadditivity of time discounting (Roelofsma and Read 2000, Read 2001 and Scholten and Read 2006a). These models are extended to allow for (i) reference points for time and wealth, and (ii) different discount functions for gains and losses. RT is able to account for all the 6 main anomalies of time discounting: gain-loss asymmetry, magnitude effect, common difference effect, delay-speedup asymmetry, apparent intransitivity of time preferences, and non-additivity of time discounting. We provide a class of utility functions compatible with RT. We show how RT can be extended to incorporate uncertainty and attribute models of intertemporal choice.Anomalies of the discounted utility model; Hyperbolic discounting; Prospect theory; gamma-delay; alpha-subadditivity; SIE value functions

Behavioral Mechanisms of Pramipexole-Induced Impulsivity: Discrimination Processes Underlying Decision-Making

Faced with an intertemporal choice, an organism that chooses a “smaller-sooner” reinforcer over a “larger-later” reinforcer is said to behave impulsively. Individual differences in intertemporal choice are effectively modeled by generalized matching law and delay discounting equations that incorporate parameters corresponding to behavioral processes such as sensitivity to reinforcer amount or delay. By simulating changes in these processes and identifying conditions under which impulsive choice is likely to result, researchers are in a position to anticipate and examine potential behavioral mechanisms underlying clinical instances of impulsivity. Pramipexole, a dopamine agonist medication, is associated with reports of impulsive behavior in populations prescribed the drug, as well as in experimental subjects administered the compound prior to intertemporal choice sessions, although the latter findings are mixed. The present set of experiments was designed (a) to systematically replicate conditions under which pramipexole increased impulsive choice, but also nonspecifically disrupted behavior, and (b) to elucidate behavioral mechanisms of pramipexole-induced impulsivity in rats. In Chapter 2, a behavioral task used previously by researchers reporting a nonspecific effect of pramipexole was modified to include procedural controls common in the intertemporal choice literature (centering response, no-delay sessions). In accord with previous findings, acute pramipexole nonspecifically disrupted choice behavior, while chronic pramipexole partially remediated elements of the disruption (i.e., decrease in initial-block choice). In Chapter 3, three experiments targeted behavioral processes critical for intertemporal choice. Experiment 1 evaluated the acute and chronic effects of pramipexole on rats’ sensitivity to relative reinforcer delays in a concurrent-chains procedure. Contrary to the predicted effect, the drug decreased this measure, indicating the possibility of impaired stimulus control. Experiments 2 and 3 assessed the drug effect on discrimination of response-reinforcer contingencies and of reinforcer amounts, respectively, and revealed deficits in accuracy of similar magnitude across both preparations. Collectively, the results of these experiments suggest that previous findings of pramipexole-induced impulsivity and nonspecific disruption of behavior can be explained as impairments in discrimination processes required for intertemporal choice. Although the generality of the present findings may be limited to experimental settings with nonhumans, they demonstrate the utility of quantitatively modeling impulsivity

A general theory of intertemporal decision-making and the perception of time

Animals and humans make decisions based on their expected outcomes. Since relevant outcomes are often delayed, perceiving delays and choosing between earlier versus later rewards (intertemporal decision-making) is an essential component of animal behavior. The myriad observations made in experiments studying intertemporal decision-making and time perception have not yet been rationalized within a single theory. Here we present a theory-Training--Integrated Maximized Estimation of Reinforcement Rate (TIMERR)--that explains a wide variety of behavioral observations made in intertemporal decision-making and the perception of time. Our theory postulates that animals make intertemporal choices to optimize expected reward rates over a limited temporal window; this window includes a past integration interval (over which experienced reward rate is estimated) and the expected delay to future reward. Using this theory, we derive a mathematical expression for the subjective representation of time. A unique contribution of our work is in finding that the past integration interval directly determines the steepness of temporal discounting and the nonlinearity of time perception. In so doing, our theory provides a single framework to understand both intertemporal decision-making and time perception.Comment: 37 pages, 4 main figures, 3 supplementary figure

Measuring time preferences

We review research that measures time preferences—i.e., preferences over intertemporal tradeoffs. We distinguish between studies using financial flows, which we call “money earlier or later” (MEL) decisions and studies that use time-dated consumption/effort. Under different structural models, we show how to translate what MEL experiments directly measure (required rates of return for financial flows) into a discount function over utils. We summarize empirical regularities found in MEL studies and the predictive power of those studies. We explain why MEL choices are driven in part by some factors that are distinct from underlying time preferences.National Institutes of Health (NIA R01AG021650 and P01AG005842) and the Pershing Square Fund for Research in the Foundations of Human Behavior

Encoding of Marginal Utility across Time in the Human Brain

Marginal utility theory prescribes the relationship between the objective property of the magnitude of rewards and their subjective value. Despite its pervasive influence, however, there is remarkably little direct empirical evidence for such a theory of value, let alone of its neurobiological basis. We show that human preferences in an intertemporal choice task are best described by a model that integrates marginally diminishing utility with temporal discounting. Using functional magnetic resonance imaging, we show that activity in the dorsal striatum encodes both the marginal utility of rewards, over and above that which can be described by their magnitude alone, and the discounting associated with increasing time. In addition, our data show that dorsal striatum may be involved in integrating subjective valuation systems inherent to time and magnitude, thereby providing an overall metric of value used to guide choice behavior. Furthermore, during choice, we show that anterior cingulate activity correlates with the degree of difficulty associated with dissonance between value and time. Our data support an integrative architecture for decision making, revealing the neural representation of distinct subcomponents of value that may contribute to impulsivity and decisiveness

A neuroeconomic theory of rational addiction and\ud nonlinear time-perception.

Neuroeconomic conditions for “rational addiction” (Becker and Murphy, 1988) have\ud been unknown. This paper derived the conditions for “rational addiction” by utilizing a\ud nonlinear time-perception theory of “hyperbolic” discounting, which is mathematically\ud equivalent to the q-exponential intertemporal choice model based on Tsallis' statistics. It\ud is shown that (i) Arrow-Pratt measure for temporal cognition corresponds to the degree\ud of irrationality (i.e., Prelec’s “decreasing impatience” parameter of temporal\ud discounting) and (ii) rationality in addicts is controlled by a nondimensionalization\ud parameter of the logarithmic time-perception function. Furthermore, the present theory\ud illustrates the possibility that addictive drugs increase impulsivity via dopaminergic\ud neuroadaptation without increasing irrationality. Future directions in the application of\ud the model to studies in neuroeconomics are discussed

Impulsivity and self-control during intertemporal decision making linked to the neural dynamics of reward value representation

A characteristic marker of impulsive decision making is the discounting of delayed rewards, demonstrated via choice preferences and choice-related brain activity. However, delay discounting may also arise from how subjective reward value is dynamically represented in the brain when anticipating an upcoming chosen reward. In the current study, brain activity was continuously monitored as human participants freely selected an immediate or delayed primary liquid reward and then waited for the specified delay before consuming it. The ventromedial prefrontal cortex (vmPFC) exhibited a characteristic pattern of activity dynamics during the delay period, as well as modulation during choice, that is consistent with the time-discounted coding of subjective value. The ventral striatum (VS) exhibited a similar activity pattern, but preferentially in impulsive individuals. A contrasting profile of delay-related and choice activation was observed in the anterior PFC (aPFC), but selectively in patient individuals. Functional connectivity analyses indicated that both vmPFC and aPFC exerted modulatory, but opposite, influences on VS activation. These results link behavioral impulsivity and self-control to dynamically evolving neural representations of future reward value, not just during choice, but also during postchoice delay periods

Cultural Neuroeconomics of Intertemporal Choice

According to theories of cultural neuroscience, Westerners and Easterners may have distinct styles of cognition (e.g., different allocation of attention). Previous research has shown that Westerners and Easterners tend to utilize analytical and holistic cognitive styles, respectively. On the other hand, little is known regarding the cultural differences in neuroeconomic behavior. For instance, economic decisions may be affected by cultural differences in neurocomputational processing underlying attention; however, this area of neuroeconomics has been largely understudied. In the present paper, we attempt to bridge this gap by considering the links between the theory of cultural neuroscience and neuroeconomic theory\ud of the role of attention in intertemporal choice. We predict that (i) Westerners are more impulsive and inconsistent in intertemporal choice in comparison to Easterners, and (ii) Westerners more steeply discount delayed monetary losses than Easterners. We examine these predictions by utilizing a novel temporal discounting model based on Tsallis' statistics (i.e. a q-exponential model). Our preliminary analysis of temporal discounting of gains and losses by Americans and Japanese confirmed the predictions from the cultural neuroeconomic theory. Future study directions, employing computational modeling via neural networks, are briefly outlined and discussed