DISCOUNTING OF DELAYED REWARDS:

Abstract— In this study, children, young adults, and older adults chose between immediate and delayed hypothetical monetary rewards The amount of the delayed reward was held constant while its delay was varied All three age groups showed delay discounting, that is, the amount of an immediate reward judged to be of equal value to the delayed reward decreased as a function of delay The rate of discounting was highest for children and lowest for older adults, predicting a life-span developmental trend toward increased selfcontrol Discounting of delayed rewards by all three age groups was well described by a single function with agesensitive parameters (all R²s>94) Thus, even though there are quantitative age differences in delay discounting, the existence of an age-invanant form of discount function suggests that the process of choosing between rewards of different amounts and delays is qualitatively similar across the life span     

DISCOUNTING OF DELAYED FOOD REWARDS IN PIGEONS AND RATS: IS THERE A MAGNITUDE EFFECT?

Temporal discounting refers to the decrease in the present, subjective value of a reward as the time to its receipt increases. Results from humans have shown that a hyperbola-like function describes the form of the discounting function when choices involve hypothetical monetary rewards. In addition, magnitude effects have been reported in which smaller reward amounts are discounted more steeply than larger amounts. The present research examines the cross-species generality of these findings using real rewards, namely food pellets, with both pigeons and rats. As with humans, an adjusting amount procedure was used to estimate the amount of immediate reward judged equal in value to a delayed reward. Different amounts of delayed food rewards (ranging from 5 to 32 pellets in pigeons and from 5 to 20 pellets in rats) were studied at delays varying from 1 s to 32 s. A simple hyperbola, similar to the hyperbola-like mathematical function that describes the discounting of hypothetical monetary rewards in humans, described the discounting of food rewards in both pigeons and rats. These results extend the generality of the mathematical model of discounting. Rates of discounting delayed food rewards were higher for pigeons than for rats. Unlike humans, however, neither pigeons nor rats showed a reliable magnitude effect: Rate of discounting did not vary systematically as a function of the amount of the delayed reward.     

Social discounting and delay discounting

Social discounting was measured as the amount of money a participant was willing to forgo to give a fixed amount (usually $75) to another person. In the first experiment, amount forgone was a hyperbolic function of the social distance between the giver and receiver. In the second experiment, degree of social discounting was an increasing function of reward magnitude whereas degree of delay discounting was a decreasing function of reward magnitude. In the third experiment, the shape of the function relating delayed rewards to equally valued immediate rewards for another person was predicted from individual delay and social discount functions. All in all, the studies show that the social discount function, like delay and probability discount functions, is hyperbolic in form. Copyright © 2007 John Wiley & Sons, Ltd.     

The absence of numbers to express the amount may affect delay discounting with humans

Human delay discounting is usually studied with experimental protocols that use symbols to express delay and amount. In order to further understand discounting, we evaluated whether the absence of numbers to represent reward amounts affects discount rate in general, and whether the magnitude effect is generalized to nonsymbolic situations in particular. In Experiment 1, human participants were exposed to a delay‐discounting task in which rewards were presented using dots to represent monetary rewards (nonsymbolic); under this condition the magnitude effect did not occur. Nevertheless, the magnitude effect was observed when equivalent reward amounts were presented using numbers (symbolic). Moreover, in estimation tasks, magnitude increments produced underestimation of large amounts. In Experiment 2, participants were exposed only to the nonsymbolic discounting task and were required to estimate reward amounts in each trial. Consistent with Experiment 1, the absence of numbers representing reward amounts produced similar discount rates of small and large rewards. These results suggest that value of nonsymbolic rewards is a nonlinear function of amount and that value attribution depends on perceived difference between the immediate and the delayed nonsymbolic rewards.     

Tests of an indifference rule in variable‐delay and double‐reward choice procedures with humans

Four hundred and fifty participants were recruited from Amazon Mechanical Turk across 3 experiments to test the predictions of a hyperbolic discounting equation in accounting for human choices involving variable delays or multiple rewards (Mazur, 1984, 1986). In Experiment 1, participants made hypothetical choices between 2 monetary alternatives, 1 consisting of a fixed delay and another consisting of 2 delays of equal probability (i.e., a variable‐delay procedure). In Experiment 2, participants made hypothetical monetary choices between a single, immediate reward and 2 rewards, 1 immediate and 1 delayed (i.e., a double‐reward procedure). Experiment 3 also used a double‐reward procedure, but with 2 delayed rewards. Participants in all 3 experiments also completed a standard delay‐discounting task. Finally, 3 reward amounts were tested in each type of task ($100, $1000, and $5000). In the double‐reward conditions (Experiments 2 and 3), the results were in good qualitative and quantitative agreement with Mazur's model (1984, 1986). In contrast, when participants made choices involving variable delays (Experiment 1), there was relatively poor qualitative and quantitative agreement with this model. These results, along with our previous findings, suggest the structure of questions in hypothetical tasks with humans can be a strong determinant of the choice pattern.     

Positive mood effects on delay discounting

Delay discounting is the process by which the value of an expected reward decreases as the delay to obtaining that reward increases. Individuals with higher discounting rates tend to prefer smaller immediate rewards over larger delayed rewards. Previous research has indicated that personality can influence an individual's discounting rates, with higher levels of Extraversion predicting a preference for immediate gratification. The current study examined how this relationship would be influenced by situational mood inductions. While main effects were observed for both Extraversion and cognitive ability in the prediction of discounting rates, a significant interaction was also observed between Extraversion and positive affect. Extraverted individuals were more likely to prefer an immediate reward when first put in a positive mood. Extraverts thus appear particularly sensitive to impulsive, incentive-reward-driven behavior by temperament and by situational factors heightening positive affect.     

Temporal discounting when the choice is between two delayed rewards

The present experiments extend the temporal discounting paradigm from choice between an immediate and a delayed reward to choice between 2 delayed rewards: a smaller amount of money available sooner and a larger amount available later. Across different amounts and delays, the data were consistently well described by a hyperbola-like discounting function, and the degree of discounting decreased systematically as the delay to the sooner reward increased. Three theoretical models (the elimination-by-aspects, present-value comparison, and common-aspect attenuation hypotheses) were evaluated. The best account of the data was provided by the common-aspect attenuation hypothesis, according to which the common aspect of the choice alternatives (i.e., the time until the sooner reward is available) receives less weight in the decision-making process.     

The neural mechanisms of inter-temporal decision-making: understanding variability

Humans and animals prefer immediate over delayed rewards (delay discounting). This preference for smaller-but-sooner over larger-but-later rewards shows substantial interindividual variability in healthy subjects. Moreover, a strong bias towards immediate reinforcement characterizes many psychiatric conditions such as addiction and attention-deficit hyperactivity disorder. We discuss the neural mechanisms underlying delay discounting and describe how interindividual variability (trait effects) in the neural instantiation of subprocesses of delay discounting (such as reward valuation, cognitive control and prospection) contributes to differences in behaviour. We next discuss different interventions that can partially remedy impulsive decision-making (state effects). Although the precise neural mechanisms underlying many of these modulating influences are only beginning to be unravelled, they point towards novel treatment approaches for disorders of impulse control.     

Temporal discounting predicts student responsiveness to exchange delays in a classroom token system

Typical assessments of temporal discounting involve presenting choices between hypothetical monetary outcomes. Participants choose between smaller immediate rewards and larger delayed rewards to determine how the passage of time affects the subjective value of reinforcement. Few studies, however, have compared such discounting to actual manipulations of reward delay. The present study examined the predictive validity of a temporal discounting procedure developed for use with children. Forty-six sixth-grade students completed a brief discounting assessment and were then exposed to a classwide intervention that involved both immediate and delayed reinforcement in a multiple baseline design across classrooms. The parameters derived from two hyperbolic models of discounting correlated significantly with actual on-task behavior under conditions of immediate and delayed exchange. Implications of temporal discounting assessments for behavioral assessment and treatment are discussed.     

Now for Me,Later for Us? Effects of Group Context on Temporal Discounting

Delayed rewards are less valuable than immediate rewards. This well‐established finding has focused almost entirely on individual outcomes. However, are delayed rewards similarly discounted if they are shared by a group? The current article reports on three experiments exploring the effect of group context on delay discounting. Results indicate that discount rates of individual and group rewards were highly correlated, but that respondents were more willing to wait (decreased discounting) for shared outcomes than for individual outcomes. An explanatory model is proposed suggesting that decreased discount rates in group contexts may be due to the way the effects of both delay and social discounting are combined. That is, in a group context, a person values both a future reward (discounted by delay) and a present reward to another person (discounted by the social distance between them). The results are explained by a combined discount function containing a delay factor and a factor representing the social distance between the decision maker and group members. Practical implications of the fact that shared consequences can increase individual self‐control are also discussed. Copyright © 2011 John Wiley & Sons, Ltd.     

Effects of reward magnitude frames on measures of delay discounting in a hypothetical money scenario

The current study analyzed the effects of three frames of reward magnitude—quantity, volume, and duration—on the rate at which college students discounted hypothetical, delayed monetary rewards. Hypothetical scenarios were presented using the fill-in-the-blank discounting questionnaire and participants made choices between immediate and delayed hypothetical monetary rewards. Scenarios framed the monetary choices as (a) quantity of dollar bills, (b) height (inches) of a stack of dollar bills, and (c) duration of time spent in a hypothetical cash machine to collect dollar bills. For each scenario, participants' subjective values were used to calculate the area under the curve (AuC). Framing resulted in a moderate effect size: The duration frame yielded significantly smaller AuC values compared to the quantity and volume frames. Thus, the framing of reward magnitude was a significant variable in controlling discounting rates for hypothetical, delayed monetary rewards. Subsequent investigations should be aware of the independent effects of the reward magnitude frames on delay discounting rates.     

Temporal discounting of monetary rewards in children and adolescents with ADHD and autism spectrum disorders

It has been difficult to differentiate attention‐deficit/hyperactivity disorder (ADHD) and autism spectrum disorder (ASD) in terms of some aspects of their cognitive profile. While both show deficits in executive functions, it has been suggested that they may differ in their response to monetary reward. For instance, children with ADHD prefer small immediate over large delayed rewards more than typically developing controls. One explanation for this is that they discount the value of rewards to a higher degree as they are moved into the future. The current study investigated whether children with ADHD can be differentiated from those with ASD in terms of reward discounting. Thirty‐nine children (8–16 y) with ADHD, 34 children with ASD and 46 typically developing controls performed a hypothetical monetary temporal discounting task. Participants were instructed to make repeated choices between small variable rewards (0, 5, 10, 20, 30€) delivered immediately and large rewards delivered after a variable delay. Children with ADHD but not ASD discounted future rewards at a higher rate than typically developing controls. These data confirm steeper discounting of future rewards in ADHD and add to a small but growing literature showing that the psychological profile of ADHD can be distinguished from that of ASD in terms of disrupted motivational processes.     

Pathological gamblers, with and without substance use disorders, discount delayed rewards at high rates

Pathological gambling is classified as a disorder of impulse control, yet little research has evaluated behavioral indices of impulsivity in gamblers. The rates at which rewards delayed in time are subjectively devalued may be a behavioral marker of impulsivity. This study evaluated delay discounting in 60 pathological gamblers and 26 control participants. Gamblers were divided into those with (n = 21) and without (n = 39) substance use disorders. A hypothetical $1,000 reward was delayed at intervals ranging from 6 hr to 25 years, and immediate rewards varied from $1 to $999. Pathological gamblers discounted delayed rewards at higher rates than control participants, and gamblers with substance use disorders discounted delayed rewards at higher rates than non-substance-abusing gamblers. These data provide further evidence that rapid discounting of delayed rewards may be a feature central to impulse control and addictive disorders, including pathological gambling.     

Observing response acquisition: preference for unpredictable appetitive rewards obtained under conditions predicted by DMOD

In five E-maze experiments, rats were given a choice between receiving reward and nonreward in a situation where stimuli were correlated with reward outcome (predictable situation) versus one where the stimuli were uncorrelated with reward outcome (unpredictable situation). Preference for the unpredictable situation occurred under the following conditions: (a) small (one 37-mg pellet), immediate rewards; (b) small, delayed (15 s) rewards, if the cues correlated with reward outcome were absent during the delay interval; (c) large (15 pellets), immediate rewards if a difficult discrimination was required; and (d) if the stimulus predicting nonreward was present at the choice point. Preference for the predictable situation was strongest if reinforcement was delayed and large or the stimulus predicting reward was present at the choice point. A weaker preference for the predictable situation occurred if reinforcement was immediate and large and a simple discrimination was required or if reinforcement was large and delayed and the cues that correlated with reward outcome were absent during the delay interval. The results support the predictions of DMOD (Daly modification of the Rescorla-Wagner model), a mathematical model of appetitive learning (Daly & Daly, 1982).     

Gambling and Delay Discounting in Japanese Students

The present study examined the relationship between gambling behavior and delay discounting with Japanese residents. Japanese university students were selected into pathological gambler and non‐gambling control groups using a Japanese version of the South Oaks Gambling Screen. In a discounting task, participants chose individually between a large delayed reward and a smaller immediate reward with varied delays. The discounting rate (k‐value) and the area under the curve were significantly higher and smaller, respectively, for the gambler group than for the control group. These findings show that Japanese gamblers discount delayed rewards more steeply than non‐gambling controls, as has been found in U.S. residents.     

Integrated versus segregated accounting and the magnitude effect in temporal discounting

Temporal discounting rates in humans generally decrease as the amount of reward increases, a phenomenon known as themagnitude effect. In the present study, we examined whether temporal discounting and the magnitude effect are related to segregation of choices in terms of gains or losses for waiting for or expediting receipt of a reward. Subjects (N = 24) responded to a series of hypothetical choices about amounts of money available either immediately or after a delay. The immediate and delayed amounts either were presented as integrated amounts in the baseline condition or were segregated as differential gains or losses for choosing delayed or expedited consumption (delay and speedup conditions, respectively). Temporal discounting rates decreased in the segregated conditions, in accord with the standard discounted utility model but contrary to the hypothesis that the subjects were choosing on the basis of reward differentials in the baseline condition. The size of the magnitude effect was comparable in the baseline and the delay conditions but decreased in the speed-up condition. These results challenge explanations of the magnitude effect in terms of an increasing proportional sensitivity property of the utility function (Loewenstein & Prelec, 1992) and the hypothesis that subjects choose on the basis of differentials even when the rewards are presented as integrated amounts.     

Delay-discounting probabilistic rewards: Rates decrease as amounts increase

The independence of delay-discounting rate and monetary reward size was tested by offering subjects (N = 621) a series of choices between immediate rewards and larger, delayed rewards. In contrast to previous studies, in which hypothetical rewards have typically been employed, subjects in the present study were entered into a lottery in which they had a chance of actually receiving one of their choices. The delayed rewards were grouped into small ($30–$35), medium ($55–$65), and large amounts ($70–$85). Using a novel parameter estimation procedure, we estimated discounting rates for all three reward sizes for each subject on the basis of his/her pattern of choices. The data indicated that the discounting rate is a decreasing function of the size of the delayed reward (p < .0001), whether hyperbolic or exponential discounting functions are assumed. In addition, a reliable gender difference was found (p = .005), with males discounting at higher rates than females, on average.     

Modeling the effect of reward amount on probability discounting

The present study with college students examined the effect of amount on the discounting of probabilistic monetary rewards. A hyperboloid function accurately described the discounting of hypothetical rewards ranging in amount from $20 to $10,000,000. The degree of discounting increased continuously with amount of probabilistic reward. This effect of amount was not due to changes in the rate parameter of the discounting function, but rather was due to increases in the exponent. These results stand in contrast to those observed with the discounting of delayed monetary rewards, in which the degree of discounting decreases with reward amount due to amount-dependent decreases in the rate parameter. Taken together, this pattern of results suggests that delay and probability discounting reflect different underlying mechanisms. That is, the fact that the exponent in the delay discounting function is independent of amount is consistent with a psychophysical scaling interpretation, whereas the finding that the exponent of the probability-discounting function is amount-dependent is inconsistent with such an interpretation. Instead, the present results are consistent with the idea that the probability-discounting function is itself the product of a value function and a weighting function. This idea was first suggested by Kahneman and Tversky (1979), although their prospect theory does not predict amount effects like those observed. The effect of amount on probability discounting was parsimoniously incorporated into our hyperboloid discounting function by assuming that the exponent was proportional to the amount raised to a power. The amount-dependent exponent of the probability-discounting function may be viewed as reflecting the effect of amount on the weighting of the probability with which the reward will be received.     

Asymmetry of the group context effect on delay discounting

Binary choice delay discounting tasks require participants to indicate preference between smaller, immediate, and larger, delayed rewards. Previous research indicates that when the delayed reward is shared with others, the delayed outcome is preferred compared with when the outcomes are for the self only, resulting in lower rates of delay discounting. The present series of studies sought to replicate and extend this finding. Study 1 compared delay discounting on a standard task in which both immediate and delayed outcomes are for the self and a group context task where the delayed outcome was shared with one other person. Replicating previous results, group context resulted in lower rates of delay discounting, and this effect was independent of how the shared outcome was presented. Study 2 compared delay discounting on a standard task and a group context task where the immediate outcome was shared. In contrast to Study 1, group context resulted in higher rates of delay discounting, suggesting that preference in intertemporal choice tracks the shared outcome. Moreover, this effect was not independent of how the shared outcome was presented. This is the first study to reveal that group context, when applied to the immediate outcome, can result in higher rates of delay discounting.