Delay or probability discounting in a model of impulsive behavior: effect of alcohol

Little is known about the acute effects of drugs of abuse on impulsivity and self-control. In this study, impulsivity was assessed in humans using a computer task that measured delay and probability discounting. Discounting describes how much the value of a reward (or punisher) is decreased when its occurrence is either delayed or uncertain. Twenty-four healthy adult volunteers ingested a moderate dose of ethanol (0.5 or 0.8 g/kg ethanol: n = 12 at each dose) or placebo before completing the discounting task. In the task the participants were given a series of choices between a small, immediate, certain amount of money and $10 that was either delayed (0, 2, 30, 180, or 365 days) or probabilistic (i.e., certainty of receipt was 1.0, .9, .75, .5, or .25). The point at which each individual was indifferent between the smaller immediate or certain reward and the $10 delayed or probabilistic reward was identified using an adjusting-amount procedure. The results indicated that (a) delay and probability discounting were well described by a hyperbolic function; (b) delay and probability discounting were positively correlated within subjects; (c) delay and probability discounting were moderately correlated with personality measures of impulsivity; and (d) alcohol had no effect on discounting.     

Social discounting and delay discounting

Social discounting was measured as the amount of money a participant was willing to forgo to give a fixed amount (usually $75) to another person. In the first experiment, amount forgone was a hyperbolic function of the social distance between the giver and receiver. In the second experiment, degree of social discounting was an increasing function of reward magnitude whereas degree of delay discounting was a decreasing function of reward magnitude. In the third experiment, the shape of the function relating delayed rewards to equally valued immediate rewards for another person was predicted from individual delay and social discount functions. All in all, the studies show that the social discount function, like delay and probability discount functions, is hyperbolic in form. Copyright © 2007 John Wiley & Sons, Ltd.     

Temporal Patterning in Probabilistic Choice

Human subjects making repeated choices between probabilistic and near-certain monetary rewards tended to be less risk-averse the shorter the inter-trial interval. Subjects were also less risk-averse when the choice-outcome sequences were clustered in threes than when each choice-outcome sequence was separated from its neighbors by equal intertrial intervals. These results further extend the analogy between delay discounting and probability discounting. Probability discounting, like delay discounting, may be described by hyperbolic function relating discounted value to expected delay to the next reward.     

Reformative self‐control and discounting of reward value by delay or effort1

Abstract: The relation between reformative self‐control and discounting of reward value by delay or effort was examined. Thirty‐two Japanese undergraduates were selected based on their scores on a standardized pencil and paper test of self‐control, the Redressive–Reformative Self‐Control Scale ( Sugiwaka, 1995 ). The subjects indicated their preference (i) between a hypothetical 100 000 yen reward available with various delays and a certain reward of variable amount available immediately; or (ii) between a hypothetical 100 000 yen reward available with various efforts and a certain reward of variable amount available without effort. The discounting of delayed reward value was described well by a hyperbolic function proposed by Mazur (1987 ). The slopes of the delayed discounting curves were steeper for subjects who scored low on the Reformative Self‐Control Scale, which is a measure of reformative self‐control and a subscale of the Redressive–Reformative Self‐Control Scale. The discounting by effort was also described well by the hyperbolic function. The slopes of the effortful discounting curves did not correlate to those of the delayed discounting curves. The slopes were not systematically different across the groups of subjects with different scores on the reformative self‐control scale.     

Modeling the effect of reward amount on probability discounting

The present study with college students examined the effect of amount on the discounting of probabilistic monetary rewards. A hyperboloid function accurately described the discounting of hypothetical rewards ranging in amount from $20 to $10,000,000. The degree of discounting increased continuously with amount of probabilistic reward. This effect of amount was not due to changes in the rate parameter of the discounting function, but rather was due to increases in the exponent. These results stand in contrast to those observed with the discounting of delayed monetary rewards, in which the degree of discounting decreases with reward amount due to amount-dependent decreases in the rate parameter. Taken together, this pattern of results suggests that delay and probability discounting reflect different underlying mechanisms. That is, the fact that the exponent in the delay discounting function is independent of amount is consistent with a psychophysical scaling interpretation, whereas the finding that the exponent of the probability-discounting function is amount-dependent is inconsistent with such an interpretation. Instead, the present results are consistent with the idea that the probability-discounting function is itself the product of a value function and a weighting function. This idea was first suggested by Kahneman and Tversky (1979), although their prospect theory does not predict amount effects like those observed. The effect of amount on probability discounting was parsimoniously incorporated into our hyperboloid discounting function by assuming that the exponent was proportional to the amount raised to a power. The amount-dependent exponent of the probability-discounting function may be viewed as reflecting the effect of amount on the weighting of the probability with which the reward will be received.     

Pathological gamblers, with and without substance use disorders, discount delayed rewards at high rates

Pathological gambling is classified as a disorder of impulse control, yet little research has evaluated behavioral indices of impulsivity in gamblers. The rates at which rewards delayed in time are subjectively devalued may be a behavioral marker of impulsivity. This study evaluated delay discounting in 60 pathological gamblers and 26 control participants. Gamblers were divided into those with (n = 21) and without (n = 39) substance use disorders. A hypothetical $1,000 reward was delayed at intervals ranging from 6 hr to 25 years, and immediate rewards varied from $1 to $999. Pathological gamblers discounted delayed rewards at higher rates than control participants, and gamblers with substance use disorders discounted delayed rewards at higher rates than non-substance-abusing gamblers. These data provide further evidence that rapid discounting of delayed rewards may be a feature central to impulse control and addictive disorders, including pathological gambling.     

LOTTERIES AND THE TIME HORIZON

Abstract— People chose between hypothetical alternatives of (a) a million-dollar lottery prize and (b) a much smaller but certain amount of money When probabilities of winning the lottery were above about 1/100,000, subjects avoided risk, for example, a 1/100,000 probability of the million-dollar prize was chosen about as often as $700 for sure But at probabilities below 1/100,000, subjects sought risk, for example, a 1/1,000,000 probability of the million-dollar prize was chosen about as often as $9 for sure This crossover from risk aversion to risk seeking is predicted by Mazur's (1987) hyperbolic delay discount function with probability expressed as average delay due to strings of losses followed by a win, together with a time horizon limiting subjective delay     

Temporal discounting and preference reversals in choice between delayed outcomes

Subjects chose between pairs of hypothetical amounts of money available after different delays. When smaller, more immediate amounts were selected over larger, more delayed amounts, the addition of a constant delay to both outcomes resulted in reversals of preference, contrary to the standard discounted utility model of economics. The delays at which preference reversed were determined for three pairs of amounts ($20 vs. $50, $100 vs. $250, and $500 vs. $1,250). The relation between the delay to the larger amount and the delay to the smaller amount at preference reversal was well fit by both linear and quadratic functions. Intercepts increased with amount, strongly suggesting that rate of discounting decreases with amount. The presence of significant negative curvature in the data from the majority of individual subjects poses problems for exponential and hyperbolic models of temporal discounting in self-control, both of which predict a linear relation between the delays to the larger and smaller amounts.     

Tests of an indifference rule in variable‐delay and double‐reward choice procedures with humans

Four hundred and fifty participants were recruited from Amazon Mechanical Turk across 3 experiments to test the predictions of a hyperbolic discounting equation in accounting for human choices involving variable delays or multiple rewards (Mazur, 1984, 1986). In Experiment 1, participants made hypothetical choices between 2 monetary alternatives, 1 consisting of a fixed delay and another consisting of 2 delays of equal probability (i.e., a variable‐delay procedure). In Experiment 2, participants made hypothetical monetary choices between a single, immediate reward and 2 rewards, 1 immediate and 1 delayed (i.e., a double‐reward procedure). Experiment 3 also used a double‐reward procedure, but with 2 delayed rewards. Participants in all 3 experiments also completed a standard delay‐discounting task. Finally, 3 reward amounts were tested in each type of task ($100, $1000, and $5000). In the double‐reward conditions (Experiments 2 and 3), the results were in good qualitative and quantitative agreement with Mazur's model (1984, 1986). In contrast, when participants made choices involving variable delays (Experiment 1), there was relatively poor qualitative and quantitative agreement with this model. These results, along with our previous findings, suggest the structure of questions in hypothetical tasks with humans can be a strong determinant of the choice pattern.     

Delay, Probability, and Social Discounting in A Public Goods Game

A human social discount function measures the value to a person of a reward to another person at a given social distance. Just as delay discounting is a hyperbolic function of delay, and probability discounting is a hyperbolic function of odds-against, social discounting is a hyperbolic function of social distance. Experiment 1 obtained individual social, delay, and probability discount functions for a hypothetical $75 reward; participants also indicated how much of an initial $100 endowment they would contribute to a common investment in a public good. Steepness of discounting correlated, across participants, among all three discount dimensions. However, only social and probability discounting were correlated with the public-good contribution; high public-good contributors were more altruistic and also less risk averse than low contributors. Experiment 2 obtained social discount functions with hypothetical $75 rewards and delay discount functions with hypothetical $1,000 rewards, as well as public-good contributions. The results replicated those of Experiment 1; steepness of the two forms of discounting correlated with each other across participants but only social discounting correlated with the public-good contribution. Most participants in Experiment 2 predicted that the average contribution would be lower than their own contribution.     

Delay-discounting probabilistic rewards: Rates decrease as amounts increase

The independence of delay-discounting rate and monetary reward size was tested by offering subjects (N = 621) a series of choices between immediate rewards and larger, delayed rewards. In contrast to previous studies, in which hypothetical rewards have typically been employed, subjects in the present study were entered into a lottery in which they had a chance of actually receiving one of their choices. The delayed rewards were grouped into small ($30–$35), medium ($55–$65), and large amounts ($70–$85). Using a novel parameter estimation procedure, we estimated discounting rates for all three reward sizes for each subject on the basis of his/her pattern of choices. The data indicated that the discounting rate is a decreasing function of the size of the delayed reward (p < .0001), whether hyperbolic or exponential discounting functions are assumed. In addition, a reliable gender difference was found (p = .005), with males discounting at higher rates than females, on average.     

Discounting by Probabilistic Waiting

In everyday life, many probabilistic situations may be characterized as probabilistic waiting. A gambler, for example, bets repeatedly at the racetrack, the casino, or the card table. The gambler may not win on the first try, but if a gamble is repeated enough times, a win is almost certain to occur eventually. If repeated gambles are structured as strings of losses ending in a win (probabilistic waiting) and the amount won is discounted by the delay caused by the series of losses, then strings with many losses will be discounted more than those with fewer losses, thereby causing subjective value of the series of gambles as a whole to increase. The current study used the opposite effect that amount has on the degree of delay and probability discounting as a marker to determine whether people evaluate situations involving probabilistic waiting as they evaluate situations involving delayed outcomes or as situations involving probabilistic outcomes. We find that the more likely a probabilistic waiting situation is to end in reward (e.g., a gamble is repeated indefinitely until reward is obtained), the more that situation conforms to delay discounting; the less likely a probabilistic waiting situation is to end in reward (e.g., a fixed, small number of gambles), the more that situation conforms to probability discounting. We argue that the former situation is applicable to pathological gambling, and that people with steep delay discount functions would therefore be more likely to have gambling problems. Copyright © 2015 John Wiley & Sons, Ltd.     

Effects of inflation on the subjective value of delayed and probabilistic rewards

In the years prior to 1994, there were very high rates of inflation in Poland, and the zloty depreciated relative to the U.S. dollar. However, the new zloty, introduced in 1995, was associated with greatly decreased rates of inflation and provided a more stable currency. We report a series of three experiments that take advantage of these changes to examine the effects of inflation on the subjective value of delayed and probabilistic rewards. Subjects were Polish citizens familiar with both zlotys and dollars. The first two experiments, conducted in 1994, used dollars and old zlotys, and the third experiment, conducted in 1996, used dollars and new zlotys. In all three experiments, the dollar and zloty rewards were of equivalent worth, according to the then current exchange rates. In Experiment 1, subjects chose between immediate and delayed rewards and, in Experiment 2, chose between certain and probabilistic rewards. The subjective value of a delayed reward was greater when its amount was specified in dollars than when it was specified in old zlotys. In contrast, the currency in which a reward was specified had no effect on the subjective value of probabilistic rewards. The results of these two experiments suggest a selective effect of inflation on decisions involving delayed rewards. This was verified in the third experiment, in which, using new zlotys, no differences in discounting were observed between the two currencies with either probabilistic or delayed rewards. Importantly, in all three experiments, the discounting of both delayed and probabilistic rewards was well described by the same simple mathematical model, suggesting that similar decision-making processes underlie both phenomena. However, the present results argue against a single-process theory in which the discounting of probabilistic rewards is derived from the discounting of delayed rewards.     

Temporal discounting when the choice is between two delayed rewards

The present experiments extend the temporal discounting paradigm from choice between an immediate and a delayed reward to choice between 2 delayed rewards: a smaller amount of money available sooner and a larger amount available later. Across different amounts and delays, the data were consistently well described by a hyperbola-like discounting function, and the degree of discounting decreased systematically as the delay to the sooner reward increased. Three theoretical models (the elimination-by-aspects, present-value comparison, and common-aspect attenuation hypotheses) were evaluated. The best account of the data was provided by the common-aspect attenuation hypothesis, according to which the common aspect of the choice alternatives (i.e., the time until the sooner reward is available) receives less weight in the decision-making process.     

Differential effects of amount on temporal and probability discounting of gains and losses

In four experiments, we compared the effects of delay, probability, and monetary amount on the subjective value of gains and losses. For delayed gains, smaller amounts were discounted more steeply than larger amounts, whereas the opposite pattern was observed with probabilistic gains. For both delayed and probabilistic losses, however, amount had much smaller and less reliable effects on discounting. Taken together, the pattern of differential magnitude effects leads to delayed gains' being discounted significantly more steeply than delayed losses, but only at smaller amounts, whereas probabilistic gains are discounted significantly more steeply than probabilistic losses, but only at larger amounts. Even though the same hyperbola-like function described both individual and group discounting of delayed and probabilistic gains and losses, the present findings suggest that different processes are involved in discounting positive and negative outcomes. Raw data may be downloaded from www.psychonomic.org/archive.     

DISCOUNTING OF DELAYED REWARDS:

Abstract— In this study, children, young adults, and older adults chose between immediate and delayed hypothetical monetary rewards The amount of the delayed reward was held constant while its delay was varied All three age groups showed delay discounting, that is, the amount of an immediate reward judged to be of equal value to the delayed reward decreased as a function of delay The rate of discounting was highest for children and lowest for older adults, predicting a life-span developmental trend toward increased selfcontrol Discounting of delayed rewards by all three age groups was well described by a single function with agesensitive parameters (all R²s>94) Thus, even though there are quantitative age differences in delay discounting, the existence of an age-invanant form of discount function suggests that the process of choosing between rewards of different amounts and delays is qualitatively similar across the life span     

Modeling Myopic Decisions: Evidence for Hyperbolic Delay-Discounting within Subjects and Amounts

Hyperbolic and exponential discounting functions were compared as models of subjects′ present valuations of delayed rewards. Previous comparisons have been limited by relying on the assumption that discounting rate is independent of reward size; we avoided this limitation by making all comparisons within reward sizes. In Experiment 1, using real rewards in a simulated auction, and in Experiment 2, using hypothetical rewards, we offered subjects five monetary rewards at six delays each and asked them to indicate the smallest amount that they would accept immediately in exchange for those rewards. Both discounting functions were then fit to the six reported amounts for each reward using nonlinear regressions. In both experiments, although both functions fit the data very well, the hyperbolic function fit better for all of the delayed rewards. Furthermore, the hyperbolic function better described the data for 20 of 21 and 14 of 18 subjects in Experiments 1 and 2, respectively.     

PREFERENCE REVERSALS DUE TO MYOPIC DISCOUNTING OF DELAYED REWARD

Abstract— A basic stationarity axiom of economic theory assumes stable preference between two deferred goods separated by a fixed time. To test this assumption, we offered subjects choices between delayed rewards, while manipulating the delays to those rewards. Preferences typically reversed with changes in delay, as predicted by hyperbolic discounting models of impulsiveness. Of 36 subjects, 34 reversed preference from a larger, later reward to a smaller, earlier reward as the delays to both rewards decreased. We conclude that the stationarity axiom is not appropriate in models of human choice.     

Delay discounting by pathological gamblers

Discounting of delayed rewards by pathological gamblers was compared to discounting of delayed rewards by matched control nongambling participants. All participants completed a hypothetical choice task in which they made repeated choices between dollars 1,000 available after a delay and an equal or lesser amount of money available immediately. The delay to the large amount of money was varied from 1 week to 10 years across conditions. Indifference points between immediate money and delayed money were identified at each delay condition by varying the amount of immediate money across choice trials. For the majority of participants, indifference points decreased monotonically across delays. Overall, gamblers discounted the delayed rewards more steeply than did control participants.