A discounting framework for choice with delayed and probabilistic rewards

When choosing between delayed or uncertain outcomes, individuals discount the value of such outcomes on the basis of the expected time to or the likelihood of their occurrence. In an integrative review of the expanding experimental literature on discounting, the authors show that although the same form of hyperbola-like function describes discounting of both delayed and probabilistic outcomes, a variety of recent findings are inconsistent with a single-process account. The authors also review studies that compare discounting in different populations and discuss the theoretical and practical implications of the findings. The present effort illustrates the value of studying choice involving both delayed and probabilistic outcomes within a general discounting framework that uses similar experimental procedures and a common analytical approach.     

Temporal discounting and preference reversals in choice between delayed outcomes

Subjects chose between pairs of hypothetical amounts of money available after different delays. When smaller, more immediate amounts were selected over larger, more delayed amounts, the addition of a constant delay to both outcomes resulted in reversals of preference, contrary to the standard discounted utility model of economics. The delays at which preference reversed were determined for three pairs of amounts ($20 vs. $50, $100 vs. $250, and $500 vs. $1,250). The relation between the delay to the larger amount and the delay to the smaller amount at preference reversal was well fit by both linear and quadratic functions. Intercepts increased with amount, strongly suggesting that rate of discounting decreases with amount. The presence of significant negative curvature in the data from the majority of individual subjects poses problems for exponential and hyperbolic models of temporal discounting in self-control, both of which predict a linear relation between the delays to the larger and smaller amounts.     

Semantic organization of study materials has opposite effects on recognition and recall

It has been well established for several decades that semantic organization of study materials greatly enhances recall by facilitating access to information during retrieval. However, the effect of organization on recognition, and its relationship to the effect on recall, is in doubt. We report the first direct comparison of the effects of categorically organizing study lists on recognition, cued recall, and free recall. We found that whereas organization improved recall, it impaired recognition. Organization had a larger effect on free recall than on cued recall. Within the categorized lists, recall was superior for items highly associated with the category; the opposite was true of recognition. In recall, organization improved the proportion of categories recalled, but it lowered the proportion of items per category recalled. A simple framework for interpreting the dissociation is offered. Possible mechanisms underlying the detrimental effect of organization on memory and prospects for future research are briefly discussed.     

Positive mood effects on delay discounting

Delay discounting is the process by which the value of an expected reward decreases as the delay to obtaining that reward increases. Individuals with higher discounting rates tend to prefer smaller immediate rewards over larger delayed rewards. Previous research has indicated that personality can influence an individual's discounting rates, with higher levels of Extraversion predicting a preference for immediate gratification. The current study examined how this relationship would be influenced by situational mood inductions. While main effects were observed for both Extraversion and cognitive ability in the prediction of discounting rates, a significant interaction was also observed between Extraversion and positive affect. Extraverted individuals were more likely to prefer an immediate reward when first put in a positive mood. Extraverts thus appear particularly sensitive to impulsive, incentive-reward-driven behavior by temperament and by situational factors heightening positive affect.     

Sensitivity of children's behavior to probabilistic reward: effects of a decreasing-ratio lottery system on math performance

Probabilistic reward has been shown to affect children's choice behavior in game-like activities. We examined the effects of a lottery system containing progressively lower exchange ratios on children's completion of math problems. Two of the 3 children completed problems above baseline levels until the chance of exchange dropped to 25%. This study describes a potentially useful method for examining reinforcement schedules in applied settings and extends previous research on probabilities of reinforcement.     

Temporal discounting when the choice is between two delayed rewards

The present experiments extend the temporal discounting paradigm from choice between an immediate and a delayed reward to choice between 2 delayed rewards: a smaller amount of money available sooner and a larger amount available later. Across different amounts and delays, the data were consistently well described by a hyperbola-like discounting function, and the degree of discounting decreased systematically as the delay to the sooner reward increased. Three theoretical models (the elimination-by-aspects, present-value comparison, and common-aspect attenuation hypotheses) were evaluated. The best account of the data was provided by the common-aspect attenuation hypothesis, according to which the common aspect of the choice alternatives (i.e., the time until the sooner reward is available) receives less weight in the decision-making process.     

Effects of inflation on the subjective value of delayed and probabilistic rewards

In the years prior to 1994, there were very high rates of inflation in Poland, and the zloty depreciated relative to the U.S. dollar. However, the new zloty, introduced in 1995, was associated with greatly decreased rates of inflation and provided a more stable currency. We report a series of three experiments that take advantage of these changes to examine the effects of inflation on the subjective value of delayed and probabilistic rewards. Subjects were Polish citizens familiar with both zlotys and dollars. The first two experiments, conducted in 1994, used dollars and old zlotys, and the third experiment, conducted in 1996, used dollars and new zlotys. In all three experiments, the dollar and zloty rewards were of equivalent worth, according to the then current exchange rates. In Experiment 1, subjects chose between immediate and delayed rewards and, in Experiment 2, chose between certain and probabilistic rewards. The subjective value of a delayed reward was greater when its amount was specified in dollars than when it was specified in old zlotys. In contrast, the currency in which a reward was specified had no effect on the subjective value of probabilistic rewards. The results of these two experiments suggest a selective effect of inflation on decisions involving delayed rewards. This was verified in the third experiment, in which, using new zlotys, no differences in discounting were observed between the two currencies with either probabilistic or delayed rewards. Importantly, in all three experiments, the discounting of both delayed and probabilistic rewards was well described by the same simple mathematical model, suggesting that similar decision-making processes underlie both phenomena. However, the present results argue against a single-process theory in which the discounting of probabilistic rewards is derived from the discounting of delayed rewards.     

The role of marking when reward is delayed

Two-choice spatial discrimination by rats is enhanced if a salient stimulus marker occurs immediately after every choice response and again after a delay interval (Lieberman, McIntosh & Thomas, 1979). Three experiments further explore this effect. Experiment 1 found that the second marker is unnecessary. Experiment 2 found that a marker presented before a response is as effective as one presented after. Both effects could be explained in terms of markers focusing attention on subsequent cues. Experiment 3, however, found that markers after choice enhance learning even when no discriminative cues are present following the marker. Markers thus appear to initiate both a backward search through memory and attention to subsequent events; both processes help to identify events that might be related to the unexpected marking stimulus.     

Incremental effects of reward on creativity

The authors examined 2 ways reward might increase creativity. First, reward contingent on creativity might increase extrinsic motivation. Studies 1 and 2 found that repeatedly giving preadolescent students reward for creative performance in 1 task increased their creativity in subsequent tasks. Study 3 reported that reward promised for creativity increased college students' creative task performance. Second, expected reward for high performance might increase creativity by enhancing perceived self-determination and, therefore, intrinsic task interest. Study 4 found that employees' intrinsic job interest mediated a positive relationship between expected reward for high performance and creative suggestions offered at work. Study 5 found that employees' perceived self-determination mediated a positive relationship between expected reward for high performance and the creativity of anonymous suggestions for helping the organization.     

Affective state and preference for immediate versus delayed reward

Fifty-two 9-year-olds were randomly assigned to success and failure treatment conditions involving acceptance or rejection of a drawing made for an art show. In a pilot study, this treatment was shown to influence significantly subjects' affective states. Just after treatment, subjects were presented with choices between pairs of small immediate and large delayed rewards. As predicted, subjects who had experienced success chose more often the large delayed reward than those who failed (p < .02). Prior expectancy for success and sex were not significantly related to choice behavior. The influence of affective state on various parameters of the choice paradigm was discussed, and other evidence was cited indicating that affective state is a major determinant of self-gratification behavior.     

Reformative self‐control and discounting of reward value by delay or effort1

Abstract: The relation between reformative self‐control and discounting of reward value by delay or effort was examined. Thirty‐two Japanese undergraduates were selected based on their scores on a standardized pencil and paper test of self‐control, the Redressive–Reformative Self‐Control Scale ( Sugiwaka, 1995 ). The subjects indicated their preference (i) between a hypothetical 100 000 yen reward available with various delays and a certain reward of variable amount available immediately; or (ii) between a hypothetical 100 000 yen reward available with various efforts and a certain reward of variable amount available without effort. The discounting of delayed reward value was described well by a hyperbolic function proposed by Mazur (1987 ). The slopes of the delayed discounting curves were steeper for subjects who scored low on the Reformative Self‐Control Scale, which is a measure of reformative self‐control and a subscale of the Redressive–Reformative Self‐Control Scale. The discounting by effort was also described well by the hyperbolic function. The slopes of the effortful discounting curves did not correlate to those of the delayed discounting curves. The slopes were not systematically different across the groups of subjects with different scores on the reformative self‐control scale.     

Intolerance of uncertainty and decisions about delayed, probabilistic rewards

Worry is the inflated concern about potential future threats and is a hallmark feature of generalized anxiety disorder. Previous theoretical work has suggested that worry may be a consequence of intolerance of uncertainty (IU). The current study seeks to explore the behavioral consequences of IU. Specifically, we examine how IU might be associated with aspects of reward-based decision making. We utilized a simple laboratory gambling task in which participants chose between small, low-probability rewards available immediately at the beginning of each trial and large, high-probability rewards only available after some variable delay. Results demonstrate that higher levels of intolerance of uncertainty were associated with a tendency to select the immediately available, but less valuable and less probable rewards. IU also predicted decision-makers' sensitivity to outcomes. We discuss the cognitive and affective mechanisms that are likely to underlie the observed decision-making behavior and the implications for anxiety disorders.     

Effects of reward on delayed reaction time task performance of hyperactive children

This experiment was designed to investigate the effects of continuous, partial, and noncontingent schedules of reward, as well as the withdrawal of rewards, on the performance of hyperactive and normal control children on a delayed reaction time task. Although noncontingent reward resulted in faster reaction times for control subjects, performance of hyperactives deteriorated under noncontingent reward and improved when it was withdrawn. Also, reaction times of controls during extinction remained superior to baseline, whereas performance of hyperactives returned to baseline level. It was suggested that these and other findings reviewed point to an unusual sensitivity to rewards in hyperactive children.     

Unpredictable chronic stress in juvenile or adult rats has opposite effects, respectively, promoting and impairing resilience

We evaluated the effects of early maternal deprivation (MD; age 7-14 days) alone or in combination with unpredictable chronic stress (UCS; MDUN; 28-84 days) on anxiety and learning in 90 days old adult rats. We hypothesized that exposure to both stressors (MDUN) would be more detrimental than exposure to one or neither. Unexpectedly, adult rats from the MDUN group did not differ from control animals, whereas adult MD animals exhibited impaired avoidance learning. We next investigated the effect of juvenile-onset (30-90 days) versus adult-onset (60-90 days) stress on avoidance learning in adulthood (90 days). We found that adult-onset chronic stress impaired avoidance learning and memory whereas juvenile-onset stress did not. Thus, the results again indicate that juvenile exposure to UCS induces resilience rather than impairment.