Time preferences in long-tailed macaques (Macaca fascicularis) and humans (Homo sapiens)

Rosati et al. (Curr Biol 17(19):1663–1668, 2007) found in a self-control test in which choice was between a smaller, immediately delivered food and a larger, delayed food, that chimpanzees preferred the larger reward (self-control); humans, however, preferred the smaller reward (impulsivity). They attributed their results to a species difference in self-control. In Experiment 1, monkeys (long-tailed macaques) were exposed to a self-control task in two conditions: where the food was hidden under differently colored bowls and where it was visible. When these two conditions were compared, choice shifted from greater preference for the impulsive alternative in the hidden condition to greater preference for the self-control alternative in the visible condition. Additionally, in both conditions, preference shifted from self-control to impulsivity over sessions. These results were explained in terms of the reversed-contingency effect (a propensity to reach for more over less when rewards are visible) and not to a capacity for self-control. In Experiment 2, humans that demonstrated preference for more over less in choice preferred the impulsive alternative when choice to either alternative was followed by the same intertrial interval—a preference that accelerates trial rates relative to preference of the self-control alternative. When trial rates were equated so that neither choice accelerated session’s end, humans demonstrated self-control. These results suggest that Rosati et al.’s demonstration of impulsivity in humans was due to participants’ desire to minimize session time.     

Time discounting and time preference in animals: A critical review

Animals are an important model for studies of impulsivity and self-control. Many studies have made use of the intertemporal choice task, which pits small rewards available sooner against larger rewards available later (typically several seconds), repeated over many trials. Preference for the sooner reward is often taken to indicate impulsivity and/or a failure of self-control. This review shows that very little evidence supports this assumption; on the contrary, ostensible discounting behavior may reflect a boundedly rational but not necessarily impulsive reward-maximizing strategy. Specifically, animals may discount weakly, or even adopt a long-term rate-maximizing strategy, but fail to fully incorporate postreward delays into their choices. This failure may reflect learning biases. Consequently, tasks that measure animal discounting may greatly overestimate the true discounting and may be confounded by processes unrelated to time preferences. If so, animals may be much more patient than is widely believed; human and animal intertemporal choices may reflect unrelated mental operations; and the shared hyperbolic shape of the human and animal discount curves, which is used to justify cross-species comparisons, may be coincidental. The discussion concludes with a consideration of alternative ways to measure self-control in animals.     

Making choices in anticipation of similar future choices can increase self-control

Two experiments tested the efficacy of linking a current choice with similar future choices as a means of increasing self-control. Participants were offered choices between smaller and sooner vs. larger and later amounts of money (Experiment 1, n = 60) or food (Experiment 2, n = 34). After a small-large pair for which the participant preferred the smaller reward was found, a choice between the same pair was offered as the 1st of 5 such choices to be offered over a period of weeks. The majority of participants in both experiments who chose between all 5 smaller and all 5 larger rewards chose the larger rewards. One third of participants in Experiment 1 who could choose independently on each pair in the series reversed their previous preference and chose the larger reward in the context of the series. These results suggest that self-control can be enhanced by viewing one's current choice as predictive of future choices.     

The predictive accuracy of intertemporal‐choice models

How do people choose between a smaller reward available sooner and a larger reward available later? Past research has evaluated models of intertemporal choice by measuring goodness of fit or identifying which decision‐making anomalies they can accommodate. An alternative criterion for model quality, which is partly antithetical to these standard criteria, is predictive accuracy. We used cross‐validation to examine how well 10 models of intertemporal choice could predict behaviour in a 100‐trial binary‐decision task. Many models achieved the apparent ceiling of 85% accuracy, even with smaller training sets. When noise was added to the training set, however, a simple logistic‐regression model we call the difference model performed particularly well. In many situations, between‐model differences in predictive accuracy may be small, contrary to long‐standing controversy over the modelling question in research on intertemporal choice, but the simplicity and robustness of the difference model recommend it to future use.     

Capuchin monkeys (Cebus apella) let lesser rewards pass them by to get better rewards

Self-control is defined as foregoing an immediate reward to gain a larger delayed reward. Methods used to test self-control comparatively include inter-temporal choice tasks, delay of gratification tasks, and accumulation tasks. To date, capuchin monkeys have shown different levels of self-control across tasks. This study introduced a new task that could be used comparatively to measure self-control in an intuitive context that involved responses that required no explicit training. Capuchin monkeys (Cebus apella) were given a choice between two food items that were presented on a mechanized, revolving tray that moved those foods sequentially toward the monkeys. A monkey could grab the first item or wait for the second, but was only allowed one item. Most monkeys in the study waited for a more highly preferred food item or a larger amount of the same food item when those came later, and they inhibited the prepotent response to grab food by not reaching out to take less-preferred foods or smaller amounts of food that passed directly in front of them first. These data confirm that the mechanisms necessary for self-control are present in capuchin monkeys and indicate that the methodology can be useful for broader comparative assessments of self-control.     

Chimpanzees (Pan troglodytes) can wait,when they choose to: a study with the hybrid delay task

Self-control has been studied in nonhuman animals using a variety of tasks. The inter-temporal choice (ITC) task presents choices between smaller–sooner (SS) and larger–later (LL) options. Using food amounts as rewards, this presents two problems: (a) choices of the LL option could either reflect self-control or instead result from animals’ difficulty with pointing to smaller amounts of food; (b) there is no way to verify whether the subjects would not revert their choice for the LL option, if given the opportunity to do so during the ensuing delay. To address these problems, we have recently introduced a new protocol, the hybrid delay task, which combines an initial ITC with a subsequent accumulation phase in which selection of the SS option leads to its immediate delivery, but choice of the LL option then leads to one-by-one presentation of those items that continues only as long as the subject does not eat any of the accumulated items. The choice of the LL option therefore only reflects self-control when the number of items obtained from LL choices during the accumulation phase is higher than what could be received in the SS option. Previous research with capuchin monkeys demonstrated that their apparent self-control responses in the ITC task may have overestimated their general self-control abilities, given their poor performance in the hybrid delay task. Here, chimpanzees instead demonstrated that their choices for the LL option in the ITC phase of the hybrid delay task were confirmed by their ability to sustain long delays during accumulation of LL rewards.     

Enhancing tolerance to delayed reinforcers: the role of intervening activities

Three participants with moderate to profound mental retardation were exposed to choices between an immediate small amount and a delayed larger amount of a preferred reinforcer. All participants initially showed a relatively high preference for the smaller reinforcer, suggesting impulsive choice making. However this preference reversed, suggesting self-control, when the larger reinforcer was available immediately and, over time, its delay was gradually increased. Results highlight the potential utility of incorporating concurrent activities into self-control training paradigms.     

Commitment, choice and self-control

When offered a choice (Choice Y) between a small immediate reward (2-sec exposure to grain) and a large reward (4-sec exposure to grain) delayed by 4 sec, pigeons invariably preferred the small, immediate reward. However, when offered a choice (Choice X) between a delay of T seconds followed by Choice Y and a delay of T seconds followed by restriction to the large delayed reward only, the pigeon's choice depended on T. When T was small, the pigeons chose the alternative leading to Choice Y (and then chose the small, immediate reward). When T was large, the pigeons chose the alternative leading to the large delayed reward only. The reversal of preference as T increases is predicted by several recent models for choice between various amounts and delays of reward. The preference for the large delayed alternative with long durations of T parallels everyday instances of advance commitment to a given course of action. Such commitment may be seen as a prototype for self-control.     

Can squirrel monkeys (Saimiri sciureus) learn self-control? A study using food array selection tests and reverse-reward contingency

Eight squirrel monkeys (Saimiri sciureus) were presented with 2 stimulus arrays, namely 1 and 4 pieces of food, but they received only the array other than the one they reached for. In this reverse-reward condition, all monkeys initially showed a strong preference for the larger array. One monkey learned to reach toward the smaller array when a large-or-none reward contingency was applied (i.e., no reward followed a reach toward the larger array, but this array was given for a reach toward the smaller array). When correction trials and time-out were added to the large-or-none procedure, all remaining monkeys except 1 learned this form of self-control. Performance was maintained when correction trials were discontinued, the original reverse-reward condition was rerun, and novel array-size pairs were presented. This study demonstrates one form of self-control in a New World primate and shows the reverse-reward procedure to be a potentially valuable method for assessing species and individual differences in self-control and numerosity-related abilities.     

Do open space environments encourage children to seek immediate gratification?

This study evaluated the effects of 14 months' exposure to open space environments on preference for delayed reward. Three hundred eighty-nine pupils in open space and conventional classes in grades 4 and 5 were individually administered one of four tests for delay of gratification. These tests consisted of a choice between an immediate, smaller reward and a delayed, larger reward. Each of the four independent studies found that approximately the same percent of pupils from each educational environment selected the delayed alternative.     

A preliminary analysis of self-control with aversive events: the effects of task magnitude and delay on the choices of children with autism

When faced with a choice between two aversive events, a person exhibits self-control by choosing a smaller, more immediate aversive event over a larger, delayed aversive event. Task demands are often aversive to children with autism and other developmental disabilities. The purpose of this study was to evaluate behavioral sensitivity to differences in the amount and delay of tasks as part of a preliminary study on self-control. Participants were 2 children with autism who engaged in problem behavior maintained by escape. Results indicated a lack of self-control with respect to choosing between two aversive tasks and suggested potential strategies for increasing self-control (i.e., choosing a small immediate task over a large delayed task).     

Teaching self-control to small groups of dually diagnosed adults

The present study examined the use of a progressive delay procedure to teach self-control to two groups of dually diagnosed adults. When given a choice between an immediate smaller reinforcer and a larger delayed reinforcer, both groups chose the smaller reinforcer during baseline. During treatment, progressive increases in work requirements for gaining access to a larger reinforcer resulted in both groups selecting larger delayed reinforcers. The results are discussed with respect to increasing cooperative work behavior and self-control.     

A procedure to teach self-control to children with attention deficit hyperactivity disorder

This study examined the use of a progressive delay procedure combined with verbal mediation to teach self-control to children with attention deficit disorder. Results showed that when participants were initially given the choice between an immediate smaller reinforcer and a larger delayed reinforcer, all participants chose the smaller reinforcer. When slight delays to obtain a larger reinforcer were instated in conjunction with intervening verbal activity, all participants demonstrated self-control regardless of the content of the verbal activity.     

Exchange delays and impulsive choice in adult humans.

Choice responding by adult humans in a discrete-trial task was examined as a function of conditions that manipulated either the delay to point delivery or the delay between points and their exchange for money. In point-delay conditions, subjects chose between an "impulsive" alternative that provided a small amount of points immediately and a "self-control" alternative that provided a larger amount of points delayed by 15, 30, or 60 s. Points were exchanged for money immediately following the session. Subjects preferred the self-control alternative. In exchange-delay conditions, subjects chose between a small amount of points exchangeable for money immediately following the session and a larger amount of points exchangeable for money after 1 day, 3 weeks, or 6 weeks. A self-control preference observed for all subjects in the 1-day exchange-delay condition reversed to exclusive impulsive preference for 4 of the 6 subjects when choice conditions involved exchange delays of 3 or 6 weeks. These results show that human choice is sensitive to the manipulation of exchange delays and that impulsive preference can be obtained with exchange delays on the order of weeks.     

Building blocks of self-control: increased tolerance for delay with bundled rewards

Impulsive choice can be defined as temporary preference for a smaller-sooner reward (SS) over a larger-later reward (LL). Hyperbolic discounting implies that impulsive choices will occur less when organisms choose between a series of SSs versus LLs all at once than when they choose between single SS versus LL pairs. Eight rats were exposed to two conditions of an intertemporal choice paradigm using sucrose solution as reward. In both conditions, the LL was 150 microl delayed by 3 s, while the SS was an immediate reward that ranged from 25-150 microl across sessions. Preference for the LL was greater when the chosen reward was automatically delivered three times in succession (bundled) than when it was chosen singly and delivered after each choice. For each of the 8 rats, the estimated SS amount that produced indifference was higher in the bundled condition than in the single condition. Because bundling in humans may be based on the perception that one's current choice is predictive of future choices, the data presented here may demonstrate an important building block of self-control.     

The effect of variable delays on self-control

Five pigeons served as subjects in an experiment that examined the effects of variable as opposed to fixed delays on preference in a self-control paradigm (choice between larger, more delayed and smaller, less delayed reinforcers). Nonindependent concurrent variable-interval schedules were used to measure choice. When delays to the larger, more delayed reinforcers were variable as opposed to fixed, the subjects showed an increased preference for that alternative (the self-control alternative). A series of regressions revealed that the hyperbolic decay model and incentive theory provided poor fits to the data, but a modified version of the generalized matching law provided an adequate fit. Together, consistent with a general prediction made by discounting models, the data supported the conclusion that variable delays can increase self-control. However, specific discounting models were not able to explain the present data well.     

Choice in a self-control paradigm: Quantification of experience-based differences

Previous quantitative models of choice in a self-control paradigm (choice between a larger, more-delayed reinforcer and a smaller, less-delayed reinforcer) have not described individual differences. Two experiments are reported that provide additional quantitative data on experience-based differences in choice between reinforcers of varying sizes and delays. In Experiment 1, seven pigeons in a self-control paradigm were exposed to a fading procedure that increased choices of the larger, more-delayed reinforcer through gradually decreasing the delay to the smaller of two equally delayed reinforcers. Three control subjects, exposed to each of the small-reinforcer delays to which the experimental subjects were exposed, but for fewer sessions, demonstrated that lengthy exposure to each of the conditions in the fading procedure may be necessary in order for the increase to occur. In Experiment 2, pigeons with and without fading-procedure exposure chose between reinforcers of varying sizes and delays scheduled according to a concurrent variable-interval variable-interval schedule. In both experiments, pigeons with fading-procedure exposure were more sensitive to variations in reinforcer size than reinforcer delay when compared with pigeons without this exposure. The data were described by the generalized matching law when the relative size of its exponents, representing subjects' relative sensitivity to reinforcer size and delay, were grouped according to subjects' experience.     

Choice with initial and terminal link reinforcement: an alternative self-control paradigm

Self-control is demonstrated when a less desirable immediate outcome is chosen to ensure a substantially better future. In a novel animal analogue of this situation, primary reinforcement was delivered in both the initial and terminal links of a concurrent chain schedule. Rats made initial link choices between equal amounts of ethanol-free or ethanol-containing milk. Choosing the ethanol-free reinforcer resulted in delivery of the larger terminal link reinforcer and was thus analogous to self-control. Self-control decreased as the delay between initial and terminal links increased. The results have implications for human choice situations where decisions are made between two immediately available reinforcement alternatives each associated with a different delayed outcome.     

Temporal discounting when the choice is between two delayed rewards

The present experiments extend the temporal discounting paradigm from choice between an immediate and a delayed reward to choice between 2 delayed rewards: a smaller amount of money available sooner and a larger amount available later. Across different amounts and delays, the data were consistently well described by a hyperbola-like discounting function, and the degree of discounting decreased systematically as the delay to the sooner reward increased. Three theoretical models (the elimination-by-aspects, present-value comparison, and common-aspect attenuation hypotheses) were evaluated. The best account of the data was provided by the common-aspect attenuation hypothesis, according to which the common aspect of the choice alternatives (i.e., the time until the sooner reward is available) receives less weight in the decision-making process.     

Failure to find a distance effect in pigeon choice: Manipulating amount and delay of reinforcement

The choice behavior of primates, including humans, displays a distance effect: Latency to choose between alternatives appears to increase with smaller differences in value. There is, so far, no demonstration of this effect in birds. Tests of distance effects in birds have been conducted in binary choice situations with a dominant alternative, where one alternative is superior to the other in all aspects that meaningfully contribute to value (e.g., provides access to the same reinforcer, but with a shorter delay). The present study considers the possibility that including dominant alternatives in choice tests precludes distance effects. Four pigeons were presented with binary choices between alternatives that varied in amount and delay. Some choices had a dominant alternative (smaller–sooner or larger–later vs. smaller–later) and some did not (smaller–sooner vs. larger–later). Across phases, only the delay to the smaller–sooner reinforcer varied. Distance effects were expected to be expressed as longer latencies as choice between smaller–sooner and larger–later reinforcers approached indifference. Despite the sensitivity of choice to differences in amount and delay, no distance effect was observed. Alternative explanations for the failure to find a distance effect in pigeon choice, including the Sequential Choice Model (SCM), are discussed.