RISKY CHOICE IN PIGEONS AND HUMANS: A CROSS‐SPECIES COMPARISON

Pigeon and human subjects were given repeated choices between variable and adjusting delays to token reinforcement that titrated in relation to a subject's recent choice patterns. Indifference curves were generated under two different procedures: immediate exchange, in which a token earned during each trial was exchanged immediately for access to the terminal reinforcer (food for pigeons, video clips for humans), and delayed exchange, in which tokens accumulated and were exchanged after 11 trials. The former was designed as an analogue of procedures typically used with nonhuman subjects, the latter as an analogue to procedures typically used with human participants. Under both procedure types, different variable‐delay schedules were manipulated systematically across conditions in ways that altered the reinforcer immediacy of the risky option. Under immediate‐exchange conditions, both humans and pigeons consistently preferred the variable delay, and indifference points were generally ordered in relation to relative reinforcer immediacies. Such risk sensitivity was greatly reduced under delayed‐exchange conditions. Choice and trial‐initiation response latencies varied directly with indifference points, suggesting that local analyses may provide useful ancillary measures of reinforcer value. On the whole, the results indicate that modifying procedural features brings choices of pigeons and humans into better accord, and that human—nonhuman differences on risky choice procedures reported in the literature may be at least partly a product of procedural differences.     

DELAY‐AMOUNT TRADEOFFS IN CHOICES BY PIGEONS AND RATS: HYPERBOLIC VERSUS EXPONENTIAL DISCOUNTING

An adjusting‐delay procedure was used to study the choices of pigeons and rats when both delay and amount of reinforcement were varied. In different conditions, the choice alternatives included one versus two reinforcers, one versus three reinforcers, and three versus two reinforcers. The delay to one alternative (the standard alternative) was kept constant in a condition, and the delay to the other (the adjusting alternative) was increased or decreased many times a session so as to estimate an indifference point—a delay at which the two alternatives were chosen about equally often. Indifference functions were constructed by plotting the adjusting delay as a function of the standard delay for each pair of reinforcer amounts. The experiments were designed to test the prediction of a hyperbolic decay equation that the slopes of the indifference functions should increase as the ratio of the two reinforcer amounts increased. Consistent with the hyperbolic equation, the slopes of the indifference functions depended on the ratios of the two reinforcer amounts for both pigeons and rats. These results were not compatible with an exponential decay equation, which predicts slopes of 1 regardless of the reinforcer amounts. Combined with other data, these findings provide further evidence that delay discounting is well described by a hyperbolic equation for both species, but not by an exponential equation. Quantitative differences in the y‐intercepts of the indifference functions from the two species suggested that the rate at which reinforcer strength decreases with increasing delay may be four or five times slower for rats than for pigeons.     

Effects of time between trials on rats' and pigeons' choices with probabilistic delayed reinforcers

Parallel experiments with rats and pigeons examined reasons for previous findings that in choices with probabilistic delayed reinforcers, rats' choices were affected by the time between trials whereas pigeons' choices were not. In both experiments, the animals chose between a standard alternative and an adjusting alternative. A choice of the standard alternative led to a short delay (1 s or 3 s), and then food might or might not be delivered. If food was not delivered, there was an "interlink interval," and then the animal was forced to continue to select the standard alternative until food was delivered. A choice of the adjusting alternative always led to food after a delay that was systematically increased and decreased over trials to estimate an indifference point--a delay at which the two alternatives were chosen about equally often. Under these conditions, the indifference points for both rats and pigeons increased as the interlink interval increased from 0 s to 20 s, indicating decreased preference for the probabilistic reinforcer with longer time between trials. The indifference points from both rats and pigeons were well described by the hyperbolic-decay model. In the last phase of each experiment, the animals were not forced to continue selecting the standard alternative if food was not delivered. Under these conditions, rats' choices were affected by the time between trials whereas pigeons' choices were not, replicating results of previous studies. The differences between the behavior of rats and pigeons appears to be the result of procedural details, not a fundamental difference in how these two species make choices with probabilistic delayed reinforcers.     

Estimation of indifference points with an adjusting-delay procedure.

In a series of conditions, pigeons chose between 1.5 s and 3 s of access to grain, each preceded by some delay. The delay that preceded the small reinforcer was constant throughout a condition. The delay that preceded the large reinforcer was increased or decreased a number of times each session in order to estimate an "indifference point," a delay at which the subject chose each alternative about equally often. The experiment was designed to determine whether variations in any of four features of this adjusting-delay procedure would systematically alter the estimated indifference points. The four features were the total trial duration, the number of center-key responses necessary to begin a trial, the number of choice trials that preceded each change in the adjusting delay, and step size--the size of each increment and decrement in the delay. Manipulation of the first three features had no systematic effects on the indifference points. As step size was increased from 0.5 s to 6 s, within-session variability of the adjusting delay steadily increased, and the 6-s step size produced larger indifference-point estimates for some subjects. The results suggest that, within certain limits, these procedural features can be altered without affecting the indifference-point estimates, but that the use of a large step size can distort the estimates. Some theoretical implications of the relative constancy of indifference points across these procedural variations are discussed.     

Transitional and steady-state choice behavior under an adjusting-delay schedule

Twelve rats made repeated choices on an adjusting-delay schedule between a smaller reinforcer (A) that was delivered immediately after a response and a larger reinforcer (B) that was delivered after a delay which increased or decreased by 20% depending on the subject's choices in successive blocks of trials. In two phases of the experiment (100 sessions and 40 sessions), reinforcer sizes were selected which enabled theoretical parameters expressing the rate of delay discounting and sensitivity to reinforcer size to be estimated from the ratio of the indifference delays obtained in the two phases. Indifference delays, calculated from adjusting delays in the last 10 sessions of each phase, were shorter when the sizes of A and B were 14 and 25 μl of a 0.6 M sucrose solution than when they were 25 and 100 μl of the same solution. The ratio of the indifference delays was significantly smaller than that predicted on the basis of an assumed linear relation between reinforcer size and instantaneous reinforcer value, consistent with a previous proposal that this relation may be hyperbolic in form. Estimates of the rate of delay discounting based on the ratio of the two indifference delays (mean, 0.08 s(-1)) were similar to values obtained previously using different intertemporal choice protocols. Estimates of the size-sensitivity parameter (mean 113 μl) were similar to estimates recently derived from performance on progressive-ratio schedules. In both phases of the experiment, adjusting delays in successive blocks of trials were analyzed using the Fourier transform. The power spectrum obtained from individual rats had a dominant frequency that corresponded to a period of oscillation of the adjusting delay between 30 and 100 trial blocks (mean, 78). Power in the dominant frequency band was highest in the early sessions of the first phase and declined with extended training. It is suggested that this experimental protocol may have utility in neurobehavioral studies of intertemporal choice.     

Tests of transitivity in choices between fixed and variable reinforcer delays.

This experiment tested for transitivity in pigeons' choices between variable-time (VT) and fixed-time (FT) schedules. In a discrete-trials procedure, a subject chose between two alternatives by making a single key peck. Each choice was between a "standard alternative," which was the same schedule throughout a condition, and an "adjusting alternative," in which the delay to reinforcement was systematically increased or decreased many times a session. These adjustments enabled an approximate indifference point to be identified--the value of the adjusting delay at which the subject chose each alternative about equally often. Each test of transitivity involved four conditions. In one, the standard alternative was a variable-time schedule with a 2-s reinforcer, and the adjusting alternative also delivered a 2-s reinforcer. A second condition was similar except that the adjusting alternative delivered a 5-s reinforcer. The indifference point from each of these conditions was then converted to a fixed-time schedule for subsequent comparisons in the third and fourth conditions, respectively. Each of these last two conditions compared one of the fixed-time schedules (based upon the previous conditions and including their different reinforcer durations) with an adjusting schedule that delivered the alternative reinforcer duration, to determine whether the obtained indifference points would be those predicted from the prior alternative-duration comparisons with the VT schedule. There was little evidence for intransitivity of choice: Averaged across subjects and replications, the obtained indifference points deviated from perfect transitivity by less than 8%, and these deviations were not statistically significant. These results contrast with those of Navarick and Fantino (1972), who found frequent violations of transitivity between periodic and aperiodic schedules using a concurrent-chains procedure with variable-interval schedules in the initial links.     

People Wait Longer when the Alternative is Risky: The Relation Between Preferences in Risky and Inter‐temporal Choice

In two experiments, we demonstrate that despite indicating indifference when probed about risk or delay in isolation, when forced to explicitly trade‐off between the two, participants prefer delayed over risky rewards. This pattern of findings sets a boundary condition for any common utility‐based comparison process involving both risk and delay. Furthermore, this change from indifference‐in‐isolation to delay‐preference‐in‐a‐trade‐off strengthens as reward amount increases. Exploratory modeling results suggest that the shift in preference can be explained by allowing for different discount rates for delay‐only choices compared with when delay is in competition with risk. This explanation is better than one in which probability weighting is different between risk‐only choices and risks considered in the presence of a delay. Together, the empirical and modeling work lays a path for future investigations of why and when people's evaluation of the properties of risky and delayed choices vary as a function of the alternatives on offer. Copyright © 2017 John Wiley & Sons, Ltd.     

Introduction to the special issue on comparative cognition

Prior research has shown that nonhumans show an extreme preference for variable‐ over fixed‐delays to reinforcement. This well‐established preference for variability occurs because a reinforcer's strength or “value” decreases according to a curvilinear function as its delay increases. The purpose of the present experiments was to investigate whether this preference for variability occurs with human participants making hypothetical choices. In three experiments, participants recruited from Amazon Mechanical Turk made choices between variable and fixed monetary rewards. In a variable‐delay procedure, participants repeatedly chose between a reward delivered either immediately or after a delay (with equal probability) and a reward after a fixed delay (Experiments 1 and 2). In a double‐reward procedure, participants made choices between an alternative consisting of two rewards, one delivered immediately and one after a delay, and a second alternative consisting of a single reward delivered after a delay (Experiments 1 and 3). Finally, all participants completed a standard delay‐discounting task. Although we observed both curvilinear discounting and magnitude effects in the standard discounting task, we found no consistent evidence of a preference for variability—as predicted by two prominent models of curvilinear discounting (i.e., a simple hyperbola and a hyperboloid)—in our variable‐delay and double‐reward procedures. This failure to observe a preference for variability may be attributed to the hypothetical, rule‐governed nature of choices in the present study. In such contexts, participants may adopt relatively simple strategies for making more complex choices.     

An alternative approach to calculating Area‐Under‐the‐Curve (AUC) in delay discounting research

Applied to delay discounting data, Area‐Under‐the‐Curve (AUC) provides an atheoretical index of the rate of delay discounting. The conventional method of calculating AUC, by summing the areas of the trapezoids formed by successive delay‐indifference point pairings, does not account for the fact that most delay discounting tasks scale delay pseudoexponentially, that is, time intervals between delays typically get larger as delays get longer. This results in a disproportionate contribution of indifference points at long delays to the total AUC, with minimal contribution from indifference points at short delays. We propose two modifications that correct for this imbalance via a base‐10 logarithmic transformation and an ordinal scaling transformation of delays. These newly proposed indices of discounting, AUC_{log d} and AUC_{or d,} address the limitation of AUC while preserving a primary strength (remaining atheoretical). Re‐examination of previously published data provides empirical support for both AUC_{log d} and AUC_{or d} . Thus, we believe theoretical and empirical arguments favor these methods as the preferred atheoretical indices of delay discounting.     

Effects of reinforcer probability, delay, and response requirements on the choices of rats and pigeons: possible species differences

In Experiment 1 with rats, a left lever press led to a 5-s delay and then a possible reinforcer. A right lever press led to an adjusting delay and then a certain reinforcer. This delay was adjusted over trials to estimate an indifference point, or a delay at which the two alternatives were chosen about equally often. Indifference points increased as the probability of reinforcement for the left lever decreased. In some conditions with a 20% chance of food, a light above the left lever was lit during the 5-s delay on all trials, but in other conditions, the light was only lit on those trials that ended with food. Unlike previous results with pigeons, the presence or absence of the delay light on no-food trials had no effect on the rats' indifference points. In other conditions, the rats showed less preference for the 20% alternative when the time between trials was longer. In Experiment 2 with rats, fixed-interval schedules were used instead of simple delays, and the presence or absence of the fixed-interval requirement on no-food trials had no effect on the indifference points. In Experiment 3 with rats and Experiment 4 with pigeons, the animals chose between a fixed-ratio 8 schedule that led to food on 33% of the trials and an adjusting-ratio schedule with food on 100% of the trials. Surprisingly, the rats showed less preference for the 33% alternative in conditions in which the ratio requirement was omitted on no-food trials. For the pigeons, the presence or absence of the ratio requirement on no-food trials had little effect. The results suggest that there may be differences between rats and pigeons in how they respond in choice situations involving delayed and probabilistic reinforcers.     

The Influence of Time Estimation and Time‐Saving Preferences on Learning to Make Temporally Dependent Decisions from Experience

In this paper, we merge research related to experiential learning, temporal perception, and the value of time and money by examining decisions where the timing of action (response) determines the outcome received. We predicted that time‐saving preferences and impatience would decrease maximization (i.e., taking action when it returned the largest reward), and that the constraints of temporal perception would compound their effects. Across three studies, participants undershot on average (i.e., responded earlier than the period of time during which a response would return the maximal reward) showed a preference for shorter‐delay options and often did not find the maximal reward. In addition, participants' reliance on temporal perception increased undershooting, increased preferences for shorter‐delay options, and reduced maximization. Nevertheless, participants who found the maximal reward continued to maximize at a high rate rather than opting for shorter delays and smaller rewards. Thus, while most participants appeared to have a preference for saving time, most behaved as reward maximizers rather than temporal discounters. Copyright © 2017 John Wiley & Sons, Ltd.     

Procrastination by pigeons with fixed-interval response requirements.

Two experiments studied the phenomenon of procrastination, in which pigeons chose a larger, more delayed response requirement over a smaller, more immediate response requirement. The response requirements were fixed-interval schedules that did not lead to an immediate food reinforcer, but that interrupted a 55-s period in which food was delivered at random times. The experiments used an adjusting-delay procedure in which the delay to the start of one fixed-interval requirement was varied over trials to estimate an indifference point--a delay at which the two alternatives were chosen about equally often. Experiment 1 found that as the delay to a shorter fixed-interval requirement was increased, the adjusting delay to a longer fixed-interval requirement also increased, and the rate of increase depended on the duration of the longer fixed-interval requirement. Experiment 2 found a strong preference for a fixed delay of 10 s to the start of a fixed-interval requirement compared to a mixed delay of either 0 or 20 s. The results help to distinguish among different equations that might describe the decreasing effectiveness of a response requirement with increasing delay, and they suggest that delayed reinforcers and delayed response requirements have symmetrical but opposite effects on choice.     

Choice between delayed reinforcers and fixed-ratio schedules requiring forceful responding.

This experiment measured pigeons' choices between delayed reinforcers and fixed-ratio schedules in which a force of approximately 0.48 N was needed to operate the response key. In ratio-delay conditions, subjects chose between a fixed-ratio schedule and an adjusting delay. The delay was increased or decreased several times a session in order to estimate an indifference point--a delay duration at which the two alternatives were chosen about equally often. Each ratio-delay condition was followed by a delay-delay condition in which subjects chose between the adjusting delay and a variable-time schedule, with the components of this schedule selected to match the ratio completion times of the preceding ratio-delay condition. The adjusting delays at the indifference point were longer when the alternative was a fixed-ratio schedule than when it was a matched variable-time schedule, which indicated a preference for the matched variable-time schedules over the fixed-ratio schedules. This preference increased in a nonlinear manner with increasing ratio size. This nonlinearity was inconsistent with a theory that states that indifference points for both time and ratio schedules can be predicted by multiplying the choice response-reinforcer intervals of the two types of schedules by different multiplicative constants. Two other theories, which predict nonlinear increases in preference for the matched variable-time schedules, are discussed.     

Assessing preference for reinforcers using demand curves, work-rate functions, and expansion paths

Much research has focused on the effects of environmental variability on foraging decisions. However, the general pattern of preference for variability in delay to reward and aversion to variability in amount of reward remains unexplained a either a mechanistic or a functional level. Starlings' preferences between a fixed and a variable option were studied in two treatments, A and D. The fixed option was the same in both treatments (20-s fixed-interval delay, five units food). In Treatment A the variable option gave two equiprobable amounts of food (20-s delay, three or seven units) and in D it gave two equiprobable delays to food (2.5-s or 60.5-s delays, five units). In both treatments the programmed ratio [amount/(intertrial interval+latency+delay)] in the fixed option equaled the arithmetic mean of the two possible ratios in the variable option (ITI = 40 s, latency = 1 s). The variable option was strongly preferred in Treatment D and was weakly avoided in Treatment A. These results are discussed in the light of two theoretical models, a form of constrained rate maximization and a version of scalar expectancy theory. The latter accommodates more of the data and is based on independently verifiable assumptions, including Weber's law.     

Alan Baron: A pioneer in translational science

Primates take longer to choose between alternatives with smaller differences in value. This effect—a particular instance of the distance effect in symbolic comparisons—has not been replicated in birds. Instead, birds appear to respond independently to each alternative, such that the latency to choose depends primarily on the alternative of highest value. Three experiments tested for the distance effect in pigeons under conditions not previously considered. Experiment 1 presented pigeons with forced‐ and binary free‐choice trials, where each alternative was one of three possible delays to reinforcement (4, 8, and 16 s). Pigeons were exposed to the choice stimuli for different amounts of time and with different sample response requirements prior to the choice response. Experiment 2 added a fourth (0‐s delay) alternative. Experiment 3 substituted the 16‐s delay with a second 4‐s delay. In all experiments, pigeons systematically chose the shortest delay to reinforcement. Latency to choose the 4‐s delay did not vary when choosing against the 8‐s or 16‐s delay, regardless of whether choice stimuli were exposed for the duration of nine pecks (Experiment 1), or whether a 0‐s delay alternative was sometimes present (Experiment 2). Latency to choose the preferred of two identical alternatives (4‐s vs. 4‐s) was shorter than the latency to choose between different alternatives (4‐s vs. 8‐s; Experiment 3); this is the opposite of a distance effect. These results show no evidence of a distance effect in pigeon choice, consistent with the hypothesis that pigeons respond independently to each choice alternative.     

Reformative self‐control and discounting of reward value by delay or effort1

Abstract: The relation between reformative self‐control and discounting of reward value by delay or effort was examined. Thirty‐two Japanese undergraduates were selected based on their scores on a standardized pencil and paper test of self‐control, the Redressive–Reformative Self‐Control Scale ( Sugiwaka, 1995 ). The subjects indicated their preference (i) between a hypothetical 100 000 yen reward available with various delays and a certain reward of variable amount available immediately; or (ii) between a hypothetical 100 000 yen reward available with various efforts and a certain reward of variable amount available without effort. The discounting of delayed reward value was described well by a hyperbolic function proposed by Mazur (1987 ). The slopes of the delayed discounting curves were steeper for subjects who scored low on the Reformative Self‐Control Scale, which is a measure of reformative self‐control and a subscale of the Redressive–Reformative Self‐Control Scale. The discounting by effort was also described well by the hyperbolic function. The slopes of the effortful discounting curves did not correlate to those of the delayed discounting curves. The slopes were not systematically different across the groups of subjects with different scores on the reformative self‐control scale.     

Evidence for a magnitude effect in temporal discounting with pigeons

A magnitude effect in human intertemporal choice is well established-larger rewards or outcomes are discounted over time at a lower rate than are smaller rewards. However, many recent studies have failed to find a corresponding effect in nonhuman animals. Here we report a magnitude effect in temporal discounting for pigeons' choices involving a tradeoff between reward delay and amount. Pigeons chose between a small reward (1-s access to food) after a 2-s delay, and a large reward (4.5-s access to food) after a 28-s delay. Across conditions, the delays to the small and large rewards were increased or decreased, respectively. Temporal discounting functions obtained through a value-estimation procedure showed clear evidence of a magnitude effect: The value of the large reward decreased more slowly with increasing delay than the value of the small reward. We linked this result to a nonlinear relationship between choice and the delays associated with the small and large rewards. The nonlinearity was contrary to the generalized matching law but was predicted by the contextual choice model. Our results confirm the existence of a magnitude effect in nonhuman temporal discounting, showing that this adaptation is not unique to humans.     

A comparison of delays and ratio requirements in self-control choice.

In a discrete-trial procedure, pigeons could choose between 2-s and 6-s access to grain by making a single key peck. In Phase 1, the pigeons obtained both reinforcers by responding on fixed-ratio schedules. In Phase 2, they received both reinforcers after simple delays, arranged by fixed-time schedules, during which no responses were required. In Phase 3, the 2-s reinforcer was available through a fixed-time schedule and the 6-s reinforcer was available through a fixed-ratio schedule. In all conditions, the size of the delay or ratio leading to the 6-s reinforcer was systematically increased or decreased several times each session, permitting estimation of an "indifference point," the schedule size at which a subject chose each alternative equally often. By varying the size of the schedule for the 2-s reinforcer across conditions, several such indifference points were obtained from both fixed-time conditions and fixed-ratio conditions. The resulting "indifference curves" from fixed-time conditions and from fixed-ratio conditions were similar in shape, and they suggested that a hyperbolic equation describes the relation between ratio size and reinforcement value as well as the relation between reinforcer delay and its reinforcement value. The results from Phase 3 showed that subjects chose fixed-time schedules over fixed-ratio schedules that generated the same average times between a choice response and reinforcement.     

Influences of delay and rate of reinforcement on discrete-trial choice

An adjusting procedure was used to measure pigeons' preferences among alternatives that differed in the duration of a delay before reinforcement and of an intertrial interval (ITI) after reinforcement. In most conditions, a peck at a red key led to a fixed delay, followed by reinforcement, a fixed ITI, and then the beginning of the next trial. A peck at a green key led to an adjustable delay, reinforcement, and then the next trial began without an ITI. The purpose of the adjusting delay was to estimate an indifference point, or a delay that made a subject approximately indifferent between the two alternatives. As the ITI for the red key increased from 0 s to 60 s, the green-key delay at the indifference point increased systematically but only slightly. The fact that there was some increase showed that pigeons' choices were controlled by more than simply the delay to the next reinforcer. One interpretation of these results is that besides delay of reinforcement, rate of reinforcement also influenced choice. However, an analysis that ignored reinforcement rate, but considered the delays between a choice response and the reinforcers on subsequent trials, was able to account for most of the obtained increases in green-key delays. It was concluded that in this type of discrete-trial situation, rate of reinforcement exerts little control over choice behavior, and perhaps none at all.     

Probability and delay of reinforcement as factors in discrete-trial choice.

Pigeons chose between two alternatives that differed in the probability of reinforcement and the delay to reinforcement. A peck on the red key always produced a delay of 5 s and then a possible reinforcer. The probability of reinforcement for responding on this key varied from .05 to 1.0 in different conditions. A response on the green key produced a delay of adjustable duration and then a possible reinforcer, with the probability of reinforcement ranging from .25 to 1.0 in different conditions. The green-key delay was increased or decreased many times per session, depending on a subject's previous choices. The purpose of these adjustments was to estimate an indifference point, or a delay that resulted in a subject's choosing each alternative about equally often. In conditions where the probability of reinforcement was five times higher on the green key, the green-key delay averaged about 12 s at the indifference point. In conditions where the probability of reinforcement was twice as high on the green key, the green-key delay at the indifference point was about 8 s with high probabilities and about 6 s with low probabilities. An analysis based on these results and those from studies on delay of reinforcement suggests that pigeons' choices are relatively insensitive to variations in the probability of reinforcement between .2 and 1.0, but quite sensitive to variations in probability between .2 and 0.