RISKY CHOICE IN PIGEONS AND HUMANS: A CROSS‐SPECIES COMPARISON

Pigeon and human subjects were given repeated choices between variable and adjusting delays to token reinforcement that titrated in relation to a subject's recent choice patterns. Indifference curves were generated under two different procedures: immediate exchange, in which a token earned during each trial was exchanged immediately for access to the terminal reinforcer (food for pigeons, video clips for humans), and delayed exchange, in which tokens accumulated and were exchanged after 11 trials. The former was designed as an analogue of procedures typically used with nonhuman subjects, the latter as an analogue to procedures typically used with human participants. Under both procedure types, different variable‐delay schedules were manipulated systematically across conditions in ways that altered the reinforcer immediacy of the risky option. Under immediate‐exchange conditions, both humans and pigeons consistently preferred the variable delay, and indifference points were generally ordered in relation to relative reinforcer immediacies. Such risk sensitivity was greatly reduced under delayed‐exchange conditions. Choice and trial‐initiation response latencies varied directly with indifference points, suggesting that local analyses may provide useful ancillary measures of reinforcer value. On the whole, the results indicate that modifying procedural features brings choices of pigeons and humans into better accord, and that human—nonhuman differences on risky choice procedures reported in the literature may be at least partly a product of procedural differences.     

Substitution effects in a generalized token economy with pigeons

Pigeons made repeated choices between earning and exchanging reinforcer‐specific tokens (green tokens exchangeable for food, red tokens exchangeable for water) and reinforcer‐general tokens (white tokens exchangeable for food or water) in a closed token economy. Food and green food tokens could be earned on one panel; water and red water tokens could be earned on a second panel; white generalized tokens could be earned on either panel. Responses on one key produced tokens according to a fixed‐ratio schedule, whereas responses on a second key produced exchange periods, during which all previously earned tokens could be exchanged for the appropriate commodity. Most conditions were conducted in a closed economy, and pigeons distributed their token allocation in ways that permitted food and water consumption. When the price of all tokens was equal and low, most pigeons preferred the generalized tokens. When token‐production prices were manipulated, pigeons reduced production of the tokens that increased in price while increasing production of the generalized tokens that remained at a fixed price. The latter is consistent with a substitution effect: Generalized tokens increased and were exchanged for the more expensive reinforcer. When food and water were made freely available outside the session, token production and exchange was sharply reduced but was not eliminated, even in conditions when it no longer produced tokens. The results join with other recent data in showing sustained generalized functions of token reinforcers, and demonstrate the utility of token‐economic methods for assessing demand for and substitution among multiple commodities in a laboratory context.     

Token reinforcement, choice, and self-control in pigeons.

Pigeons were exposed to self-control procedures that involved illumination of light-emitting diodes (LEDs) as a form of token reinforcement. In a discrete-trials arrangement, subjects chose between one and three LEDs; each LED was exchangeable for 2-s access to food during distinct posttrial exchange periods. In Experiment 1, subjects generally preferred the immediate presentation of a single LED over the delayed presentation of three LEDs, but differences in the delay to the exchange period between the two options prevented a clear assessment of the relative influence of LED delay and exchange-period delay as determinants of choice. In Experiment 2, in which delays to the exchange period from either alternative were equal in most conditions, all subjects preferred the delayed three LEDs more often than in Experiment-1. In Experiment 3, subjects preferred the option that resulted in a greater amount of food more often if the choices also produced LEDs than if they did not. In Experiment 4, preference for the delayed three LEDs was obtained when delays to the exchange period were equal, but reversed in favor of an immediate single LED when the latter choice also resulted in quicker access to exchange periods. The overall pattern of results suggests that (a) delay to the exchange period is a more critical determinant of choice than is delay to token presentation; (b) tokens may function as conditioned reinforcers, although their discriminative properties may be responsible for the self-control that occurs under token reinforcer arrangements; and (c) previously reported differences in the self-control choices of humans and pigeons may have resulted at least in part from the procedural conventions of using token reinforcers with human subjects and food reinforcers with pigeon subjects.     

Determinants of pigeons' choices in token-based self-control procedures

Four pigeons were exposed to a token-based self-control procedure with stimulus lights serving as token reinforcers. Smaller-reinforcer choices produced one token immediately; larger-reinforcer choices produced three tokens following a delay. Each token could be exchanged for 2-s access to food during a signaled exchange period each trial. The main variables of interest were the exchange delays (delays from the choice to the exchange stimulus) and the food delays (also timed from the choice), which were varied separately and together across blocks of sessions. When exchange delays and food delays were shorter following smaller-reinforcer choices, strong preference for the smaller reinforcer was observed. When exchange delays and food delays were equal for both options, strong preference for the larger reinforcer was observed. When food delays were equal for both options but exchange delays were shorter for smaller-reinforcer choices, preference for the larger reinforcer generally was less extreme than under conditions in which both exchange and food delays were equal. When exchange delays were equal for both options but food delays were shorter for smaller-reinforcer choices, preference for the smaller reinforcer generally was less extreme than under conditions in which both exchange and food delays favored smaller-reinforcer choices. On the whole, the results were consistent with prior research on token-based self-control procedures in showing that choices are governed by reinforcer immediacy when exchange and food delays are unequal and by reinforcer amount when exchange and food delays are equal. Further, by decoupling the exchange delays from food delays, the results tentatively support a role for the exchange stimulus as a conditioned reinforcer.     

Stock optimizing in choice when a token deposit is the operant.

Each of 2 monkeys typically earned their daily food ration by depositing tokens in one of two slots. Tokens deposited in one slot dropped into a bin where they were kept (token kept). Deposits to a second slot dropped into a bin where they could be obtained again (token returned). In Experiment 1, a fixed-ratio (FR) 5 schedule that provided two food pellets was associated with each slot. Both monkeys preferred the token-returned slot. In Experiment 2, both subjects chose between unequal FR schedules with the token-returned slot always associated with the leaner schedule. When the FRs were 2 versus 3 and 2 versus 6, preferences were maintained for the token-returned slot; however, when the ratios were 2 versus 12, preference shifted to the token-kept slot. In Experiment 3, both monkeys chose between equal-valued concurrent variable-interval variable-interval schedules. Both monkeys preferred the slot that returned tokens. In Experiment 4, both monkeys chose between FRs that typically differed in size by a factor of 10. Both monkeys preferred the FR schedule that provided more food per trial. These data show that monkeys will choose so as to increase the number of reinforcers earned (stock optimizing) even when this preference reduces the rate of reinforcement (all reinforcers divided by session time).     

Does overall reinforcer rate affect discrimination of time‐based contingencies?

Overall reinforcer rate appears to affect choice. The mechanism for such an effect is uncertain, but may relate to reinforcer rate changing the discrimination of the relation between stimuli and reinforcers. We assessed whether a quantitative model based on a stimulus‐control approach could be used to account for the effects of overall reinforcer rate on choice under changing time‐based contingencies. On a two‐key concurrent schedule, the likely availability of a reinforcer reversed when a fixed time had elapsed since the last reinforcer, and the overall reinforcer rate was varied across conditions. Changes in the overall reinforcer rate produced a change in response bias, and some indication of a change in discrimination. These changes in bias and discrimination always occurred quickly, usually within the first session of a condition. The stimulus‐control approach provided an excellent account of the data, suggesting that changes in overall reinforcer rate affect choice because they alter the frequency of reinforcers obtained at different times, or in different stimulus contexts, and thus change the discriminated relation between stimuli and reinforcers. These findings support the notion that temporal and spatial discriminations can be understood in terms of discrimination of reinforcers across time and space.     

ISOLATING BEHAVIORAL MECHANISMS OF INTERTEMPORAL CHOICE: NICOTINE EFFECT ON DELAY DISCOUNTING AND AMOUNT SENSITIVITY

Many drugs of abuse produce changes in impulsive choice, that is, choice for a smaller—sooner reinforcer over a larger—later reinforcer. Because the alternatives differ in both delay and amount, it is not clear whether these drug effects are due to the differences in reinforcer delay or amount. To isolate the effects of delay, we used a titrating delay procedure. In phase 1, 9 rats made discrete choices between variable delays (1 or 19 s, equal probability of each) and a delay to a single food pellet. The computer titrated the delay to a single food pellet until the rats were indifferent between the two options. This indifference delay was used as the starting value for the titrating delay for all future sessions. We next evaluated the acute effects of nicotine (subcutaneous 1.0, 0.3, 0.1, and 0.03 mg/kg) on choice. If nicotine increases delay discounting, it should have increased preference for the variable delay. Instead, nicotine had very little effect on choice. In a second phase, the titrated delay alternative produced three food pellets instead of one, which was again produced by the variable delay (1 s or 19 s) alternative. Under this procedure, nicotine increased preference for the one pellet alternative. Nicotine‐induced changes in impulsive choice are therefore likely due to differences in reinforcer amount rather than differences in reinforcer delay. In addition, it may be necessary to include an amount sensitivity parameter in any mathematical model of choice when the alternatives differ in reinforcer amount.     

Preferences for token exchange‐production schedules: Effects of task difficulty and token‐production schedules

Previous research suggests that accumulated exchange‐production schedules promote increased work completion and are more preferred than distributed exchange‐production schedules. The purpose of this study was to evaluate whether the response effort or token‐production schedules associated with token delivery influenced preferences for exchange‐production schedules. Tokens exchanged under accumulated schedules were more preferred, relative to distributed schedules, when tokens were earned for completing easy tasks. When participants earned tokens for completing difficult tasks, two of three participants preferred accumulated exchange‐production schedules (Experiment 1). Under dense token‐production schedules, accumulated exchange‐production schedules were preferred, but participant's preferences switched to distributed schedules under increasing token‐production (i.e., leaner) schedules (Experiment 2).     

Non‐generalized and generalized conditioned reinforcers: establishment and validation

Conditioned reinforcers, or tokens, are frequently used in clinical settings; however, little applied research has focused on establishing and validating their effectiveness. Following pairing of a novel token with an edible reinforcer in the present study, preference and reinforcer assessments verified that the tokens functioned as conditioned reinforcers. Results indicated that the tokens were established as conditioned reinforcers that matched the reinforcer value of the primary reinforcers with which they had been paired. The effects of motivating operations (MOs) on the effectiveness of these conditioned reinforcers were then evaluated. Conditioned reinforcer effectiveness decreased during satiation conditions. Subsequently it was found that increasing the number of backup reinforcers with which the token was paired resulted in the effectiveness maintaining during satiation conditions. Copyright © 2008 John Wiley & Sons, Ltd.     

Children's choice: Sensitivity to changes in reinforcer density

Two experiments were carried out in which children's sensitivity to changes in reinforcer density (number of reinforcers per session) was measured in a choice paradigm. In Experiment 1, 24 girls (ages 6, 9, and 12 years) performed on concurrent-chain schedules of reinforcement. The initial links were variable-interval 10-s schedules. One terminal link always gave three tokens after 30 s, but the parameters associated with the other were varied. Independent manipulations of reinforcer size (two tokens or four tokens) and prereinforcement delay (25 s or 65 s) led to equal changes in the relative density of tokens that could be earned on the schedules. Subjects at all ages were sensitive to changes in reinforcer density brought about by changes in reinforcer size, whereas only 3 12-year-olds showed sensitivity to the changes brought about by manipulation of prereinforcer delay. In Experiment 2, titration procedures were used to test the extent of this insensitivity to delay in 32 6- and 12-year-old children. In these procedures, a repeated choice of the large reinforcer increased the delay to its delivery, and a repeated choice of the small reinforcer reduced the delay to the delivery of the large reinforcer. Whereas 6-year-old boys and girls tended to maintain a strong preference for the large reinforcer, so increasing the delay to its delivery, 12-year-olds tended to distribute their responses to both alternatives, thus producing a stable level of delay to the large reinforcer. The results from the two experiments support the idea of two stages in the development of adaptive intertemporal choice.     

Risky choice as a function of amount and variance in food supply.

In Experiment 1, 4 rats earned their daily food ration by choosing on a trials basis between a "risky" and a "riskless" lever. The risky lever produced either 15 45-mg food pellets or no pellets, and on average provided five pellets per choice. The riskless lever always produced three pellets. Across conditions, the number of trials per session was varied. Body weight and choice of the risky lever decreased as the number of trials per session decreased, even though body weight could only be defended by increased choice of the risky lever. In Experiment 2, trials per session were fixed, but the number of pellets delivered by the risky and riskless levers was either at the same level as in Experiment 1 or tripled from those levels. Now choice of the risky lever was inversely related to the size of reinforcement and to body weight. The results of these experiments show that risk aversion covaries with the amount of food available in a session and the daily variance in the amount of food earned.     

REINFORCER ACCUMULATION IN A TOKEN-REINFORCEMENT CONTEXT WITH PIGEONS

Four pigeons were exposed to a token-reinforcement procedure with stimulus lights serving as tokens. Responses on one key (the token-production key) produced tokens that could be exchanged for food during an exchange period. Exchange periods could be produced by satisfying a ratio requirement on a second key (the exchange-production key). The exchange-production key was available any time after one token had been produced, permitting up to 12 tokens to accumulate prior to exchange. Token accumulation, measured in terms of both frequency (percent cycles with accumulation) and magnitude (mean number of tokens accumulated), decreased as the token-production ratio increased from 1 to 10 across conditions (with exchange-production ratio held constant), and increased as the exchange-production ratio increased from 1 to 250 across conditions (with token-production ratio held constant). When tokens were removed, accumulation decreased markedly compared to conditions with tokens and the same schedules. These data show that token accumulation is an orderly function of token-production and exchange-production schedules, and they are broadly consistent with a unit-price model based on local and global responses per reinforcer.     

Risk-sensitive choice in humans as a function of an earnings budget

Risky choice in 3 adult humans was investigated across procedural manipulations designed to model energy-budget manipulations conducted with nonhumans. Subjects were presented with repeated choices between a fixed and a variable number of points. An energy budget was simulated by use of an earnings budget, defined as the number of points needed within a block of trials for points to be exchanged for money. During positive earnings-budget conditions, exclusive preference for the fixed option met the earnings requirement. During negative earnings-budget conditions, exclusive preference for the certain option did not meet the earnings requirement, but choice for the variable option met the requirement probabilistically. Choice was generally risk averse (the fixed option was preferred) when the earnings budget was positive and risk prone (the variable option was preferred) when the earnings budget was negative. Furthermore, choice was most risk prone during negative earnings-budget conditions in which the earnings requirement was most stringent. Local choice patterns were also frequently consistent with the predictions of a dynamic optimization model, indicating that choice was simultaneously sensitive to short-term choice contingencies, current point earnings, and the earnings requirement. Overall, these results show that the patterns of risky choice generated by energy-budget variables can also be produced by choice contingencies that do not involve immediate survival, and that risky choice in humans may be similar to that shown in nonhumans when choice is studied under analogous experimental conditions.     

The effects of token reinforcement on attending in a young child with autism

The benefits of token economies have been widely established, however there are very few empirical studies on the effects of token reinforcement on the behavior of young children with autism. The establishment of conditioned reinforcers such as tokens may be important in interventions for children with autism. Token reinforcement was used to increase the attending behavior of a young child with autism during discrete trials instruction for academic and communication skills. A reversal design showed that token reinforcement sustained attending only when the back‐up reinforcer was available and was accessed immediately. The results extend and corroborate findings on token reinforcement reported in other applied settings and are also consistent with findings from basic research on conditioned reinforcement. Copyright © 2006 John Wiley & Sons, Ltd.     

Stimulus Equivalence: Effects Of A Default-response Option On Emergence Of Untrained Stimulus Relations

Human subjects were exposed to a concurrent-chains schedule in which reinforcer amounts, delays, or both were varied in the terminal links, and consummatory responses were required to receive points that were later exchangeable for money. Two independent variable-interval 30-s schedules were in effect during the initial links, and delay periods were defined by fixed-time schedules. In Experiment 1, subjects were exposed to three different pairs of reinforcer amounts and delays, and sensitivity to reinforcer amount and delay was determined based on the generalized matching law. The relative responding (choice) of most subjects was more sensitive to reinforcer amount than to reinforcer delay. In Experiment 2, subjects chose between immediate smaller reinforcers and delayed larger reinforcers in five conditions with and without timeout periods that followed a shorter delay, in which reinforcer amounts and delays were combined to make different predictions based on local reinforcement density (i.e., points per delay) or overall reinforcement density (i.e., points per total time). In most conditions, subjects' choices were qualitatively in accord with the predictions from the overall reinforcement density calculated by the ratio of reinforcer amount and total time. Therefore, the overall reinforcement density appears to influence the preference of humans in the present self-control choice situation.     

Numerical judgments by chimpanzees (Pan troglodytes) in a token economy

We presented four chimpanzees with a series of tasks that involved comparing two token sets or comparing a token set to a quantity of food. Selected tokens could be exchanged for food items on a one-to-one basis. Chimpanzees successfully selected the larger numerical set for comparisons of 1 to 5 items when both sets were visible and when sets were presented through one-by-one addition of tokens into two opaque containers. Two of four chimpanzees used the number of tokens and food items to guide responding in all conditions, rather than relying on token color, size, total amount, or duration of set presentation. These results demonstrate that judgments of simultaneous and sequential sets of stimuli are made by some chimpanzees on the basis of the numerousness of sets rather than other non-numerical dimensions. The tokens were treated as equivalent to food items on the basis of their numerousness, and the chimpanzees maximized reward by choosing the larger number of items in all situations.     

Choice, foraging, and reinforcer duration.

Pigeons were exposed to a foraging schedule characterized by three different states, beginning with a search state in which completion of a variable interval on a white key led to a choice state. In the choice state the subject could, by appropriate responding on a fixed ratio of three, either accept or reject the schedule offered. If the subject accepted the schedule, it entered a handling state in which the appropriate reinforcer amount was presented according to a variable-interval schedule. In Experiment 1 the shorter duration reinforcer was more likely to be accepted the longer the duration of the search state and the shorter the equal durations of the handling states. In Experiment 2 the shorter duration reinforcer was more likely to be accepted the longer the handling time preceding the longer duration reinforcer. All of the results were in qualitative--and some were in quantitative--agreement with those predicted by the delay-reduction hypothesis and the optimal-diet model.     

EFFECTS OF A SIGNALED DELAY TO REINFORCEMENT IN THE PREVIOUS AND UPCOMING RATIOS ON BETWEEN-RATIO PAUSING IN FIXED-RATIO SCHEDULES

Domestic hens responded under multiple fixed‐ratio fixed‐ratio schedules with equal fixed ratios. One component provided immediate reinforcement and the other provided reinforcement after a delay, signaled by the offset of the key light. The components were presented quasirandomly so that all four possible transitions occurred in each session. The delay was varied over 0, 4, 8, 16, and 32 s with fixed‐ratio 5 schedules, and over 0, 8 and 32 s with fixed‐ratio 1, 15 and 40 schedules. Main effects of fixed‐ratio value and delay duration were detected on between‐ratio pauses. Pauses were longer when the multiple‐schedule stimulus correlated with a delayed‐reinforcer component was presented, with the longest pauses occurring at the transition from a component with an immediate reinforcer to one with a delayed reinforcer. Pause durations were shortest during immediate components. Overall, both the presence or absence of a delay in the upcoming component, and the presence or absence of a delay in the preceding component affected pause length, but the upcoming delay had the larger effect. Thus changes in delay had similar effects to past reports of the effects of changes in response force, response requirement, and reinforcer magnitude in multiple fixed‐ratio fixed‐ratio schedules.     

Self-control in male and female rats

Eight male and 8 female Wistar rats were exposed to a discrete-trial procedure in which they chose between the presentation of a small (one pellet) or a large (three pellets) reinforcer. The delay to the small and large reinforcer was 6.0 s in the first condition of Experiment 1. Subjects consistently chose the large reinforcer. When the delay to the small reinforcer was decreased to 0.1 s in the next experimental condition, all subjects continued to choose the large 6.0-s delayed reinforcer. When the contingencies correlated with the two levers were reversed in the next experimental condition, the majority of subjects (5 males and 6 females) still chose the large delayed reinforcer over the small immediately presented reinforcer. The delay to the small reinforcer was maintained at 6.0 s, but the delay to the large reinforcer was varied among 9.0, 15.0, 24.0, and 36.0 s in Experiment 2, in which 4 males and 4 females participated. Most subjects consistently chose the large increasingly delayed reinforcer, although choice for the small 6.0-s delayed reinforcer developed in some females when the large reinforcer was delayed for 24.0 or 36.0 s. These choice patterns were not predicted from a literal application of a model that says choice should favor the alternative correlated with the higher (amount/delay) ratio.     

Token transfer between mother and offspring chimpanzees (Pan troglodytes): mother–offspring interaction in a competitive situation

Chimpanzees can flexibly use tokens in cognitive tasks, but it is still unknown if they can share and/or compete over tokens as they do for food. This study aimed to evaluate the interactions spontaneously occurring between mother and offspring chimpanzees when tokens exchangeable for food were provided. Forty tokens were scattered on the floor in an experimental playroom. Three mother and offspring chimpanzee pairs were tested. Each token was exchangeable for a piece of food in a vending machine installed on the wall of the playroom. In the beginning of the study, both mother and offspring took tokens and exchanged them for pieces of food independently. Later, two offspring started to take more tokens than their mothers. At that time, the offspring whimpered or cried more often than during earlier sessions. This behavior compelled the mothers to abstain from taking tokens. The mothers sometimes shared their tokens with their offspring, or were tolerant of their offspring taking their tokens from their hand. For one pair, the offspring sometimes shared tokens with her mother when her mother begged for the tokens. These results suggest that chimpanzees’ cognitive abilities enable them not only to use tokens, but also to compete for tokens, as they do for food. The results also suggest that token sharing between mother and offspring may be bidirectional and that transfer of tokens mainly occurs as a result of begging, although on some occasion offspring were able to obtain a token directly from his/her mother through tolerated scrounging.