RISKY CHOICE IN PIGEONS: PREFERENCE FOR AMOUNT VARIABILITY USING A TOKEN-REINFORCEMENT SYSTEM

Pigeons were given repeated choices between variable and fixed numbers of token reinforcers (stimulus lamps arrayed above the response keys), with each earned token exchangeable for food. The number of tokens provided by the fixed‐amount option remained constant within blocks of sessions, but varied parametrically across phases, assuming values of 2, 4, 6, or 8 tokens per choice. The number of tokens provided by the variable‐amount option varied between 0 and 12 tokens per choice, arranged according to an exponential or rectangular distribution. In general, the pigeons strongly preferred the variable option when the fixed option provided equal or greater numbers of tokens than the variable amount. Preference for the variable amount decreased only when the alternatives provided widely disparate amounts favoring the fixed amount. When tokens were removed from the experimental context, preference for the variable option was reduced or eliminated, suggesting that the token presentation played a key role in maintaining risk‐prone choice patterns. Choice latencies varied inversely with preferences, suggesting that local analyses may provide useful ancillary measures of reinforcer value. Overall, the results indicate that systematic risk sensitivity can be attained with respect to reinforcer amount, and that tokens may be critical in the development of such preferences.     

REINFORCER ACCUMULATION IN A TOKEN-REINFORCEMENT CONTEXT WITH PIGEONS

Four pigeons were exposed to a token-reinforcement procedure with stimulus lights serving as tokens. Responses on one key (the token-production key) produced tokens that could be exchanged for food during an exchange period. Exchange periods could be produced by satisfying a ratio requirement on a second key (the exchange-production key). The exchange-production key was available any time after one token had been produced, permitting up to 12 tokens to accumulate prior to exchange. Token accumulation, measured in terms of both frequency (percent cycles with accumulation) and magnitude (mean number of tokens accumulated), decreased as the token-production ratio increased from 1 to 10 across conditions (with exchange-production ratio held constant), and increased as the exchange-production ratio increased from 1 to 250 across conditions (with token-production ratio held constant). When tokens were removed, accumulation decreased markedly compared to conditions with tokens and the same schedules. These data show that token accumulation is an orderly function of token-production and exchange-production schedules, and they are broadly consistent with a unit-price model based on local and global responses per reinforcer.     

Determinants of pigeons' choices in token-based self-control procedures

Four pigeons were exposed to a token-based self-control procedure with stimulus lights serving as token reinforcers. Smaller-reinforcer choices produced one token immediately; larger-reinforcer choices produced three tokens following a delay. Each token could be exchanged for 2-s access to food during a signaled exchange period each trial. The main variables of interest were the exchange delays (delays from the choice to the exchange stimulus) and the food delays (also timed from the choice), which were varied separately and together across blocks of sessions. When exchange delays and food delays were shorter following smaller-reinforcer choices, strong preference for the smaller reinforcer was observed. When exchange delays and food delays were equal for both options, strong preference for the larger reinforcer was observed. When food delays were equal for both options but exchange delays were shorter for smaller-reinforcer choices, preference for the larger reinforcer generally was less extreme than under conditions in which both exchange and food delays were equal. When exchange delays were equal for both options but food delays were shorter for smaller-reinforcer choices, preference for the smaller reinforcer generally was less extreme than under conditions in which both exchange and food delays favored smaller-reinforcer choices. On the whole, the results were consistent with prior research on token-based self-control procedures in showing that choices are governed by reinforcer immediacy when exchange and food delays are unequal and by reinforcer amount when exchange and food delays are equal. Further, by decoupling the exchange delays from food delays, the results tentatively support a role for the exchange stimulus as a conditioned reinforcer.     

RISKY CHOICE IN PIGEONS AND HUMANS: A CROSS‐SPECIES COMPARISON

Pigeon and human subjects were given repeated choices between variable and adjusting delays to token reinforcement that titrated in relation to a subject's recent choice patterns. Indifference curves were generated under two different procedures: immediate exchange, in which a token earned during each trial was exchanged immediately for access to the terminal reinforcer (food for pigeons, video clips for humans), and delayed exchange, in which tokens accumulated and were exchanged after 11 trials. The former was designed as an analogue of procedures typically used with nonhuman subjects, the latter as an analogue to procedures typically used with human participants. Under both procedure types, different variable‐delay schedules were manipulated systematically across conditions in ways that altered the reinforcer immediacy of the risky option. Under immediate‐exchange conditions, both humans and pigeons consistently preferred the variable delay, and indifference points were generally ordered in relation to relative reinforcer immediacies. Such risk sensitivity was greatly reduced under delayed‐exchange conditions. Choice and trial‐initiation response latencies varied directly with indifference points, suggesting that local analyses may provide useful ancillary measures of reinforcer value. On the whole, the results indicate that modifying procedural features brings choices of pigeons and humans into better accord, and that human—nonhuman differences on risky choice procedures reported in the literature may be at least partly a product of procedural differences.     

Preferences for token exchange‐production schedules: Effects of task difficulty and token‐production schedules

Previous research suggests that accumulated exchange‐production schedules promote increased work completion and are more preferred than distributed exchange‐production schedules. The purpose of this study was to evaluate whether the response effort or token‐production schedules associated with token delivery influenced preferences for exchange‐production schedules. Tokens exchanged under accumulated schedules were more preferred, relative to distributed schedules, when tokens were earned for completing easy tasks. When participants earned tokens for completing difficult tasks, two of three participants preferred accumulated exchange‐production schedules (Experiment 1). Under dense token‐production schedules, accumulated exchange‐production schedules were preferred, but participant's preferences switched to distributed schedules under increasing token‐production (i.e., leaner) schedules (Experiment 2).     

Second-order schedules of token reinforcement with pigeons: effects of fixed- and variable-ratio exchange schedules

Pigeons' key pecks produced food under second-order schedules of token reinforcement, with light-emitting diodes serving as token reinforcers. In Experiment 1, tokens were earned according to a fixed-ratio 50 schedule and were exchanged for food according to either fixed-ratio or variable-ratio exchange schedules, with schedule type varied across conditions. In Experiment 2, schedule type was varied within sessions using a multiple schedule. In one component, tokens were earned according to a fixed-ratio 50 schedule and exchanged according to a variable-ratio schedule. In the other component, tokens were earned according to a variable-ratio 50 schedule and exchanged according to a fixed-ratio schedule. In both experiments, the number of responses per exchange was varied parametrically across conditions, ranging from 50 to 400 responses. Response rates decreased systematically with increases in the fixed-ratio exchange schedules, but were much less affected by changes in the variable-ratio exchange schedules. Response rates were consistently higher under variable-ratio exchange schedules than tinder comparable fixed-ratio exchange schedules, especially at higher exchange ratios. These response-rate differences were due both to greater pre-ratio pausing and to lower local rates tinder the fixed-ratio exchange schedules. Local response rates increased with proximity to food under the higher fixed-ratio exchange schedules, indicative of discriminative control by the tokens.     

Second-order schedules of token reinforcement with pigeons: implications for unit price

Four pigeons were exposed to second-order schedules of token reinforcement, with stimulus lights serving as token reinforcers. Tokens were earned according to a fixed-ratio (token-production) schedule, with the opportunity to exchange tokens for food (exchange period) occurring after a fixed number had been produced (exchange-production ratio). The token-production and exchange-production ratios were manipulated systematically across conditions. Response rates varied inversely with the token-production ratio at each exchange-production ratio. Response rates also varied inversely with the exchange-production ratio at each token-production ratio, particularly at the higher token-production ratios. At higher token-production and exchange-production ratios, response rates increased in token-production segments closer to exchange periods and food. Some conditions were conducted in a closed economy, in which the pigeons earned all their daily ration of food within the session. Relative to comparable open-economy conditions, response rates in the closed economy were less affected by changes in token-production ratio, resulting in higher levels of food intake and body weight. Some of the results are consistent with the economic concept of unit price, a cost-benefit ratio comprised of responses per unit of food delivery, but most are well accounted for by a consideration of the number of responses required to produce exchange periods, without regard to the amount of reinforcement available during those exchange periods.     

Parametric analysis of delayed primary and conditioned reinforcers

We examined the effects of delayed reinforcement on the responding of individuals with intellectual disabilities. Three conditions were evaluated: (a) food reinforcement, (b) token reinforcement with a postsession exchange opportunity, and (c) token reinforcement with a posttrial exchange opportunity. Within each condition, we assessed responding given (a) a no‐reinforcement baseline, (b) immediate reinforcement, and (c) delayed reinforcement, in which responses produced a reinforcer after 1 of 6 delays. Results suggest that delayed food produced greater response persistence than did delayed tokens.     

Using token reinforcement to increase walking for adults with intellectual disabilities

The purpose of this study was to evaluate the effectiveness of token reinforcement, using an ABAB reversal design, for increasing distance walked for adults with mild to moderate intellectual disabilities at an adult day‐training center. Five participants earned tokens for walking 50‐m laps and exchanged tokens for back‐up reinforcers that had been identified through preference assessments. Token reinforcement resulted in a substantial increase from baseline in laps walked for 4 participants.     

Second-order schedules of token reinforcement: comparisons of performance under fixed-ratio and variable-ratio exchange schedules

Rats' lever pressing produced tokens according to a 20-response fixed-ratio schedule. Sequences of token schedules were reinforced under a second-order schedule by presentation of periods when tokens could be exchanged for food pellets. When the exchange period schedule was a six-response fixed ratio, patterns of completing the component token schedules were bivalued, with relatively long and frequent pauses marking the initiation of each new sequence. Altering the exchange period schedule to a six-response variable ratio resulted in sharp reductions in the frequency and duration of these initial pauses, and increases in overall rates of lever pressing. These results are comparable to those ordinarily obtained under simple fixed-ratio and variable-ratio schedules.     

Token reinforcement, choice, and self-control in pigeons.

Pigeons were exposed to self-control procedures that involved illumination of light-emitting diodes (LEDs) as a form of token reinforcement. In a discrete-trials arrangement, subjects chose between one and three LEDs; each LED was exchangeable for 2-s access to food during distinct posttrial exchange periods. In Experiment 1, subjects generally preferred the immediate presentation of a single LED over the delayed presentation of three LEDs, but differences in the delay to the exchange period between the two options prevented a clear assessment of the relative influence of LED delay and exchange-period delay as determinants of choice. In Experiment 2, in which delays to the exchange period from either alternative were equal in most conditions, all subjects preferred the delayed three LEDs more often than in Experiment-1. In Experiment 3, subjects preferred the option that resulted in a greater amount of food more often if the choices also produced LEDs than if they did not. In Experiment 4, preference for the delayed three LEDs was obtained when delays to the exchange period were equal, but reversed in favor of an immediate single LED when the latter choice also resulted in quicker access to exchange periods. The overall pattern of results suggests that (a) delay to the exchange period is a more critical determinant of choice than is delay to token presentation; (b) tokens may function as conditioned reinforcers, although their discriminative properties may be responsible for the self-control that occurs under token reinforcer arrangements; and (c) previously reported differences in the self-control choices of humans and pigeons may have resulted at least in part from the procedural conventions of using token reinforcers with human subjects and food reinforcers with pigeon subjects.     

Unit price and choice in a token-reinforcement context

Pigeons were exposed to multiple and concurrent second-order schedules of token reinforcement, with stimulus lights serving as token reinforcers. Tokens were produced and exchanged for food according to various fixed-ratio schedules, yielding equal and unequal unit prices (responses per unit food delivery). On one schedule (termed the standard schedule), the unit price was held constant across conditions. On a second schedule (the alternative schedule), the unit price was either the same or different from the standard. Under conditions with unequal unit prices, near-exclusive preference for the lower unit price was obtained. Under conditions with equal unit prices, the direction and degree of preference depended on ratio size (number of responses per exchange period). When this ratio differed, strong preferences for the smaller ratio were observed. When this ratio was equal, preferences were nearer indifference. Response rates on the multiple schedule were generally consistent with the preference data in showing sensitivity to ratio size. Results are discussed in terms of a unit-price model that includes handling and reinforcer immediacy as additional costs. On the whole, results show that preferences were determined primarily by delay to the exchange period.     

Stock optimizing in choice when a token deposit is the operant.

Each of 2 monkeys typically earned their daily food ration by depositing tokens in one of two slots. Tokens deposited in one slot dropped into a bin where they were kept (token kept). Deposits to a second slot dropped into a bin where they could be obtained again (token returned). In Experiment 1, a fixed-ratio (FR) 5 schedule that provided two food pellets was associated with each slot. Both monkeys preferred the token-returned slot. In Experiment 2, both subjects chose between unequal FR schedules with the token-returned slot always associated with the leaner schedule. When the FRs were 2 versus 3 and 2 versus 6, preferences were maintained for the token-returned slot; however, when the ratios were 2 versus 12, preference shifted to the token-kept slot. In Experiment 3, both monkeys chose between equal-valued concurrent variable-interval variable-interval schedules. Both monkeys preferred the slot that returned tokens. In Experiment 4, both monkeys chose between FRs that typically differed in size by a factor of 10. Both monkeys preferred the FR schedule that provided more food per trial. These data show that monkeys will choose so as to increase the number of reinforcers earned (stock optimizing) even when this preference reduces the rate of reinforcement (all reinforcers divided by session time).     

Token transfer between mother and offspring chimpanzees (Pan troglodytes): mother–offspring interaction in a competitive situation

Chimpanzees can flexibly use tokens in cognitive tasks, but it is still unknown if they can share and/or compete over tokens as they do for food. This study aimed to evaluate the interactions spontaneously occurring between mother and offspring chimpanzees when tokens exchangeable for food were provided. Forty tokens were scattered on the floor in an experimental playroom. Three mother and offspring chimpanzee pairs were tested. Each token was exchangeable for a piece of food in a vending machine installed on the wall of the playroom. In the beginning of the study, both mother and offspring took tokens and exchanged them for pieces of food independently. Later, two offspring started to take more tokens than their mothers. At that time, the offspring whimpered or cried more often than during earlier sessions. This behavior compelled the mothers to abstain from taking tokens. The mothers sometimes shared their tokens with their offspring, or were tolerant of their offspring taking their tokens from their hand. For one pair, the offspring sometimes shared tokens with her mother when her mother begged for the tokens. These results suggest that chimpanzees’ cognitive abilities enable them not only to use tokens, but also to compete for tokens, as they do for food. The results also suggest that token sharing between mother and offspring may be bidirectional and that transfer of tokens mainly occurs as a result of begging, although on some occasion offspring were able to obtain a token directly from his/her mother through tolerated scrounging.     

A TOKEN SYSTEM FOR A CLASS OF UNDERACHIEVING HYPERACTIVE CHILDREN

Using a BAB design, a token system requiring cooperative interaction was used to change the reading and vocabulary performance of an 18-member class of third-grade hyperactive boys. Four different colored tokens, which could be exchanged for 15 minutes of play on electrovideo games, were earned by successful completion of two tasks that involved learning to read and to use new vocabulary words in sentences, and two tasks in which the student served as a proctor to a student who had not yet completed those tasks. The mean number of tasks completed during the intervention periods rose to over nine times the number completed during reversal. Additionally, the average completion rate for the school district's standardized weekly reading level examinations rose from four to eight fold during the token conditions. All 18 students responded to the token program by increasing their academic performance.     

Assessing the value of choice in a token system

Responding of 4 children was assessed under conditions in which (a) no programmed contingencies were arranged for target behavior, (b) responding produced tokens that could be exchanged for a single highly preferred edible item, and (c) responding produced a token that could be exchanged for a variety of preferred edible items. After assessing the effects of these contingencies, the preferences of 3 participants were assessed using a concurrent-chains schedule. Preference for the opportunity to choose from the same or qualitatively different edible items varied across participants, and findings were generally consistent with those of Tiger, Hanley, and Hernandez (2006).     

Paired-associate learning by normal children and retardates with relevant redundant compound stimuli

During daily sessions four first-grade boys each played a two-choice marble dropping game. When one colored light was presented a plain token was dispensed as soon as S dropped a marble, and when another light was on a token with a hole in its center was delivered. During choice trials that were interspersed among single-color trials S had to choose one of the two different colored holes. At the end of each session S traded tokens with holes for pennies and then turned in the remaining plain tokens which were not exchanged for money. After S had developed a preference for the response that earned the tokens with holes (tokens that were backed up with pennies) E instructed S to try to get as many plain tokens (unexchangeable tokens) as he could. All Ss immediately switched to the response that earned unexchangeable tokens but as soon as E left they returned to choosing the backed up response. When the adult reentered, Ss again chose to work for unexchangeable tokens. Two different Es (male and female) served and the instructions of each were effective in controlling responding as long as the adult was present, except for one S who did not respond to one E, but did respond as described for the other.     

Second-order schedules of token reinforcement: effects of varying the schedule of food presentation

In the initial link of a complex schedule, one discriminative stimulus was presented and lever pressing produced tokens on fixed-ratio schedules. In the terminal link, signalled by a second discriminative stimulus, deposits of the tokens produced food. With two rats, the terminal link was presented after each sixth component schedule of token reinforcement was completed. With the other two rats, the terminal link was presented following the first component schedule completed after a fixed interval. During the terminal link, each token deposit initially produced food. The schedule of food presentation was subsequently increased such that an increasing number of token deposits in the terminal link was required for each food presentation. Rates of lever pressing in the initial link were inversely related to the schedule of food presentation in the terminal link. These results are similar to those of experiments that have varied schedules of food presentation in chained schedules. Rates and patterns of responding controlled throughout the initial link were more similar to those ordinarily controlled by second-order brief-stimulus schedules than to those controlled by comparable extended chained schedules.     

Numerical judgments by chimpanzees (Pan troglodytes) in a token economy

We presented four chimpanzees with a series of tasks that involved comparing two token sets or comparing a token set to a quantity of food. Selected tokens could be exchanged for food items on a one-to-one basis. Chimpanzees successfully selected the larger numerical set for comparisons of 1 to 5 items when both sets were visible and when sets were presented through one-by-one addition of tokens into two opaque containers. Two of four chimpanzees used the number of tokens and food items to guide responding in all conditions, rather than relying on token color, size, total amount, or duration of set presentation. These results demonstrate that judgments of simultaneous and sequential sets of stimuli are made by some chimpanzees on the basis of the numerousness of sets rather than other non-numerical dimensions. The tokens were treated as equivalent to food items on the basis of their numerousness, and the chimpanzees maximized reward by choosing the larger number of items in all situations.