Second-order schedules of token reinforcement with pigeons: effects of fixed- and variable-ratio exchange schedules

Pigeons' key pecks produced food under second-order schedules of token reinforcement, with light-emitting diodes serving as token reinforcers. In Experiment 1, tokens were earned according to a fixed-ratio 50 schedule and were exchanged for food according to either fixed-ratio or variable-ratio exchange schedules, with schedule type varied across conditions. In Experiment 2, schedule type was varied within sessions using a multiple schedule. In one component, tokens were earned according to a fixed-ratio 50 schedule and exchanged according to a variable-ratio schedule. In the other component, tokens were earned according to a variable-ratio 50 schedule and exchanged according to a fixed-ratio schedule. In both experiments, the number of responses per exchange was varied parametrically across conditions, ranging from 50 to 400 responses. Response rates decreased systematically with increases in the fixed-ratio exchange schedules, but were much less affected by changes in the variable-ratio exchange schedules. Response rates were consistently higher under variable-ratio exchange schedules than tinder comparable fixed-ratio exchange schedules, especially at higher exchange ratios. These response-rate differences were due both to greater pre-ratio pausing and to lower local rates tinder the fixed-ratio exchange schedules. Local response rates increased with proximity to food under the higher fixed-ratio exchange schedules, indicative of discriminative control by the tokens.     

Substitution effects in a generalized token economy with pigeons

Pigeons made repeated choices between earning and exchanging reinforcer‐specific tokens (green tokens exchangeable for food, red tokens exchangeable for water) and reinforcer‐general tokens (white tokens exchangeable for food or water) in a closed token economy. Food and green food tokens could be earned on one panel; water and red water tokens could be earned on a second panel; white generalized tokens could be earned on either panel. Responses on one key produced tokens according to a fixed‐ratio schedule, whereas responses on a second key produced exchange periods, during which all previously earned tokens could be exchanged for the appropriate commodity. Most conditions were conducted in a closed economy, and pigeons distributed their token allocation in ways that permitted food and water consumption. When the price of all tokens was equal and low, most pigeons preferred the generalized tokens. When token‐production prices were manipulated, pigeons reduced production of the tokens that increased in price while increasing production of the generalized tokens that remained at a fixed price. The latter is consistent with a substitution effect: Generalized tokens increased and were exchanged for the more expensive reinforcer. When food and water were made freely available outside the session, token production and exchange was sharply reduced but was not eliminated, even in conditions when it no longer produced tokens. The results join with other recent data in showing sustained generalized functions of token reinforcers, and demonstrate the utility of token‐economic methods for assessing demand for and substitution among multiple commodities in a laboratory context.     

Stock optimizing in choice when a token deposit is the operant.

Each of 2 monkeys typically earned their daily food ration by depositing tokens in one of two slots. Tokens deposited in one slot dropped into a bin where they were kept (token kept). Deposits to a second slot dropped into a bin where they could be obtained again (token returned). In Experiment 1, a fixed-ratio (FR) 5 schedule that provided two food pellets was associated with each slot. Both monkeys preferred the token-returned slot. In Experiment 2, both subjects chose between unequal FR schedules with the token-returned slot always associated with the leaner schedule. When the FRs were 2 versus 3 and 2 versus 6, preferences were maintained for the token-returned slot; however, when the ratios were 2 versus 12, preference shifted to the token-kept slot. In Experiment 3, both monkeys chose between equal-valued concurrent variable-interval variable-interval schedules. Both monkeys preferred the slot that returned tokens. In Experiment 4, both monkeys chose between FRs that typically differed in size by a factor of 10. Both monkeys preferred the FR schedule that provided more food per trial. These data show that monkeys will choose so as to increase the number of reinforcers earned (stock optimizing) even when this preference reduces the rate of reinforcement (all reinforcers divided by session time).     

Second-order schedules of token reinforcement: comparisons of performance under fixed-ratio and variable-ratio exchange schedules

Rats' lever pressing produced tokens according to a 20-response fixed-ratio schedule. Sequences of token schedules were reinforced under a second-order schedule by presentation of periods when tokens could be exchanged for food pellets. When the exchange period schedule was a six-response fixed ratio, patterns of completing the component token schedules were bivalued, with relatively long and frequent pauses marking the initiation of each new sequence. Altering the exchange period schedule to a six-response variable ratio resulted in sharp reductions in the frequency and duration of these initial pauses, and increases in overall rates of lever pressing. These results are comparable to those ordinarily obtained under simple fixed-ratio and variable-ratio schedules.     

RISKY CHOICE IN PIGEONS AND HUMANS: A CROSS‐SPECIES COMPARISON

Pigeon and human subjects were given repeated choices between variable and adjusting delays to token reinforcement that titrated in relation to a subject's recent choice patterns. Indifference curves were generated under two different procedures: immediate exchange, in which a token earned during each trial was exchanged immediately for access to the terminal reinforcer (food for pigeons, video clips for humans), and delayed exchange, in which tokens accumulated and were exchanged after 11 trials. The former was designed as an analogue of procedures typically used with nonhuman subjects, the latter as an analogue to procedures typically used with human participants. Under both procedure types, different variable‐delay schedules were manipulated systematically across conditions in ways that altered the reinforcer immediacy of the risky option. Under immediate‐exchange conditions, both humans and pigeons consistently preferred the variable delay, and indifference points were generally ordered in relation to relative reinforcer immediacies. Such risk sensitivity was greatly reduced under delayed‐exchange conditions. Choice and trial‐initiation response latencies varied directly with indifference points, suggesting that local analyses may provide useful ancillary measures of reinforcer value. On the whole, the results indicate that modifying procedural features brings choices of pigeons and humans into better accord, and that human—nonhuman differences on risky choice procedures reported in the literature may be at least partly a product of procedural differences.     

Exchange schedules in token economies: A preliminary investigation of second‐order schedule effects

Basic research shows that token‐production and exchange‐production schedules in token economies affect each other as second‐order schedules (i.e., the exchange‐production schedule's requirements affect responding toward the token‐production schedule). This relationship has not been investigated with children in academic settings despite the widespread use of token economies in this context. This study compared the effects of fixed‐ratio (FR) and variable‐ratio (VR) exchange‐production schedules of equal ratios (2, 5, and 10) on responding toward an FR 1 token‐production schedule with a child diagnosed with autism. A concurrent chains assessment was also conducted to assess the participant's relative preference for FR and VR exchange‐production schedule arrangements within her typical discrete trial training. Results showed no difference in response rate between the two schedule types. However, the concurrent chains assessment revealed an exclusive preference for the VR arrangement.     

Second-order schedules of token reinforcement with pigeons: implications for unit price

Four pigeons were exposed to second-order schedules of token reinforcement, with stimulus lights serving as token reinforcers. Tokens were earned according to a fixed-ratio (token-production) schedule, with the opportunity to exchange tokens for food (exchange period) occurring after a fixed number had been produced (exchange-production ratio). The token-production and exchange-production ratios were manipulated systematically across conditions. Response rates varied inversely with the token-production ratio at each exchange-production ratio. Response rates also varied inversely with the exchange-production ratio at each token-production ratio, particularly at the higher token-production ratios. At higher token-production and exchange-production ratios, response rates increased in token-production segments closer to exchange periods and food. Some conditions were conducted in a closed economy, in which the pigeons earned all their daily ration of food within the session. Relative to comparable open-economy conditions, response rates in the closed economy were less affected by changes in token-production ratio, resulting in higher levels of food intake and body weight. Some of the results are consistent with the economic concept of unit price, a cost-benefit ratio comprised of responses per unit of food delivery, but most are well accounted for by a consideration of the number of responses required to produce exchange periods, without regard to the amount of reinforcement available during those exchange periods.     

RISKY CHOICE IN PIGEONS: PREFERENCE FOR AMOUNT VARIABILITY USING A TOKEN-REINFORCEMENT SYSTEM

Pigeons were given repeated choices between variable and fixed numbers of token reinforcers (stimulus lamps arrayed above the response keys), with each earned token exchangeable for food. The number of tokens provided by the fixed‐amount option remained constant within blocks of sessions, but varied parametrically across phases, assuming values of 2, 4, 6, or 8 tokens per choice. The number of tokens provided by the variable‐amount option varied between 0 and 12 tokens per choice, arranged according to an exponential or rectangular distribution. In general, the pigeons strongly preferred the variable option when the fixed option provided equal or greater numbers of tokens than the variable amount. Preference for the variable amount decreased only when the alternatives provided widely disparate amounts favoring the fixed amount. When tokens were removed from the experimental context, preference for the variable option was reduced or eliminated, suggesting that the token presentation played a key role in maintaining risk‐prone choice patterns. Choice latencies varied inversely with preferences, suggesting that local analyses may provide useful ancillary measures of reinforcer value. Overall, the results indicate that systematic risk sensitivity can be attained with respect to reinforcer amount, and that tokens may be critical in the development of such preferences.     

REINFORCER ACCUMULATION IN A TOKEN-REINFORCEMENT CONTEXT WITH PIGEONS

Four pigeons were exposed to a token-reinforcement procedure with stimulus lights serving as tokens. Responses on one key (the token-production key) produced tokens that could be exchanged for food during an exchange period. Exchange periods could be produced by satisfying a ratio requirement on a second key (the exchange-production key). The exchange-production key was available any time after one token had been produced, permitting up to 12 tokens to accumulate prior to exchange. Token accumulation, measured in terms of both frequency (percent cycles with accumulation) and magnitude (mean number of tokens accumulated), decreased as the token-production ratio increased from 1 to 10 across conditions (with exchange-production ratio held constant), and increased as the exchange-production ratio increased from 1 to 250 across conditions (with token-production ratio held constant). When tokens were removed, accumulation decreased markedly compared to conditions with tokens and the same schedules. These data show that token accumulation is an orderly function of token-production and exchange-production schedules, and they are broadly consistent with a unit-price model based on local and global responses per reinforcer.     

Token reinforcement, choice, and self-control in pigeons.

Pigeons were exposed to self-control procedures that involved illumination of light-emitting diodes (LEDs) as a form of token reinforcement. In a discrete-trials arrangement, subjects chose between one and three LEDs; each LED was exchangeable for 2-s access to food during distinct posttrial exchange periods. In Experiment 1, subjects generally preferred the immediate presentation of a single LED over the delayed presentation of three LEDs, but differences in the delay to the exchange period between the two options prevented a clear assessment of the relative influence of LED delay and exchange-period delay as determinants of choice. In Experiment 2, in which delays to the exchange period from either alternative were equal in most conditions, all subjects preferred the delayed three LEDs more often than in Experiment-1. In Experiment 3, subjects preferred the option that resulted in a greater amount of food more often if the choices also produced LEDs than if they did not. In Experiment 4, preference for the delayed three LEDs was obtained when delays to the exchange period were equal, but reversed in favor of an immediate single LED when the latter choice also resulted in quicker access to exchange periods. The overall pattern of results suggests that (a) delay to the exchange period is a more critical determinant of choice than is delay to token presentation; (b) tokens may function as conditioned reinforcers, although their discriminative properties may be responsible for the self-control that occurs under token reinforcer arrangements; and (c) previously reported differences in the self-control choices of humans and pigeons may have resulted at least in part from the procedural conventions of using token reinforcers with human subjects and food reinforcers with pigeon subjects.     

The role of signals in two variations of differential‐reinforcement‐of‐low‐rate procedures

Differential‐reinforcement‐of‐low‐rate (DRL) schedules are used to decrease the overall rate of, but not eliminate, a target response. Two variations of DRL, spaced‐responding and full‐session, exist. Preliminary comparative analyses suggest that the two schedules function differently when unsignaled. We compared response rates under these two DRL variations with and without signals. In Experiment 1, five preschool students played a game in which points were earned under DRL schedules. In some sessions, a stimulus signaled when responses would be reinforced (S+) or not reinforced (S‐). In others, only an S‐ was present. Signals (S+/S‐) facilitated and maintained responding in both types of DRL schedules. In Experiment 2, we modified the signals with five different preschoolers. Instead of an S‐ only, we did not present any signals. Elimination and high variability of the target response were observed with the S‐ only and absence of S+/S‐, respectively. Signaled DRL schedules are recommended for application.     

Stock optimizing: maximizing reinforcers per session on a variable-interval schedule.

In Experiment 1, 2 monkeys earned their daily food ration by pressing a key that delivered food according to a variable-interval 3-min schedule. In Phases 1 and 4, sessions ended after 3 hr. In Phases 2 and 3, sessions ended after a fixed number of responses that reduced food intake and body weights from levels during Phases 1 and 4. Monkeys responded at higher rates and emitted more responses per food delivery when the food earned in a session was reduced. In Experiment 2, monkeys earned their daily food ration by depositing tokens into the response panel. Deposits delivered food according to a variable-interval 3-min schedule. When the token supply was unlimited (Phases 1, 3, and 5), sessions ended after 3 hr. In Phases 2 and 4, sessions ended after 150 tokens were deposited, resulting in a decrease in food intake and body weight. Both monkeys responded at lower rates and emitted fewer responses per food delivery when the food earned in a session was reduced. Experiment 1's results are consistent with a strength account, according to which the phases that reduced body weights increased food's value and therefore increased subjects' response rates. The results of Experiment 2 are consistent with an optimizing strategy, because lowering response rates when food is restricted defends body weight on variable-interval schedules. These contrasting results may be attributed to the discriminability of the contingency between response number and the end of a session being greater in Experiment 2 than in Experiment 1. In consequence, subjects lowered their response rates in order to increase the number of reinforcers per session (stock optimizing).     

Cooperative responding in rats maintained by fixed‐ and variable‐ratio schedules

The present study investigated the effects of fixed‐ratio (FR) and variable‐ratio (VR) reinforcement schedules on patterns of cooperative responding in pairs of rats. Experiment 1 arranged FR 1, FR 10, and VR 10 schedules to establish cooperative responding (water delivery depended on the joint responding of two rats). Cooperative response rates and proportions were higher under intermittent schedules than under continuous reinforcement. The FR 10 schedule generated a break‐and‐run pattern, whereas the VR 10 schedule generated a relatively high and constant rate pattern. Experiment 2 evaluated the effects of parametric manipulations of FR and VR schedules on cooperative responding. Rates and proportions of cooperative responding generally increased between ratio sizes of 1 and 5 but showed no consistent trend as the ratio increased from 5 to 10. Experiment 3 contrasted cooperative responding between an FR6 schedule and a yoked control schedule. Coordinated behavior occurred at a higher rate under the former schedule. The present study showed that external consequences and the schedules under which the delivery of these consequences are based, select patterns of coordinated behavior.     

Assessing the value of choice in a token system

Responding of 4 children was assessed under conditions in which (a) no programmed contingencies were arranged for target behavior, (b) responding produced tokens that could be exchanged for a single highly preferred edible item, and (c) responding produced a token that could be exchanged for a variety of preferred edible items. After assessing the effects of these contingencies, the preferences of 3 participants were assessed using a concurrent-chains schedule. Preference for the opportunity to choose from the same or qualitatively different edible items varied across participants, and findings were generally consistent with those of Tiger, Hanley, and Hernandez (2006).     

Using token reinforcement to increase walking for adults with intellectual disabilities

The purpose of this study was to evaluate the effectiveness of token reinforcement, using an ABAB reversal design, for increasing distance walked for adults with mild to moderate intellectual disabilities at an adult day‐training center. Five participants earned tokens for walking 50‐m laps and exchanged tokens for back‐up reinforcers that had been identified through preference assessments. Token reinforcement resulted in a substantial increase from baseline in laps walked for 4 participants.     

A comparison of token and tandem schedules of reinforcement on response patterns for adolescents with autism

Token schedules of reinforcement are ubiquitous in clinical settings, yet little research has thoroughly evaluated the effects of token schedules on responding in clinical settings. Basic research has shown token schedules of reinforcement produce lower response rates and larger pre‐ratio pauses compared to tandem schedules. The purpose of the current study was to determine whether the same effects are produced with adolescents with autism or related disorders. We examined response patterns under otherwise identical FR token and FR tandem schedules. Tokens suppressed responding for one participant only under high schedule values and for a second participant under common clinical schedule values; no difference in responding occurred between token and tandem schedules for two participants. These results support the systematic evaluation of token schedules of reinforcement in clinical settings. Additional applied research is needed on token schedules to further our understanding of the underlying mechanisms that contribute to the overall effectiveness of token economies.     

Second-order schedules of token reinforcement: effects of varying the schedule of food presentation

In the initial link of a complex schedule, one discriminative stimulus was presented and lever pressing produced tokens on fixed-ratio schedules. In the terminal link, signalled by a second discriminative stimulus, deposits of the tokens produced food. With two rats, the terminal link was presented after each sixth component schedule of token reinforcement was completed. With the other two rats, the terminal link was presented following the first component schedule completed after a fixed interval. During the terminal link, each token deposit initially produced food. The schedule of food presentation was subsequently increased such that an increasing number of token deposits in the terminal link was required for each food presentation. Rates of lever pressing in the initial link were inversely related to the schedule of food presentation in the terminal link. These results are similar to those of experiments that have varied schedules of food presentation in chained schedules. Rates and patterns of responding controlled throughout the initial link were more similar to those ordinarily controlled by second-order brief-stimulus schedules than to those controlled by comparable extended chained schedules.     

The choices and preferences of nursery school children

After a period of equal reinforcement for choices of any job revealed which of 10 jobs each of 24 preschool children preferred and did not prefer, token payment became dependent on particular choices. Some children received tokens only for choosing previously non-preferred jobs, others for choosing previously preferred jobs. When tokens depended on choosing the nonpreferred jobs, those came to be preferred. When tokens depended on choosing the preferred jobs, the preferences were strengthened. The effects were replicated both within and between subjects, except in the case of one boy who consistently avoided token pay.     

Token mediated tool exchange between tufted capuchin monkeys (<Emphasis Type="Italic">Cebus apella</Emphasis>)

Three experiments were conducted to test whether a pair of tufted capuchin monkeys (Cebus apella) could generalize their ability to exchange tokens and tool objects with a human experimenter to similar exchanges with a conspecific partner. Monkeys were tested in side-by-side enclosures, one enclosure containing a tool-use apparatus and one or more token(s), and the other enclosure containing one or more tool object(s). The monkeys willingly transferred tokens and tools to a conspecific with little practice. Following a small amount of training, we also found that the monkeys would select situation-appropriate tokens to exchange for specific tools, but did not select appropriate tool objects in response to another monkey’s token transfers. Implications regarding role reversal are discussed.     

Evaluation of absolute and relative reinforcer value using progressive-ratio schedules

We evaluated behavior exhibited by individuals with developmental disabilities using progressive-ratio (PR) schedules. High- and low-preference stimuli were determined based on the results of a paired-stimulus preference assessment and were evaluated in subsequent reinforcer and PR assessments using concurrent and single schedules of presentation. In Experiment 1, results showed that for 2 of 3 participants, stimuli determined to be low-preference functioned as reinforcers when evaluated independent of high-preference stimuli. Further, the results from Experiment 2 showed that low-preference stimuli also functioned as reinforcers under gradually increasing PR requirements. Results suggest that for cases in which a high-preference stimulus is unavailable or impractical, the contingent delivery of relatively less preferred stimuli may maintain appropriate behavior, even as schedule requirements increase.