Second-order schedules of token reinforcement with pigeons: implications for unit price

Four pigeons were exposed to second-order schedules of token reinforcement, with stimulus lights serving as token reinforcers. Tokens were earned according to a fixed-ratio (token-production) schedule, with the opportunity to exchange tokens for food (exchange period) occurring after a fixed number had been produced (exchange-production ratio). The token-production and exchange-production ratios were manipulated systematically across conditions. Response rates varied inversely with the token-production ratio at each exchange-production ratio. Response rates also varied inversely with the exchange-production ratio at each token-production ratio, particularly at the higher token-production ratios. At higher token-production and exchange-production ratios, response rates increased in token-production segments closer to exchange periods and food. Some conditions were conducted in a closed economy, in which the pigeons earned all their daily ration of food within the session. Relative to comparable open-economy conditions, response rates in the closed economy were less affected by changes in token-production ratio, resulting in higher levels of food intake and body weight. Some of the results are consistent with the economic concept of unit price, a cost-benefit ratio comprised of responses per unit of food delivery, but most are well accounted for by a consideration of the number of responses required to produce exchange periods, without regard to the amount of reinforcement available during those exchange periods.     

Rate Dependence and Token Reinforcement? A Preliminary Analysis

The Psychological Record - Changes in token exchange-production schedules may alter behavior when token-production schedules and token-exchange schedules are held constant. Although research...     

Second-order schedules of token reinforcement with pigeons: effects of fixed- and variable-ratio exchange schedules

Pigeons' key pecks produced food under second-order schedules of token reinforcement, with light-emitting diodes serving as token reinforcers. In Experiment 1, tokens were earned according to a fixed-ratio 50 schedule and were exchanged for food according to either fixed-ratio or variable-ratio exchange schedules, with schedule type varied across conditions. In Experiment 2, schedule type was varied within sessions using a multiple schedule. In one component, tokens were earned according to a fixed-ratio 50 schedule and exchanged according to a variable-ratio schedule. In the other component, tokens were earned according to a variable-ratio 50 schedule and exchanged according to a fixed-ratio schedule. In both experiments, the number of responses per exchange was varied parametrically across conditions, ranging from 50 to 400 responses. Response rates decreased systematically with increases in the fixed-ratio exchange schedules, but were much less affected by changes in the variable-ratio exchange schedules. Response rates were consistently higher under variable-ratio exchange schedules than tinder comparable fixed-ratio exchange schedules, especially at higher exchange ratios. These response-rate differences were due both to greater pre-ratio pausing and to lower local rates tinder the fixed-ratio exchange schedules. Local response rates increased with proximity to food under the higher fixed-ratio exchange schedules, indicative of discriminative control by the tokens.     

Substitution effects in a generalized token economy with pigeons

Pigeons made repeated choices between earning and exchanging reinforcer‐specific tokens (green tokens exchangeable for food, red tokens exchangeable for water) and reinforcer‐general tokens (white tokens exchangeable for food or water) in a closed token economy. Food and green food tokens could be earned on one panel; water and red water tokens could be earned on a second panel; white generalized tokens could be earned on either panel. Responses on one key produced tokens according to a fixed‐ratio schedule, whereas responses on a second key produced exchange periods, during which all previously earned tokens could be exchanged for the appropriate commodity. Most conditions were conducted in a closed economy, and pigeons distributed their token allocation in ways that permitted food and water consumption. When the price of all tokens was equal and low, most pigeons preferred the generalized tokens. When token‐production prices were manipulated, pigeons reduced production of the tokens that increased in price while increasing production of the generalized tokens that remained at a fixed price. The latter is consistent with a substitution effect: Generalized tokens increased and were exchanged for the more expensive reinforcer. When food and water were made freely available outside the session, token production and exchange was sharply reduced but was not eliminated, even in conditions when it no longer produced tokens. The results join with other recent data in showing sustained generalized functions of token reinforcers, and demonstrate the utility of token‐economic methods for assessing demand for and substitution among multiple commodities in a laboratory context.     

Second-order schedules of token reinforcement: comparisons of performance under fixed-ratio and variable-ratio exchange schedules

Rats' lever pressing produced tokens according to a 20-response fixed-ratio schedule. Sequences of token schedules were reinforced under a second-order schedule by presentation of periods when tokens could be exchanged for food pellets. When the exchange period schedule was a six-response fixed ratio, patterns of completing the component token schedules were bivalued, with relatively long and frequent pauses marking the initiation of each new sequence. Altering the exchange period schedule to a six-response variable ratio resulted in sharp reductions in the frequency and duration of these initial pauses, and increases in overall rates of lever pressing. These results are comparable to those ordinarily obtained under simple fixed-ratio and variable-ratio schedules.     

Unit price and choice in a token-reinforcement context

Pigeons were exposed to multiple and concurrent second-order schedules of token reinforcement, with stimulus lights serving as token reinforcers. Tokens were produced and exchanged for food according to various fixed-ratio schedules, yielding equal and unequal unit prices (responses per unit food delivery). On one schedule (termed the standard schedule), the unit price was held constant across conditions. On a second schedule (the alternative schedule), the unit price was either the same or different from the standard. Under conditions with unequal unit prices, near-exclusive preference for the lower unit price was obtained. Under conditions with equal unit prices, the direction and degree of preference depended on ratio size (number of responses per exchange period). When this ratio differed, strong preferences for the smaller ratio were observed. When this ratio was equal, preferences were nearer indifference. Response rates on the multiple schedule were generally consistent with the preference data in showing sensitivity to ratio size. Results are discussed in terms of a unit-price model that includes handling and reinforcer immediacy as additional costs. On the whole, results show that preferences were determined primarily by delay to the exchange period.     

Preferences for token exchange‐production schedules: Effects of task difficulty and token‐production schedules

Previous research suggests that accumulated exchange‐production schedules promote increased work completion and are more preferred than distributed exchange‐production schedules. The purpose of this study was to evaluate whether the response effort or token‐production schedules associated with token delivery influenced preferences for exchange‐production schedules. Tokens exchanged under accumulated schedules were more preferred, relative to distributed schedules, when tokens were earned for completing easy tasks. When participants earned tokens for completing difficult tasks, two of three participants preferred accumulated exchange‐production schedules (Experiment 1). Under dense token‐production schedules, accumulated exchange‐production schedules were preferred, but participant's preferences switched to distributed schedules under increasing token‐production (i.e., leaner) schedules (Experiment 2).     

Second-order schedules of token reinforcement: effects of varying the schedule of food presentation

In the initial link of a complex schedule, one discriminative stimulus was presented and lever pressing produced tokens on fixed-ratio schedules. In the terminal link, signalled by a second discriminative stimulus, deposits of the tokens produced food. With two rats, the terminal link was presented after each sixth component schedule of token reinforcement was completed. With the other two rats, the terminal link was presented following the first component schedule completed after a fixed interval. During the terminal link, each token deposit initially produced food. The schedule of food presentation was subsequently increased such that an increasing number of token deposits in the terminal link was required for each food presentation. Rates of lever pressing in the initial link were inversely related to the schedule of food presentation in the terminal link. These results are similar to those of experiments that have varied schedules of food presentation in chained schedules. Rates and patterns of responding controlled throughout the initial link were more similar to those ordinarily controlled by second-order brief-stimulus schedules than to those controlled by comparable extended chained schedules.     

Token Economies: Using Basic Experimental Research to Guide Practical Applications

This paper highlights the applicability of patterns seen within basic experimental research in relation to contemporary application of token economies. Token economies are one of the most widely used interventions to promote behavior change, and this procedure has evolved to be effective across many settings, behaviors, and individuals. Due to this widespread use, casual implementation of the token economy might result in inconsistencies in responding and therefore an overall skepticism in the procedure itself. We present multiple barriers that encumber practical application of token economies, including insufficient conditioning and pairing of tokens, determining quality of backup reinforcers, unforeseen effects of motivating operations, teaching the token exchange, effects of higher-order reinforcement schedules, ratio strain, and use of response cost procedures. To assist practitioners in implementing more effective treatments, for each barrier we revisit the often overlooked basic research involving features of conditioned reinforcement and reinforcement schedules. It is important to translate the often complex implications of basic research so that practitioners can use this information to improve their own practice as well as their confidence in disseminating use of this evidence-based treatment. To further guide practitioners in using this knowledge in everyday settings, we also provide recommendations specific to each barrier as well as relevant applied research and practical examples.     

Successive independence and behavioral contrast in a closed economy.

Two pigeons had access to multiple concurrent schedules of reinforcement for 24 hours per day in their home cages. The variable-interval schedules comprising the multiple concurrent schedules were varied across 16 conditions. In three sets of conditions, one schedule was varied while its concurrent alternative and the concurrent schedules in the other component were held constant. Behavioral contrast was observed; that is, as the rate of reinforcement arranged by the varied schedule decreased, response rates on the constant schedules typically increased. These conditions formed part of two larger sets of conditions in which the concurrent schedules in one multiple-schedule component remained constant while the concurrent schedules in the other component were varied. Successive independence was found, in that behavior allocation during the constant component did not vary as a function of the reinforcer ratios in the varied component. Successive independence between components in multiple concurrent schedules is a robust result that occurs in closed economies and under conditions that promote behavioral contrast.     

RISKY CHOICE IN PIGEONS: PREFERENCE FOR AMOUNT VARIABILITY USING A TOKEN-REINFORCEMENT SYSTEM

Pigeons were given repeated choices between variable and fixed numbers of token reinforcers (stimulus lamps arrayed above the response keys), with each earned token exchangeable for food. The number of tokens provided by the fixed‐amount option remained constant within blocks of sessions, but varied parametrically across phases, assuming values of 2, 4, 6, or 8 tokens per choice. The number of tokens provided by the variable‐amount option varied between 0 and 12 tokens per choice, arranged according to an exponential or rectangular distribution. In general, the pigeons strongly preferred the variable option when the fixed option provided equal or greater numbers of tokens than the variable amount. Preference for the variable amount decreased only when the alternatives provided widely disparate amounts favoring the fixed amount. When tokens were removed from the experimental context, preference for the variable option was reduced or eliminated, suggesting that the token presentation played a key role in maintaining risk‐prone choice patterns. Choice latencies varied inversely with preferences, suggesting that local analyses may provide useful ancillary measures of reinforcer value. Overall, the results indicate that systematic risk sensitivity can be attained with respect to reinforcer amount, and that tokens may be critical in the development of such preferences.     

Determinants of pigeons' choices in token-based self-control procedures

Four pigeons were exposed to a token-based self-control procedure with stimulus lights serving as token reinforcers. Smaller-reinforcer choices produced one token immediately; larger-reinforcer choices produced three tokens following a delay. Each token could be exchanged for 2-s access to food during a signaled exchange period each trial. The main variables of interest were the exchange delays (delays from the choice to the exchange stimulus) and the food delays (also timed from the choice), which were varied separately and together across blocks of sessions. When exchange delays and food delays were shorter following smaller-reinforcer choices, strong preference for the smaller reinforcer was observed. When exchange delays and food delays were equal for both options, strong preference for the larger reinforcer was observed. When food delays were equal for both options but exchange delays were shorter for smaller-reinforcer choices, preference for the larger reinforcer generally was less extreme than under conditions in which both exchange and food delays were equal. When exchange delays were equal for both options but food delays were shorter for smaller-reinforcer choices, preference for the smaller reinforcer generally was less extreme than under conditions in which both exchange and food delays favored smaller-reinforcer choices. On the whole, the results were consistent with prior research on token-based self-control procedures in showing that choices are governed by reinforcer immediacy when exchange and food delays are unequal and by reinforcer amount when exchange and food delays are equal. Further, by decoupling the exchange delays from food delays, the results tentatively support a role for the exchange stimulus as a conditioned reinforcer.     

Token mediated tool exchange between tufted capuchin monkeys (<Emphasis Type="Italic">Cebus apella</Emphasis>)

Three experiments were conducted to test whether a pair of tufted capuchin monkeys (Cebus apella) could generalize their ability to exchange tokens and tool objects with a human experimenter to similar exchanges with a conspecific partner. Monkeys were tested in side-by-side enclosures, one enclosure containing a tool-use apparatus and one or more token(s), and the other enclosure containing one or more tool object(s). The monkeys willingly transferred tokens and tools to a conspecific with little practice. Following a small amount of training, we also found that the monkeys would select situation-appropriate tokens to exchange for specific tools, but did not select appropriate tool objects in response to another monkey’s token transfers. Implications regarding role reversal are discussed.     

THE EFFECTIVENESS OF REINFORCEMENT,RESPONSE COST,AND MIXED PROGRAMS ON CLASSROOM BEHAVIORS1

Six elementary school children served as subjects in an experiment comparing the relative effectiveness of (1) token reinforcement, in which children received tokens for attending and for correct arithmetic performance; (2) response cost, in which children received “free” tokens at the start of a period but lost them for inattention and for arithmetic performance below a specified level; and (3) a combination of both token reinforcement and response cost. During training, the six subjects received all three procedures in counterbalanced sequence. The effects of the three procedures were assessed by a within-subject comparison divided into three phases: (i) baseline, (ii) training, (iii) withdrawal of tokens. Introduction of the three token procedures markedly increased the two dependent measures. However, there were no differences across the procedures in the amount of change produced in either attending behavior or arithmetic performance. During baseline, the subjects averaged 29% attending behavior and 6.4 correct problems. These levels increased to 85% for attending behavior and 11.4 correct problems for arithmetic performance during training. Removal of all token procedures significantly decreased attending behavior (to an average of 65%), but produced a nonsignificant reduction in arithmetic performance (to an average of 7.6 correct problems). There was evidence that this lack of differential effects of the three token procedures was not due to an inability to discriminate among them. Furthermore, the subjects were evenly divided in their preference for the three procedures.     

Parametric analysis of delayed primary and conditioned reinforcers

We examined the effects of delayed reinforcement on the responding of individuals with intellectual disabilities. Three conditions were evaluated: (a) food reinforcement, (b) token reinforcement with a postsession exchange opportunity, and (c) token reinforcement with a posttrial exchange opportunity. Within each condition, we assessed responding given (a) a no‐reinforcement baseline, (b) immediate reinforcement, and (c) delayed reinforcement, in which responses produced a reinforcer after 1 of 6 delays. Results suggest that delayed food produced greater response persistence than did delayed tokens.     

Exchange schedules in token economies: A preliminary investigation of second‐order schedule effects

Basic research shows that token‐production and exchange‐production schedules in token economies affect each other as second‐order schedules (i.e., the exchange‐production schedule's requirements affect responding toward the token‐production schedule). This relationship has not been investigated with children in academic settings despite the widespread use of token economies in this context. This study compared the effects of fixed‐ratio (FR) and variable‐ratio (VR) exchange‐production schedules of equal ratios (2, 5, and 10) on responding toward an FR 1 token‐production schedule with a child diagnosed with autism. A concurrent chains assessment was also conducted to assess the participant's relative preference for FR and VR exchange‐production schedule arrangements within her typical discrete trial training. Results showed no difference in response rate between the two schedule types. However, the concurrent chains assessment revealed an exclusive preference for the VR arrangement.     

An experimental evaluation of tokens as conditioned reinforcers in retarded children

Since the publication of experimental reports by Wolfe (1936) and Cowles (1937), that tokens could be established as conditioned reinforcers, many accounts have appeared on the implementation of token systems in human subjects. Tokens have been successfully used as prosthetic motivational devices in a great variety of situations and subjects, such as retarded children (Birnbrauer, Wolf, Kidder and Tague, 1965), adult psychotics (Ayllon and Azrin, 1969), delinquent children and adolescents (Tyler, 1967; Phillips and Wolf, 1968), low achievers in schools (Clark, Lachowicz and Wolf, 1968), emotionally disturbed children (Hewett, Taylor and Artuso, 1969) and in normal classroom settings (O'Leary, Becker, Evans and Saudargas, 1969). In these reports, administration of tokens usually produced an increase in the frequency of desirable social and productive behavior. However, none of these studies showed that the effect of the tokens depended on their role as conditioned reinforcers based on the standard exchange systems being used. No assessment was provided about the separate contribution of the social reinforcement operations involved in dispensing tokens in a social setting.A suggestion about the possible confounding effects of social reinforcement in token administration comes from the failure to establish token systems in subjects who are partially unresponsive to social reinforcement (Hamblin et al., 1970; Ribes and Souza e Silva, in preparation). The present research was conducted with the aim of isolating the effects of conditioned reinforcement as such—apart from those produced by the social reinforcement provided in the delivery of tokens.     

Assessing the value of choice in a token system

Responding of 4 children was assessed under conditions in which (a) no programmed contingencies were arranged for target behavior, (b) responding produced tokens that could be exchanged for a single highly preferred edible item, and (c) responding produced a token that could be exchanged for a variety of preferred edible items. After assessing the effects of these contingencies, the preferences of 3 participants were assessed using a concurrent-chains schedule. Preference for the opportunity to choose from the same or qualitatively different edible items varied across participants, and findings were generally consistent with those of Tiger, Hanley, and Hernandez (2006).     

RISKY CHOICE IN PIGEONS AND HUMANS: A CROSS‐SPECIES COMPARISON

Pigeon and human subjects were given repeated choices between variable and adjusting delays to token reinforcement that titrated in relation to a subject's recent choice patterns. Indifference curves were generated under two different procedures: immediate exchange, in which a token earned during each trial was exchanged immediately for access to the terminal reinforcer (food for pigeons, video clips for humans), and delayed exchange, in which tokens accumulated and were exchanged after 11 trials. The former was designed as an analogue of procedures typically used with nonhuman subjects, the latter as an analogue to procedures typically used with human participants. Under both procedure types, different variable‐delay schedules were manipulated systematically across conditions in ways that altered the reinforcer immediacy of the risky option. Under immediate‐exchange conditions, both humans and pigeons consistently preferred the variable delay, and indifference points were generally ordered in relation to relative reinforcer immediacies. Such risk sensitivity was greatly reduced under delayed‐exchange conditions. Choice and trial‐initiation response latencies varied directly with indifference points, suggesting that local analyses may provide useful ancillary measures of reinforcer value. On the whole, the results indicate that modifying procedural features brings choices of pigeons and humans into better accord, and that human—nonhuman differences on risky choice procedures reported in the literature may be at least partly a product of procedural differences.     

Functions of Clinical Psychological Service in California Juvenile and Adult Courts

Subjects provided free responses to six concepts representing varying degrees of involvement. Responses were obtained from each subject under normal and negative affect conditions. Group data were used. Under negative affect conditions, significant correlations were found between concept involvement and type/token ratio (TTR), tokens per type, response variation index (RVI), and total types. Correlations were in the direction of increased redundancy as involvement decreased and, conversely, increased response variation as involvement increased. These relationships held only under negative affect conditions. Implications were discussed for the problems of communication under negative affect conditions and the assessment of individuals’ internal states from verbal behavior. Finally, a sociolinguistic view of the “language community” concept was offered.