Second-order schedules and the problem of conditioned reinforcement

Thirteen pigeons were exposed to a variety of second-order schedules in which responding under a component schedule was reinforced according to a schedule of reinforcement. Under different conditions, completion of each component resulted in either (1) the brief presentation of a stimulus also present during reinforcement (pairing operation), (2) the brief presentation of a stimulus not present during reinforcement (nonpairing operation), or (3) no brief stimulus presentation (tandem). Brief-stimulus presentations engendered a pattern of responding within components similar to that engendered by food. Patterning was observed when fixed-interval and fixed-ratio components were maintained under fixed- and variable-ratio and fixed- and variable-interval schedules. There were no apparent differences in performance under pairing and nonpairing conditions in any study. The properties of the stimuli presented in brief-stimulus operations produced different effects on response patterning. In one study, similar effects on performance were found whether brief-stimulus presentations were response-produced or delivered independently of responding. Response patterning did not occur when the component schedule under which a nonpaired stimulus was produced occurred independently of the food schedule. The results suggest a reevaluation of the role of conditioned reinforcement in second-order schedule performance. The similarity of behavior under pairing and nonpairing operations is consistent with two hypotheses: (1) the major effect is due to the discriminative properties of the brief stimulus; (2) the scheduling operation under which the paired or nonpaired stimulus is presented can establish it as a reinforcer.     

Behavioral contrast and response independent reinforcement

Four pigeons received pre-training that included presentation of the reinforcer independently of behavior and then baseline training on a variable-interval schedule of reinforcement. With the introduction of a multiple schedule, in which the first stimulus was associated with a response contingent and a second stimulus with a response independent, 1-min variable-interval schedule, a reduction in response rate was obtained in the second component, which was not accompanied by a behavioral contrast effect in the first component. A further three pigeons were given the same pre-training and baseline training before the introduction of an otherwise identical multiple schedule, in which no reinforcement occured in the second component. Behavioral contrast was obtained from all three subjects. The results indicated that under conditions of constant reinforcement density a reduction in responding is not a sufficient condition for the occurrence of behavioral contrast.     

Key-peck durations under behavioral contrast and differential reinforcement

Pigeons were maintained on a multiple schedule in which both components were variable-interval one-minute schedules. When they were switched to a condition in which one component was extinction, behavioral contrast was observed. The median durations of the key pecks in the unchanged component did not decrease in size. The results are incompatible with a theory of behavioral contrast which considers the added pecks to be short-duration responses. In a second experiment, pigeons were required to emit short-duration key pecks in one component of a multiple schedule, and long-duration pecks in the other. Two of three pigeons learned to emit responses appropriate to the requirements of the component in effect, suggesting that the duration of the key-peck response is sensitive to differential reinforcement.     

Varying response effort in the treatment of pica maintained by automatic reinforcement

Pica is a life-threatening behavior displayed by many individuals with developmental disabilities. In the current study, automatic reinforcement maintained the pica of 3 participants. Following functional analyses of pica, response-effort manipulations were conducted in which the effort to obtain pica or alternative items was varied systematically. Several general relations emerged as a result of the study. First, levels of pica were reduced relative to baseline when alternative items were available independent of the effort required to obtain alternative items or pica. Second, increasing the effort for alternative items resulted in increases in pica relative to when effort for alternative items was low. Third, increasing response effort for pica produced reductions in pica relative to baseline when alternative items were unavailable. Fourth, the highest levels of pica occurred when the effort to engage in pica was low or medium and no alternative items were available. These findings are discussed in terms of the relative effects of quality of reinforcement and response effort on behavior.     

From contrast to reinforcement: role of response contingency in anticipatory contrast

Intake of a 0.15% saccharin solution is suppressed if access to the saccharin is followed by access to 32% sucrose in brief daily pairings. The present series of four experiments was concerned with factors that lead to this anticipatory contrast effect (suppressed saccharin intake) rather than a reinforcement effect. In Experiment 1, anticipatory contrast was obtained with an autoshaping procedure (no lick requirement on the initial tube), and degree of contrast did not vary as a function of intersolution interval in the range of 0-15 s. Experiments 2 and 3 showed that requirements of 10, 100, 200, or 400 licks on the first tube available led to a reinforcement effect in latency, but a requirement of 0 licks (autoshaping procedure) led to a contrast effect in licks and latency. In Experiment 4, a group with a 200-contingent-lick requirement showed a reinforcement effect in latency, but a group yoked to this contingent group showed a contrast effect in both latency and licks. Overall, the results suggest that anticipatory contrast occurs under conditions of a "relaxed" instrumental contingency. The data are discussed in terms of control of behavior by stimulus-stimulus, response-stimulus, and stimulus-response associations, and the results are related to behavioral contrast, to flavor-outcome associations, and to "misbehavior" produced by Pavlovian-instrumental interactions.     

Second-order schedules of token reinforcement: effects of varying the schedule of food presentation

In the initial link of a complex schedule, one discriminative stimulus was presented and lever pressing produced tokens on fixed-ratio schedules. In the terminal link, signalled by a second discriminative stimulus, deposits of the tokens produced food. With two rats, the terminal link was presented after each sixth component schedule of token reinforcement was completed. With the other two rats, the terminal link was presented following the first component schedule completed after a fixed interval. During the terminal link, each token deposit initially produced food. The schedule of food presentation was subsequently increased such that an increasing number of token deposits in the terminal link was required for each food presentation. Rates of lever pressing in the initial link were inversely related to the schedule of food presentation in the terminal link. These results are similar to those of experiments that have varied schedules of food presentation in chained schedules. Rates and patterns of responding controlled throughout the initial link were more similar to those ordinarily controlled by second-order brief-stimulus schedules than to those controlled by comparable extended chained schedules.     

Response rate, reinforcement frequency, and behavioral contrast

Pigeons responded for food on a multiple schedule in which periods of green-key illumination alternated with periods of red-key illumination. When behavior had stabilized with a variable-interval 2-min schedule of reinforcement operating during both stimuli, low rates of responding (interresponse times greater than 2 sec) were differentially reinforced during the green component. Conditions during the red stimulus were unchanged. Response rates during the green component fell without changing the frequency of reinforcement but there were no unequivocal contrast effects during the red stimulus. The frequency of reinforcement during the green component was then reduced by changing to a variable-interval 8-min schedule without reducing the response rates in that component, which were held at a low level by the spacing requirement. Again, the conditions during the red stimulus were unchanged but response rates during that stimulus increased. These results show that reductions in reinforcement frequency, independently of response rate, can produce interactions in multiple schedules.     

The effects of continuous and partial schedules of reinforcement on effort,performance, and satisfaction

Three schedules of pay (hourly, fixed ratio, and variable ratio—variable amount) were used in a job simulation setting. Three groups of 20 subjects each worked for 1 week on programmed texts under each of the three schedules. Results indicated that performance was superior under the contingent schedules, but there were no differences between FR and VR VA. A hard criterion of effort, however, showed that FR was superior to VR VA. Attitudes did not differ across conditions. It was concluded that, based on this and other studies, there is no real evidence that partial schedules are more effective in maintaining task performance than continuous FR (piece-rate) schedules.     

Second-order schedules of token reinforcement with pigeons: implications for unit price

Four pigeons were exposed to second-order schedules of token reinforcement, with stimulus lights serving as token reinforcers. Tokens were earned according to a fixed-ratio (token-production) schedule, with the opportunity to exchange tokens for food (exchange period) occurring after a fixed number had been produced (exchange-production ratio). The token-production and exchange-production ratios were manipulated systematically across conditions. Response rates varied inversely with the token-production ratio at each exchange-production ratio. Response rates also varied inversely with the exchange-production ratio at each token-production ratio, particularly at the higher token-production ratios. At higher token-production and exchange-production ratios, response rates increased in token-production segments closer to exchange periods and food. Some conditions were conducted in a closed economy, in which the pigeons earned all their daily ration of food within the session. Relative to comparable open-economy conditions, response rates in the closed economy were less affected by changes in token-production ratio, resulting in higher levels of food intake and body weight. Some of the results are consistent with the economic concept of unit price, a cost-benefit ratio comprised of responses per unit of food delivery, but most are well accounted for by a consideration of the number of responses required to produce exchange periods, without regard to the amount of reinforcement available during those exchange periods.     

Behavioral contrast as a function of the temporal location of reinforcement

Pigeons were trained on a multiple variable-interval variable-interval schedule of reinforcement. One component was then changed to a variation of a fixed-interval schedule in which the same rate of reinforcement was obtained as previously but the location of the reinforcer was fixed within the component. The effects of different temporal locations were compared. An increase in response rate for the unchanged variable-interval component (behavioral contrast) occurred when the reinforcer was located in the middle or at the end of the FI component, but response suppression occurred when it was located at the beginning of the component. The pattern of results cannot be explained by any previous theories of contrast. The overall response rates, and the pattern of local rates within the components, were consistent with the hypothesis that the major determinant of the contrast effect was the transition to a lower reinforcement rate following the unchanged component.     

Influence of visually induced expectation on perceived motor effort: A visual-proprioceptive interaction at the Santa Cruz Mystery Spot

It is known that dense objects seem heavier than larger, less dense objects of the same weight. We have investigated a related illusion, in which visual context biases the apparent weight of a single object. The apparatus is a cabin on a steep hillside near Santa Cruz, CA, tilted 17° from vertical. From its ceiling hangs a weight on a chain. The cabin’s tilt makes the weight appear suspended at an angle. Pushing the weight toward the visually based vertical is perceived as difficult, whereas pushing it away from the visual vertical is perceived as easy. Seven subjects pushed the weight in both directions, judging required effort on a double-anchored 1–10 scale. All experienced the effort illusion, with no significant subject effect. When subjects’ eyes were closed, the effect was smaller but still present. Apparently proprioceptive and skin inputs, equal for both directions, are ignored or underweighted as visually based expectations influence perceived effort.     

Behavioral contrast and relative reinforcement frequency in two multiple schedules

After preliminary variable-interval training, one group of pigeons was trained on a series of multiple variable-interval low-rate reinforcement schedules, while another group was trained on a series of multiple variable-interval fixed-ratio reinforcement schedules. Contrast effects were observed as variable-interval baseline rate changed in a direction away from the change in reinforcement frequency in the other component. The effects of the variable-interval component on performance in the low-rate and fixed-ratio reinforcement components in the multiple schedules were assessed by comparing the birds' performances on each of these schedules alone. Fixed-ratio reinforcement schedules showed a susceptibility to contrast effects, low-rate reinforcement schedules did not. The rate of reinforcement in fixed-ratio schedules at which no interaction occurred in the multiple schedules was higher than that in variable-interval 1-min schedules, suggesting that pigeons may prefer time-based, rather than response-based, reinforcement.     

Second-order schedules of token reinforcement with pigeons: effects of fixed- and variable-ratio exchange schedules

Pigeons' key pecks produced food under second-order schedules of token reinforcement, with light-emitting diodes serving as token reinforcers. In Experiment 1, tokens were earned according to a fixed-ratio 50 schedule and were exchanged for food according to either fixed-ratio or variable-ratio exchange schedules, with schedule type varied across conditions. In Experiment 2, schedule type was varied within sessions using a multiple schedule. In one component, tokens were earned according to a fixed-ratio 50 schedule and exchanged according to a variable-ratio schedule. In the other component, tokens were earned according to a variable-ratio 50 schedule and exchanged according to a fixed-ratio schedule. In both experiments, the number of responses per exchange was varied parametrically across conditions, ranging from 50 to 400 responses. Response rates decreased systematically with increases in the fixed-ratio exchange schedules, but were much less affected by changes in the variable-ratio exchange schedules. Response rates were consistently higher under variable-ratio exchange schedules than tinder comparable fixed-ratio exchange schedules, especially at higher exchange ratios. These response-rate differences were due both to greater pre-ratio pausing and to lower local rates tinder the fixed-ratio exchange schedules. Local response rates increased with proximity to food under the higher fixed-ratio exchange schedules, indicative of discriminative control by the tokens.     

Second-order schedules of token reinforcement: comparisons of performance under fixed-ratio and variable-ratio exchange schedules

Rats' lever pressing produced tokens according to a 20-response fixed-ratio schedule. Sequences of token schedules were reinforced under a second-order schedule by presentation of periods when tokens could be exchanged for food pellets. When the exchange period schedule was a six-response fixed ratio, patterns of completing the component token schedules were bivalued, with relatively long and frequent pauses marking the initiation of each new sequence. Altering the exchange period schedule to a six-response variable ratio resulted in sharp reductions in the frequency and duration of these initial pauses, and increases in overall rates of lever pressing. These results are comparable to those ordinarily obtained under simple fixed-ratio and variable-ratio schedules.     

Successive negative contrast after partial reinforcement in the consummatory behavior of rats

Rats given access to a 32% sucrose solution later reject a 4% solution significantly more than controls that have only received the 4% solution. In Experiment 1, this consummatory successive negative contrast (cSNC) effect was attenuated by previous exposure to 50% partial reinforcement. Furthermore, recovery from cSNC was also facilitated by partial reinforcement. In Experiment 2, the attenuating effects of partial reinforcement on cSNC were eliminated by administration of the benzodiazepine anxiolytic chlordiazepoxide (5 mg/kg) before nonreinforced trials. In Experiment 3, the attenuating effect of partial reinforcement was greater after a shift from 32 to 6% solution, than after a shift from 32 to 2% solution. The parallels between the effects of partial reinforcement on consummatory and instrumental behavior are discussed.     

Time-dependent contrast effects in a multiple schedule of food reinforcement

Four rats were rewarded for running in a wheel under two alternating conditions of food reinforcement. These periods of frequent and infrequent reinforcement, each accompanied by a particular stimulus, were presented a number of times in each daily session. Following shifts from high to low frequency of reinforcement, responding decreased suddenly and markedly, and then recovered within the next few minutes. The magnitude of this temporary depression was an increasing function of the duration of the immediately preceding component of high-frequency reinforcement. A transient elevation in performance, which did not vary with the duration of the prior component, was noted in two subjects following shifts from low to high frequency of reinforcement. The elevation and depression effects did not appear simultaneously during the 48 experimental sessions. A possible relation between the difficulty of the discrimination and the extent of contrast effects is discussed.     

Behavioural contrast does not occur with continuous reinforcement in the positive component

An experiment reported by Jenkins (1961) obtained positive behavioural contrast in a discrimination with continuous reinforcement of the positive stimulus. A large body of research, however, has failed to obtain positive contrast with this type of schedule. Jenkins's experiment was repeated with an added control group matched to the discrimination group in number of nonreinforced components. No positive contrast was obtained, and it was concluded that Jenkins's result could not be interpreted as positive contrast. The implications of this result for some theoretical accounts of positive contrast were discussed.