Signalled reinforcement in multiple and concurrent schedules

Five pigeons were exposed to multiple and concurrent variable-interval, variable-interval reinforcement schedules in which reinforcement availability in one component was never signalled. During certain phases of the experiment, reinforcement availability in the other component was signalled. Behavioral contrast was observed in seven of eight instances when reinforcement availability in the multiple schedules was signalled. Under the concurrent schedules in which reinforcement availability was signalled, the subjects did not always allocate more time to (prefer) the component containing non-signalled reinforcement, as would be predicted by an account of behavioral contrast holding that contrast results from the introduction of a less-preferred condition in one component of a multiple schedule.     

Time allocation in concurrent schedules: the effect of signalled reinforcement

The responses of five pigeons were reinforced on concurrent variable-interval variable-interval reinforcement schedules in which changeover key responses changed the stimulus and reinforcement schedules associated with the food key. While the reinforcement availability in one component remained unchanged throughout the experiment, the reinforcement availability in the other component was, during several conditions, signalled by the onset of an additional discriminative stimulus. During unsignalled conditions, both the relative frequency of responding and the relative time spent in each component approximated the obtained relative reinforcement frequency in each component. The effect of signalling reinforcer availability in one component was to (1) reduce responding in the signalled component to near-zero levels, and (2) increase the relative time in the unsignalled component, without a corresponding increase in the obtained relative reinforcement frequency. The magnitude of the increase in relative time in the unsignalled component decreased as the overall frequency of reinforcement increased. This deviation in the matching relation between relative time and the obtained relative reinforcement frequency was eliminated if the overall reinforcement frequency was increased before the signal was introduced and then, without removing the signal, gradually reduced.     

Effects on concurrent performances of a stimulus correlated with reinforcer availability

A multiple schedule was arranged in which each component consisted of two, concurrent variable-interval schedules of reinforcement. A changeover-key procedure was used, and the components of the multiple schedule were distinguished (initially) by the color of the changeover key. During one component of the multiple schedule, the availability of a reinforcer arranged by one of the variable-interval schedules was marked by an exteroceptive stimulus, provided that that variable-interval schedule was not at the time assigned to the main key. During the other component of the multiple schedule, no reinforcer-correlated stimuli were ever presented. During the latter component of the multiple schedule, the distribution of responses and time for the concurrent variable-interval schedules suggested control by the distribution of reinforcements. During the former component, most main-key responses were emitted on the key in the presence of which reinforcer-correlated stimuli were presented. Changeover rate in the presence of that key color was depressed. The discriminative control over the changeover was easily established and was reversible.     

Negative behavioral contrast on multiple treadle-press schedules

Eight pigeons pressed treadles for food reinforcers delivered by several multiple variable-interval schedules. The rate of reinforcement for responding during one component schedule was held constant at 30 reinforcers per hour. The rate of reinforcement for responding during the other component varied from 0 to 120 or 240 reinforcers per hour. The schedules were presented in different orders for different subjects. The rate of responding emitted during the variable component schedule varied directly with the rate of reinforcement it provided. The rate of responding during the constant component did not increase consistently when the rate of reinforcement obtained from the variable component decreased from 30 to 0 reinforcers per hr. The rate of responding emitted during the constant component decreased when the rate of reinforcement obtained from the variable component increased from 30 reinforcers per hour to a higher rate. That is, negative but not positive behavioral contrast occurred. The failure to find positive contrast is consistent with one of the predictions of the additive theories of behavioral contrast. Finding negative contrast has ambiguous implications for the additive theories.     

The effects of unsignalled delayed reinforcement

Pigeons' pecks were reinforced according to a variable-interval schedule. A delay-of-reinforcement procedure was then added to the schedule, or a yoked-control procedure was arranged where the reinforcers occurred independently of responding according to the same variable-interval schedule. During the delay-of-reinforcement procedure, the first peck after a reinforcer was scheduled began a delay timer and the reinforcer was delivered at the end of the interval. No stimulus change signalled the delay interval and responses could occur during it, so that the obtained delays were often shorter than those scheduled. Responding under this procedure was highly variable but, in general, behavior was substantially reduced even with the shortest delay used, 3 sec. In addition, the rates maintained by delayed reinforcement were only slightly greater than those maintained by the yoked-control procedure, suggesting that adventitious pairings of response and reinforcer were responsible for some of the maintenance of behavior that did occur. The results challenge recent conceptions of reinforcement as involving response-reinforcer correlations and re-emphasize the role of temporal proximity between response and reinforcer.     

The momentum of human behavior in a natural setting

Adults with mental retardation in a group home received popcorn or coffee reinforcers for sorting plastic dinnerware. In Part 1 of the experiment, reinforcers were dispensed according to a variable-interval 60-s schedule for sorting dinnerware of one color and according to a variable-interval 240-s schedule for sorting dinnerware of a different color in successive components of a multiple schedule. Sorting rates were similar in baseline, but when a video program was shown concurrently, sorting of dinnerware was more resistant to distraction when correlated with a higher rate of reinforcement. In Part 2 of the experiment, popcorn or coffee reinforcers were contingent upon sorting both colors of dinnerware according to variable-interval 60-s schedules, but additional reinforcers were given independently of sorting according to a variable-time 30-s schedule during one dinnerware-color component. Baseline sorting rate was lower but resistance to distraction by the video program was greater in the component with additional variable-time reinforcers. These results demonstrate that resistance to distraction depends on the rate of reinforcers obtained in the presence of component stimuli but is independent of baseline response rates and response–reinforcer contingencies. Moreover, these results are similar to those obtained in laboratory studies with pigeons, demonstrating that the determination of resistance to change by stimulus–reinforcer relations is not confined to controlled laboratory settings or unique to the pigeon.     

Multiple determinants of the effects of reinforcement magnitude on free-operant response rates

Four experiments examined the effects of increasing the number of food pellets given to hungry rats for a lever-press response. On a simple variable-interval 60-s schedule, increased number of pellets depressed response rates (Experiment 1). In Experiment 2, the decrease in response rate as a function of increased reinforcement magnitude was demonstrated on a variable-interval 30-s schedule, but enhanced rates of response were obtained with the same increase in reinforcement magnitude on a variable-ratio 30 schedule. In Experiment 3, higher rates of responding were maintained by the component of a concurrent variable-interval 60-s variable-interval 60-s schedule associated with a higher reinforcement magnitude. In Experiment 4, higher rates of response were produced in the component of a multiple variable-interval 60-s variable-interval 60-s schedule associated with the higher reinforcement magnitude. It is suggested that on simple schedules greater reinforcer magnitudes shape the reinforced pattern of responding more effectively than do smaller reinforcement magnitudes. This effect is, however, overridden by another process, such a contrast, when two magnitudes are presented within a single session on two-component schedules.     

Effects of reinforcer rate and reinforcer quality on time allocation: Extensions of matching theory to educational settings

We examined how 3 special education students allocated their responding across two concurrently available tasks associated with unequal rates and equal versus unequal qualities of reinforcement. The students completed math problems from two alternative sets on concurrent variable-interval (VI) 30-s VI 120-s schedules of reinforcement. During the equal-quality reinforcer condition, high-quality (nickels) and low-quality items ("program money" in the school's token economy) were alternated across sessions as the reinforcer for both sets of problems. During the unequal-quality reinforcer condition, the low-quality reinforcer was used for the set of problems on the VI 30-s schedule, and the high-quality reinforcer was used for the set of problems on the VI 120-s schedule. Equal- and unequal-quality reinforcer conditions were alternated using a reversal design. Results showed that sensitivity to the features of the VI reinforcement schedules developed only after the reinforcement intervals were signaled through countdown timers. Thereafter, when reinforcer quality was equal, the time allocated to concurrent response alternatives was approximately proportional to obtained reinforcement, as predicted by the matching law. However the matching relation was disrupted when, as occurs in most natural choice situations, the quality of the reinforcers differed across the response options.     

Some effects of relative reinforcement rate and changeover delay in response-independent concurrent schedules of reinforcement

Reinforcements were arranged independently of the pigeon's behavior by concurrent variable-interval schedules. The reinforcements arranged by one of the schedules occurred when the chamber was illuminated with amber light, and the reinforcements arranged by the other schedule occurred when the chamber was illuminated with blue light. Both schedules functioned concurrently, but reinforcers were delivered by each only in the presence of the appropriate stimulus condition. A response on a white key, the only key in the chamber, alternated the stimulus condition and the effective schedule. The results of this procedure were similar to those obtained with concurrent response-dependent variable-interval schedules of reinforcement. The proportion of the total session time spent in the presence of a schedule component approximated the proportion of the total number of reinforcements in the component. Changeover rate was a decreasing function of the changeover delay and of the difference between the relative rates of reinforcement for each pair of concurrent schedules.     

Changeover behavior and preference in concurrent schedules

Pigeons were trained on a multiple schedule of reinforcement in which separate concurrent schedules occurred in each of two components. Key pecking was reinforced with milo. During one component, a variable-interval 40-s schedule was concurrent with a variable-interval 20-s schedule; during the other component, a variable-interval 40-s schedule was concurrent with a variable-interval 80-s schedule. During probe tests, the stimuli correlated with the two variable-interval 40-s schedules were presented simultaneously to assess preference, measured by the relative response rates to the two stimuli. In Experiment 1, the concurrently available variable-interval 20-s schedule operated normally; that is, reinforcer availability was not signaled. Following this baseline training, relative response rate during the probes favored the variable-interval 40-s alternative that had been paired with the lower valued schedule (i.e., with the variable-interval 80-s schedule). In Experiment 2, a signal for reinforcer availability was added to the high-value alternative (i.e., to the variable-interval 20-s schedule), thus reducing the rate of key pecking maintained by that schedule but leaving the reinforcement rate unchanged. Following that baseline training, relative response rates during probes favored the variable-interval 40-s alternative that had been paired with the higher valued schedule. The reversal in the pattern of preference implies that the pattern of changeover behavior established during training, and not reinforcement rate, determined the preference patterns obtained on the probe tests.     

EFFECTS OF PROBLEM DIFFICULTY AND REINFORCER QUALITY ON TIME ALLOCATED TO CONCURRENT ARITHMETIC PROBLEMS

Students with learning difficulties participated in two studies that analyzed the effects of problem difficulty and reinforcer quality upon time allocated to two sets of arithmetic problems reinforced according to a concurrent variable-interval 30-s variable-interval 120-s schedule. In Study 1, high- and low-difficulty arithmetic problems were systematically combined with rich and lean concurrent schedules (nickels used as reinforcers) across conditions using a single-subject design. The pairing of the high-difficulty problems with the richer schedule failed to offset time allocated to that alternative. Study 2 investigated the interactive effects of problem difficulty and reinforcer quality (nickels vs. program money) upon time allocation to arithmetic problems maintained by the concurrent schedules of reinforcement. Unlike problem difficulty, the pairing of the lesser quality reinforcer (program money) with the richer schedule reduced the time allocated to that alternative. The magnitude of this effect was greatest when combined with the low-difficulty problems. These studies have important implications for a matching law analysis of asymmetrical reinforcement variables that influence time allocation.     

Stimulus-reinforcer contingencies and local behavioral contrast

Four pigeons were exposed to a series of multiple schedules of variable-interval reinforcement in which pecks were required on one key (operant key) and components were signalled on a second key (signal key). Four additional pigeons experienced identical conditions, except that a yoking procedure delivered food on variable-time schedules, with no key pecks required. One of the components of the multiple schedule was constant throughout the experiment as a variable-interval (or variable-time) 30-second schedule. Operant-key responding during the constant component was uniform throughout the component, uninfluenced by changes in the duration of the variable component, and only slightly influenced by changes in reinforcement frequency correlated with the variable component. By comparison, signal-key response rate during the constant component was highest at the onset of the component, was higher when the variable component was 60-sec long than when it was 1-sec long, and was higher when no reinforcement occurred in the variable component than when reinforcement was scheduled in the variable component. These characteristics of signal-key pecking matched characteristics of local positive behavioral contrast. These data are taken to support the “additivity theory” of behavioral contrast and to suggest that Pavlovian stimulus-reinforcer relations contribute primarily to the phenomenon of local positive contrast.     

Response rate as a function of amount of reinforcement for a signalled concurrent response

Pigeons were exposed to two equal, concurrent variable-interval schedules of reinforcement on two response keys. One key was continuously illuminated. Pecking on that key produced reinforcements of constant duration. The other key was normally dark, except that availability of reinforcement was signalled by illuminating the key. The duration of access to a grain reinforcer was varied on the key that signalled reinforcement. Rate of response on the first key, the one that did not signal reinforcement, was found to vary inversely with duration of signalled reinforcement on the other key. The latency between the signal and the peck that produced signalled reinforcement remained about constant. These results show that responding on one key in concurrent variable-interval schedules depends on the reinforcement delivered by both schedules and is independent of responding on the other key.     

Contextual influences on resistance to disruption in children with intellectual disabilities

Training context can influence resistance to disruption under differing reinforcement schedules. With nonhumans, when relatively lean and rich reinforcement schedules are experienced in the context of a multiple schedule, greater resistance is found in the rich than the lean component, as described by behavioral momentum theory. By contrast, when the schedules are experienced in separated blocks of sessions (i.e., as single schedules), resistance is not consistently greater in either component. In the current study, two groups of 6 children with intellectual disabilities responded to stimuli presented in relatively lean or rich components. For both, reinforcers were delivered according to the same variable-interval reinforcement schedule; additionally, the rich component included the delivery of response-independent reinforcers. The Within group was trained on a multiple schedule in which lean and rich components alternated regularly within sessions; the Blocked group was trained on two single schedules in which sessions with either the lean or rich schedule were conducted in successive blocks. Disruption tests presented a concurrently available alternative stimulus disrupter signaling the availability of tangible reinforcers. All 6 Within participants showed greater resistance to disruption in the rich component, consistent with behavioral momentum theory. By contrast, there was no consistent or significant difference in resistance for Blocked participants. This finding is potentially relevant to the development of interventions in applied settings, where such interventions often approximate single schedules and include response-independent reinforcers.     

Contrast and reallocation of extraneous reinforcers between multiple-schedule components

Four pigeons responded in components of multiple schedules in which two responses were available and reinforced with food. Pecks on the left key (“main” key) were reinforced at a constant rate in one component and at a rate that varied over conditions in the other component. When reinforcer rate was varied, behavioral contrast occurred in the constant component. On the right key (“extra” key), five variable-interval schedules and one variable-ratio schedule, presented conjointly, arranged reinforcers for responses in all conditions. These conjoint schedules were common to both multiple-schedule components—rather than unique to particular components—and reinforcers from these schedules could therefore be arranged in one component and obtained during the other component. In this way, the additional reinforcers were analogous to the “extraneous” reinforcers thought to maintain behavior other than pecking in conventional multiple schedules. Response rate on the extra key did not change systematically over conditions in the constant component, and in the varied component extra responding was inversely related to main-key reinforcement. All subjects obtained more extra-key reinforcers in whichever component arranged fewer main-key reinforcers. Consistent with the theory that reallocation of extraneous reinforcers may cause behavioral contrast, absolute reinforcer rate for the extra key in the constant component was low in conditions that produced positive contrast on the main key and high in those that produced negative contrast. Also consistent with this theory, behavioral contrast was reduced in two conditions that canceled extra-key reinforcers that had been arranged but not obtained at the end of components. Thus, a constraint on reallocation markedly reduced the extent of contrast.     

Matching and contrast on several concurrent treadle-press schedules

Four White King pigeons pressed treadles for food reinforcement on several concurrent variable-interval variable-interval schedules. The rate of reinforcement available for responding in one of the two component schedules was held constant at 30 reinforcers per hour. The rate of reinforcement available for responding in the other was varied from 120 to 60 to 15, and then to 30 reinforcers per hour. The relative rate of responding in each component schedule equalled the relative rate of reinforcement that the component provided. And, behavioral contrast, defined as an inverse relationship between the rate of responding in the constant component and the rate of reinforcement obtained by responding in the other component, occurred for all schedules.     

Matching, contrast, and equalizing in the concurrent lever-press responding of rats

Five rats pressed levers for food reinforces delivered by several concurrent variable-interval variable-interval schedules. The rate of reinforcement available for responding on one component schedule was held constant at 60 reinforcers per hour. The rate of reinforcement available for responding on the other schedule varied from 30 to 240 reinforcers per hour. The behavior of the rats resembled the behavior of pigeons pecking keys for food reinforcers. The ratio of the overall rates of responding emitted under, and the ratio of the time spent responding under, the two components of each concurrent schedule were approximately equal to the ratio of the overall rates of reinforcement obtained from the components. The overall rate of responding emitted under, and the time spent responding under, the variable component schedule varied directly with the overall rate of reinforcement from that schedule. The overall rate of responding emitted under, and the time spent responding under, the constant component schedule varied inversely with the overall rate of reinforcement obtained from the variable component. The local rates of responding emitted under, and the local rates of reinforcement obtained from, the two components did not differ consistently across subjects. But they were not exactly equal either.     

Response-rate invariance in concurrent schedules: effects of different changeover contingencies

In a two-key chamber, one key (the food key) was either red or green with different variable-interval schedules operating concurrently in each color and a second key (the changeover key) served to change the food-key color. Three pigeons were trained with either a 2-sec changeover delay or a 0-sec changeover delay and three birds with a fixed-ratio 2 on the changeover key instead of a changeover delay. The proportion of time spent in red approximated the proportion of reinforcers delivered in red for all birds. When the procedure was changed so that reinforcers were signalled in the green schedule, rates of reinforcement were unaltered, but the pigeons spent virtually the whole session in red. Changeovers to green were allowed only when a reinforcer was assigned by the schedule associated with green. For all pigeons with the fixed–ratio requirement on the changeover key or with a 0-sec changeover delay, the overall rate of red-key responses was higher during the signalling condition than during unsignalled, or baseline, condition. The present data question the generality of previous reports that the rate of one response is independent of the amount of time allocated to the alternative response.     

Secondary reinforcement and number of primary reinforcements

Pigeons' pecks on either of two concurrently available response keys produced secondary reinforcers according to independent one-minute variable-interval schedules. Different secondary reinforcers, in the presence of which the rates of primary reinforcement were equal, were associated with each key. The rate of pecking maintained by each secondary reinforcer varied directly, but nonproportionally, with the number of primary reinforcements given in the presence of the secondary reinforcer.     

Behavioral contrast of time allocation

Pigeons' standing on a platform produced food reinforcement according to two-component multiple schedules in which either both components consisted of the same variable-interval schedule or one of these was replaced with a component without reinforcement (extinction). The components of the multiple schedule alternated every 30 sec, and were signalled by changes in the color of diffuse overhead illumination. Changing the schedule of one of the components to extinction increased the percentage of time spent on the platform during the unchanged component (behavioral contrast). This result casts doubt on accounts that attribute behavioral contrast to variations in the rate of noninstrumental elicited responses.