Stimulus and reinforcer relativity in multiple schedules: Local and dimensional effects on sensitivity to reinforcement

Pigeons' responses in two successive components of multiple schedules were reinforced according to variable-interval schedules of reinforcement that varied over five different conditions. Within each session of all conditions, line orientations of 0°, 30°, or 45° in Component 1 alternated with orientations of 45°, 60°, or 90° in Component 2. Response rates were recorded in three successive subintervals of each component. Ratios were taken between the response rate in each Component 1 line orientation and the response rate in each Component 2 orientation. These ratios were found to be power functions of the corresponding ratios of obtained reinforcement rates. Sensitivity of response ratios to changes in reinforcer ratios, given by the value of the exponent of the power function, increased systematically with increasing disparity between the dimensional values of orientation stimuli. In addition, sensitivity decreased systematically over successive subintervals of components, that is, with increasing time since component alternation. Dimensional and local (subinterval) effects interacted in that sensitivity increased with stimulus disparity to a far greater extent in the first subinterval than later in components. The data could be described by a combination of rectangular hyperbolae which attributed the interaction between local and dimensional effects to limits set by local effects on the extent that stimulus differences could affect sensitivity.     

Local contrast and maintained generalization.

Pigeons received variable-interval reinforcement for key pecking during presentations of horizontal and vertical line-orientation stimuli, while pecks during five intermediate orientations were extinguished. Lowest peck rates were observed during presentations of negative stimuli adjacent to the positive orientations while peck rate during 45 degrees (the intermediate negative orientation) was relatively high, i.e., there were negative contrast shoulders. When peck rates were manipulated in the positive orientations, peck rate in neithboring orientations changed in the opposite direction. Contrast shoulders faded after prolonged training. A second type of contrast, local contrast, was correlated with similarity of preceding stimulus and different average peck rates during different stages of the discrimination process. The data suggest that sequential local contrast accompanying the formation of a discrimination contributes to the form of generalization gradients. Blough's model of stimulus control predicts the changes in gradient form described here, but may not accurately depict the underlying process responsible for gradient form.     

Behavioral contrast in one component of a multiple schedule as a function of the reinforcement conditions operating in the following component

The key pecks of four pigeons were reinforced on a variable-interval 5-min schedule which operated in each of the four components of a multiple schedule, indicated by red, green, yellow, and blue stimuli and presented in such an order that the red stimulus always preceded the yellow and the green stimulus always preceded the blue. After establishing baseline rates, the reinforcement schedule associated with the blue and yellow components was altered so that one was now an extinction schedule and the other was a variable-interval 1-min schedule. In a second experimental stage, the blue stimulus was interchanged with the yellow so that the red stimulus preceded the blue and the green stimulus preceded the yellow. In both experimental stages the response rate in the variable-interval 5-min component that preceded the extinction component was higher than the response rate in the variable-interval 5-min component that preceded the variable-interval 1-min component. The results were discussed in relation to the importance of stimulus ordering in experiments concerned with investigating behavioral contrast.     

Contrast and reallocation of extraneous reinforcers as a function of component duration and baseline rate of reinforcement

Four pigeons responded on multiple schedules arranged on a “main” key in a two-key experimental chamber. A constant schedule component was alternated with another component that was varied over conditions. On an extra response key, conjoint schedules of reinforcement that operated in both components were arranged concurrently with the multiple schedule on the main key. On the main key, changes in reinforcement rate in the varied component were inversely related to changes in response rates in the constant component (behavioral contrast). On the extra key, some reinforcers were reallocated between components, depending on the schedules in effect on the main key in the varied component. In the varied component, the obtained rates of reinforcement on the extra key were inversely related to main-key reinforcement rate. In the constant component, extra-key reinforcer rates were positively related to main-key reinforcer rates obtained in the varied component, and were not a function of response rates on the extra key. In two comparisons, the rate at which components alternated and the value of the main-key schedule in the constant component were varied. Consistent with earlier work, long components reduced the extent of contrast. Reductions in contrast as a function of component duration were accompanied by similar reductions in the extent of reinforcer reallocation on the extra key. In the second comparison, lowering the rate of reinforcement in the constant component increased the rate at which extra-key reinforcers were obtained, reduced the extent of reinforcer reallocation, and reduced contrast. Overall, the results are consistent with the suggestion that some contrast effects are due to the changes in extraneous reinforcement during the constant component, and that manipulations of component duration, and manipulations of the rate of reinforcement in the constant component, affect contrast because they influence the extent of extraneous reinforcer real-location.     

Contrast effects in multiple fixed-interval reinforcement schedules

Pigeons were exposed to a multiple fixed-interval one-minute fixed-interval three-minute schedule of reinforcement following training on either a multiple fixed-interval one-minute fixed-interval one-minute schedule or a multiple fixed-interval three-minute fixed-interval three-minute schedule. For all birds, large negative local contrast effects developed during the first of four three-minute intervals in a component; response rate was depressed and postreinforcement pause lengthened in this interval. Positive local contrast effects were evident during the first of 12 one-minute intervals in a component for five of six birds; at asymptote, the pause was very short and response rate slightly elevated during this interval. Overall positive contrast was generally transient and varied considerably across subjects, while overall negative contrast effects, if they occurred, appeared only after a large number of sessions.     

On the effects of component durations and component reinforcement rates in multiple schedules

Four experiments, each using the same six pigeons, investigated the effects of varying component durations and component reinforcement rates in multiple variable-interval schedules. Experiment 1 used unequal component durations in which one component was five times the duration of the other, and the shorter component was varied over conditions from 120 seconds to 5 seconds. The schedules were varied over five values for each pair of component durations. Sensitivity to reinforcement rate changes was the same at all component durations. In Experiment 2, both component durations were 5 seconds, and the schedules were again varied using both one and two response keys. Sensitivity to reinforcement was not different from the values found in Experiment 1. In Experiment 3, various manipulations, including body-weight changes, reinforcer duration changes, blackouts, hopper lights correlated with keylights, and overall reinforcement rate changes were carried out. No reliable increase in reinforcement sensitivity resulted from any manipulation. Finally, in Experiment 4, reinforcement rates in the two components were kept constant and unequal, and the component durations were varied. Shorter components produced significantly increased response rates normally in the higher reinforcement rate component, but schedule reversals at short component durations eliminated the response rate increases. The effects of component duration on multiple schedule performance cannot be interpreted as changing sensitivity to reinforcement nor to changing bias.     

Matching to relative reinforcement frequency in multiple schedules with a short component duration

Three pigeons performed on two-component multiple variable-interval variable-interval schedules of reinforcement. There were two independent variables: component duration and the relative frequency of reinforcement in a component. The component duration, which was always the same in both components, was varied over experimental conditions from 2 to 180 sec. Over these conditions, the relative frequency of reinforcement in a component was either 0.2 or 0.8 (±0.03). As the component duration was shortened, the relative frequency of responding in a component approached a value equal to the relative frequency of reinforcement in that component. When the relative frequency of reinforcement was varied over conditions in which the component duration was fixed at 5 sec, the relative frequency of responding in a component closely approximated the relative frequency of reinforcement in that component. That is, the familiar matching relationship, obtained previously only with concurrent schedules, was obtained in multiple schedules with a short component duration.     

The effects of component duration on multiple-schedule performance in closed and open economies.

Pigeons responded on multiple variable-interval variable-interval schedules of reinforcement in an open and a closed economy. Equal duration components were increased in duration while the component rates of reinforcement were held constant, the component schedules were reversed, and component duration was decreased. In the open economy, daily sessions were limited to 1 hr, and subjects were maintained at 80% of their free-feeding weights through supplemental feeding when necessary in their home cages. In the closed economy, subjects were housed in their experimental chambers and no deprivation regimen was enforced. Relative response rate decreased as components were lengthened in the open economy, whereas in the closed economy relative rate increased as components were lengthened. Response proportions overmatched reinforcer proportions to a greater extent at long component durations in the closed economy, but there was no systematic effect of component duration on responding in the open economy.     

Behavioral contrast in fixed-interval components: effects of extinction-component duration.

Seven albino rats were exposed to a multiple schedule of reinforcement in which the two components (fixed interval and extinction) alternated such that a presentation of the extinction component followed each fixed-interval reinforcement. In baseline sessions, the duration of the extinction component was constant and always one-third of the fixed-interval value. Probe sessions contained a probe segment in which the duration of the extinction component was increased; the response rate in fixed-interval components during the probe segment was compared with the response rate in the segments preceding and following the probe. The effect of increasing the duration of the extinction component was studied under three values of fixed interval: 30 s, 120 s, and 18 s, in three successive conditions. Response rate within fixed intervals was a direct function of duration of the extinction component. Pausing at the beginning of the fixed interval decreased as extinction duration increased. These effects were larger and more consistent for the shorter fixed-interval values (18 s and 30 s). These results indicate a functional relation between relative component duration and responding. For the component providing more frequent reinforcement, this could be stated as an inverse relationship between relative component duration and response rate. This relation is similar to findings regarding the ratio of trial and intertrial duration in Pavlovian conditioning procedures, and suggests that behavioral contrast may be related to Pavlovian contingencies underlying the multiple schedule.     

Contrast and autoshaping in multiple schedules varying reinforcer rate and duration

Thirteen master pigeons were exposed to multiple schedules in which reinforcement frequency (Experiment I) or duration (Experiment II) was varied. In Phases 1 and 3 of Experiment I, the values of the first and second components' random-interval schedules were 33 and 99 seconds, respectively. In Phase 2, these values were 99 seconds for both components. In Experiment II, a random-interval 33-second schedule was associated with each component. During Phases 1 and 3, the first and second components had hopper durations of 7.5 and 2.5 seconds respectively. During Phase 2, both components' hopper durations were 2.5 seconds. In each experiment, positive contrast obtained for about half the master subjects. The rest showed a rate increase in both components (positive induction). Each master subject's key colors and reinforcers were synchronously presented on a response-independent basis to a yoked control. Richer component key-pecking occurred during each experiment's Phases 1 and 3 among half these subjects. However, none responded during the contrast condition (unchanged component of each experiment's Phase 2). From this it is inferred that autoshaping did not contribute to the contrast and induction findings among master birds. Little evidence of local contrast (highest rate at beginning of richer component) was found in any subject. These data show that (a) contrast can occur independently from autoshaping, (b) contrast assays during equal-valued components may produce induction, (c) local contrast in multiple schedules often does not occur, and (d) differential hopper durations can produce autoshaping and contrast.     

Stimulus control of respondent and operant key pecking: A single key procedure

Pigeons' responses to a uniformly illuminated response key were either reinforced on a variable-interval one-minute schedule of reinforcement or extinguished for one-minute periods. When 1.5 second signals were presented at the beginning of each component, so as to differentially predict reinforcement, the pigeons pecked at the signals, at rates higher than rates during the remainder of the component. When the brief signals were not differentially predictive of reinforcement, pecking in their presence decreased to near zero levels. Similar results were obtained with signals based upon colors and upon line orientations. Changes in rates of (unreinforced) pecking occurred during the signal whether pigeons responded differentially during the remainder of the component or not. Experiment II demonstrated that the presence of the signal correlated with extinction was not necessary for pecking to develop at the signal which preceded the component in which responding was intermittently reinforced. The experiments demonstrated a clear dissociation of respondent control from operant control of a response. In addition, operant behavior was shown to be relatively insensitive to differing rates of reinforcement, as compared to the sensitivity of respondent behavior to differing rates of reinforcement produced by the very same operant behavior.     

Discrimination of a response-independent component in a multiple schedule

Pigeons were trained to respond in non-differential reinforcement pre-discrimination training, with a multiple variable-interval 1-min variable-interval 1-min schedule. Each bird then received discrimination training with a multiple variable-interval 1-min variable-time 1-min schedule. Thus, discrimination training was between response-dependent (variable-interval) and response-independent (variable-time) schedules with the rate of reinforcement equated. In Experiment I, only three sessions of non-differential reinforcement preceded discrimination training and for half the birds, a 0° line was correlated with the response-dependent schedule; for the remaining birds the 0° line was correlated with the response-independent schedule. Post-discrimination gradients of excitatory stimulus control were obtained from the former group, while the latter group showed little evidence of post-discrimination stimulus control by the 0° line. Differential responding to the variable-time schedule was not accompanied by behavioral contrast to the variable-interval schedule. In Experiment II, 20 sessions of non-differential reinforcement preceded discrimination training and the 0° line was correlated with variable-time reinforcement for each bird. Differential responding to the 0° line was accompanied by negative induction to the variable-interval schedule and by inhibitory stimulus control about the 0° line during a post-discrimination generalization test.     

Stimulus-reinforcer contingencies and local behavioral contrast

Four pigeons were exposed to a series of multiple schedules of variable-interval reinforcement in which pecks were required on one key (operant key) and components were signalled on a second key (signal key). Four additional pigeons experienced identical conditions, except that a yoking procedure delivered food on variable-time schedules, with no key pecks required. One of the components of the multiple schedule was constant throughout the experiment as a variable-interval (or variable-time) 30-second schedule. Operant-key responding during the constant component was uniform throughout the component, uninfluenced by changes in the duration of the variable component, and only slightly influenced by changes in reinforcement frequency correlated with the variable component. By comparison, signal-key response rate during the constant component was highest at the onset of the component, was higher when the variable component was 60-sec long than when it was 1-sec long, and was higher when no reinforcement occurred in the variable component than when reinforcement was scheduled in the variable component. These characteristics of signal-key pecking matched characteristics of local positive behavioral contrast. These data are taken to support the “additivity theory” of behavioral contrast and to suggest that Pavlovian stimulus-reinforcer relations contribute primarily to the phenomenon of local positive contrast.     

Alternative reinforcement effects on fixed-interval performance

Pigeons' key pecks were reinforced with food on a fixed-interval schedule. Food also was available at variable time periods either independently of responding or for not key pecking (a differential-reinforcement-of-other-behavior schedule). The latter condition arranged reinforcement following the first pause of t seconds after it became available according to a variable-time schedule. This schedule allowed separation of the effects of pause requirements ≤ five-seconds and reinforcement frequency. The time spent pausing increased as the duration of the pause required for reinforcement increased from 0 to 30 seconds and as the frequency of reinforcement for pausing increased from 0 to 2 reinforcers per minute. Key pecking was more evenly distributed within each fixed interval with shorter required pauses and with more frequent reinforcement for pausing. The results complement those obtained with other concurrent schedules in which the same operant response was reinforced in both components.     

Contrast effects in maintained generalization gradients

In Experiment I, pigeons were given equal reinforcement (variable-interval 1-min) for responding during randomized presentations of eight line-orientation stimuli. Then, only responding in the vertical orientation was reinforced. Stable generalization gradients soon formed and persistent behavioral and local (transient) contrast effects appeared. Local contrast effects were not a function of relative reinforcement frequency or of any other variable known to produce contrast. Instead, they were related to average response rates associated with each stimulus. Experiment II showed that local contrast effects represent increases and decreases in response rates relative to baseline responding, and that these effects are relative; a given stimulus might enhance responding during a subsequent presentation of one stimulus, but depress responding when followed by another. These data indicate that discrimination learning is not adequately described as the acquisition of excitatory properties by some stimuli and inhibitory properties by others. A more adequate account implies that stimuli exert both excitatory and inhibitory effects related to their value.     

Behavioral aftereffects of reinforcement and its omission as a function of reinforcement magnitude

Rats responded on a multiple fixed-interval fixed-interval schedule of reinforcement. Each complete cycle of the multiple schedule was separated from the next by a relatively long period of timeout from all schedule contingencies. A response at the end of the second component of each cycle was always reinforced with an invariant reinforcement magnitude, while reinforcement magnitude and reinforcement omission were systematically varied in the first component. Response rate in the first component was a monotonic function of reinforcement magnitude in that component. These changes in response rate in the first component did not affect response rate in the second component. When reinforcement was omitted on 50% of occasions in the first component, following reinforcement there was a reduction in response rate in the second component that was monotonically related to reinforcement magnitude. Following reinforcement omission there was an increase in response rate in the second component that was unrelated to reinforcement magnitude. When reinforcement was omitted on 100% of occasions in the first component, behavioral contrast was observed.     

Contrast effects in response rate and accuracy of delayed matching to sample

Behavioural contrast is an inverse relation between the response rate in one component of a multiple schedule and the reinforcer rate in an alternated component. To explore possible contrast effects in accuracy as well as response rate, four pigeons were trained in multiple schedules where key pecking produced delayed matching-to-sample trials on a variable-interval schedule. Reinforcer probability for correct matches was constant at .3 in one component, and the conditions of reinforcement were varied in the second component. In Experiment 1, the varied component arranged the same contingencies as the constant component but with reinforcer probabilities of .9 or .1 across conditions. In the varied component, both response rate and accuracy of delayed matching were directly related to reinforcer probability; in the constant component, however, contrast effects on response rate were weak, and there was no evidence of contrast in accuracy of matching. In Experiment 2, the varied component was either variable interval with immediate food reinforcement or extinction. Reliable contrast effects were obtained in both response rate and in accuracy of matching in the constant component, and their magnitudes were correlated within and between subjects. The results of Experiment 2 join previous findings of covariation in the effects of reinforcement on free-operant responding and accuracy of discrimination.     

Signal duration and suppression of operant responding by free reinforcement

Pigeons were trained on a VI (variable interval) schedule of food presentation with a superimposed schedule of response-independent food. Substantial suppression of the operant response rate occurred when the free food was presented without a signal. When the free food was preceded by a short (4 sec) signal, the degree of suppression was similar to that with unsignaled free food. But when the signal was lengthened to 12 sec, the degree of suppression was substantially reduced. Experiment 3 assessed the effect of signal duration using a baseline schedule of delayed reinforcement, in which contingent reinforcers were themselves preceded by a signal. The signal preceding the free reinforcers was then either the same as or different from this contingent signal. Signal duration effects occurred only when the two types of signals were different. These differences as a function of signal duration have implications for both “context-blocking” and “comparator” interpretations of the effects of noncontingent reinforcement in both Pavlovian and operant procedures.     

Contrast and undermatching with regular or irregular alternation of components

Behavioral contrast and response-ratio sensitivity to reinforcement were compared in multiple schedules in which components alternated strictly or according to a pseudorandom sequence. Average component durations in the two regimes were always 60 s, and order of presentation of component alternation regimes was counterbalanced across subjects. In Part 1, the reinforcer rate in one component was reduced from 60 per hour to zero, while that in the other component was unchanged. Positive behavioral contrast occurred in the constant component in that response rates increased, but neither the reliability nor the magnitude of contrast was affected by the manner in which components alternated. Part 2 was similar, except that a number of different reinforcer rates were used in the varied component. Neither contrast nor sensitivity of response ratios to changes in reinforcer ratios depended on the regime of component alternation. Thus, the predictability in time of future reinforcement conditions, which is a feature of regular multiple scheduling, does not appear to be a determinant of multiple-schedule performance.     

Delayed reinforcement in a multiple schedule

Three rats and a pigeon were first trained on a two-component multiple schedule in which reinforcement in the two components occurred immediately after a response. Later, reinforcement in one component was delayed by a few seconds. During both stages of the experiment, reinforcement was scheduled by equal variable- (pigeon) or random-interval (rats) schedules in the two components. The main effect of the delayed reinforcement was to increase the rate of responding in the unchanged (non-delay) component. This behavioral contrast effect did not appear in all cases to be dependent upon a reduction in the rate of responding or the frequency of reinforcement in the delay component. This finding suggests that a reduction in response rate and/or reinforcement frequency in one component of a multiple schedule may not be a necessary prerequisite for the occurrence of behavioral contrast. This finding is, however, consistent with an explanation that suggests that behavioral contrast results from the introduction of a less-preferred condition in one component of a multiple schedule, since it is known that animals “prefer” immediate to delayed reinforcement.