Stimulus-reinforcer contingencies and local behavioral contrast

Four pigeons were exposed to a series of multiple schedules of variable-interval reinforcement in which pecks were required on one key (operant key) and components were signalled on a second key (signal key). Four additional pigeons experienced identical conditions, except that a yoking procedure delivered food on variable-time schedules, with no key pecks required. One of the components of the multiple schedule was constant throughout the experiment as a variable-interval (or variable-time) 30-second schedule. Operant-key responding during the constant component was uniform throughout the component, uninfluenced by changes in the duration of the variable component, and only slightly influenced by changes in reinforcement frequency correlated with the variable component. By comparison, signal-key response rate during the constant component was highest at the onset of the component, was higher when the variable component was 60-sec long than when it was 1-sec long, and was higher when no reinforcement occurred in the variable component than when reinforcement was scheduled in the variable component. These characteristics of signal-key pecking matched characteristics of local positive behavioral contrast. These data are taken to support the “additivity theory” of behavioral contrast and to suggest that Pavlovian stimulus-reinforcer relations contribute primarily to the phenomenon of local positive contrast.     

Variable-time reinforcement in multiple and concurrent schedules

Experiment I examined the role of a reduced rate of responding in the occurrence of behavioral contrast. Four rats and a pigeon were exposed to a two-component multiple schedule in which one component was always a variable-interval schedule. The second component was, at different times, either a variable-time schedule in which food was delivered independently of responding, or extinction. Both extinction and the variable-time schedule reduced the rate of responding in the second component. Behavioral contrast was observed, however, only when extinction was scheduled in the second component. Experiment II examined preference, as measured by time allocation in concurrent schedules for a variable-interval schedule relative to a variable-time schedule. Two rats displayed a lack of preference between the two schedules. The results of these experiments support a preference interpretation of behavioral contrast, which holds that behavioral contrast is the result of the introduction of a less-preferred condition in one component of a multiple schedule.     

Signalled reinforcement and multiple schedules

The responses of four pigeons were first reinforced in the presence of two different wave-lengths (green and red) on a two-ply multiple schedule with identical variable-interval 3-min schedules of reinforcement associated with each component. While the constant-component reinforcement schedule remained unchanged during the experiment, the schedule associated with the variable component was changed to (1) signalled variable time, (2) unsignalled variable time, or (3) signalled variable interval. The probability with which the availability of the reinforcer was signalled in the variable-interval schedules was either 0.5 or 1.0. Positive contrast occurred in both signalled variable-interval and variable-time schedules, but only when the availability of all the variable-component reinforcers was signalled. Signalling the availability of only 50% of the reinforcers in signalled variable-interval schedules resulted in negative induction. The present data suggest that positive behavioral contrast resulting from signalled reinforcer availability is due to the presence of an extinction-correlated stimulus.     

Melloration and maximization of reinforcement minus costs of behavior

Eight pigeons were exposed to independent concurrent schedules. Concurrent variable-interval 60-second variable-interval 60-second schedules were presented to one group of four subjects. Following baseline training, a limited hold was added to one of the schedules and the duration of the hold was decreased in successive conditions. Concurrent variable-interval 120-second variable-interval 40-second schedules were presented to another group of four subjects. These subjects were first exposed to decreasing durations of a limited hold in the variable-interval 40-second component. After replication of the baseline, a limited hold in the variable-interval 120-second component was decreased in duration. The initial durations of the holds were determined from the subjects' responding in the baseline conditions. A duration was chosen such that approximately 25% of the scheduled reinforcers would be canceled if responding remained unchanged.

Approximate matching of time proportions and reinforcement proportions was observed when the limited hold was added to the variable-interval 60-second schedule and when the limited hold was added to the variable-interval 40-second schedule. Time proportions were less extreme than reinforcement proportions when the limited hold operated in a variable-interval 120-second schedule. Overall reinforcement rates tended to decrease with continued training in concurrent schedules with a limited hold. Absolute deviations from time matching also decreased. The results provide evidence against the principle of reinforcement maximization, and support Herrnstein and Vaughan's (1980) melioration hypothesis.

     

Multiple determinants of the effects of reinforcement magnitude on free-operant response rates

Four experiments examined the effects of increasing the number of food pellets given to hungry rats for a lever-press response. On a simple variable-interval 60-s schedule, increased number of pellets depressed response rates (Experiment 1). In Experiment 2, the decrease in response rate as a function of increased reinforcement magnitude was demonstrated on a variable-interval 30-s schedule, but enhanced rates of response were obtained with the same increase in reinforcement magnitude on a variable-ratio 30 schedule. In Experiment 3, higher rates of responding were maintained by the component of a concurrent variable-interval 60-s variable-interval 60-s schedule associated with a higher reinforcement magnitude. In Experiment 4, higher rates of response were produced in the component of a multiple variable-interval 60-s variable-interval 60-s schedule associated with the higher reinforcement magnitude. It is suggested that on simple schedules greater reinforcer magnitudes shape the reinforced pattern of responding more effectively than do smaller reinforcement magnitudes. This effect is, however, overridden by another process, such a contrast, when two magnitudes are presented within a single session on two-component schedules.     

Stimulus control of behavioral history.

Pigeons were exposed to two different reinforcement schedules under different stimulus conditions in each of two daily sessions separated by 6 hr (Experiments 1 and 2) or in a single session (Experiment 3). Following this, either a fixed-interval (Experiment 1) or a variable-interval schedule (Experiments 2 and 3) was effected in both stimulus conditions. In the first two experiments, exposure to fixed-ratio or differential-reinforcement-of-low-rate schedules led to response-rate, but not pattern, differences in subsequent performance on fixed- or variable-interval schedules that persisted for up to 60 sessions. The effects of reinforcement-schedule history on fixed-interval schedule performance generally were more persistent. In Experiment 3, a history of high and low response rates in different components of a multiple schedule resulted in subsequent response-rate differences under identical variable-interval schedules. Higher response rates initially occurred in the component previously correlated with high response rates. For 3 of 4 subjects, the differences persisted for 20 or more sessions. Previous demonstrations of behavioral history effects have been confined largely to between-subject comparisons. By contrast, the present results demonstrate strong behavioral effects of schedule histories under stimulus control within individual subjects.     

Concurrent schedules: Spatial separation of response alternatives

Four pigeons were exposed to independent concurrent variable-interval 20-second variable-interval 60-second schedules of reinforcement. A transparent partition was inserted midway between the two response keys. The length of the partition was systematically manipulated. Increasing partition length produced a decrease in changeover rate in Experiment 1. Over-matching was observed with a partition length of 20 centimeters. In Experiment 2 a four-second limited hold was added to the schedules. Increasing partition length produced a decrease in changeover rate that exceeded the decrease observed in Experiment 1. This manipulation produced nearly exclusive choice of the variable-interval 20-second component. The present results, together with results obtained in related research, suggest that deviation from matching is a function of procedural variables that determine the consequences of a changeover response.     

Positive contrast, negative induction, and inhibitory stimulus control in the rat

In Experiment I, 24 rats were trained on a multiple variable-interval variable-interval schedule with a doorlight and white noise serving as component cues. Two groups were then shifted to a multiple extinction variable-interval schedule, and a third group was maintained on the multiple variable-interval variable-interval schedule. The multiple extinction variable-interval condition produced positive contrast when either the light or noise signalled extinction, and both of these cues acquired inhibitory stimulus control as measured by a combined cue test. In Experiment II, the multiple variable-interval variable-interval condition was shifted to multiple extinction variable-interval for one group, to multiple variable-time variable-interval for a second group, and was unchanged for the third group. The two experimental conditions produced identical patterns of response-rate reduction in the altered component, but the multiple extinction variable-interval condition produced positive contrast, whereas the multiple variable-time variable-interval condition did not. Subsequent combined cue and resistance to reinforcement tests revealed that the cue signalling extinction acquired stronger inhibitory stimulus control than the cue signalling variable time.     

ALTERNATIVE REINFORCEMENT INCREASES RESISTANCE TO CHANGE: PAVLOVIAN OR OPERANT CONTINGENCIES?

Two multiple-schedule experiments with pigeons examined the effect of adding food reinforcement from an alternative source on the resistance of the reinforced response (target response) to the decremental effects of satiation and extinction. In Experiment 1, key pecks were reinforced by food in two components according to variable-interval schedules and, in some conditions, food was delivered according to variable-time schedules in one of the components. The rate of key pecking in a component was negatively related to the proportion of reinforcers from the alternative (variable-time) source. Resistance to satiation and extinction, in contrast, was positively related to the overall rate of reinforcement in the component. Experiment 2 was conceptually similar except that the alternative reinforcers were contingent on a specific concurrent response. Again, the rate of the target response varied as a function of its relative reinforcement, but its resistance to satiation and extinction varied directly with the overall rate of reinforcement in the component stimulus regardless of its relative reinforcement. Together the results of the two experiments suggest that the relative reinforcement of a response (the operant contingency) determines its rate, whereas the stimulus-reinforcement contingency (a Pavlovian contingency) determines its resistance to change.     

Positive behavioral contrast when pigeons press treadles during multiple schedules

Pigeons were placed on multiple variable-interval 15-second variable-interval 15-second and on multiple variable-interval 15-second extinction schedules in which treadle presses produced food reinforcers. Positive behavioral contrast occurred. That is, rates of responding were higher during the variable-interval 15-second component when the other component was extinction than when it was another variable-interval 15-second schedule. These results contradict the findings of other studies, which failed to find positive contrast when pigeons pressed treadles for food reinforcers. They may also question the additive theories of behavioral contrast that predict that contrast should not occur in this situation.     

Matching and maximizing with variable-time schedules.

Pigeons were offered choices between a variable-time schedule that arranged reinforcers throughout the session and a variable-time schedule that arranged reinforcers only when the pigeon was spending time on it. The subjects could maximize the overall rate of reinforcement in this situation by biasing their time allocation towards the latter schedule. This arrangement provides an alternative to concurrent variable-interval variable-ratio schedules for testing whether animals maximize overall rates or match relative rates, and has the advantage of being free of the asymmetrical response requirements present with those schedules. The results were contrary to those predicted by maximizing: The bias it predicts did not appear.     

Stimulus control of schedule-induced activity in pigeons during multiple schedules

Stimulus control of schedule-induced general activity was demonstrated with pigeons using multiple schedules of response-independent food delivery. In Experiment 1, the introduction of food during a multiple variable-time 30-second variable-time 30-second schedule produced a tenfold increase in activity above the no-food baseline. Each pigeon developed stable differential activity rates during the components (correlated with red and green lights) of a multiple variable-time 30-second extinction schedule. Lengthening the extinction component from 1 to 7 minutes increased the rate differences and produced a reliable pattern of responding during S− (the stimulus correlated with extinction): Activity rate was high immediately following the change from S+ (the stimulus correlated with variable-time 30-second) to S−, then decreased abruptly and remained low throughout the middle of the interval, and subsequently showed a positively accelerated increase until the stimulus changed to S+. In Experiment 2, three pigeons were exposed to a mixed variable-time extinction schedule prior to a multiple variable-time extinction schedule. Auditory rather than visual stimuli were used to determine the generality of Experiment 1 results. The multiple- versus mixed-schedule results indicated that stimulus control of activity occurred for two of the birds, but rate differences between S+ and S− were much less than those demonstrated with visual stimuli. A direct comparison of visual and auditory stimulus control in Experiment 3 supported this conclusion. These parallels between the stimulus control of reinforced responding and that of schedule-induced activity suggest that the stimulus control of induced activity may be a factor in operant stimulus control.     

Dependency, temporal contiguity, and response-independent reinforcement

A comparison was made of the effects of variable-interval, variable-time, and tandem variable-interval fixed-time schedules on key-peck responding of pigeons. The variable-interval component of the tandem schedule retained the response-reinforcement dependency; the fixed-time component allowed the temporal proximity between responding and reinforcement to vary, constrained only by the duration of the fixed-time interval. Response rates were highest during the variable-interval and lowest during the variable-time schedule. Intermediate response rates occurred during the tandem schedule. The results of a yoked control condition showed that the effects of the tandem schedule were not due simply to changes in reinforcement distribution or frequency. The results suggest that substantial reductions in responding occur when reinforcement is response-dependent but not necessarily contiguous with the response required to produce reinforcement.     

Successive independence and behavioral contrast in a closed economy.

Two pigeons had access to multiple concurrent schedules of reinforcement for 24 hours per day in their home cages. The variable-interval schedules comprising the multiple concurrent schedules were varied across 16 conditions. In three sets of conditions, one schedule was varied while its concurrent alternative and the concurrent schedules in the other component were held constant. Behavioral contrast was observed; that is, as the rate of reinforcement arranged by the varied schedule decreased, response rates on the constant schedules typically increased. These conditions formed part of two larger sets of conditions in which the concurrent schedules in one multiple-schedule component remained constant while the concurrent schedules in the other component were varied. Successive independence was found, in that behavior allocation during the constant component did not vary as a function of the reinforcer ratios in the varied component. Successive independence between components in multiple concurrent schedules is a robust result that occurs in closed economies and under conditions that promote behavioral contrast.     

Drug effects on responding maintained by stimulus-reinforcer and response-reinforcer contingencies.

The effects of pentobarbital and d-amphetamine were assessed on key pecking by pigeons under conventional single-key multiple schedules and under two-key multiple schedules in which discriminative stimuli appeared on one key (stimulus key) while pecks on a second key (constant key) produced food. Pecks on the stimulus key had no scheduled consequences. A 60-second variable-interval schedule operated in one component of each multiple schedule: either extinction or a 60-second variable-time schedule operated in the alternate component. When the alternate-component schedule was extinction, a high rate of responding was maintained in the variable-interval component of the single-key schedule; responding on both keys was maintained in the variable-interval component of the two-key schedule. Pentobarbital increased responding in the variable-interval component of the single-key schedule and increased stimulus-key, but not constant-key responding in that component of the two-key schedule. When the alternate-component schedule was changed to variable time, responding declined in the variable-interval component of the single-key schedule; stimulus-key responding was no longer maintained under the two-key schedule. Pentobarbital decreased responding in the variable-interval component of both schedules. With an exception, d-amphetamine only decreased responding in the variable-interval component of the single- and two-key schedules both when the alternate-component schedule was extinction and when it was variable time. The results suggest that the effects of pentobarbital, but not d-amphetamine, depend on the nature of the contingency (stimulus-reinforcer, response-reinforcer) that maintains responding.     

Behavioral contrast in competitive and noncompetitive environments

Three experiments examined the effects of opportunities for an alternative response (drinking) on positive behavioral contrast of rats' food-reinforced bar pressing. In both Experiments 1 and 2 the baseline multiple variable-interval schedules were rich (variable interval 10-s), and contrast was examined both with and without a water bottle present. In Experiment 1, the rats were not water deprived. When one component of the multiple schedule was changed to extinction, the rate of bar pressing increased in the constant component (positive behavioral contrast). The magnitude of contrast was larger when the bottle was absent than when it was present, as predicted by the matching law. Drinking did not shift from the constant variable-interval component to the extinction component, as might have been expected from competition theory. In Experiment 2, the rats were water deprived. Contrast was larger when the bottle was present than when it was absent, and drinking did shift to the extinction component, as predicted by competition theory. In Experiment 3, water-deprived rats responded on leaner multiple variable-interval schedules (60-s) in the presence of a water bottle. When one component was changed to extinction, contrast did not occur, and drinking did not shift to the extinction component. The present results suggest that there are at least two different sources of behavioral contrast: “competitive” contrast, observed when an alternative response occurs with high probability, and “noncompetitive” contrast, observed when an alternative response occurs with low probability. The results, in conjunction with earlier studies, also suggest that the form of the alternative response and the rate of food reinforcement provided by the multiple schedule combine to determine the amount of contrast.     

Discrimination of a response-independent component in a multiple schedule

Pigeons were trained to respond in non-differential reinforcement pre-discrimination training, with a multiple variable-interval 1-min variable-interval 1-min schedule. Each bird then received discrimination training with a multiple variable-interval 1-min variable-time 1-min schedule. Thus, discrimination training was between response-dependent (variable-interval) and response-independent (variable-time) schedules with the rate of reinforcement equated. In Experiment I, only three sessions of non-differential reinforcement preceded discrimination training and for half the birds, a 0° line was correlated with the response-dependent schedule; for the remaining birds the 0° line was correlated with the response-independent schedule. Post-discrimination gradients of excitatory stimulus control were obtained from the former group, while the latter group showed little evidence of post-discrimination stimulus control by the 0° line. Differential responding to the variable-time schedule was not accompanied by behavioral contrast to the variable-interval schedule. In Experiment II, 20 sessions of non-differential reinforcement preceded discrimination training and the 0° line was correlated with variable-time reinforcement for each bird. Differential responding to the 0° line was accompanied by negative induction to the variable-interval schedule and by inhibitory stimulus control about the 0° line during a post-discrimination generalization test.     

Behavioral interactions in multiple variable-interval schedules

In Experiment I, two groups of four pigeons each were exposed to multiple schedules in which one component was always a variable-interval schedule with a mean interreinforcement interval of 30 or 180 seconds. The other component was either an equal variable-interval schedule or extinction. Response rates in the unchanged component always increased when reinforcement was no longer scheduled in the changed component, and decreased in seven of eight cases when the variable-interval schedule was re-introduced. The per cent rate change in the unchanged component was inversely related to the frequency of reinforcement and to the ongoing response rate in the unchanged component. Rate changes in the unchanged component were not consistently correlated with changes in any single feature of the relative-frequency interresponse-time distributions. In Experiment II, the same pigeons were exposed to variable-interval schedules and multiple variable-interval variable-interval schedules with equal mean interreinforcement intervals. Response rates were similar under both conditions.     

Successive discrimination training with equated reinforcement frequencies: failure to obtain behavioral contrast

In two experiments, pigeons were trained on two-component multiple schedules in which responding in one component (S₁) was always maintained by a variable-interval schedule. In Experiment I, low response rates were reinforced in the second (S₂) component for six master subjects. This schedule was adjusted to equate reinforcement frequencies in the two components. These subjects were compared to yoked partners, for which reinforcement in the S₂ component was made available on a variable-interval schedule whose value was determined by the master subjects. A similar procedure was used in Experiment II, where the S₂ schedule for master subjects made reinforcers contingent on the absence of responding. No evidence was found in either experiment for a behavioral contrast effect in the S₁ component attributable to response reduction in the S₂ component. A reliable contrast effect was obtained from a group of pigeons given extinction conditions in the S₂ component, which was compared to a group maintained throughout on a multiple variable-interval schedule. The results suggest that previous indications of behavioral contrast in similar situations were probably caused by uneven reinforcement distributions or reflect uncontrolled fluctuations in response rates.     

Signalled reinforcement in multiple and concurrent schedules

Five pigeons were exposed to multiple and concurrent variable-interval, variable-interval reinforcement schedules in which reinforcement availability in one component was never signalled. During certain phases of the experiment, reinforcement availability in the other component was signalled. Behavioral contrast was observed in seven of eight instances when reinforcement availability in the multiple schedules was signalled. Under the concurrent schedules in which reinforcement availability was signalled, the subjects did not always allocate more time to (prefer) the component containing non-signalled reinforcement, as would be predicted by an account of behavioral contrast holding that contrast results from the introduction of a less-preferred condition in one component of a multiple schedule.