Discrimination of a response-independent component in a multiple schedule

Pigeons were trained to respond in non-differential reinforcement pre-discrimination training, with a multiple variable-interval 1-min variable-interval 1-min schedule. Each bird then received discrimination training with a multiple variable-interval 1-min variable-time 1-min schedule. Thus, discrimination training was between response-dependent (variable-interval) and response-independent (variable-time) schedules with the rate of reinforcement equated. In Experiment I, only three sessions of non-differential reinforcement preceded discrimination training and for half the birds, a 0° line was correlated with the response-dependent schedule; for the remaining birds the 0° line was correlated with the response-independent schedule. Post-discrimination gradients of excitatory stimulus control were obtained from the former group, while the latter group showed little evidence of post-discrimination stimulus control by the 0° line. Differential responding to the variable-time schedule was not accompanied by behavioral contrast to the variable-interval schedule. In Experiment II, 20 sessions of non-differential reinforcement preceded discrimination training and the 0° line was correlated with variable-time reinforcement for each bird. Differential responding to the 0° line was accompanied by negative induction to the variable-interval schedule and by inhibitory stimulus control about the 0° line during a post-discrimination generalization test.     

Some determinants of inhibitory stimulus control

Interspersed reinforcement and extinction during discrimination learning generate a U-shaped gradient of inhibition about the stimulus correlated with extinction. The present work showed that extinction is not a necessary determinant of inhibitory stimulus control. In Exp. I, a reduction in the rate of reinforcement, through a shift from a multiple variable-interval 1-min variable-interval 1-min schedule to a multiple variable-interval 1-min variable-interval 5-min schedule, resulted in a post-discrimination line orientation gradient of inhibition about the stimulus correlated with the variable-interval 5-min schedule. In Exp. II, the rates of reinforcement, correlated with a pair of stimuli, were held constant during a shift from a multiple variable-interval 1-min variable-interval 1-min schedule to a multiple variable-interval 1-min differential-reinforcement-of-low-rate schedule. Inhibitory stimulus control about the stimulus correlated with the differential reinforcement of low rate was obtained. In both experiments, a reduction in the rate of responding during one stimulus and behavioral contrast during the other stimulus preceded the observation of inhibitory stimulus control.     

Discrimination learning, the peak shift, and behavioral contrast

A discrimination between two successively alternating stimuli was trained under conditions that maintained equal frequencies of reinforcement in the presence of each of the discriminative stimuli (S1 and S2) but that also reduced the rate of responding to S2. These conditions included a multiple variable-interval differential-reinforcement-of-low-rate schedule and a multiple variable-interval variable-interval schedule in which responses to S2 were punished. Whenever the rate of responding to S2 was reduced, rate of responding to S1 (behavioral contrast) increased, and the peak of a subsequently obtained generalization gradient did not occur at the expected location (between S1 and S2) but was displaced away from S2, below S1. Discrimination training in which the frequencies of reinforcement earned in S1 and S2 were not equal (variable-interval 1-min variable-interval 5-min training) produced contrast and the peak shift only if the rate of responding to S2 had been reduced, as after non-differential reinforcement in which variable-interval 1-min schedules were correlated with S1 and with S2. It was concluded that a sufficient condition for the occurrence of behavioral contrast and the peak shift was reduction of the rate of responding to one of two alternating discriminative stimuli and that a peak shift will occur only if contrast had occurred during discrimination training.     

Dimensional stimulus control following brief wavelength training

Pigeons with extensive training pecking a key illuminated by a white line then had brief training with the key illuminated by 555 nanometers. This was immediately followed by a wavelength generalization test in extinction. Dimensional stimulus control about the training wavelength increased with the duration and number of reinforcements given on variable-interval 30-sec and variable-interval 10-sec schedules in Experiment I. In Experiment II, dimensional stimulus control was obtained after only 4 min of wavelength training from birds with prior and independent discrimination training. Experiment III provided groups equated in number of reinforcers with groups in Experiment I and two 8-min duration groups. Analyses, which included results from both Experiments I and III, showed that dimensional stimulus control increased: (a) more rapidly as a function of the duration of variable-interval 10-sec than variable-interval 30-sec reinforcement; (b) at the same rate across variable-interval reinforcement schedules, as a function of the number of reinforcers available during brief wavelength training.     

Peak shift as a function of multiple schedules of reinforcement

Pigeons were trained to respond to two stimuli on the wavelength continuum, 550 nm and 570 nm, each correlated with an independent schedule of reinforcement. The multiple schedule component in effect during 550 nm (S1) was always a variable-interval 1-min. During the 570-nm stimulus (S2) the second component of the schedule was either variable-interval 30-sec, 1-min, 2-min, 5-min, or extinction for different groups of birds. Generalization gradients were obtained after this training, with the following results: (1) response rate to S1 during training was related to the reinforcement frequency associated with S2; the distribution of responding during generalization testing was a function of the schedules of reinforcement used during training and the response rates they produced. Decreases in the relative frequency of reinforcement correlated with S2 resulted in increases in the distribution shift of responses away from S2 during generalization testing.     

Behavioral contrast and response independent reinforcement

Four pigeons received pre-training that included presentation of the reinforcer independently of behavior and then baseline training on a variable-interval schedule of reinforcement. With the introduction of a multiple schedule, in which the first stimulus was associated with a response contingent and a second stimulus with a response independent, 1-min variable-interval schedule, a reduction in response rate was obtained in the second component, which was not accompanied by a behavioral contrast effect in the first component. A further three pigeons were given the same pre-training and baseline training before the introduction of an otherwise identical multiple schedule, in which no reinforcement occured in the second component. Behavioral contrast was obtained from all three subjects. The results indicated that under conditions of constant reinforcement density a reduction in responding is not a sufficient condition for the occurrence of behavioral contrast.     

Behavioral contrast in one component of a multiple schedule as a function of the reinforcement conditions operating in the following component

The key pecks of four pigeons were reinforced on a variable-interval 5-min schedule which operated in each of the four components of a multiple schedule, indicated by red, green, yellow, and blue stimuli and presented in such an order that the red stimulus always preceded the yellow and the green stimulus always preceded the blue. After establishing baseline rates, the reinforcement schedule associated with the blue and yellow components was altered so that one was now an extinction schedule and the other was a variable-interval 1-min schedule. In a second experimental stage, the blue stimulus was interchanged with the yellow so that the red stimulus preceded the blue and the green stimulus preceded the yellow. In both experimental stages the response rate in the variable-interval 5-min component that preceded the extinction component was higher than the response rate in the variable-interval 5-min component that preceded the variable-interval 1-min component. The results were discussed in relation to the importance of stimulus ordering in experiments concerned with investigating behavioral contrast.     

Aversive properties of the negative stimulus in a successive discrimination

Experiment I sought to determine if the stimulus correlated with extinction in a successive discrimination was an aversive stimulus. An escape response provided an index of aversive control. Two groups of pigeons were exposed to a multiple variable-interval 30-sec extinction schedule. For the experimental group, a single peck on a second key produced a timeout during which all lights in the chamber were dark. For the control group, pecks on the second key had no contingency. The rate of responding on the timeout key during extinction for the experimental group was higher than that of the control group during all sessions of discrimination training except the first. In Exp. II, green was correlated with variable interval 30-sec and red was correlated with variable-interval 5-min. Timeouts were obtained from variable-interval 5-min. There were more timeouts from extinction in Exp. I than from variable-interval 5-min in Exp. II. Experiment III showed that not presenting the positive stimulus reduced the number of timeouts from the negative stimulus for the two birds from Exp. I that had the highest rate of timeouts from extinction, but had little effect on the two birds that had the lowest rate of timeouts. These results suggest that in a multiple schedule, the stimulus correlated with extinction, or the lower response rate, functions as a conditioned aversive stimulus. Explanations of the timeout response in terms of extinction produced variability, displaced aggression, and stimulus change, were considered but found inadequate.     

Behavioral contrast and relative reinforcement frequency in two multiple schedules

After preliminary variable-interval training, one group of pigeons was trained on a series of multiple variable-interval low-rate reinforcement schedules, while another group was trained on a series of multiple variable-interval fixed-ratio reinforcement schedules. Contrast effects were observed as variable-interval baseline rate changed in a direction away from the change in reinforcement frequency in the other component. The effects of the variable-interval component on performance in the low-rate and fixed-ratio reinforcement components in the multiple schedules were assessed by comparing the birds' performances on each of these schedules alone. Fixed-ratio reinforcement schedules showed a susceptibility to contrast effects, low-rate reinforcement schedules did not. The rate of reinforcement in fixed-ratio schedules at which no interaction occurred in the multiple schedules was higher than that in variable-interval 1-min schedules, suggesting that pigeons may prefer time-based, rather than response-based, reinforcement.     

Effects of conditioned reinforcement frequency in an intermittent free-feeding situation,

Key-pecking intermittently produced a set of brief exteroceptive stimulus changes under two-component multiple schedules of conditioned reinforcement. Throughout the study, free access to grain was concurrently provided on an intermittent basis via a variable-interval tape. Free food presentations scheduled by the tape were delivered if no peck had been emitted for 6 sec, and the brief stimulus changes produced by responding under the multiple schedules were those which accompanied food presentation. The second component of each multiple schedule was always associated with a 1-min, variable-interval schedule of conditioned reinforcement. The schedule associated with the first component was systematically varied and conditioned reinforcement was either absent (extinction) or programmed on a 1-, 3-, 6-, or 12-min variable-interval schedule. Under these conditions, rate of responding in the manipulated component decreased monotonically with a decrease in the frequency of conditioned reinforcement. In addition, contrast effects were often obtained in the constant, second component. These results are similar to those obtained with similar multiple schedules of primary reinforcement.     

The effects of brief stimuli presented under a multiple schedule of second-order schedules

The effects of briefly presented stimuli paired or not paired with food reinforcement were investigated in the pigeon on a multiple schedule containing second-order schedules. A stimulus paired with food reinforcement was presented on a variable-interval schedule in one unit of the multiple schedule and either a stimulus not paired with food reinforcement or no stimuli were scheduled in the other unit. Response rates were highest when behavior was followed by the food-paired stimulus. Presentation of the food-paired stimulus at completion of each 1-min variable-interval component maintained a steady rate of responding between consecutive food presentations. Pausing following food reinforcement was greatest in the second-order schedule not containing the paired stimulus. Reversing the stimulus pairings led to a reversal of the relative response rates and patterns of responding for each stimulus.     

Behavioral contrast without response-rate reduction

Behavioral contrast was obtained in two experiments, which both employed a standard free-operant successive discrimination (a multiple variable-interval extinction schedule), without the occurrence of reductions of response rate in the extinction component. In Experiment I, one group of four pigeons was trained on a multiple schedule in which one stimulus was associated with a variable-interval schedule and the second stimulus with response-independent reinforcement on a free variable-interval schedule. Though by the end of this training three pigeons were responding very little to the second stimulus, when this stimulus was associated with extinction all subjects showed a contrast effect. In Experiment II, eight pigeons were trained extensively to respond to a single stimulus on a variable-interval schedule, before a second stimulus associated with extinction was introduced. This second stimulus was dissimilar to the initial stimulus and five pigeons never responded in its presence. Nevertheless, all pigeons showed a contrast effect and there was no evidence that the effect was smaller in errorless subjects or smaller than in a subsequent discrimination where all subjects made many errors. Both experiments indicated that response reduction in one component of a multiple schedule is not a necessary condition for the occurrence of contrast.     

Testing for inhibitory stimulus control with S- superimposed on S+

Pigeons learned a successive discrimination between a positive stimulus (red) correlated with a variable-interval 1-min reinforcement schedule and a negative stimulus (vertical line) correlated with either a variable-interval 5-min schedule or extinction. Transfer tests measured the rate of responding to the positive stimulus alone, to various orientations of the negative stimulus, and to the same line orientations superimposed on the positive stimulus. Although there were no gradients with minima at the training value for the negative stimulus dimension, the addition of the negative stimulus dimension to the positive stimulus always resulted in a lower response rate than that for the positive stimulus alone. The results demonstrate that an operational definition of inhibitory stimulus control that requires increased responding to stimuli more distant from a negative stimulus (along some dimension) is not always consistent with a definition that requires the suppression of responding in the presence of one stimulus, the positive stimulus, by the simultaneous presentation of another, the negative stimulus.     

Stimulus control and delayed reinforcement

Pigeons were tested for generalization along the line-orientation dimension, after being trained on various two-component multiple schedules. The first component contained either a variable-interval 1-min schedule of immediate reinforcement or an extinction schedule and was associated with a plain white key (S1). The second component contained a variable-interval 1-min schedule of delayed reinforcement and was associated with a black line on a white background (S2). The major results showed that (a) decremental gradients were obtained around the stimulus associated with the delayed reinforcement component when S1 was associated with extinction, but that incremental gradients were obtained when S1 was associated with immediate reinforcement, (b) the subjects' pretraining did not affect the generalization gradients if sufficient training on the terminal multiple schedule was provided, and (c) changing the S1 schedule from immediate reinforcement to extinction produced behavioral contrast if reinforcement was delayed for 10 sec during S2, but not if it was delayed for 20 sec.     

Antecedent reinforcement contingencies in the stimulus control of an auditory discrimination

In order to assess possible confounding of discriminative stimulus effects with those produced by the reinforcing stimulus, three groups of four rats each were trained for 45 hr on a variable-interval 1-min reinforcement program. Two groups were run on a multiple variable-interval extinction schedule in which the reinforcement stimulus (S^D) and the nonreinforcement stimulus (S^Δ) were two intensities of a 4-kHz (cps) tone separated by 40 or 10 db. The third group was run on a mixed schedule with a single intensity constantly present. The mixed-schedule animals showed no discrimination of the reinforcement program. Under the multiple schedule, the highest S^Δ rates were obtained after S^D intervals, regardless of the reinforcement availability in the S^D interval. These local rate variations in S^Δ were small in proportion to those produced by the S^D versus S^Δ intensities.     

Schedule-induced licking during multiple schedules

Schedule-induced polydipsia was studied in rats bar pressing under two-component multiple schedules of food reinforcement. The first component of the multiple schedule was a variable-interval 1-min schedule throughout the experiment. The schedule comprising the second component was varied over blocks of sessions in terms of rate and magnitude of reinforcement, and was either variable-interval 3-min (one pellet), variable-interval 3-min (three pellets), variable-interval 1-min (one pellet), or extinction. Water intake per session varied with the rate of reinforcement in the schedule comprising the second component and was highest when the schedule was variable-interval 1-min. Both bar-pressing behavior and licking behavior showed behavioral interactions between the two components of the multiple schedules. With magnitude of reinforcement held constant, a matching relationship was observed between lick rate and reinforcement rate; the relative frequency of licks in the constant component matched the relative frequency of reinforcement in that component. Bar pressing, however, showed only a moderate degree of relativity matching. During the schedule-induced licking, a burst of licking followed each delivery of a pellet (post-prandial drinking). The duration of these bursts of licking was observed to be a function of the inter-reinforcement interval.     

Response rate, reinforcement frequency, and behavioral contrast

Pigeons responded for food on a multiple schedule in which periods of green-key illumination alternated with periods of red-key illumination. When behavior had stabilized with a variable-interval 2-min schedule of reinforcement operating during both stimuli, low rates of responding (interresponse times greater than 2 sec) were differentially reinforced during the green component. Conditions during the red stimulus were unchanged. Response rates during the green component fell without changing the frequency of reinforcement but there were no unequivocal contrast effects during the red stimulus. The frequency of reinforcement during the green component was then reduced by changing to a variable-interval 8-min schedule without reducing the response rates in that component, which were held at a low level by the spacing requirement. Again, the conditions during the red stimulus were unchanged but response rates during that stimulus increased. These results show that reductions in reinforcement frequency, independently of response rate, can produce interactions in multiple schedules.     

Failure to obtain positive contrast when pigeons press a bar

In Experiment 1, two groups of pigeons were exposed to multiple variable-interval 1-min variable-interval 1-min schedules of reinforcement. The required response was a key peck for one group, and a foot press of a bar for the other. When the procedure was shifted to multiple variable-interval 1-min extinction, positive contrast occurred with pigeons that key pecked, whereas negative induction occurred with those that bar pressed. Moreover, the absence of contrast could not be ascribed to the lack of inhibition, since negative generalization gradients after bar press training were U-shaped in Experiment 2. The results are discussed in terms of possible relationships between positive contrast and the phenomenon of autoshaping.     

Blocking the development of stimulus control when stimuli indicate periods of nonreinforcement

To learn whether prior discrimination training based on one stimulus would block learning about a subsequently added stimulus, rats were first trained to press a bar on a variable-interval schedule of food reinforcement. Occasional stimuli were presented during which no reinforcement was available. Responding became suppressed in the presence of these stimuli. Stimuli could be noise, light, or a compound of noise plus light. A group trained with noise in Phase 1, then trained with the compound in Phase 2, showed less suppression to light in a subsequent test than a group that had the same compound training in Phase 2 but only variable-interval training in Phase 1. This showed that prior training with noise blocked the development of control by light during compound training. Two further groups showed that noise training following compound training did not have the same effect on control by light.     

Stimulus generalization and delay of reinforcement during one component of a multiple schedule

The key pecking of six pigeons was reinforced according to a variable-interval 1-min schedule during each of two successively presented stimuli. When the key was illuminated by a black line on a white background, reinforcement was delayed for 10 sec. When the key was illuminated by a plain white light, reinforcement was not delayed. All subjects responded at a lower rate during the presentation of the black line. A subsequent generalization test along the line-orientation dimension produced a U-shaped gradient, with the nadir located at or near the training stimulus, for each subject. These gradients suggested that the lower rate of response during the stimulus associated with delayed reinforcement may have been due to an inhibition of responding.