Some effects of relative reinforcement rate and changeover delay in response-independent concurrent schedules of reinforcement

Reinforcements were arranged independently of the pigeon's behavior by concurrent variable-interval schedules. The reinforcements arranged by one of the schedules occurred when the chamber was illuminated with amber light, and the reinforcements arranged by the other schedule occurred when the chamber was illuminated with blue light. Both schedules functioned concurrently, but reinforcers were delivered by each only in the presence of the appropriate stimulus condition. A response on a white key, the only key in the chamber, alternated the stimulus condition and the effective schedule. The results of this procedure were similar to those obtained with concurrent response-dependent variable-interval schedules of reinforcement. The proportion of the total session time spent in the presence of a schedule component approximated the proportion of the total number of reinforcements in the component. Changeover rate was a decreasing function of the changeover delay and of the difference between the relative rates of reinforcement for each pair of concurrent schedules.     

Concurrent second-order schedules of reinforcement

Responses on one key (the main key) of a two-key chamber produced food according to a second-order variable-interval schedule with fixed-interval schedule components. A response on a second key (the changeover key) alternated colors on the main key and provided a second independent second-order variable-interval schedule with fixed-interval components. The fixed-interval component on one variable-interval schedule was held constant at 8 sec, while the fixed interval on the other variable-interval schedule was varied from 0 to 32 sec. Under some conditions, a brief stimulus terminated each fixed interval and generated fixed-interval patterns; in other conditions, the brief stimulus was omitted. Relative response rate and relative time deviated substantially from scheduled relative reinforcement rate and, to a lesser extent, from obtained relative reinforcement rate under both brief-stimulus and no-stimulus conditions. Matching was observed with equal components on both schedules; with unequal components, increasingly greater proportions of time and responses than the matching relation would predict were spent on the variable-interval schedule containing the shorter component. Preference for the shorter fixed interval was typically more extreme under brief-stimulus than under no-stimulus schedules. The results limit the extension of the matching relation typically observed under simple concurrent variable-interval schedules to concurrent second-order variable-interval schedules.     

Matching under nonindependent variable-ratio schedules of drug reinforcement.

Response-contingent deliveries of oral pentobarbital maintained responding of 3 rhesus monkeys during daily 3-hr sessions. Deliveries of pentobarbital were arranged under nonindependent concurrent variable-ratio variable-ratio schedules. Responses to either schedule counted toward completion of both variable-ratio schedule requirements. This schedule is similar in some respects to conventional concurrent variable-interval variable-interval schedules, in which passage of time counts toward completion of the interval value on both schedules. Restricted nonindependent concurrent variable-ratio variable-ratio schedules were also studied. On that schedule, when a drug delivery was assigned to one spout, it had to be collected before responses on the opposite spout again counted toward completion of the schedule requirements. Relative reinforcer magnitude was varied by changing the drug concentration on one schedule while keeping the drug concentration constant on the other variable-ratio schedule. Under both types of concurrent variable-ratio schedules, the relative rate of responding corresponded to the relative drug intake. Unlike earlier studies of concurrent variable-interval variable-interval intravenous cocaine reinforcement, preference was proportionate to concentration, and exclusive preferences did not develop. The relationship between relative rate of responding and relative drug intake was well described by the generalized matching law.     

Behavioral interactions in multiple variable-interval schedules

In Experiment I, two groups of four pigeons each were exposed to multiple schedules in which one component was always a variable-interval schedule with a mean interreinforcement interval of 30 or 180 seconds. The other component was either an equal variable-interval schedule or extinction. Response rates in the unchanged component always increased when reinforcement was no longer scheduled in the changed component, and decreased in seven of eight cases when the variable-interval schedule was re-introduced. The per cent rate change in the unchanged component was inversely related to the frequency of reinforcement and to the ongoing response rate in the unchanged component. Rate changes in the unchanged component were not consistently correlated with changes in any single feature of the relative-frequency interresponse-time distributions. In Experiment II, the same pigeons were exposed to variable-interval schedules and multiple variable-interval variable-interval schedules with equal mean interreinforcement intervals. Response rates were similar under both conditions.     

The role of discriminative stimuli in concurrent performances

Key pecking in pigeons was examined under concurrent and parallel arrangements of two independent and simultaneously available variable-interval schedules. Pecks on the changeover key alternated the schedule of reinforcement for responses on the main key. Under concurrent schedules, discriminative stimuli were paired with the reinforcement schedule arranged in each component and changeover responses also alternated these stimuli. Under parallel schedules, changeover responses alternated the effective reinforcement schedule, but did not change the discriminative stimulus. On concurrent procedures, changeover response rate was inversely related to the difference in reinforcement rate between the two components, whereas on parallel schedules no consistent relationship was found. With both schedules, absolute response and reinforcement rates were positively related, although for a given set of reinforcement frequencies, rates were often higher on the concurrent schedules. On concurrent schedules, relative response rates and relative times were equal to relative reinforcement rates. On parallel schedules these ratios were positively related, but response and time ratios were much smaller than were obtained with comparable concurrent schedules. This inequality was most pronounced when absolute reinforcement frequencies were lowest.     

Matching and contrast on several concurrent treadle-press schedules

Four White King pigeons pressed treadles for food reinforcement on several concurrent variable-interval variable-interval schedules. The rate of reinforcement available for responding in one of the two component schedules was held constant at 30 reinforcers per hour. The rate of reinforcement available for responding in the other was varied from 120 to 60 to 15, and then to 30 reinforcers per hour. The relative rate of responding in each component schedule equalled the relative rate of reinforcement that the component provided. And, behavioral contrast, defined as an inverse relationship between the rate of responding in the constant component and the rate of reinforcement obtained by responding in the other component, occurred for all schedules.     

Melloration and maximization of reinforcement minus costs of behavior

Eight pigeons were exposed to independent concurrent schedules. Concurrent variable-interval 60-second variable-interval 60-second schedules were presented to one group of four subjects. Following baseline training, a limited hold was added to one of the schedules and the duration of the hold was decreased in successive conditions. Concurrent variable-interval 120-second variable-interval 40-second schedules were presented to another group of four subjects. These subjects were first exposed to decreasing durations of a limited hold in the variable-interval 40-second component. After replication of the baseline, a limited hold in the variable-interval 120-second component was decreased in duration. The initial durations of the holds were determined from the subjects' responding in the baseline conditions. A duration was chosen such that approximately 25% of the scheduled reinforcers would be canceled if responding remained unchanged.

Approximate matching of time proportions and reinforcement proportions was observed when the limited hold was added to the variable-interval 60-second schedule and when the limited hold was added to the variable-interval 40-second schedule. Time proportions were less extreme than reinforcement proportions when the limited hold operated in a variable-interval 120-second schedule. Overall reinforcement rates tended to decrease with continued training in concurrent schedules with a limited hold. Absolute deviations from time matching also decreased. The results provide evidence against the principle of reinforcement maximization, and support Herrnstein and Vaughan's (1980) melioration hypothesis.

     

Choice between concurrent schedules

Six pigeons pecked for food in a three-key experiment. A subject at any time could choose the left or right key and receive reinforcement according to one two-key concurrent variable-interval variable-interval schedule of reinforcement, or it could peck the center key. A peck on the center key arranged the complementary two-key concurrent variable-interval variable-interval schedule on the left and right keys. The two different two-key concurrent schedules arranged reinforcements concurrently and were signalled by two different colors of key lights. Choice behavior in the presence of a given color conformed to the usual relationship in two-key concurrent schedules: the relative frequency of responding on a key approximately equalled the relative frequency of reinforcement on that key. Preference for a two-key concurrent schedule, which was equivalent to preference for a color, was measured by the percentage of all responses on the left and right keys in the presence of that color: this percentage approximately equalled the percentage of all reinforcements that were delivered in the presence of that color. Thus, choice between concurrent schedules conforms approximately to the same relationship as does choice between alternatives in a single concurrent schedule.     

Fixed-ratio punishment by timeout of concurrent variable-interval behavior

Pigeons' responding was maintained by two concurrently available variable-interval reinforcement schedules. A fixed-ratio punishment schedule of timeout periods from the concurrent reinforcement schedules was arranged for responding during one of the variable-interval schedules. The greater the probability of a timeout after a response on the punished variable-interval schedule (the smaller the fixed ratio that produced timeout), the greater the decline in the relative punished response rates. Relative reinforcement rates remained invariant when relative response rates declined. Both behavioral contrast and induction effects were observed on the unpunished variable-interval schedule as a function of timeout punishment of the other schedule.     

Schedule-induced licking during multiple schedules

Schedule-induced polydipsia was studied in rats bar pressing under two-component multiple schedules of food reinforcement. The first component of the multiple schedule was a variable-interval 1-min schedule throughout the experiment. The schedule comprising the second component was varied over blocks of sessions in terms of rate and magnitude of reinforcement, and was either variable-interval 3-min (one pellet), variable-interval 3-min (three pellets), variable-interval 1-min (one pellet), or extinction. Water intake per session varied with the rate of reinforcement in the schedule comprising the second component and was highest when the schedule was variable-interval 1-min. Both bar-pressing behavior and licking behavior showed behavioral interactions between the two components of the multiple schedules. With magnitude of reinforcement held constant, a matching relationship was observed between lick rate and reinforcement rate; the relative frequency of licks in the constant component matched the relative frequency of reinforcement in that component. Bar pressing, however, showed only a moderate degree of relativity matching. During the schedule-induced licking, a burst of licking followed each delivery of a pellet (post-prandial drinking). The duration of these bursts of licking was observed to be a function of the inter-reinforcement interval.     

Changeover behavior and preference in concurrent schedules

Pigeons were trained on a multiple schedule of reinforcement in which separate concurrent schedules occurred in each of two components. Key pecking was reinforced with milo. During one component, a variable-interval 40-s schedule was concurrent with a variable-interval 20-s schedule; during the other component, a variable-interval 40-s schedule was concurrent with a variable-interval 80-s schedule. During probe tests, the stimuli correlated with the two variable-interval 40-s schedules were presented simultaneously to assess preference, measured by the relative response rates to the two stimuli. In Experiment 1, the concurrently available variable-interval 20-s schedule operated normally; that is, reinforcer availability was not signaled. Following this baseline training, relative response rate during the probes favored the variable-interval 40-s alternative that had been paired with the lower valued schedule (i.e., with the variable-interval 80-s schedule). In Experiment 2, a signal for reinforcer availability was added to the high-value alternative (i.e., to the variable-interval 20-s schedule), thus reducing the rate of key pecking maintained by that schedule but leaving the reinforcement rate unchanged. Following that baseline training, relative response rates during probes favored the variable-interval 40-s alternative that had been paired with the higher valued schedule. The reversal in the pattern of preference implies that the pattern of changeover behavior established during training, and not reinforcement rate, determined the preference patterns obtained on the probe tests.     

Multiple and concurrent schedule performance: independence from concurrent and successive schedule contexts

Six pigeons were trained on multiple variable-interval schedules and performance was measured in the presence or absence of another variable-interval schedule (the common schedule) arranged concurrently with both components. Manipulations included varying the rate of reinforcement on the common schedule, leaving the common schedule unchanged while the components of the multiple schedule were varied, varying the multiple schedule components in the absence of the common schedule, and varying one component of the multiple schedule while the other component and the common schedule were unchanged. The normal rate-increasing and rate-decreasing effects of reinforcement rate increase were found, except that changing one multiple schedule component did not affect the response rate in the successively available common schedule component. Both concurrent and multiple schedule performance undermatched obtained reinforcement-rate ratios, but the degree of undermatching in multiple schedules was reliably greater. Allocation of responses between multiple schedule components was unaffected by the concurrent availability of reinforcement, and allocation of responses between concurrent schedules was unaffected by the successive availability of different reinforcement rates.     

Signalled reinforcement in multiple and concurrent schedules

Five pigeons were exposed to multiple and concurrent variable-interval, variable-interval reinforcement schedules in which reinforcement availability in one component was never signalled. During certain phases of the experiment, reinforcement availability in the other component was signalled. Behavioral contrast was observed in seven of eight instances when reinforcement availability in the multiple schedules was signalled. Under the concurrent schedules in which reinforcement availability was signalled, the subjects did not always allocate more time to (prefer) the component containing non-signalled reinforcement, as would be predicted by an account of behavioral contrast holding that contrast results from the introduction of a less-preferred condition in one component of a multiple schedule.     

Duration and rate of reinforcement as determinants of concurrent responding

The duration and frequency of food presentation were varied in concurrent variable-interval variable-interval schedules of reinforcement. In the first experiment, in which pigeons were exposed to a succession of eight different schedules, neither relative duration nor relative frequency of reinforcement had as great an effect on response distribution as they have when they are manipulated separately. These results supported those previously reported by Todorov (1973) and Schneider (1973). In a second experiment, each of seven pigeons was exposed to only one concurrent schedule in which the frequency and/or duration of reinforcement differed on the two keys. Under these conditions, each pigeon's relative rate of response closely matched the relative total access to food that each schedule provided. This result suggests that previous failures to obtain matching may be due to factors such as an insufficient length of exposure to each schedule or to the pigeons' repeated exposure to different concurrent schedules.     

Performance of humans in variable-interval avoidance schedules programmed singly, and concurrently with variable-interval schedules of positive reinforcement

Performance maintained under single variable-interval avoidance schedules, single variable-interval schedules of positive reinforcement, and concurrent schedules consisting of a variable-interval avoidance component and a variable-interval positive reinforcement component, was studied in three human subjects, using points exchangeable for money as the reinforcer. Response rate in the single variable-interval avoidance schedules was an increasing function of the frequency of monetary loss avoidance. Response rate in the single variable-interval positive reinforcement schedules was an increasing function of the frequency of obtained monetary reinforcement. In the concurrent avoidance/reinforcement schedules, the rate of responding in the avoidance component increased, and the rate of responding in the positive reinforcement schedule decreased (with one exception) as a function of the frequency of loss avoidance in the avoidance component. The logarithms of the ratios of the response rates in the two components, and the logarithms of the ratios of the times spent in the two components, were linearly related to the logarithms of the ratios of the frequency of loss avoidance in the avoidance component to the frequency of reinforcement in the positive reinforcement component. All three subjects exhibited marked undermatching of response rate ratios to reinforcement frequency ratios. The results are discussed in the context of Herrnstein's quantitative model of operant performance.     

Hill-climbing by pigeons

Pigeons were exposed to two types of concurrent operant-reinforcement schedules in order to determine what choice rules determine behavior on these schedules. In the first set of experiments, concurrent variable-interval, variable-interval schedules, key-peck responses to either of two alternative schedules produced food reinforcement after a random time interval. The frequency of food-reinforcement availability for the two schedules was varied over different ranges for different birds. In the second series of experiments, concurrent variable-ratio, variable-interval schedules, key-peck responses to one schedule produced food reinforcement after a random time interval, whereas food reinforcement occurred for an alternative schedule only after a random number of responses. Results from both experiments showed that pigeons consistently follow a behavioral strategy in which the alternative schedule chosen at any time is the one which offers the highest momentary reinforcement probability (momentary maximizing). The quality of momentary maximizing was somewhat higher and more consistent when both alternative reinforcement schedules were time-based than when one schedule was time-based and the alternative response-count based. Previous attempts to provide evidence for the existence of momentary maximizing were shown to be based upon faulty assumptions about the behavior implied by momentary maximizing and resultant inappropriate measures of behavior.     

Multiple determinants of the effects of reinforcement magnitude on free-operant response rates

Four experiments examined the effects of increasing the number of food pellets given to hungry rats for a lever-press response. On a simple variable-interval 60-s schedule, increased number of pellets depressed response rates (Experiment 1). In Experiment 2, the decrease in response rate as a function of increased reinforcement magnitude was demonstrated on a variable-interval 30-s schedule, but enhanced rates of response were obtained with the same increase in reinforcement magnitude on a variable-ratio 30 schedule. In Experiment 3, higher rates of responding were maintained by the component of a concurrent variable-interval 60-s variable-interval 60-s schedule associated with a higher reinforcement magnitude. In Experiment 4, higher rates of response were produced in the component of a multiple variable-interval 60-s variable-interval 60-s schedule associated with the higher reinforcement magnitude. It is suggested that on simple schedules greater reinforcer magnitudes shape the reinforced pattern of responding more effectively than do smaller reinforcement magnitudes. This effect is, however, overridden by another process, such a contrast, when two magnitudes are presented within a single session on two-component schedules.     

More on concurrent interval-ratio schedules: a replication and review.

It has been suggested that the failure to maximize reinforcement on concurrent variable-interval, variable-ratio schedules may be misleading. Inasmuch as response costs are not directly measured, it is possible that subjects are optimally balancing the benefits of reinforcement against the costs of responding. To evaluate this hypothesis, pigeons were tested in a procedure in which interval and ratio schedules had equal response costs. On a concurrent variable time (VT), variable ratio-time (VRT) schedule, the VT schedule runs throughout the session and the VRT schedule is controlled by responses to a changeover key that switches from one schedule to the other. Reinforcement is presented independent of response. This schedule retains the essential features of concurrent VI VR, but eliminates differential response costs for the two alternatives. It therefore also eliminates at least one significant ambiguity about the reinforcement maximizing performance. Pigeons did not maximize rate of reinforcement on this procedure. Instead, their times spent on the alternative schedules matched the relative rates of reinforcement, even when schedule parameters were such that matching earned the lowest possible overall rate of reinforcement. It was further shown that the observed matching was not a procedural artifact arising from the constraints built into the schedule.     

Interaction of frequency and magnitude of reinforcement on concurrent performances

Frequency and magnitude of reinforcement were varied in concurrent variable-interval variable-interval schedules of reinforcement. The relative response rate to the two stimuli did not support the notion that choice approximately matches relative total access to food (the product of frequency and magnitude of reinforcement in one schedule divided by the sum of products of frequency and magnitude in both schedules). Relative response rates matched relative reinforcement value when that measure was adjusted to give more emphasis to reinforcement frequency than to reinforcement duration.     

Molar And Molecular Control In Variable-interval And Variable-ratio Schedules

Response rates are typically higher under variable-ratio than under variable-interval schedules of reinforcement, perhaps because of differences in the dependence of reinforcement rate on response rate or because of differences in the reinforcement of long interresponse times. A variable-interval-with-added-linear-feedback schedule is a variable-interval schedule that provides a response rate/reinforcement rate correlation by permitting the minimum interfood interval to decrease with rapid responding. Four rats were exposed to variable-ratio 15, 30, and 60 food reinforcement schedules, variable-interval 15-, 30-, and 60-s food reinforcement schedules, and two versions of variable-interval-with-added-linear-feedback 15-, 30-, and 60-s food reinforcement schedules. Response rates on the variable-interval-with-added-linear-feedback schedule were similar to those on the variable-interval schedule; all three schedules led to lower response rates than those on the variable-ratio schedules, especially when the schedule values were 30. Also, reinforced interresponse times on the variable-interval-with-added-linear-feedback schedule were similar to those on variable interval and much longer than those produced by variable ratio. The results were interpreted as supporting the hypothesis that response rates on variable-interval schedules in rats are lower than those on comparable variable-ratio schedules, primarily because the former schedules reinforce long interresponse times.