Response-rate differences in variable-interval and variable-ratio schedules: An old problem revisited

In Experiment 1, a variable-ratio 10 schedule became, successively, a variable-interval schedule with only the minimum interreinforcement intervals yoked to the variable ratio, or a variable-interval schedule with both interreinforcement intervals and reinforced interresponse times yoked to the variable ratio. Response rates in the variable-interval schedule with both interreinforcement interval and reinforced interresponse time yoking fell between the higher rates maintained by the variable-ratio schedule and the lower rates maintained by the variable-interval schedule with only interreinforcement interval yoking. In Experiment 2, a tandem variable-interval 15-s variable-ratio 5 schedule became a yoked tandem variable-ratio 5 variable-interval x-s schedule, and a tandem variable-interval 30-s variable-ratio 10 schedule became a yoked tandem variable-ratio 10 variable-interval x-s schedule. In the yoked tandem schedules, the minimum interreinforcement intervals in the variable-interval components were those that equated overall interreinforcement times in the two phases. Response rates did not decline in the yoked schedules even when the reinforced interresponse times became longer. Experiment 1 suggests that both reinforced interresponse times and response rate–reinforcement rate correlations determine response-rate differences in variable-ratio 10 and yoked variable-interval schedules in rats. Experiment 2 suggests a minimal role for the reinforced interresponse time in determining response rates on tandem variable-interval 30-s variable-ratio 10 and yoked tandem variable-ratio 10 variable-interval x-s schedules in rats.     

Preference for unsegmented interreinforcement intervals in concurrent chains

Five pigeons were trained under concurrent-chain schedules in which a pair of independent, concurrent variable-interval 60-s schedules were presented in the initial link and either both variable-interval or both fixed-interval schedules were presented in the terminal link. Except for the baseline, one of the terminal-link schedules was always a two-component chained schedule and the other was either a simple or a tandem schedule of equal mean interreinforcement interval. The values of the fixed-interval schedules were either 15 s or 60 s; that of the variable-interval schedules was always 60 s. A 1.5-s changeover delay operated during the initial link in some conditions. The pigeons preferred a simple or a tandem schedule to a chain. For the fixed-interval schedules, this preference was greater when the fixed interval was 60 s than when it was 15 s. For the variable-interval schedules, the preferences were less pronounced and occurred only when the changeover delay was in effect. For a given type of schedule and interreinforcement interval, similar preferences were obtained whether the nonchained schedule was a tandem or simple schedule. The changeover delay generally inflated preference and lowered the changeover rate, especially when the terminal-link schedules were either short (15 s) or aperiodic (variable-interval). The results were consistent with the notion that segmenting the interreinforcement interval of a schedule into a chain lowers the preference for it.     

A local-rate-of-response and interresponse-time analysis of behavioral contrast

Three pigeons were exposed to a two-component multiple schedule in which a variable-interval 3-min schedule was always in effect in one component. The schedule in the other component was either variable-interval 3-min or extinction in alternate blocks of sessions. When the schedule was changed from multiple variable-interval 3-min variable-interval 3-min to multiple variable-interval 3-min extinction in the second and fourth phases of the experiment, overall response rates in the unchanged variable-interval 3-min component increased in two pigeons. Response rate declined when the schedule was changed to multiple variable-interval 3-min variable-interval 3-min again. Correlated with increases in overall response rate in the unchanged component were increases in local response rates at the beginning of the unchanged component and immediately after food presentation. Local rates 40 sec after food presentation did not increase greatly in the presence of the multiple variable-interval 3-min extinction schedule. An interresponse time analysis of three local rate samples showed small increases in the relative frequency of short-duration interresponse times at the beginning of the unchanged component and immediately after food presentation. Neither the postreinforcement pause nor the latency to the first response in the unchanged component changed systematically.     

Sustained behavioral contrast in children

Children were exposed to a multiple schedule involving equal variable-interval schedules in each of two components and a multiple schedule involving a variable-interval schedule in one component and an extinction schedule in the other. Response rates were equal in both components when each involved a variable-interval schedule. Response rates differed in the two components of the multiple variable-interval extinction schedule. Response rates were higher in the variable-interval schedule when the accompanying schedule was extinction than when it was variable interval. The increase in response rate in the variable-interval component, simultaneous with the decrease in response rate in the extinction component, illustrated sustained behavioral contrast, and was the first evidence of this phenomenon in children.     

Interactions in multiple schedules: negative induction with squirrel monkeys

In Experiment I, lever pressing by squirrel monkeys was maintained under a sequence of variable-interval, multiple variable-interval variable-interval, and multiple variable-interval extinction schedules of food presentation. Negative induction (decreased responding in the unchanged component) occurred when one component of the multiple variable-interval variable-interval schedule was changed to extinction. Negative induction was transient over sessions; responding in the unchanged component usually recovered to a rate similar to that under the multiple variable-interval variable-interval schedule. Negative induction was not accompanied by consistent changes in the patterns of local responding within the unchanged component, and did not depend on whether component schedules were associated with localized (lever lights) or diffuse visual stimuli (houselights), or on whether the unchanged component was a 60- or 180-sec variable-interval schedule. In Experiment II, responding was maintained under a sequence of variable-interval and multiple variable-interval timeout schedules of food presentation. Negative induction occurred when responding declined gradually in the timeout component but not when responding declined abruptly. The nature of interactions in multiple schedules may depend on the species; negative induction was observed with squirrel monkeys under conditions similar to those that produce positive contrast with pigeons.     

Signalled reinforcement in multiple and concurrent schedules

Five pigeons were exposed to multiple and concurrent variable-interval, variable-interval reinforcement schedules in which reinforcement availability in one component was never signalled. During certain phases of the experiment, reinforcement availability in the other component was signalled. Behavioral contrast was observed in seven of eight instances when reinforcement availability in the multiple schedules was signalled. Under the concurrent schedules in which reinforcement availability was signalled, the subjects did not always allocate more time to (prefer) the component containing non-signalled reinforcement, as would be predicted by an account of behavioral contrast holding that contrast results from the introduction of a less-preferred condition in one component of a multiple schedule.     

Positive interaction (induction) in multiple variable-interval, differential-reinforcement-of-high-rate schedules

The effect of increases in the rate of responding in one component of a multiple schedule upon the rate of responding in a second component was investigated. Pigeons were exposed to a multiple schedule where both components were initially variable-interval schedules having the same parameter value. After rate of key pecking stabilized, one component was changed to a schedule that differentially reinforced high rates of responding. Rate of reinforcement in this varied component was adjusted to remain equal to rate of reinforcement in the constant (variable-interval) component. Four of five pigeons showed a maintained increase in rate of responding during both the constant and varied components, even though rates of reinforcement did not change.     

Dependency, temporal contiguity, and response-independent reinforcement

A comparison was made of the effects of variable-interval, variable-time, and tandem variable-interval fixed-time schedules on key-peck responding of pigeons. The variable-interval component of the tandem schedule retained the response-reinforcement dependency; the fixed-time component allowed the temporal proximity between responding and reinforcement to vary, constrained only by the duration of the fixed-time interval. Response rates were highest during the variable-interval and lowest during the variable-time schedule. Intermediate response rates occurred during the tandem schedule. The results of a yoked control condition showed that the effects of the tandem schedule were not due simply to changes in reinforcement distribution or frequency. The results suggest that substantial reductions in responding occur when reinforcement is response-dependent but not necessarily contiguous with the response required to produce reinforcement.     

Multiple and concurrent schedule performance: independence from concurrent and successive schedule contexts

Six pigeons were trained on multiple variable-interval schedules and performance was measured in the presence or absence of another variable-interval schedule (the common schedule) arranged concurrently with both components. Manipulations included varying the rate of reinforcement on the common schedule, leaving the common schedule unchanged while the components of the multiple schedule were varied, varying the multiple schedule components in the absence of the common schedule, and varying one component of the multiple schedule while the other component and the common schedule were unchanged. The normal rate-increasing and rate-decreasing effects of reinforcement rate increase were found, except that changing one multiple schedule component did not affect the response rate in the successively available common schedule component. Both concurrent and multiple schedule performance undermatched obtained reinforcement-rate ratios, but the degree of undermatching in multiple schedules was reliably greater. Allocation of responses between multiple schedule components was unaffected by the concurrent availability of reinforcement, and allocation of responses between concurrent schedules was unaffected by the successive availability of different reinforcement rates.     

Yoked variable-ratio and variable-interval responding in pigeons

Pigeons' key pecks were maintained by variable-ratio or variable-interval schedules of food reinforcement. For pairs of pigeons in one group, variable-ratio reinforcement was arranged for one pigeon's pecks; for the second pigeon, reinforcement was arranged according to a variable-interval schedule yoked to the interreinforcement times produced by the first pigeon. For pairs of pigeons in another group, variable-interval reinforcement was arranged for one pigeon's pecks; for the second pigeon, reinforcement was arranged according to a variable-ratio schedule yoked to the interreinforcement responses produced by the first pigeon. For each pair, the yoking procedure was maintained for four or five consecutive sessions of 50 reinforcements each. In more than three-quarters of the pairs, variable-ratio response rates were higher than variable-interval rates within two sessions; in all cases, the rate difference developed within four sessions.     

The relation between response rates and reinforcement rates in a multiple schedule

In a multiple schedule, exteroceptive stimuli change when the reinforcement schedule is changed. Each performance in a multiple schedule may be considered concurrent with other behavior. Accordingly, two variable-interval schedules of reinforcement were arranged in a multiple schedule, and a third, common variable-interval schedule was programmed concurrently with each of the first two. A quantitative statement was derived that relates as a ratio the response rates for the first two (multiple) variable-interval schedules. The value of the ratio depends on the rates of reinforcement provided by those schedules and the reinforcement rate provided by the common variable-interval schedule. The following implications of the expression were evaluated in an experiment with pigeons: (a) if the reinforcement rates for the multiple variable-interval schedules are equal, then the ratio of response rates is unity at all reinforcement rates of the common schedule; (b) if the reinforcement rates for the multiple schedules are unequal, then the ratio of response rates increases as the reinforcement rate provided by the common schedule increases; (c) the limit of the ratio is equal to the ratio of the reinforcement rates. Satisfactory confirmation was obtained for the first two implications, but the third was left in doubt.     

Response strength in multiple periodic and aperiodic schedules

Responding in multiple periodic and aperiodic schedules of equal mean reinforcement rate was examined during extinction, satiation, and in the presence of various free-food schedules. In Experiments I and II, pigeons were trained on multiple variable-interval–fixed-interval schedules. Decreases in the rate of responding due to extinction, satiation, or food schedules were approximately equal regardless of the temporal pattern of reinforcer presentation. In Experiment III, pigeons responded on a two-component multiple schedule in which each component was a two-member homogeneous response chain terminating in a fixed-interval schedule during one component and in a variable-interval schedule during the other. The length of both terminal links was varied over a series of conditions. Initial-link responding in the fixed-interval component was reduced more by increasing terminal-link length than was initial-link responding in the variable-interval component. However, no differences in resistance to satiation and extinction were obtained across the fixed and variable components. If the relative decrease in responding produced by satiation and extinction is used as an index of the “value” of the conditions maintaining responding, then these data suggest that fixed and variable schedules of equal mean length are equally valued. This conclusion, however, is not consistent with findings of preference for variable over fixed schedules obtained in studies using concurrent-chain procedures.     

Melloration and maximization of reinforcement minus costs of behavior

Eight pigeons were exposed to independent concurrent schedules. Concurrent variable-interval 60-second variable-interval 60-second schedules were presented to one group of four subjects. Following baseline training, a limited hold was added to one of the schedules and the duration of the hold was decreased in successive conditions. Concurrent variable-interval 120-second variable-interval 40-second schedules were presented to another group of four subjects. These subjects were first exposed to decreasing durations of a limited hold in the variable-interval 40-second component. After replication of the baseline, a limited hold in the variable-interval 120-second component was decreased in duration. The initial durations of the holds were determined from the subjects' responding in the baseline conditions. A duration was chosen such that approximately 25% of the scheduled reinforcers would be canceled if responding remained unchanged.

Approximate matching of time proportions and reinforcement proportions was observed when the limited hold was added to the variable-interval 60-second schedule and when the limited hold was added to the variable-interval 40-second schedule. Time proportions were less extreme than reinforcement proportions when the limited hold operated in a variable-interval 120-second schedule. Overall reinforcement rates tended to decrease with continued training in concurrent schedules with a limited hold. Absolute deviations from time matching also decreased. The results provide evidence against the principle of reinforcement maximization, and support Herrnstein and Vaughan's (1980) melioration hypothesis.

     

On the effects of component durations and component reinforcement rates in multiple schedules

Four experiments, each using the same six pigeons, investigated the effects of varying component durations and component reinforcement rates in multiple variable-interval schedules. Experiment 1 used unequal component durations in which one component was five times the duration of the other, and the shorter component was varied over conditions from 120 seconds to 5 seconds. The schedules were varied over five values for each pair of component durations. Sensitivity to reinforcement rate changes was the same at all component durations. In Experiment 2, both component durations were 5 seconds, and the schedules were again varied using both one and two response keys. Sensitivity to reinforcement was not different from the values found in Experiment 1. In Experiment 3, various manipulations, including body-weight changes, reinforcer duration changes, blackouts, hopper lights correlated with keylights, and overall reinforcement rate changes were carried out. No reliable increase in reinforcement sensitivity resulted from any manipulation. Finally, in Experiment 4, reinforcement rates in the two components were kept constant and unequal, and the component durations were varied. Shorter components produced significantly increased response rates normally in the higher reinforcement rate component, but schedule reversals at short component durations eliminated the response rate increases. The effects of component duration on multiple schedule performance cannot be interpreted as changing sensitivity to reinforcement nor to changing bias.     

Successive discrimination training with equated reinforcement frequencies: failure to obtain behavioral contrast

In two experiments, pigeons were trained on two-component multiple schedules in which responding in one component (S₁) was always maintained by a variable-interval schedule. In Experiment I, low response rates were reinforced in the second (S₂) component for six master subjects. This schedule was adjusted to equate reinforcement frequencies in the two components. These subjects were compared to yoked partners, for which reinforcement in the S₂ component was made available on a variable-interval schedule whose value was determined by the master subjects. A similar procedure was used in Experiment II, where the S₂ schedule for master subjects made reinforcers contingent on the absence of responding. No evidence was found in either experiment for a behavioral contrast effect in the S₁ component attributable to response reduction in the S₂ component. A reliable contrast effect was obtained from a group of pigeons given extinction conditions in the S₂ component, which was compared to a group maintained throughout on a multiple variable-interval schedule. The results suggest that previous indications of behavioral contrast in similar situations were probably caused by uneven reinforcement distributions or reflect uncontrolled fluctuations in response rates.     

Relative and absolute density of reinforcement as factors influencing the peak shift

Four groups of six pigeons each were given nondifferential training on multiple variable-interval variable-interval reinforcement schedules and then were switched to differential training involving a multiple schedule in which reinforcement density was reduced in one of the two components. The multiple schedules used in the four groups had mean interreinforcement intervals of 1 min and 1 min in the two components changed to 1 min, 5 min; 2.5 min, 2.5 min changed to 2.5 min, 5 min; 12 sec, 12 sec changed to 12 sec, 24 sec; and 12 sec, 12 sec changed to 12 sec, 60 sec. In subsequently administered wavelength generalization tests, some peak shifts were observed in each condition and occurred occasionally in the absence of behavioral contrast or rate reduction in the less-reinforced component. The best predictor of peak shift was a high proportion of total responses emitted during the more-reinforced component at the end of differential training.     

Choice and transformed interreinforcement intervals

Pigeons chose between two aperiodic, time-based schedules of reinforcement. The arithmetic mean interreinforcement interval of the first schedule was short, but the harmonic mean was long, whereas the arithmetic mean interreinforcement interval of the second schedule was long, but the harmonic mean was short. The pigeons preferred the schedule with the shorter harmonic mean in a concurrent-chains procedure when a terminal link ended after the first scheduled reinforcer had been gained on a terminal-link entry, but reversed their preferences, such that they preferred the schedule with the shorter arithmetic mean, when the terminal links ended after a fixed duration of exposure to the schedule. Moreover, the pigeons preferred the schedule with the shorter arithmetic mean in a two-key concurrent variable-interval variable-interval procedure, as well as in a concurrent variable-time variable-time, changeover-key procedure. The data suggest that an aggregate property of a schedule may not yield valid information about the responding that schedule will maintain as a choice alternative.     

Tests of behavior momentum in simple and multiple schedules with rats and pigeons.

Four experiments examined the relationship between rate of reinforcement and resistance to change in rats' and pigeons' responses under simple and multiple schedules of reinforcement. In Experiment 1, 28 rats responded under either simple fixed-ratio, variable-ratio, fixed-interval, or variable-interval schedules; in Experiment 2, 3 pigeons responded under simple fixed-ratio schedules. Under each schedule, rate of reinforcement varied across four successive conditions. In Experiment 3, 14 rats responded under either a multiple fixed-ratio schedule or a multiple fixed-interval schedule, each with two components that differed in rate of reinforcement. In Experiment 4, 7 pigeons responded under either a multiple fixed-ratio or a multiple fixed-interval schedule, each with three components that also differed in rate of reinforcement. Under each condition of each experiment, resistance to change was studied by measuring schedule-controlled performance under conditions with prefeeding, response-independent food during the schedule or during timeouts that separated components of the multiple schedules, and by measuring behavior under extinction. There were no consistent differences between rats and pigeons. There was no direct relationship between rates of reinforcement and resistance to change when rates of reinforcement varied across successive conditions in the simple schedules. By comparison, in the multiple schedules there was a direct relationship between rates of reinforcement and resistance to change during most tests of resistance to change. The major exception was delivering response-independent food during the schedule; this disrupted responding, but there was no direct relationship between rates of reinforcement and resistance to change in simple- or multiple-schedule contexts. The data suggest that rate of reinforcement determines resistance to change in multiple schedules, but that this relationship does not hold under simple schedules.     

Multiple determinants of the effects of reinforcement magnitude on free-operant response rates

Four experiments examined the effects of increasing the number of food pellets given to hungry rats for a lever-press response. On a simple variable-interval 60-s schedule, increased number of pellets depressed response rates (Experiment 1). In Experiment 2, the decrease in response rate as a function of increased reinforcement magnitude was demonstrated on a variable-interval 30-s schedule, but enhanced rates of response were obtained with the same increase in reinforcement magnitude on a variable-ratio 30 schedule. In Experiment 3, higher rates of responding were maintained by the component of a concurrent variable-interval 60-s variable-interval 60-s schedule associated with a higher reinforcement magnitude. In Experiment 4, higher rates of response were produced in the component of a multiple variable-interval 60-s variable-interval 60-s schedule associated with the higher reinforcement magnitude. It is suggested that on simple schedules greater reinforcer magnitudes shape the reinforced pattern of responding more effectively than do smaller reinforcement magnitudes. This effect is, however, overridden by another process, such a contrast, when two magnitudes are presented within a single session on two-component schedules.     

An analysis of reinforcement history effects

Four pigeons were exposed to two tandem variable-interval differential-reinforcement-of-low-rate schedules under different stimulus conditions. The values of the tandem schedules were adjusted so that reinforcement rates in one stimulus condition were higher than those in the other, even though response rates in the two conditions were nearly identical. Following this, a fixed-interval schedule of either shorter or longer values than, or equal to the baseline schedule, was introduced in the two stimulus conditions respectively. Response rates during those fixed-interval schedules typically were higher in the presence of the stimuli previously correlated with the lower reinforcement rates than were those in the presence of the stimuli previously correlated with the higher reinforcement rates. Such effects of the reinforcement history were most prominent when the value of the fixed-interval schedule was shorter. The results are consistent with both incentive contrast and response strength conceptualizations of related effects. They also suggest methods for disentangling the effects of reinforcement rate on subsequent responding, from the response rate with which it is confounded in many conventional schedules of reinforcement.