Response-rate differences in variable-interval and variable-ratio schedules: An old problem revisited

In Experiment 1, a variable-ratio 10 schedule became, successively, a variable-interval schedule with only the minimum interreinforcement intervals yoked to the variable ratio, or a variable-interval schedule with both interreinforcement intervals and reinforced interresponse times yoked to the variable ratio. Response rates in the variable-interval schedule with both interreinforcement interval and reinforced interresponse time yoking fell between the higher rates maintained by the variable-ratio schedule and the lower rates maintained by the variable-interval schedule with only interreinforcement interval yoking. In Experiment 2, a tandem variable-interval 15-s variable-ratio 5 schedule became a yoked tandem variable-ratio 5 variable-interval x-s schedule, and a tandem variable-interval 30-s variable-ratio 10 schedule became a yoked tandem variable-ratio 10 variable-interval x-s schedule. In the yoked tandem schedules, the minimum interreinforcement intervals in the variable-interval components were those that equated overall interreinforcement times in the two phases. Response rates did not decline in the yoked schedules even when the reinforced interresponse times became longer. Experiment 1 suggests that both reinforced interresponse times and response rate–reinforcement rate correlations determine response-rate differences in variable-ratio 10 and yoked variable-interval schedules in rats. Experiment 2 suggests a minimal role for the reinforced interresponse time in determining response rates on tandem variable-interval 30-s variable-ratio 10 and yoked tandem variable-ratio 10 variable-interval x-s schedules in rats.     

A test of the effectiveness of the differential-reinforcement-of-low-rate schedule

Pigeons and rats were used in a yoked-control design that equated the reinforcement distributions of differential-reinforcement-of-low-rate and variable-interval schedules. Both a between-subjects design and a within-subjects design found response rate higher for the variable-interval schedule than for the differential-reinforcement-of-low-rate schedule, thus demonstrating the effectiveness of the differential-reinforcement-of-low-rate contingency. The interresponse-time distributions were unimodal for all subjects under the variable-interval schedule and bimodal for pigeons under the differential-reinforcement-of-low-rate schedule. The interresponse-time distributions for rats under the differential-reinforcement-of-low-rate schedule were also bimodal in three of four cases but the height of the modes at the shorter interresponse times were small in both absolute value and in relation to the height of the modes at the shorter interresponse times of the pigeons' distributions.     

The reinforcement of least-frequent interresponse times

A new schedule of reinforcement was used to maintain key-pecking by pigeons. The schedule reinforced only pecks terminating interresponse times which occurred least often relative to the exponential distribution of interresponse times to be expected from an ideal random generator. Two schedule parameters were varied: (1) the rate constant of the controlling exponential distribution and (2) the probability that a response would be reinforced, given that it met the interresponse-time contingency. Response rate changed quickly and markedly with changes in the rate constant; it changed only slightly with a fourfold change in the reinforcement probability. The schedule produced stable rates and high intra- and inter-subject reliability, yet interresponse time distributions were approximately exponential. Such local interresponse time variability in the context of good overall control suggests that the schedule may be used to generate stable, predictable, yet sensitive baseline rates. Implications for the measurement of rate are discussed.     

Behavioral interactions in multiple variable-interval schedules

In Experiment I, two groups of four pigeons each were exposed to multiple schedules in which one component was always a variable-interval schedule with a mean interreinforcement interval of 30 or 180 seconds. The other component was either an equal variable-interval schedule or extinction. Response rates in the unchanged component always increased when reinforcement was no longer scheduled in the changed component, and decreased in seven of eight cases when the variable-interval schedule was re-introduced. The per cent rate change in the unchanged component was inversely related to the frequency of reinforcement and to the ongoing response rate in the unchanged component. Rate changes in the unchanged component were not consistently correlated with changes in any single feature of the relative-frequency interresponse-time distributions. In Experiment II, the same pigeons were exposed to variable-interval schedules and multiple variable-interval variable-interval schedules with equal mean interreinforcement intervals. Response rates were similar under both conditions.     

Dynamics in the fine structure of schedule-controlled behavior.

The variability in the behavioral equilibrium established by six basic schedules was characterized. The measures were the pause preceding the first response in each interreinforcement interval; the mean rate of responding in each interreinforcement interval; and the relative frequency of each interresponse time. The temporal windows ranged across the 780-session exposure, across a session, and across the interreinforcement interval. A display of individual interresponse times as a function of time in the interreinforcement interval indicated clear recurrent responding at somewhat less than 3 Hz in every bird, even after extended exposure to a schedule and regardless of the contingency. No strong sequential dependencies in the interresponse-time distributions were identified. A simulator, based on a simple recurrent pulser, was presented that produced output similar to the obtained data. An archival data base of the behavior chronically maintained by the simple schedules was also generated.     

Parametric manipulation of interresponse-time contingency independent of reinforcement rate

Pecking of pigeons was reinforced under a modified interval-percentile procedure that allowed independent manipulation of overall reinforcement rate and the degree to which reinforcement depended on interresponse-time duration. Increasing the contingency, as measured by the phi coefficient, between reinforcement and long interresponse times while controlling the overall rate of reinforcement systematically increased the frequency of those interresponse times and decreased response rate under both of the reinforcement rates studied. Increasing reinforcement rate also generally increased response rate, particularly under weaker interresponse-time contingencies. Random-interval schedules with comparable reinforcement rates generated response rates and interresponse-time distributions similar to those obtained with moderate-to-high interresponse-time reinforcement contingencies. These results suggest that interresponse-time reinforcement contingencies inherent in random-interval and constant-probability variable-interval schedules exercise substantial control over responding independent of overall reinforcement rate effects. The interresponse-time reinforcement contingencies inherent in these schedules may actually mask the effects of overall reinforcement rate; thus differences in response rate as a function of reinforcement rate when interresponse-time reinforcement is eliminated may be underestimated.     

The reinforcement of short interresponse times

Five contingencies were superimposed successively on a variable-interval schedule of reinforcement. In each of the resulting conditions, a different short, interresponse time was reinforced and an interresponse-time distribution was obtained from each of three pigeons. The lower bound of the reinforced interresponse times ranged from 0.3 to 2.4 sec. The resulting distributions were combined, according to a rationale based upon concurrent operants, induction, and a property of variable-interval schedules, to describe the interresponse-time distributions from a variable-interval schedule.     

Choice between response rates

Three pigeons were required to peck a single key at a higher and a lower rate, corresponding to two classes of shorter and longer concurrently reinforced interresponse times. Food reinforcers arranged by a single variable-interval schedule were randomly allocated to the two reinforced interresponse times. The absolute durations of reinforced interresponse times were varied while the total reinforcements per hour was held constant and the relative duration, i.e., the relative reciprocal, of the shorter reinforcer class was held constant at 0.70. Preference for the higher rate of responding, as measured by the relative frequency of responses terminating interresponse times in the shorter reinforced class, depended on the absolute reinforced response rates. Preference for the higher reinforced rate increased from a level of near-indifference (0.50) at high reinforced response rates, through the matching level (0.70) at intermediate reinforced response rates, to a virtually exclusive preference (>0.90) at low reinforced response rates. These results resemble corresponding preference functions obtained with two-key concurrent-chains schedules and thereby provide another sense in which it may be said that interresponse-time distributions from interval schedules estimate preference functions for the component response rates corresponding to different classes of reinforced interresponse times.     

A quantitative interresponse-time analysis of DRL performance differentiates similar effects of the antidepressant desipramine and the novel anxiolytic gepirone.

We describe an interresponse-time analysis of performance on a differential-reinforcement-of-low-rate 72-s schedule. This analysis compares the obtained interresponse-time distribution of individual rats to a corresponding random interresponse-time distribution. The random interresponse-time distribution is a negative exponential probability function; it predicts the relative distribution of interresponse times if the rat emitted the same number of responses randomly (i.e., with a constant probability) with respect to time. The analysis provides quantitative measures of peak location and dispersion of the interresponse times toward random performance. In Experiment 1, an unexpected outcome of this analysis was that the rats would have obtained more reinforcers had they responded at the same rate but randomly. Based on the interresponse-time analysis in Experiment 1, it was shown that rats trained on the differential-reinforcement-of-low-rate 72-s schedule could increase the number of reinforcers obtained in two ways: first, by a coherent shift of the interresponse-time distribution toward longer durations and, second, by dispersal of the interresponse times toward a random interresponse-time distribution. Experiment 2 applied the analysis described in Experiment 1 to the effects of desipramine and gepirone. Both drugs decreased response rate and increased reinforcement rate, but their effects on the distribution of interresponse times were different. The increase in reinforcement rate observed with desipramine was accompanied by a coherent shift of the reinforcement rate observed with gepirone was accompanied by dispersal of the interresponse-time distribution toward the random negative exponential prediction.     

Mechanisms underlying the effects of unsignaled delayed reinforcement on key pecking of pigeons under variable-interval schedules.

Three experiments were conducted to test an interpretation of the response-rate-reducing effects of unsignaled nonresetting delays to reinforcement in pigeons. According to this interpretation, rates of key pecking decrease under these conditions because key pecks alternate with hopper-observing behavior. In Experiment 1, 4 pigeons pecked a food key that raised the hopper provided that pecks on a different variable-interval-schedule key met the requirements of a variable-interval 60-s schedule. The stimuli associated with the availability of the hopper (i.e., houselight and keylight off, food key illuminated, feedback following food-key pecks) were gradually removed across phases while the dependent relation between hopper availability and variable-interval-schedule key pecks was maintained. Rates of pecking the variable-interval-schedule key decreased to low levels and rates of food-key pecks increased when variable-interval-schedule key pecks did not produce hopper-correlated stimuli. In Experiment 2, pigeons initially pecked a single key under a variable-interval 60-s schedule. Then the dependent relation between hopper presentation and key pecks was eliminated by arranging a variable-time 60-s schedule. When rates of pecking had decreased to low levels, conditions were changed so that pecks during the final 5 s of each interval changed the keylight color from green to amber. When pecking produced these hopper-correlated stimuli, pecking occurred at high rates, despite the absence of a peck-food dependency. When peck-produced changes in keylight color were uncorrelated with food, rates of pecking fell to low levels. In Experiment 3, details (obtained delays, interresponse-time distributions, eating times) of the transition from high to low response rates produced by the introduction of a 3-s unsignaled delay were tracked from session to session in 3 pigeons that had been initially trained to peck under a conventional variable-interval 60-s schedule. Decreases in response rates soon after the transition to delayed reinforcement were accompanied by decreases in eating times and alterations in interresponse-time distributions. As response rates decreased and became stable, eating times increased and their variability decreased. These findings support an interpretation of the effects of delayed reinforcement that emphasizes the importance of hopper-observing behavior.     

Synthetic variable-interval schedules of reinforcement

Three pigeons pecked for food on a synthetic variable-interval schedule of reinforcement that had two independent parts: a variable-interval schedule that arranged a distribution of interreinforcement intervals, and a device that randomly assigned each reinforcement to one of 10 classes of interresponse times. The frequencies of reinforcement for the 10 classes of interresponse times were systematically varied, while the overall frequency of reinforcement was held within a comparatively narrow range. The 10 classes extended either from 0.1 to 0.6 sec in 0.05-sec intervals, or from 1.0 to 6.0 sec in 0.5-sec intervals. In the former case, some control by reinforcement was obtained, but it was weak and no simple relationships were discernible. In the latter case, the relative frequency of an interresponse time was a generally increasing function of its relative frequency of reinforcement, and two simple controlling relationships were found. First, the function relating interresponse times per opportunity to reinforcements per opportunity was, over a restricted range, approximately linear with a slope of unity. Second, when all 10 classes of interresponse times were reinforced equally often, the relative frequency of an interresponse time approximately equalled the relative reciprocal of its length.     

Maximizing present value: A model to explain why moderate response rates obtain on variable-interval schedules

In Phases 1 and 3, two Japanese monkeys responded on a multiple variable-ratio 80 variable-interval X schedule, where the value of X was adjusted to ensure equal between-schedule reinforcement rates. Components strictly alternated following the delivery of a food pellet, and each session ended following 50 components. Phase 2 differed from the others only in that the 50 pellets previously earned during the session were delivered together at session's end. Variable-ratio response rates did not decrease across phases, but variable-interval response rates decreased substantially during the Phase 2 procedure. This rate decrease was attributed to the food-at-session's-end manipulation removing the greater immediacy of reinforcement provided by short interresponse times relative to long interresponse times. Without this time preference for short interresponse times, the variable-interval interresponse-time reinforcement feedback function largely controlled response emission, dictating a response-rate reduction. This result was explained in terms of the economic notion of “maximizing present value.”     

A molecular analysis of multiple schedule interactions: negative contrast

The present experiments investigated the relationship between changes in the relative reinforced interresponse-time distributions and the occurrence of positive and negative contrast in multiple variable-interval—variable-interval and multiple variable-interval—extinction schedules of reinforcement. Experiment I demonstrated that changes in the interresponse-time distributions were consistently correlated with response-rate changes referred to as positive and negative contrast. Corresponding changes in the reinforced interresponse-time distributions suggested that negative contrast resulted as an inductive effect of selectively reinforcing long interresponse times in the altered component at the moment the baseline schedule was reintroduced. Experiment II demonstrated that the magnitude of the negative-contrast effect could be significantly decreased if the altered component schedule was modified in order to prevent the reinforcement of these interresponse times during the first few sessions of baseline recovery. The results supported a proposal that interresponse time—reinforcer relations may act as amplifiers or attenuators of negative contrast.     

Interresponse-time punishment: a basis for shock-maintained behavior.

Lever pressing of squirrel monkeys postponed brief electric shock according to a free-operant shock-postponement procedure. Pressing also produced shock with a probability proportional to the duration of the current interresponse time in some conditions, or to the fifth ordinally-preceding interresponse time in others. These conditions provided equal frequencies and temporal distributions of response-produced shocks either contingent on or independent of the current interresponse-time duration, respectively. Shock delivered contingent on the current interresponse-time duration resulted in shorter mean interresponse times and higher overall response rates that shock delivered independent of the current interresponse time. In subsequent conditions, response-produced shocks were sufficient to maintain responding following suspension of the postponement procedure only when those shocks were contingent on the current interresponse time. Presenting shock independent of the current interresponse time, conversely, suppressed response rate and ultimately led to cessation of responding in the absence of a conjoint shock-postponement procedure. These results demonstrate interresponse-time punishment in the absence of any indirect avoidance contingencies based on overall shock-frequency reduction, and strongly support similar interpretation at the more local level of shock-frequency reduction correlated with particular interresponse times. Differential punishment of long interresponse times also provides both an a priori basis for predicting whether a schedule of shock presentation will maintain or suppress responding and a framework for interpreting many of the functional relations between overall response rate and parameters of consequent shock presentation. Finally, these results and others indicate the importance of response-consequence contiguity above and beyong any notion of noncontiguous contingency in the control of behavior.     

Changes in functional response units with briefly delayed reinforcement

In two experiments, key-peck responding of pigeons was compared under variable-interval schedules that arranged immediate reinforcement and ones that arranged unsignaled delays of reinforcement. Responses during the nominal unsignaled delay periods had no effect on the reinforcer presentations. In Experiment 1, the unsignaled delays were studied using variable-interval schedules as baselines. Relative to the immediate reinforcement condition, 0.5-s unsignaled delays decreased the duration of the reinforced interresponse times and increased the overall frequency of short (<0.5-s) interresponse times. Longer, 5.0-s unsignaled delays increased the duration of the reinforced interresponse times and decreased the overall frequency of the short interresponse times. In Experiment 2, similar effects to those of Experiment 1 were obtained when the 0.5-s unsignaled delays were imposed upon a baseline schedule that explicitly arranged reinforcement of short interresponse times and therefore already generated a large number of short interresponse times. The results support earlier suggestions that the unsignaled 0.5-s delays change the functional response unit from a single key peck to a multiple key-peck unit. These findings are discussed in terms of the mechanisms by which contingencies control response structure in the absence of specific structural requirements.     

Collateral responding during differential reinforcement of low rates

Two pigeons were trained to peck either of two response keys for food, under two different variable-interval schedules. When responding stabilized, the schedule on the left key (reinforcement-key) was changed to a differential-reinforcement-of-low-rates schedule, and responses on the right key (extinction-key) were no longer reinforced. The mean interresponse time of responses on the reinforcement-key approximated the temporal requirement of the reinforcement schedule on that key. Collateral responding on the extinction-key was maintained by one of the birds. A “run” of these collateral responses was defined as a sequence of responses on the extinction-key occurring between two responses on the reinforcement-key. For this one bird, collateral behavior, measured by mean time per run and mean number of responses per run, was an increasing function of the temporal requirements of the reinforcement schedule on the reinforcement key, and it was strongly positively correlated with the mean interresponse time of responses on the reinforcement-key. However, from an analysis of the results, the collateral behavior did not appear to have mediated the temporal spacing of responses on the reinforcement-key.     

Yoked variable-ratio and variable-interval responding in pigeons

Pigeons' key pecks were maintained by variable-ratio or variable-interval schedules of food reinforcement. For pairs of pigeons in one group, variable-ratio reinforcement was arranged for one pigeon's pecks; for the second pigeon, reinforcement was arranged according to a variable-interval schedule yoked to the interreinforcement times produced by the first pigeon. For pairs of pigeons in another group, variable-interval reinforcement was arranged for one pigeon's pecks; for the second pigeon, reinforcement was arranged according to a variable-ratio schedule yoked to the interreinforcement responses produced by the first pigeon. For each pair, the yoking procedure was maintained for four or five consecutive sessions of 50 reinforcements each. In more than three-quarters of the pairs, variable-ratio response rates were higher than variable-interval rates within two sessions; in all cases, the rate difference developed within four sessions.     

Interresponse-time analysis of stimulus control in multiple schedules

Interresponse-time distributions were recorded in two components of multiple variable-interval schedules that were varied over several conditions. Values of the exponent for power functions relating ratios of interresponse times emitted per opportunity to ratios of reinforcers obtained in the two components varied with interresponse-time class interval. The exponent (sensitivity to reinforcement) afforded a measure of stimulus control exerted by the discriminative stimuli. Exponents were near zero for short interresponse times, consistent with previous conclusions that responses following short interresponse times are controlled by response-produced or proprioceptive stimuli. Values of exponents increased with longer interresponse times, indicating strong control by exteroceptive stimuli over responses following interresponse times of approximately one second or longer.     

Stimulus control and the response-reinforcement contingency

Pigeons were trained under a schedule in which reinforcement was made available at varying periods of time after a prior reinforcement. The first key peck after a reinforcer was available began a timer and a second key peck, which exceeded a specified minimal time interval, produced the reinforcer. It was shown that a contingency which contains a minimal interresponse time does not necessarily weaken stimulus control by an exteroceptive stimulus.     

Two-key concurrent paced variable-interval paced variable-interval schedules of reinforcement

Nine pigeons were used in two experiments in which a response was reinforced if a variable-interval schedule had assigned a reinforcement and if the response terminated an interresponse time within a certain interval, or class, of interresponse times. One such class was scheduled on one key, and a second class was scheduled on a second key. The procedure was, therefore, a two-key concurrent paced variable-interval paced variable-interval schedule. In Exp. I, the lengths of the two reinforced interresponse times were varied. The relative frequency of responding on a key approximately equalled the relative reciprocal of the length of the interresponse time reinforced on that key. In Exp. II, the relative frequency and relative magnitude of reinforcement were varied. The relative frequency of responding on the key for which the shorter interresponse time was reinforced was a monotonically increasing, negatively accelerated function of the relative frequency of reinforcement on that key. The relative frequency of responding depended on the relative magnitude of reinforcement in approximately the same way as it depended on the relative frequency of reinforcement. The relative frequency of responding on the key for which the shorter interresponse time was reinforced depended on the lengths of the two reinforced interresponse times and on the relative frequency and relative magnitude of reinforcement in the same way as the relative frequency of the shorter interresponse time depended on these variables in previous one-key concurrent schedules of reinforcement for two interresponse times.