Periodicities within a fixed-interval session

Within-session periodicities in number of responses per interval and postreinforcement pauses were investigated on fixed-interval schedules of 1, 2, and 3 minutes with rats. Postreinforcement pause values and the number of responses in successive intervals were not systematically related. The direction of change of these variables from one pair of intervals to the next revealed periodicities in that the direction of change varied more than would be expected by chance. A response prevention technique used to manipulate the length of time spent responding in an interval had little effect on the postreinforcement pause value of the next interval except when only a single response was permitted in an interval. This procedure tended to reduce the postreinforcement pause value of the next interval to an abnormally low level.     

The effects of interval duration on temporal tracking and alternation learning

On cyclic-interval reinforcement schedules, animals typically show a postreinforcement pause that is a function of the immediately preceding time interval (temporal tracking). Animals, however, do not track single-alternation schedules-when two different intervals are presented in strict alternation on successive trials. In this experiment, pigeons were first trained with a cyclic schedule consisting of alternating blocks of 12 short intervals (5 s or 30 s) and 12 long intervals (180 s), followed by three different single-alternation interval schedules: (a) 30 s and 180 s, (b) 5 s and 180 s, and (c) 5 s and 30 s. Pigeons tracked both schedules with alternating blocks of 12 intervals. With the single-alternation schedules, when the short interval duration was 5 s, regardless of the duration of the longer interval, pigeons learned the alternation pattern, and their pause anticipated the upcoming interval. When the shorter interval was 30 s, even when the ratio of short to long intervals was kept at 6:1, pigeons did not initially show anticipatory pausing-a violation of the principle of timescale invariance.     

Temporal control of periodic schedules: signal properties of reinforcement and blackout

Pigeons were exposed to periodic food-reinforcement schedules in which intervals ended with equal probability in either reinforcement or brief blackout. The effects on the pattern of key pecking of sequential probability of reinforcement, interval duration, and time to reinforcement opportunity were investigated in three experiments. The major results were: (1) at short absolute interval durations, time to reinforcement opportunity determined both postreinforcement and postblackout pause (time to first key peck within an interval); (2) at long intervals, postblackout pause was consistently shorter than postreinforcement pause, even if both events signalled the same time to the next reinforcement opportunity (omission effect); (3) when reinforcement and blackout signalled different times to the next reinforcement opportunity, within the same experiment, there was some evidence for interactions analogous to behavioral contrast.     

Aftereffects of reinforcement on variable-ratio schedules

On each of variable-ratio 10, 40, and 80 schedules of reinforcement, when rats' lever-pressing rates were stable, the concentration of a liquid reinforcer was varied within sessions. The duration of the postreinforcement pause was an increasing function of the reinforcer concentration, this effect being more marked the higher the schedule parameter. The running rate, calculated by excluding the postreinforcement pause, was unaffected by concentration. The duration of the postreinforcement pause increased with the schedule parameter, but the proportion of the interreinforcement interval taken up by the pause decreased. Consequently, the overall response rate was an increasing function of the schedule parameter; i.e., it was inversely related to reinforcement frequency, contrary to the law of effect. The running rate, however, decreased with the reinforcement frequency, in accord with the law of effect. When 50% of reinforcements were randomly omitted, the postomission pause was shorter than the postreinforcement pause, but the running rate of responses was not affected.     

Sequential patterns in post-reinforcement pauses on fixed-interval schedules of food

Responding by one pigeon was reinforced with food on fixed-interval schedules of 30, 60, and 300 sec duration. A second pigeon was studied under fixed-interval durations of 60 and 300 sec. For both birds, the average post-reinforcement pause was one-half the duration of the fixed interval. Autocorrelation coefficients revealed first-order sequential dependencies in series of post-reinforcement pauses. On the 300-sec fixed-interval schedule, successive post-reinforcement tended to alternate between long and short durations. At the shorter fixed-interval durations there was less evidence of alternation sequences. A second experiment was conducted to determine if the time intervals between the first response after reinforcement and the next reinforcement (the work periods) were responsible for the alternation patterns in the series of post-reinforcement pauses. To evaluate the role of the work period, several procedures were used to modify the work period from that obtained on the fixed-interval 300-sec schedule. Adding a schedule to the fixed-interval schedule that set the minimum amount of time that could elapse between the first response after reinforcement and the next reinforcement eliminated the alternation pattern. Control schedules indicated that the elimination of alternation patterns resulted from constraints on the work period per se and not from confounded changes in the interreinforcement intervals.     

FI length and performance on an FI FR chain schedule of reinforcement

In a chained FI FR schedule, manipulating the length of the FI component produced changes confined almost entirely to the FI performance; increasing the interval length increased the total number of responses emitted per reinforcement. The configuration of the fixed-interval scallop was clearly modified as the interval length was increased, with the larger intervals becoming flatter (i.e., a larger proportion of the total responses earlier in an interval). Measurement of the postreinforcement pause is suggested as a possible indicator of fixed-interval scalloping.     

The effects of number of responses on the postreinforcement pause in fixed-interval schedules

The present study manipulated the number of responses in a modified fixed-interval schedule by imposing a blackout after each unreinforced response during the interval. The blackout duration was varied, and the duration of the fixed interval was held constant. The subjects were initially exposed to a fixed-interval 300-sec schedule. Blackout durations of 0, 10, and 50 sec were used. Following this, a fixed-interval 30-sec schedule was used with blackout durations of 0, 1, and 5 sec. Under the fixed-interval 300-sec schedule, the number of interreinforcement responses varied over a wider range than occurred under the fixed-interval 30-sec schedule. The duration of the postreinforcement pause decreased as blackout durations were increased and number of responses decreased on the fixed-interval 300-sec schedule, but pause length did not vary with changes in blackout duration and number of responses for the fixed-interval 30-sec schedule. The differences in the effects of blackout duration and response manipulation on the two fixed-interval schedules were attributed to relatively greater changes in the number of interreinforcement responses for the fixed-interval 300-sec schedule.     

Temporal control of behavior: schedule interactions

In Experiment I the response that terminated the postreinforcement pauses occurring under a fixed-interval 60-second schedule was reinforced, if the pause duration exceeded 30 seconds. The percentage of such pauses, rather than increasing, decreased. There were complex effects on the discriminative control of the pause by the reinforcer terminating the previous fixed interval, depending on whether the fixed interval and the added reinforcer were the same or different. In Experiments II(a) and II(b), each reinforcement initiated an alternative fixed-interval interresponse-time-greater-than-t-sec schedule, the schedule values being systematically varied. When the response following a pause exceeding a given duration was reinforced, fewer such pauses occurred than when they were not reinforced, i.e., on the comparable simple fixed-interval schedule. There was no systematic relationship between mean interrinforcement interval and duration of the postreinforcement pause. The pause duration initiated by reinforcement was directly related to the dependency controlling the shortest pause at that time, regardless of changes in mean interreinforcement interval.     

Species differences in temporal control of behavior

Temporal control of rats' and pigeons' responding was analyzed and compared in detail on fixed-interval and fixed-time schedules with parameters of 30, 60, and 120 seconds. On fixed-time schedules, rats' responding decreased greatly or ceased, whereas pigeons continued to respond, especially on low schedule values. The running rate of responses (calculated by excluding the postreinforcement pause) was related to the duration of the preceding postreinforcement pause for rats but not for pigeons. Changes in response rate in successive segments of the interval were best described by normal curves. The relationship between midpoints of the normal curves and schedule value was a power function, with an exponent of less than one for pigeons but greater than one for rats. These differences could be explained in terms of a basic difference between the key-peck and lever-press responses, the two being differently affected by the response-eliciting properties of food.     

Temporal control in rats: analysis of nonlocalized effects from short interfood intervals.

The present experiment analyzed temporal control of postreinforcement pause duration during within-session changes in the criterion for reinforcement (interfood interval, IFI). Analysis of interval-by-interval changes in the pause revealed localized and nonlocalized effects from short intervals that caused specific changes in performance. In Phase 1, rats were presented with five consecutive 15-s IFIs intercalated into a series of 60-s IFIs. The 15-s set decreased the pause in adjacent and more remote 60-s intervals. In Phase 2, two sets of 15-s intervals were intercalated. The spacing between the two sets varied so that 0, 5, 10, or 15 60-s IFIs separated the sets. The postreinforcement pause tracked all changes in the IFI duration, and the localized effect from a short set extended beyond the next interval to the next few 60-s IFIs. Effects from one set, however, did not combine with a second set: Changes in the pause after two sets were the same regardless of the spacing between sets.     

Effects of reinforcement magnitude on interval and ratio schedules

Rats' lever pressing was studied on three schedules of reinforcement: fixed interval, response-initiated fixed interval, and fixed ratio. In testing, concentration of the milk reinforcer was varied within each session. On all schedules, duration of the postreinforcement pause was an increasing function of the concentration of the preceding reinforcer. The running rate (response rate calculated by excluding the postreinforcement pauses) increased linearly as a function of the preceding magnitude of reinforcement on fixed interval, showed slight increases for two of the three animals on response-initiated fixed interval, and did not change systematically on fixed ratio. In all cases, the overall response rate either declined or showed no effect of concentration. The major effect of increasing the reinforcement magnitude was in determining the duration of the following postreinforcement pause, and changes in the response rate reflected this main effect.     

The conditions for temporal tracking under interval schedules of reinforcement

On many cyclic-interval schedules, animals adjust their postreinforcement pause to follow the interval duration (temporal tracking). Six pigeons were trained on a series of square-wave (2-valued) interval schedules (e.g., 12 fixed-interval [FI] 60, 4 FI 180). Experiment 1 showed that pigeons track square-wave schedules, except those with a single long interval per cycle. Experiments 2 and 3 established that tracking and nontracking are learned and both can transfer from one cyclic schedule to another. Experiment 4 demonstrated that pigeons track a schedule with a single short interval per cycle, suggesting that a dual process--cuing and tracking--is necessary to explain behavior on these schedules. These findings suggest a potential explanation for earlier results that reported a failure to track square-wave schedules.     

TEMPORAL CONTROL ON INTERVAL SCHEDULES: WHAT DETERMINES THE POSTREINFORCEMENT PAUSE?

On fixed-interval or response-initiated delay schedules of reinforcement, the average pause following food presentation is proportional to the interfood interval. Moreover, when a number of intervals of different durations occur in a programmed cyclic series, postreinforcement pauses track the changes in interval value. What controls the duration of postreinforcement pauses under these conditions? Staddon, Wynne, and Higa (1991), in their linear waiting model, propose control by the preceding interfood interval. Another possibility is that delay to reinforcement, signaled by a key peck and/or stimulus change, determines the subsequent pause. The experiments reported here examined the role of these two possible time markers by studying the performance of pigeons under a chained cyclic fixed-interval procedure. The data support the linear waiting model, but suggest that more than the immediately preceding interfood interval plays a role in temporal control.     

Fixed-interval performance: The dynamics of behavior and the interval length

Postreinforcement pauses from successive intervals under various fixed-interval schedules (ranging from 15 seconds to 480 seconds in length) were subjected to lag-1 autocorrelation analysis. Results from both rats and pigeons suggested that there was a consistent tendency for pause values in successive intervals to be weakly positively related. This tendency did not appear to change systematically with interval length and was exhibited both when the reinforcer magnitude was constant and when it was variable at different interval values. The findings do not support suggestions that the dynamic properties of performance under fixed-interval schedules vary systematically with interval length, and are in the opposite direction from some previous findings suggesting that measures of behavior (such as post-reinforcement pause length or number of responses) in successive intervals are inversely related.     

Food deliveries during the pause on fixed-interval schedules

Pigeons were trained on fixed-interval schedules of food delivery. In Experiments I and II, the fixed interval was initiated by the previous fixed-interval reinforcer; in Experiment III, the fixed interval was initiated by the first key peck following the preceding fixed-interval reinforcer (a chain fixed-ratio one, fixed-interval schedule). During the postreinforcement pause, variable-time schedules delivered food independent of any specific response. Rate of food delivery during the pause had only small effects on pause duration in Experiments I and II. In Experiment III, however, pause duration increased systematically with the rate of food delivery during the pause. These data suggest that the momentary proximity to reinforcement delivered via the fixed-interval schedule exerts potent control over pause termination. Additional analysis revealed that pause termination was unaffected by the intermittent delivery of food during the pause. Such data suggest that the temporal control by fixed-interval schedules is highly resistant to interference.     

EFFECT OF REINFORCER MAGNITUDE ON PERFORMANCE MAINTAINED BY PROGRESSIVE‐RATIO SCHEDULES

This experiment examined the relationship between reinforcer magnitude and quantitative measures of performance on progressive‐ratio schedules. Fifteen rats were trained under a progressive‐ratio schedule in seven phases of the experiment in which the volume of a 0.6‐M sucrose solution reinforcer was varied within the range 6–300 μl. Overall response rates in successive ratios conformed to a bitonic equation derived from Killeen's (1994) Mathematical Principles of Reinforcement. The “specific activation” parameter, a, which is presumed to reflect the incentive value of the reinforcer, was a monotonically increasing function of reinforcer volume; the “response time” parameter, δ, which defines the minimum response time, increased as a function of reinforcer volume; the “currency” parameter, b, which is presumed to reflect the coupling of responses to the reinforcer, declined as a function of volume. Running response rate (response rate calculated after exclusion of the postreinforcement pause) decayed monotonically as a function of ratio size; the index of curvature of this function increased as a function of reinforcer volume. Postreinforcement pause increased as a function of ratio size. Estimates of a derived from overall response rates and postreinforcement pauses showed a modest positive correlation across conditions and between animals. Implications of the results for the quantification of reinforcer value and for the use of progressive‐ratio schedules in behavioral neuroscience are discussed.     

Temporal control of behavior and the power law

The performance of rats and pigeons under fixed-interval schedules was studied in two experiments. The duration of postreinforcement pause was a declining proportion of fixed-interval duration. For pigeons this was true both when the duration of the reinforcer was fixed and when it was increased in direct proportion to increases in fixed-interval duration; the longer reinforcer durations did, however, lengthen the postreinforcement pause at higher schedule values. A quantitative analysis of data from Experiments 1 and 2 and from other studies showed that fractional exponent power functions described the relationship between postreinforcement pause and fixed-interval value; similar functions have previously been observed in studies of temporal differentiation. It was concluded that power functions reflect a direct causal, rather than artifactual, relationship between performance and the temporal requirements of reinforcement schedules.     

Development and maintenance of attack in pigeons during variable-interval reinforcement of key pecking

Key-peck responses of two pigeons were maintained on variable-interval schedules of food reinforcement in the presence of a stuffed pigeon to study the characteristics of attack induced by that schedule. The mean interval of the schedule was increased from 15 sec to 600 sec in eight steps before an intermediate interval was reintroduced. The principal characteristics of attack were: (1) substantial attack first occurred on a variable-interval schedule of 90 sec in one pigeon and at 180 sec in the other, (2) the highest attack rates occurred on variable-interval schedules of 300 sec and 600 sec, (3) attack rate generally increased to a maximum and then decreased to a lower level across sessions at each schedule, (4) attacks developed a postreinforcement locus across the initial sessions on all schedules and, except on variable-interval schedules of 300 and 600 sec, occurred primarily in the postreinforcement period during extended training, (5) attack rates and key-peck rates were not recovered when the intermediate-length schedules were reintroduced, and (6) attack rate and key-peck rates were negatively correlated. Except for the fact that the maximum attack rates occurred at interfood intervals of 300 and 600 sec, and that attack and key-peck rates were negatively correlated, these findings have counter-parts in experiments with other reinforcement schedules.     

The role of verbal behavior in human learning: infant performance on fixed-interval schedules

The performances of two infants less than one year old were investigated on fixed-interval schedules. When the infants touched a cylinder either music or food was presented according to fixed-interval schedules ranging in value from 10 to 50 seconds. With respect to two principal criteria, namely, pattern of responding and sensitivity to the schedule parameter, the subjects' behavior closely resembled that of animals but differed markedly from that of older children and adults. Negatively accelerated responding in the course of the fixed interval in the early sessions gave way to a scalloped pattern, consisting of a pause after reinforcement followed by an accelerated response rate. This scalloped pattern was the final form of responding on all schedule values. Analysis of data after performance had stabilized showed that postreinforcement pause was a negatively accelerated increasing function, and running rate (calculated after excluding the postreinforcement pause) was a declining function, of schedule value. On each schedule, the durations of mean successive interresponse times declined in the course of the fixed interval and were directly related to schedule value. The results supported Lowe's (1979) suggestion that verbal behavior may be responsible for major differences in the schedule performance of older humans and animals.