Sequential patterns in post-reinforcement pauses on fixed-interval schedules of food

Responding by one pigeon was reinforced with food on fixed-interval schedules of 30, 60, and 300 sec duration. A second pigeon was studied under fixed-interval durations of 60 and 300 sec. For both birds, the average post-reinforcement pause was one-half the duration of the fixed interval. Autocorrelation coefficients revealed first-order sequential dependencies in series of post-reinforcement pauses. On the 300-sec fixed-interval schedule, successive post-reinforcement tended to alternate between long and short durations. At the shorter fixed-interval durations there was less evidence of alternation sequences. A second experiment was conducted to determine if the time intervals between the first response after reinforcement and the next reinforcement (the work periods) were responsible for the alternation patterns in the series of post-reinforcement pauses. To evaluate the role of the work period, several procedures were used to modify the work period from that obtained on the fixed-interval 300-sec schedule. Adding a schedule to the fixed-interval schedule that set the minimum amount of time that could elapse between the first response after reinforcement and the next reinforcement eliminated the alternation pattern. Control schedules indicated that the elimination of alternation patterns resulted from constraints on the work period per se and not from confounded changes in the interreinforcement intervals.     

Separating the effects of interreinforcement time and number of interreinforcement responses

The relative importance of interreinforcement time and interreinforcement responses was evaluated by varying each independently. To do this, a blackout was presented after each nonreinforced response under both fixed-ratio and fixed-interval schedules of reinforcement. Manipulating the blackout duration under the fixed-ratio schedule caused interreinforcement time to vary without affecting the number of interreinforcement responses. Pigeons' post-reinforcement and post-blackout response latencies were found to increase linearly with interreinforcement time. Under the fixed-interval schedule, the same blackout manipulations changed the number of interreinforcement responses without affecting interreinforcement time. Post-reinforcement and post-blackout response latencies under this condition were approximately constant. These results suggest that responding is controlled by interreinforcement time and is not influenced by the number of responses emitted between reinforcements.     

Changing the response unit from a single peck to a fixed number of pecks in fixed-interval schedules

Each of three pigeons was studied first under a standard fixed-interval schedule. With the fixed interval held constant, the schedule was changed to a second-order schedule in which the response unit was the behavior on a small fixed-ratio schedule (first a fixed-ratio 10 and then a fixed-ratio 20 schedule). That is, every completion of the fixed-ratio schedule produced a 0.7-sec darkening of the key and reset the response count to zero for the next ratio. The first fixed-ratio completed after the fixed-interval schedule elapsed produced the 0.7-sec blackout followed immediately by food. These manipulations were carried out under two different fixed-interval durations for each bird ranging from 3 min to 12 min. The standard fixed-interval schedules produced the typical pause after reinforcement followed by responding at a moderate rate until the next reinforcement. The second-order schedules also engendered a pause after reinforcement, but responding occurred in bursts separated by brief pauses after each blackout. For a particular fixed-interval duration, post-reinforcement pauses increased slightly as the number of pecks in the response unit increased despite large differences in the rate and pattern of key pecking. Post-reinforcement pause increased with the fixed-interval duration under all response units. These data confirm that the allocation of time between pausing and responding is relatively independent of the rate and topography of responding after the pause.     

Temporal control of behavior: schedule interactions

In Experiment I the response that terminated the postreinforcement pauses occurring under a fixed-interval 60-second schedule was reinforced, if the pause duration exceeded 30 seconds. The percentage of such pauses, rather than increasing, decreased. There were complex effects on the discriminative control of the pause by the reinforcer terminating the previous fixed interval, depending on whether the fixed interval and the added reinforcer were the same or different. In Experiments II(a) and II(b), each reinforcement initiated an alternative fixed-interval interresponse-time-greater-than-t-sec schedule, the schedule values being systematically varied. When the response following a pause exceeding a given duration was reinforced, fewer such pauses occurred than when they were not reinforced, i.e., on the comparable simple fixed-interval schedule. There was no systematic relationship between mean interrinforcement interval and duration of the postreinforcement pause. The pause duration initiated by reinforcement was directly related to the dependency controlling the shortest pause at that time, regardless of changes in mean interreinforcement interval.     

The response-reinforcement dependency in fixed-interval schedules of reinforcement

Pigeons were exposed to four different schedules of food reinforcement that arranged a fixed minimum time interval between reinforcements (60 sec or 300 sec). The first was a standard fixed-interval schedule. The second was a schedule in which food was presented automatically at the end of the fixed time interval as long as a response had occurred earlier. The third and fourth schedules were identical to the first two except that the first response after reinforcement changed the color on the key. When the schedule required a peck after the interval elapsed, the response pattern consisted of a pause after reinforcement followed by responding at a high rate until reinforcement. When a response was not required after the termination of the interval, the pattern consisted of a pause after reinforcement, followed by responses and then by a subsequent pause until reinforcement. Having the first response after reinforcement change the color on the key had little effect on performance. Post-reinforcement pause duration varied with the minimum interreinforcement interval but was unaffected by whether or not a response was required after the interval elapsed.     

Conjunctive schedules of reinforcement: III. A fixed-interval adjusting fixed-ratio schedule

Key pecking of three pigeons was studied under a conjunctive schedule that specified both a fixed-interval and an adjusting fixed-ratio requirement. The fixed-interval schedule was 6 min for one pigeon and 3 min for the other two. The size of the ratio requirement was determined within each cycle of the fixed interval by the duration of the pause before responding began. The fixed-ratio value was at maximum at the start of each fixed interval and decreased linearly until the first response occurred (adjusting fixed-ratio schedule). A peck produced food when the number of responses remaining on the fixed-ratio schedule was completed and when the fixed interval had elapsed. If no response occurred during the interval, the fixed-ratio requirement decreased to one and a single response after the interval elapsed produced food. The initial value of the adjusting fixed-ratio schedule was studied over a range of 0 to 900. Increases in the adjusting fixed-ratio schedule to about 300 responses increased both pause duration and running response rate and also modified the pattern of responding from that obtained under the fixed-interval schedule. Higher values of the adjusting fixed ratio generally decreased pause duration and running response rate and also disrupted responding. Interreinforcement time under the conjunctive schedule was increased substantially when the adjusting fixed-ratio size exceeded 300 responses.     

Temporal tracking on cyclic-interval reinforcement schedules

Pigeons were exposed to four cycles per session of a schedule in which the duration of successive interreinforcement intervals differed by t-sec. A cycle was composed of seven increasing and seven decreasing intervals, from 2t to 8t sec in length. In Exp. 1, postreinforcement pause tracked interval duration on five cyclic schedules, with values of t ranging from 2 to 40 sec. Tracking was better at shorter t values, and when discriminative stimuli signalled increasing and decreasing parts of the cycle. Pooled data for the whole experiment showed postreinforcement pause to bear a power function relationship to interval length, with a smaller exponent than the comparable function for fixed-interval schedules. Tests in a second experiment showed that pigeons trained on an arithmetic progression could also track schedules in which successive intervals followed either a logarithmic or a geometric progression, although tracking was more stable in the logarithmic case.     

Reinforcement omission on fixed-interval schedules

Experiments with pigeons and rats showed that: (1) When a brief blackout was presented in lieu of reinforcement at the end of 25% of intervals on a fixed-interval 2-min schedule, response rate was reliably and persistently higher during the following 2-min intervals (omission effect). This effect was largely due to a decrease in time to first response after reinforcement omission. (2) When blackout duration was varied, within sessions, over the range 2 to 32 sec, time to first response was inversely related to the duration of the preceding blackout, for pigeons, and for rats during the first few sessions after the transition from FI 2-min to FI 2-min with reinforcement omission. Post-blackout pause was independent of blackout duration for rats at asymptote. These results were interpreted in terms of differential depressive effects of reinforcement and blackout on subsequent responding.     

Responding in the squirrel monkey under second-order schedules of shock delivery

Lever-pressing responses were maintained in the squirrel monkey when the only consequence of responding was the delivery of a response-produced electric shock, or alternatively, a brief visual stimulus that was occasionally followed by an electric shock. When shock was produced by the first response occurring after 8 min (8-min fixed-interval schedule), a period of no responding at the beginning of the interval was followed by a gradual increase in response rate during the interval. Similar rates and patterns of responding were maintained when a 1-sec visual stimulus was produced by the first response occurring after 8 min and shock delivery followed the brief stimulus. Subsequently, patterns of positively accelerated responding were engendered during individual fixed-interval components when the first response occurring after 4 min produced a 1-sec visual stimulus and shock delivery followed the second, and later the fourth, presentation of the 1-sec stimulus. When the duration of the brief stimulus was varied over a 100-fold range from 0.1 to 10.0 sec (1) mean response rates decreased monotonically as stimulus duration increased, and (2) patterns of positively accelerated responding were least variable and response rates during the initial part of each 4-min interval were lowest at a stimulus duration of 1 sec.     

Conjunctive schedules of reinforcement II: response requirements and stimulus effects

Responding of three pigeons was maintained under conjunctive fixed-ratio, fixed-interval schedules where a key peck produced food after both schedule requirements were completed. The individual schedule requirements were then successively removed and reinstated with responding maintained under the following conditions: conjunctive fixed-ratio, fixed-time; fixed-time; and fixed-interval schedules. Patterns of responding changed in accord with the successive removal of the schedule requirements. Compared to the conjunctive fixed-ratio, fixed-interval schedule, pause duration increased and response rate decreased under conjunctive fixed-ratio, fixed-time schedules and under fixed-time schedules alone. Overall mean rates of responding were highest and pause duration lowest under fixed-interval schedules. When changes in the keylight colors were correlated with completion of the fixed-ratio, the end of the fixed-interval, or both of these conditions, the pattern of responding was modified and indicated a greater degree of control by the individual schedules. Although two birds showed large increases in interreinforcement time when they were initially exposed to the conjunctive schedule, when responding stabilized this measure was largely invariant for all birds across most schedule conditions.     

Latency and frequency of responding under discrete-trial fixed-interval schedules of reinforcement

Pigeons' key pecking was studied under a number of discrete-trial fixed-interval schedules of food reinforcement. Discrete trials were presented by briefly illuminating the keylight repetitively throughout the interreinforcement interval. A response latency counterpart to the fixed-interval scallop was found, latency showing a gradual, negatively accelerated decrease across the interval. This latency pattern was largely invariant across changes in fixed-interval length, number of trials per interval, and maximum trial duration. Frequency of responding during early trials in the intervals varied, however, with different schedule parameters, being directly related to fixed-interval length, inversely related to number of trials, and complexly affected by conjoint variations of fixed-interval length and number of trials. Response latency thus was found to be simply related to elapsed time during the interval while response frequency was complexly determined by other factors as well.     

Component duration and relative response rates in multiple schedules

Pigeons were trained on a multiple variable-interval 30-sec, variable-interval 90-sec schedule with each component presented alternately for an equal (on the average) duration. This average duration of exposure to each component was varied from 5 to 300 sec. The main concern was with rate of response in the variable-interval 30-sec component relative to rate of response in the variable-interval 90-sec component. In all cases, rate of response was higher in the variable-interval 30 sec component, but the discrepancy in the rate produced by the two schedules tended to be greatest when the duration of component presentation was brief. The mean proportion of responses emitted during the variable-interval 30-sec component (responses in variable-interval 30-sec component divided by total responses) varied from about 0.60 to 0.71, where 0.75 would be expected on the basis of a matching rule, and 0.59 was that obtained by Lander and Irwin (1968). These results are in agreement with data reported by Shimp and Wheatley (1971) from a similar experiment.     

Aftereffects of reinforcement on variable-ratio schedules

On each of variable-ratio 10, 40, and 80 schedules of reinforcement, when rats' lever-pressing rates were stable, the concentration of a liquid reinforcer was varied within sessions. The duration of the postreinforcement pause was an increasing function of the reinforcer concentration, this effect being more marked the higher the schedule parameter. The running rate, calculated by excluding the postreinforcement pause, was unaffected by concentration. The duration of the postreinforcement pause increased with the schedule parameter, but the proportion of the interreinforcement interval taken up by the pause decreased. Consequently, the overall response rate was an increasing function of the schedule parameter; i.e., it was inversely related to reinforcement frequency, contrary to the law of effect. The running rate, however, decreased with the reinforcement frequency, in accord with the law of effect. When 50% of reinforcements were randomly omitted, the postomission pause was shorter than the postreinforcement pause, but the running rate of responses was not affected.     

Control of responding by stimulus duration

Pigeons were trained on a procedure in which the key was white for 30 sec, alternating with periods of darkness, or timeout. In a nondifferential training procedure, timeout duration was held constant at either 9 or 21 sec for different animals, and pecks on the white key were reinforced on a variable-interval 36-sec schedule. After 30 sessions an extinction generalization test was conducted where the duration of the timeout was varied from 3 to 27 sec. This test showed no differences in responding following timeouts of different durations. In a differential training procedure, timeout durations of either 9 or 21 sec were randomly scheduled for each animal. The variable-internal schedule was in effect following the same timeout duration as in the prior nondifferential procedure. No pecks were reinforced after the other timeout duration. In 40 sessions, differences in response rates following the two durations gradually developed. A maintained generalization procedure was then imposed in which timeout durations were varied from 3 to 27 sec, with the variable-interval schedule in effect following only the same duration as in the previous procedures. The first maintained generalization session showed that the prior differential training had established control of the animals' behavior by the timeout duration. In continued training on the maintained generalization procedure, control by the timeout duration decreased.     

Context, observing behavior, and conditioned reinforcement

Pigeons made observing responses for stimuli signalling either a fixed-interval 30-sec schedule or a fixed-ratio x schedule, where x was either 20, 30, 100, 140, or 200 and the schedules alternated at random after reinforcement. If observing responses did not occur, food-producing responses occurred to a stimulus common to both reinforcement schedules. When the fixed-interval schedule was paired with a low-value fixed ratio, i.e., 20 or 30, the presentation of the stimulus reliably signalling the fixed-ratio schedule reinforced observing behavior, but the presentation of the stimulus reliably signalling the fixed-interval schedule did not. The converse was the case when the fixed-interval schedule was paired with a large-valued fixed ratio, i.e., 100, 140, or 200. The results demonstrated that the occasional presentation of the stimulus signalling the shorter interreinforcement interval was necessary for the maintenance of observing behavior. The reinforcement relationship was a function of the schedule context and was reversed by changing the context. Taken together, the results show that the establishment and measurement of conditioned reinforcement is dependent upon the context or environment in which stimuli reliably correlated with differential events occur.     

Fixed-interval schedules of intravenous cocaine presentation in rats

Fixed-interval schedules of intravenous cocaine presentation were examined as a function of injection dose (0.32 to 0.64 mg/kg) and interval duration (200 to 400 sec) in two rats. Cocaine was found to exert a dose-related temporal control over the initiation of responding that was unaffected by the fixed-interval contingency. Fixed-interval pause duration was linearly related to injection dose and was the same duration as the interresponse time found on continuous reinforcement schedules of cocaine presentation. The fixed-interval pause remained constant with changes in interval duration. Characteristic fixed-interval patterns of responding were observed. However, overall response rates were inversely related to injection dose and directly related to interval duration. Running response rates varied unsystematically with both variables. These findings are at variance with results typically found in studies of fixed-interval food and electric shock presentation.     

A comparison of measures of responding under fixed-interval schedules

Average response rate, post-reinforcement pause, elapsed time to the fourth response, average quarter-life, and running rate were examined to see how they reflected changes in fixed-interval performance. Rats were exposed to a mixed schedule of water presentation comprising fixed-interval schedules of two durations. Changes in responding were produced by varying the duration of the shorter component. The five measures were derived only from the longer schedule component. Post-reinforcement pause, elapsed time to the fourth response in the interval, and quarter-life all showed high, positive inter-correlations (0.78<r<0.99). Running rate and post-reinforcement pause were not as highly correlated. Quarter-life reliably reflected changes in fixed-interval performance but changes in the quarter-life value did not necessarily result from similar changes in fixed-interval response pattern. The two measures that adequately described changes in response patterning were post-reinforcement pause and running rate. These two measures also had the advantage of being simple both computationally and in terms of the instrumentation involved in their recording.     

Effects Of Fixed-interval Schedule And Reinforcer Duration On Responding Reinforced By The Opportunity To Run

Two experiments investigated the effects of schedule value and reinforcer duration on responding for the opportunity to run on fixed-interval (FI) schedules in rats. In the first experiment, 8 male Wistar rats were exposed to FI 15-s, 30-s, and 60-s schedules of wheel-running reinforcement. The operant was lever pressing, and the consequence was the opportunity to run for 60 s. In the second experiment, 8 male Long-Evans rats were exposed to reinforcer durations of 15 s, 30 s, and 90 s. The schedule of reinforcement was an FI 60-s schedule. Results showed that postreinforcement pause and wheel-running rates varied systematically with reinforcer duration but not schedule value. Local lever-pressing rates decreased with reinforcer duration. Overall lever-pressing rates decreased with reinforcer duration but increased with schedule value. Although the reinforcer-duration effect is consistent with previous research, the lack a schedule effect appears to be the result of long post-reinforcement pauses following wheel-running reinforcement that render the manipulation of the interval requirement ineffective.     

Contrast effects in multiple fixed-interval reinforcement schedules

Pigeons were exposed to a multiple fixed-interval one-minute fixed-interval three-minute schedule of reinforcement following training on either a multiple fixed-interval one-minute fixed-interval one-minute schedule or a multiple fixed-interval three-minute fixed-interval three-minute schedule. For all birds, large negative local contrast effects developed during the first of four three-minute intervals in a component; response rate was depressed and postreinforcement pause lengthened in this interval. Positive local contrast effects were evident during the first of 12 one-minute intervals in a component for five of six birds; at asymptote, the pause was very short and response rate slightly elevated during this interval. Overall positive contrast was generally transient and varied considerably across subjects, while overall negative contrast effects, if they occurred, appeared only after a large number of sessions.     

The effects of number of responses on pause length with temporal variables controlled

A change in the size of a fixed-ratio schedule involves a simultaneous change in number of responses, in time to complete the ratio (work time), and in the interval between successive reinforcements (interreinforcement interval). Previous studies have suggested the importance of work time and the interreinforcement interval in controlling the length of the post-reinforcement pause. The present study sought to determine whether number of responses is also a significant factor. Pigeons were trained on a multiple fixed-ratio x fixed-ratio 2 plus timeout schedule in which the size of the fixed-ratio x was manipulated. When the work times (Experiment I) or interreinforcement intervals (Experiment II) were equated for the two components, the pause before the fixed-ratio x was longer than the pause before the fixed-ratio 2 plus timeout. As fixed-ratio x size increased, the relative difference in the lengths of the two types of pauses also increased. Because the fixed-ratio x component contained a larger number of responses than the fixed-ratio 2 plus timeout component, the relatively longer pause preceding the fixed-ratio x indicates that number of responses played a significant role in determining the length of the post-reinforcement pause.