Preference for fixed-interval schedules of reinforcement

Pigeons were trained on a two-link concurrent chain schedule in which responding on either of two keys in the initial link occasionally produced a terminal link, signaled by a change in the color of that key and a darkening of the other. Further responding on the lighted key was reinforced with food according to a fixed-interval schedule. For one of the keys, this fixed interval was always 20 sec, while for the other it was held at values of 5, 14, 30, or 60 sec for several weeks. In the initial link, all pigeons responded relatively more often on the key with the shorter fixed interval than was predicted by the matching hypothesis. Responding in the initial link showed a large negative recency effect: pigeons responded less frequently on the key that provided their last reinforcement than predicted from the overall response rates.     

Alternative fixed-ratio fixed-interval schedules of reinforcement

Five rats were trained under alternative fixed-ratio fixed-interval schedules, in which food reinforcement was provided for the completion of either a fixed-ratio or a fixed-interval requirement, whichever was met first. Overall response rate and running rate (the rate of responding after the postreinforcement pause) decreased for all subjects as the fixed-ratio value increased. As the proportion of reinforcements obtained from the fixed-ratio component increased and the alternative schedule approached a simple fixed ratio, overall response rate and running rate both increased; conversely, as the proportion of reinforcements obtained from the fixed-interval component increased and the alternative schedule approached a simple fixed interval, response rates decreased. Postreinforcement pause length increased linearly as the average time between reinforcements increased, regardless of the schedule parameters. A break-run pattern of responding was predominant at low- and medium-valued fixed ratios. All subjects displayed at least occasional positively accelerated responding within interreinforcement intervals at higher fixed-ratio values.     

Responding under discrete-trial fixed-interval schedules of reinforcement

A fixed-interval schedule of reinforcement was modified by dividing each interval into 4-sec trial periods. No more than one response could occur during each trial because the operandum was inactivated for the remainder of any trial in which a response occurred. For example, under a 28-sec schedule, no more than seven responses could be emitted between reinforcements. Probabilities of responding by pigeons under six values of this discrete-trial fixed-interval schedule were best described by a two-state model: responding was either absent or infrequent immediately after reinforcement; then, at some variable time after reinforcement, there was an abrupt transition to a high and constant probability of responding on each trial. Performances under the discrete-trial procedure were less affected by uncontrolled sources of variance than performances under equivalent free-operant fixed-interval schedules.     

Contrast effects in multiple fixed-interval reinforcement schedules

Pigeons were exposed to a multiple fixed-interval one-minute fixed-interval three-minute schedule of reinforcement following training on either a multiple fixed-interval one-minute fixed-interval one-minute schedule or a multiple fixed-interval three-minute fixed-interval three-minute schedule. For all birds, large negative local contrast effects developed during the first of four three-minute intervals in a component; response rate was depressed and postreinforcement pause lengthened in this interval. Positive local contrast effects were evident during the first of 12 one-minute intervals in a component for five of six birds; at asymptote, the pause was very short and response rate slightly elevated during this interval. Overall positive contrast was generally transient and varied considerably across subjects, while overall negative contrast effects, if they occurred, appeared only after a large number of sessions.     

Order and chaos in fixed-interval schedules of reinforcement

Fixed-interval schedule performance is characterized by high levels of variability. Responding is absent at the onset of the interval and gradually increases in frequency until reinforcer delivery. Measures of behavior also vary drastically and unpredictably between successive intervals. Recent advances in the study of nonlinear dynamics have allowed researchers to study irregular and unpredictable behavior in a number of fields. This paper reviews several concepts and techniques from nonlinear dynamics and examines their utility in predicting the behavior of pigeons responding to a fixed-interval schedule of reinforcement. The analysis provided fairly accurate a priori accounts of response rates, accounting for 92.8% of the variance when predicting response rate 1 second in the future and 64% of the variance when predicting response rates for each second over the entire next interreinforcer interval. The nonlinear dynamics account suggests that even the “noisiest” behavior might be the product of purely deterministic mechanisms.     

The response-reinforcement dependency in fixed-interval schedules of reinforcement

Pigeons were exposed to four different schedules of food reinforcement that arranged a fixed minimum time interval between reinforcements (60 sec or 300 sec). The first was a standard fixed-interval schedule. The second was a schedule in which food was presented automatically at the end of the fixed time interval as long as a response had occurred earlier. The third and fourth schedules were identical to the first two except that the first response after reinforcement changed the color on the key. When the schedule required a peck after the interval elapsed, the response pattern consisted of a pause after reinforcement followed by responding at a high rate until reinforcement. When a response was not required after the termination of the interval, the pattern consisted of a pause after reinforcement, followed by responses and then by a subsequent pause until reinforcement. Having the first response after reinforcement change the color on the key had little effect on performance. Post-reinforcement pause duration varied with the minimum interreinforcement interval but was unaffected by whether or not a response was required after the interval elapsed.     

Human performance on conjunctive fixed-interval fixed-ratio schedules

Eighteen young adults performed a lever-pulling task for money. Subjects were initially exposed to a fixed-interval 80-second schedule and subsequently to one of three conjunctive schedules in which the added fixed-ratio requirement was set at either 10, 80, or 120 responses. Three fixed-interval response patterns emerged: high constant rate, intermediate rate, or low rate, with most subjects displaying the last. Conjunctive performance was related to the subjects' prior fixed-interval patterns and the conjunctive ratio requirements. Low-rate subjects tended to optimize reinforcement (maximum reinforcers for minimum responses) on conjunctive schedules. Response rate was directly related to ratio requirements. Subjects' performance closely corresponded to their verbal statements of the contingencies.     

Alternative reinforcement effects on fixed-interval performance

Pigeons' key pecks were reinforced with food on a fixed-interval schedule. Food also was available at variable time periods either independently of responding or for not key pecking (a differential-reinforcement-of-other-behavior schedule). The latter condition arranged reinforcement following the first pause of t seconds after it became available according to a variable-time schedule. This schedule allowed separation of the effects of pause requirements ≤ five-seconds and reinforcement frequency. The time spent pausing increased as the duration of the pause required for reinforcement increased from 0 to 30 seconds and as the frequency of reinforcement for pausing increased from 0 to 2 reinforcers per minute. Key pecking was more evenly distributed within each fixed interval with shorter required pauses and with more frequent reinforcement for pausing. The results complement those obtained with other concurrent schedules in which the same operant response was reinforced in both components.     

Effects of concurrent schedules on human fixed-interval performance

Young adults performed a lever-pressing task for money on two schedules of reinforcement: concurrent fixed-interval 1 min—differential-reinforcement-of-low-rate 20-sec, and concurrent fixed-interval 1-min—fixed ratio 100 responses. All subjects were trained on both schedules. Fixed-interval performance concurrent with the differential reinforcement procedure was characterized by high constant rates with no post-reinforcement pauses. Fixed-interval performance concurrent with fixed ratio was characterized by low rates and lengthy post-reinforcement pauses. These results differ from those obtained in prior studies on the effects of conditioning history upon subsequent fixed-interval performance. The prior work, using non-concurrent procedures, had shown that fixed-interval performance following differential reinforcement of low rates was characterized by post-reinforcement pauses and low rates, while fixed-interval performance following fixed ratio exhibited high constant rates and no post-reinforcement pause. The present results suggest that alternative concurrent contingencies are another major determinant of human fixed-interval performance.     

Variability of response location on fixed-ratio and fixed-interval schedules of reinforcement

Variability of response location was studied in monkeys performing in a six-lever chamber. Fixed-ratio schedules, ranging from FR 1 to FR 300, generated a high degree of stereotypy of response location. In contrast, fixed-interval schedules of comparable reinforcement frequencies (0.06 to 4 minutes) generated much greater variability. These results failed to confirm any simple relationship between response variability and intermittence of reinforcement. Rather, variability seems to be determined by the particular characteristics of the reinforcement schedule.     

Conjunctive schedules of reinforcement: III. A fixed-interval adjusting fixed-ratio schedule

Key pecking of three pigeons was studied under a conjunctive schedule that specified both a fixed-interval and an adjusting fixed-ratio requirement. The fixed-interval schedule was 6 min for one pigeon and 3 min for the other two. The size of the ratio requirement was determined within each cycle of the fixed interval by the duration of the pause before responding began. The fixed-ratio value was at maximum at the start of each fixed interval and decreased linearly until the first response occurred (adjusting fixed-ratio schedule). A peck produced food when the number of responses remaining on the fixed-ratio schedule was completed and when the fixed interval had elapsed. If no response occurred during the interval, the fixed-ratio requirement decreased to one and a single response after the interval elapsed produced food. The initial value of the adjusting fixed-ratio schedule was studied over a range of 0 to 900. Increases in the adjusting fixed-ratio schedule to about 300 responses increased both pause duration and running response rate and also modified the pattern of responding from that obtained under the fixed-interval schedule. Higher values of the adjusting fixed ratio generally decreased pause duration and running response rate and also disrupted responding. Interreinforcement time under the conjunctive schedule was increased substantially when the adjusting fixed-ratio size exceeded 300 responses.     

The relations among measures of performance on fixed-interval schedules

Quantitative measures of the performances of seven rats and two pigeons under FI schedules of reinforcement were obtained. For the rats (under FI 2 and FI 100 sec) the mean response rate and two measures of the temporal distribution of responses within the interval (quarter-life and an Index of Curvature) were computed for individual intervals. The measures of curvature were highly correlated with each other, whereas the response rate was only moderately correlated with either of them. Similar results were found for comparisons of the same measures on a session-by-session basis. The performances of the pigeons (under FI 10) were analyzed to yield response rate, quarter-life and elapsed time to the first, fifth and tenth response. Response rate was only moderately correlated with quarter-life, whereas quarter-life and time to the fifth or tenth response were highly correlated. Measures of temporal distribution based on an average of the intervals of a daily session were highly similar to the means of those measures calculated from the individual intervals.     

Latency and frequency of responding under discrete-trial fixed-interval schedules of reinforcement

Pigeons' key pecking was studied under a number of discrete-trial fixed-interval schedules of food reinforcement. Discrete trials were presented by briefly illuminating the keylight repetitively throughout the interreinforcement interval. A response latency counterpart to the fixed-interval scallop was found, latency showing a gradual, negatively accelerated decrease across the interval. This latency pattern was largely invariant across changes in fixed-interval length, number of trials per interval, and maximum trial duration. Frequency of responding during early trials in the intervals varied, however, with different schedule parameters, being directly related to fixed-interval length, inversely related to number of trials, and complexly affected by conjoint variations of fixed-interval length and number of trials. Response latency thus was found to be simply related to elapsed time during the interval while response frequency was complexly determined by other factors as well.     

Clock control of human performance on avoidance and fixed-interval schedules

The avoidance and fixed-interval performances of human subjects were studied in two experiments. Addition of time-correlated stimuli (added clock) improved behavioral efficiency, since response rates decreased without decreases in reinforcement rates. Response-dependent display of the clock maintained a second, observing response and reductions in clock duration weakened such observing behavior. Generally, the reinforcing properties of the clock were more apparent with the avoidance than with the fixed-interval schedule, a finding attributed to temporal cues already provided by delivery of the fixed-interval reinforcers. Reduced rates of the main response when the clock was dependent on an observing response were more than offset by rates of the observing response in the majority of subjects. Thus, the results do not support an interpretation of the reinforcing properties of added clocks simply in terms of work reduction.     

Human fixed-interval performance with concurrently programmed schedules: A parametric analysis

Young adults pressed a lever for points, exchangeable for money, programmed on concurrent schedules in which one component was a fixed-interval and the other component either a fixed-ratio (Experiment 1) or a differential-reinforcement-of-low-rate (Experiment 2). Two general patterns of fixed-interval responding, postreinforcement pause or constant rate, occurred in both experiments as a function of the parameter values of each component. Also patterns of interaction between the component schedules developed, in which responding or point delivery on one component appeared to be discriminative for responding on the other component. Once a pattern of responding was established, it tended to persist when the parameter values of the schedule were changed. On many schedules, subjects with an experimental history responded differently than did naive subjects, although certain schedule values were resistant to the history effects. The role of verbal strategies in mediating history effects was discussed.     

Effects of concurrent response-independent reinforcement on fixed-interval schedule performance

In three experiments, behavior maintained by fixed-interval schedules changed when response-independent reinforcement was delivered concurrently according to fixed- or variable-time schedules. In Experiment I, a pattern of positively accelerated responding during fixed interval was changed to a linear pattern when response-independent reinforcement occurred under a variable-time schedule. Overall response rates (total responses/total time) decreased as the frequency of response-independent reinforcement increased. Experiment II showed that the response-rate changes in the first experiment were controlled by the response-reinforcer relation, but the changes in patterns of responding were similar whether concurrently available reinforcement at varying times was response-dependent or response-independent. In the final experiment, the addition of response-independent reinforcement at fixed times to a fixed-interval schedule resulted in changes in both local and overall response rates and in the occurrence of positively accelerated responding between reinforcements. These results suggest that the temporal distribution of reinforcers determines response patterns and that both the response-reinforcement dependency and the schedule of reinforcement determine overall response rates during concurrently scheduled response-dependent and response-independent reinforcement.     

Diversity and substitutability of adjunctive activities under fixed-interval schedules of food reinforcement

Six rats received food contingent on pressing a lever on fixed-ratio 1, fixed-interval 30-second, and fixed-interval 60-second schedules, with concurrent access to a drinking spout, a running wheel, and a block of wood. Drinking, running, and chewing were monitored automatically, and these and other activities were observed directly during selected sessions. Because all sessions ended after delivery of 60 pellets, total time available for activities other than eating increased over the three schedules. Time spent contacting the lever and visiting the food tray increased in proportion to total available time, whereas the time spent in other activities changed in a complex manner such that drinking was the dominant adjunctive behavior in the 30-second condition, and running or chewing the dominant adjunctive behavior in five of six rats in the 60-second condition. General activity and grooming also occupied significant amounts of time. In a subsequent part of the experiment, running and chewing were prevented, and the majority of other activities, especially drinking and grooming, increased. The results show that (a) FI schedules of food reinforcement are accompanied by a wide variety of adjunctive activities; (b) the preferred activity differs according to the schedule duration; and (c) the extent to which activities substitute for one another is limited by the tendency for different activities to occupy different parts of the interreinforcement interval.     

Reinforcement omission on fixed-interval schedules

Experiments with pigeons and rats showed that: (1) When a brief blackout was presented in lieu of reinforcement at the end of 25% of intervals on a fixed-interval 2-min schedule, response rate was reliably and persistently higher during the following 2-min intervals (omission effect). This effect was largely due to a decrease in time to first response after reinforcement omission. (2) When blackout duration was varied, within sessions, over the range 2 to 32 sec, time to first response was inversely related to the duration of the preceding blackout, for pigeons, and for rats during the first few sessions after the transition from FI 2-min to FI 2-min with reinforcement omission. Post-blackout pause was independent of blackout duration for rats at asymptote. These results were interpreted in terms of differential depressive effects of reinforcement and blackout on subsequent responding.     

A mathematical model of learning under schedules of interresponse time reinforcement

A model is proposed for free responding under schedules of interresponse time reinforcement. Each response generates a pattern of stimuli that changes in an orderly way over time. At any time the changing pattern may be sampled, and the state of conditioning of the sampled pattern determines whether or not a response actually occurs. The prediction of this model for the asymptotic distribution of interresponse times is derived. The predictions are tested against data from ratio, interval, and DRL schedules of reinforcement. The fit of the model is sufficiently good to make it worth-while continuing investigation of the model.