Properties of behavior under random interval reinforcement schedules

In a temporally defined system of reinforcement schedules, the fixed interval case is defined when reinforcement probability, P, is equal to unity for the first response in any cycle length, T; when P is less than 1.0, random interval schedules emerge wherein T/P specifies the expected interval between reinforcements. Key-pecking rates were found to be: (a) inversely related to T/P; (b) higher at T=1.0 second than at other T parameter values; (c) low and linear at several T and T/P values. The mean post-reinforcement pause, if initially small, increased, and if initially large, decreased, as T/P increased.     

Response persistence under ratio and interval reinforcement schedules

In Experiment 1, rats were exposed to progressive-ratio schedules of food reinforcement while other rats were exposed simultaneously to yoked-interval schedules that arranged equivalent interreinforcer intervals but required only a single response at the end of the interval for food delivery. In Experiment 2, a within-subject yoked-control procedure was employed in which pigeons were exposed to alternating sessions (one per day) of progressive-ratio schedules and yoked-interval schedules as described above. In both experiments, responding under the yoked-interval schedule persisted beyond the point at which responding under the progressive-ratio schedule had ceased. The progressive-ratio schedules controlled break-and-run distributions, and the yoked-interval schedules controlled more even distributions of responses in time. Response rates decreased and postreinforcement pauses increased over time within individual sessions under both schedules. The results suggest that responding maintained by interval schedules is more persistent than that maintained by ratio schedules. The limitations and implications of this conclusion are discussed in the context of other investigations of response strength and behavioral momentum.     

Random interval schedules of reinforcement

A method for generating a reinforcement schedule that closely approximates idealized VI schedules in which reinforcement assignments occur randomly in time (RI schedules) is described. Response rates of pigeons exposed for 20 sessions to this schedule appeared very similar to response rates characteristic of arithmetic series VIs. The distribution function describing these schedules was derived and its relations to other VI distributions, as well as to FI and random ratio (RR) were shown.     

Varying reinforcement magnitude on interval schedules

In Experiment 1, rats were trained on either a random-interval or a variable-interval 60-sec schedule of reinforcement, and reinforcement magnitude was varied across conditions between one and four pellets. Although the two schedules maintained different patterns of behaviour, patterns and rates of responding were not systematically affected by the variation in reinforcement magnitude. In Experiment 2, a regulated probability interval schedule that generated similar rates of reinforcement to those of the schedules of Experiment 1 was used, with the pattern of behaviour generated resembling that typical of a random-interval schedule. Changing reinforcement magnitude again produced few systematic changes in behaviour. In Experiment 3, a variable-ratio schedule was used within a procedure that otherwise resembled that of Experiments 1 and 2. Increasing the reinforcement magnitude now decreased the rates of responding, and examination of the patterns of responding showed that this came about because rates of responding were higher early in the interreinforcer interval in the one-pellet condition. These experiments demonstrate the insensitivity of behaviour under interval schedules to changes in reinforcement magnitude and suggest the operation of mechanisms different from those engaged by ratio schedules and discretetrial learning procedures.     

A comparison of two models for performance under fixed interval schedules of reinforcement

Two models are proposed for responding under fixed-interval schedules of reinforcement. The first model is a Poisson model and seems suitable for situations in which responding produces a classical “FI scallop”. A second model is then developed to describe “break and run” performance, which is also known to occur under some Fixed Interval schedules. The models do not however give any indication of the circumstances under which a particular mode of responding should arise. A comparison of the models to a small set of data collected from rats performing under an FI 60 sec schedule indicates that for the data considered, the second model (a State model) produced by far the best fit.     

Temporal tracking on cyclic-interval reinforcement schedules

Pigeons were exposed to four cycles per session of a schedule in which the duration of successive interreinforcement intervals differed by t-sec. A cycle was composed of seven increasing and seven decreasing intervals, from 2t to 8t sec in length. In Exp. 1, postreinforcement pause tracked interval duration on five cyclic schedules, with values of t ranging from 2 to 40 sec. Tracking was better at shorter t values, and when discriminative stimuli signalled increasing and decreasing parts of the cycle. Pooled data for the whole experiment showed postreinforcement pause to bear a power function relationship to interval length, with a smaller exponent than the comparable function for fixed-interval schedules. Tests in a second experiment showed that pigeons trained on an arithmetic progression could also track schedules in which successive intervals followed either a logarithmic or a geometric progression, although tracking was more stable in the logarithmic case.     

Fixed interval schedules and delay of reinforcement

In a fixed interval schedule of reinforcement the only responses to be reinforced are those made when a certain time interval has elapsed since the previous reinforcement. The behaviour of three rats on such a schedule was compared with their behaviour on a schedule where a response made at any time during the interval was reinforced by setting up a reward which was delivered when the interval had elapsed. Response rates were higher in the ordinary fixed interval schedule than in its modified version, and it is argued that this rules out attempts to explain the maintenance of fixed interval performance by delayed reinforcement. Despite the clear difference in response rates, there was considerable similarity between the post-reinforcement pauses developed in the two schedules, and this suggests that pausing is influenced more by temporal than by response contingencies.     

Effects of reinforcement magnitude on interval and ratio schedules

Rats' lever pressing was studied on three schedules of reinforcement: fixed interval, response-initiated fixed interval, and fixed ratio. In testing, concentration of the milk reinforcer was varied within each session. On all schedules, duration of the postreinforcement pause was an increasing function of the concentration of the preceding reinforcer. The running rate (response rate calculated by excluding the postreinforcement pauses) increased linearly as a function of the preceding magnitude of reinforcement on fixed interval, showed slight increases for two of the three animals on response-initiated fixed interval, and did not change systematically on fixed ratio. In all cases, the overall response rate either declined or showed no effect of concentration. The major effect of increasing the reinforcement magnitude was in determining the duration of the following postreinforcement pause, and changes in the response rate reflected this main effect.     

Rate and temporal pattern of key pecking under autoshaping and omission schedules of reinforcement

The role of response-reinforcer contiguity on autoshaped key pecking in pigeons was studied by scheduling response-dependent nonreinforcement at the beginning or the end of brief (8-sec) discrete trials. Schedules that permitted chance conjunctions of key pecking and food sustained high rates of responding, whereas those that prevented the occurrence of key peck-food intervals shorter than 4 sec sustained low response rates. In addition, selective reinforcement schedules supported accelerating or decelerating rates of responding within individual trials. These effects were traceable to response-reinforcer (operant), but not stimulus-reinforcer (respondent) factors.     

Independence of concurrent responding maintained by interval schedules of reinforcement

A pigeon's responses were reinforced on a variable-interval schedule on one key; and, concurrently, either a multiple or a fixed-interval schedule of reinforcement was in effect on a second key. These concurrent schedules, conc VI 3 (mult VI 3 EXT) or conc VI 3 FI 6, were programmed with or without a changeover delay (COD). Because the COD provided that responses on one key could not be followed by reinforced responses on the other key, responding on one key was not likely to accidentally come under the control of the reinforcement schedule on the other. When the COD was used, the performances on each key were comparable to the performances maintained when these interval schedules are programmed separately. The VI schedule maintained a relatively constant rate of responding, even though the rate of responding on the second key varied in a manner appropriate to the schedule on the second key. The mult VI 3 EXT schedule maintained two separate rates of responding: a relatively high rate during the VI 3 component, and almost no responding during the EXT component. The FI schedule maintained the gradually increasing rate of responding within each interval that is characteristic of the performance maintained by this schedule. The concurrent performances, however, did include certain interactions involving the local characteristics of responding and the over-all rates of responding maintained by the various schedules. The relevance of the present findings to an inter-response time analysis of VI responding, a chaining account of FI responding, and the concept of the reflex reserve was discussed.     

Performance on ratio and interval schedules with matched reinforcement rates

In the first study, rats were trained to pull a chain on a schedule (RPI) that regulates the probability of reinforcement to maintain a constant average reinforcement rate without differentially reinforcing long inter-response times (IRTs). Although the response rate was sensitive to the overall rate of reinforcement, performance was unaffected by variations between 1 and 50 in the IRT memory size used in programming the schedule. In the second study, two groups of animals performed on either a random-interval (RI) schedule or a RPI schedule, with reinforcement rates determined by those generated by a third group performing on a random ratio (RR) 20 schedule. The RI group responded at a lower rate than the RPI group, which, in turn, responded at a lower rate than the RR group, even though the three groups experienced comparable rates of reinforcement. The fact that the RPI group responded at a lower rate than the RR group suggests that the standard response rate difference observed between ratio and interval schedules, which have been matched for reinforcement rate, cannot be attributed solely to the fact that conventional interval schedules differentially reinforce long IRTs.     

Responding under discrete-trial fixed-interval schedules of reinforcement

A fixed-interval schedule of reinforcement was modified by dividing each interval into 4-sec trial periods. No more than one response could occur during each trial because the operandum was inactivated for the remainder of any trial in which a response occurred. For example, under a 28-sec schedule, no more than seven responses could be emitted between reinforcements. Probabilities of responding by pigeons under six values of this discrete-trial fixed-interval schedule were best described by a two-state model: responding was either absent or infrequent immediately after reinforcement; then, at some variable time after reinforcement, there was an abrupt transition to a high and constant probability of responding on each trial. Performances under the discrete-trial procedure were less affected by uncontrolled sources of variance than performances under equivalent free-operant fixed-interval schedules.     

The effect of reinforcement magnitude upon responding under fixed-ratio schedules

Responding under fixed-ratio schedules was studied as a function of two durations of food presentation. Latency of the first response after food presentation (post-reinforcement pause) was consistently shorter when food was presented for the longer duration. Only one of the four pigeons studied showed a consistently higher response rate, exclusive of post-reinforcement pause, as a function of the longer access to food. When ratio size was reduced, pause durations decreased, and the differences related to the two durations of food presentations became progressively smaller.     

Group behavior of rats under schedules of reinforcement

Groups of three rats were placed in a chamber containing one response lever and one water dispenser. A variety of schedule conditions were explored including fixed ratio, extinction, satiation, fixed interval, fixed time, differential reinforcement of low rates, and discrimination learning. Each group was treated as a single unit, with the collective lever responses emitted by the three rats being the main dependent variable. Group responding was found to be controlled by the reinforcement schedules in an orderly and consistent manner. However, the groups often paused less and responded faster than individual rats working under identical conditions.     

The effects of interval duration on temporal tracking and alternation learning

On cyclic-interval reinforcement schedules, animals typically show a postreinforcement pause that is a function of the immediately preceding time interval (temporal tracking). Animals, however, do not track single-alternation schedules-when two different intervals are presented in strict alternation on successive trials. In this experiment, pigeons were first trained with a cyclic schedule consisting of alternating blocks of 12 short intervals (5 s or 30 s) and 12 long intervals (180 s), followed by three different single-alternation interval schedules: (a) 30 s and 180 s, (b) 5 s and 180 s, and (c) 5 s and 30 s. Pigeons tracked both schedules with alternating blocks of 12 intervals. With the single-alternation schedules, when the short interval duration was 5 s, regardless of the duration of the longer interval, pigeons learned the alternation pattern, and their pause anticipated the upcoming interval. When the shorter interval was 30 s, even when the ratio of short to long intervals was kept at 6:1, pigeons did not initially show anticipatory pausing-a violation of the principle of timescale invariance.     

A quantitative analysis of the responding maintained by interval schedules of reinforcement

Interval schedules of reinforcement maintained pigeons' key-pecking in six experiments. Each schedule was specified in terms of mean interval, which determined the maximum rate of reinforcement possible, and distribution of intervals, which ranged from many-valued (variable-interval) to single-valued (fixed-interval). In Exp. 1, the relative durations of a sequence of intervals from an arithmetic progression were held constant while the mean interval was varied. Rate of responding was a monotonically increasing, negatively accelerated function of rate of reinforcement over a range from 8.4 to 300 reinforcements per hour. The rate of responding also increased as time passed within the individual intervals of a given schedule. In Exp. 2 and 3, several variable-interval schedules made up of different sequences of intervals were examined. In each schedule, the rate of responding at a particular time within an interval was shown to depend at least in part on the local rate of reinforcement at that time, derived from a measure of the probability of reinforcement at that time and the proximity of potential reinforcements at other times. The functional relationship between rate of responding and rate of reinforcement at different times within the intervals of a single schedule was similar to that obtained across different schedules in Exp. 1. Experiments 4, 5, and 6 examined fixed-interval and two-valued (mixed fixed-interval fixed-interval) schedules, and demonstrated that reinforcement at one time in an interval had substantial effects on responding maintained at other times. It was concluded that the rate of responding maintained by a given interval schedule depends not on the overall rate of reinforcement provided but rather on the summation of different local effects of reinforcement at different times within intervals.     

A comparison of ratio and interval reinforcement schedules with comparable interreinforcement times

Pigeons were trained to peck keys on fixed-ratio and fixed-interval schedules of food reinforcement. Both schedules produced a pattern of behavior characterized as pause and run, but the relation of pausing to time between reinforcers differed for the two schedules even when mean time between reinforcers was the same. Pausing in the fixed ratio occupied less of the time between reinforcers for shorter interreinforcer times. For two of three birds, the relation was reversed at longer interreinforcer times. As an interreinforcer time elapsed, there was an increasing tendency to return to responding for the fixed interval, but a roughly constant tendency to return to responding for the fixed-ratio schedule. In Experiment 1 these observations were made for both single-reinforcement schedules and multiple schedules of fixed-ratio and fixed-interval reinforcement. In Experiment 2 the observations were extended to a comparison of fixed-ratio versus variable-interval reinforcement schedules, where the distribution of interreinforcement times in the variable interval approximated that for the fixed ratio.