Effects of reinforcement magnitude on interval and ratio schedules

Rats' lever pressing was studied on three schedules of reinforcement: fixed interval, response-initiated fixed interval, and fixed ratio. In testing, concentration of the milk reinforcer was varied within each session. On all schedules, duration of the postreinforcement pause was an increasing function of the concentration of the preceding reinforcer. The running rate (response rate calculated by excluding the postreinforcement pauses) increased linearly as a function of the preceding magnitude of reinforcement on fixed interval, showed slight increases for two of the three animals on response-initiated fixed interval, and did not change systematically on fixed ratio. In all cases, the overall response rate either declined or showed no effect of concentration. The major effect of increasing the reinforcement magnitude was in determining the duration of the following postreinforcement pause, and changes in the response rate reflected this main effect.     

Random interval schedules of reinforcement

A method for generating a reinforcement schedule that closely approximates idealized VI schedules in which reinforcement assignments occur randomly in time (RI schedules) is described. Response rates of pigeons exposed for 20 sessions to this schedule appeared very similar to response rates characteristic of arithmetic series VIs. The distribution function describing these schedules was derived and its relations to other VI distributions, as well as to FI and random ratio (RR) were shown.     

Fixed interval schedules and delay of reinforcement

In a fixed interval schedule of reinforcement the only responses to be reinforced are those made when a certain time interval has elapsed since the previous reinforcement. The behaviour of three rats on such a schedule was compared with their behaviour on a schedule where a response made at any time during the interval was reinforced by setting up a reward which was delivered when the interval had elapsed. Response rates were higher in the ordinary fixed interval schedule than in its modified version, and it is argued that this rules out attempts to explain the maintenance of fixed interval performance by delayed reinforcement. Despite the clear difference in response rates, there was considerable similarity between the post-reinforcement pauses developed in the two schedules, and this suggests that pausing is influenced more by temporal than by response contingencies.     

Effects of reinforcement magnitude on pigeons' preference for different fixed-ratio schedules of reinforcement

In concurrent, two-member chains, the completion of one or the other of two initial percentage fixed-interval 90-sec links produced a terminal link in which the completion of a fixed ratio produced food reinforcement. The fixed ratios and the duration of reinforcement in the terminal links were varied. Relative response rate in initial links was proportional to the relative reinforcement duration per ratio response (reinforcement duration divided by fixed ratio) in terminal links. The rate of responding in the terminal fixed-ratio links was insensitive to both ratio size and reinforcement duration and therefore did not vary sufficiently to distinguish between responses per reinforcement and immediacy of reinforcement as controlling variables in terminal links.     

Inhibiting function of reinforcement: magnitude effects on variable-interval schedules

In two experiments, the performance of rats under constant-probability and arithmetic variable-interval schedules respectively was compared when the concentration of a liquid reinforcer was varied within sessions; in other sessions, half of the reinforcers were randomly omitted. When the discriminative function of the reinforcer as a signal for a decrease in the probability of reinforcement was attenuated (the constant-probability schedule) the postreinforcement pause duration was nevertheless an increasing function of reinforcer magnitude. This relationship was also present, but more marked, when the temporal discriminative function of the reinforcer was enhanced (the arithmetic schedule). These results suggested that reinforcement has an unconditioned suppressive effect on the reinforced response distinct from any discriminative function it may acquire. The reinforcement-omission effect, where response rate accelerates following omission, was observed when the reinforcer functioned as an effective temporal discriminative stimulus, but not when such temporal control was absent.     

Properties of behavior under random interval reinforcement schedules

In a temporally defined system of reinforcement schedules, the fixed interval case is defined when reinforcement probability, P, is equal to unity for the first response in any cycle length, T; when P is less than 1.0, random interval schedules emerge wherein T/P specifies the expected interval between reinforcements. Key-pecking rates were found to be: (a) inversely related to T/P; (b) higher at T=1.0 second than at other T parameter values; (c) low and linear at several T and T/P values. The mean post-reinforcement pause, if initially small, increased, and if initially large, decreased, as T/P increased.     

Response persistence under ratio and interval reinforcement schedules

In Experiment 1, rats were exposed to progressive-ratio schedules of food reinforcement while other rats were exposed simultaneously to yoked-interval schedules that arranged equivalent interreinforcer intervals but required only a single response at the end of the interval for food delivery. In Experiment 2, a within-subject yoked-control procedure was employed in which pigeons were exposed to alternating sessions (one per day) of progressive-ratio schedules and yoked-interval schedules as described above. In both experiments, responding under the yoked-interval schedule persisted beyond the point at which responding under the progressive-ratio schedule had ceased. The progressive-ratio schedules controlled break-and-run distributions, and the yoked-interval schedules controlled more even distributions of responses in time. Response rates decreased and postreinforcement pauses increased over time within individual sessions under both schedules. The results suggest that responding maintained by interval schedules is more persistent than that maintained by ratio schedules. The limitations and implications of this conclusion are discussed in the context of other investigations of response strength and behavioral momentum.     

Performance on ratio and interval schedules with matched reinforcement rates

In the first study, rats were trained to pull a chain on a schedule (RPI) that regulates the probability of reinforcement to maintain a constant average reinforcement rate without differentially reinforcing long inter-response times (IRTs). Although the response rate was sensitive to the overall rate of reinforcement, performance was unaffected by variations between 1 and 50 in the IRT memory size used in programming the schedule. In the second study, two groups of animals performed on either a random-interval (RI) schedule or a RPI schedule, with reinforcement rates determined by those generated by a third group performing on a random ratio (RR) 20 schedule. The RI group responded at a lower rate than the RPI group, which, in turn, responded at a lower rate than the RR group, even though the three groups experienced comparable rates of reinforcement. The fact that the RPI group responded at a lower rate than the RR group suggests that the standard response rate difference observed between ratio and interval schedules, which have been matched for reinforcement rate, cannot be attributed solely to the fact that conventional interval schedules differentially reinforce long IRTs.     

The effect of reinforcement magnitude upon responding under fixed-ratio schedules

Responding under fixed-ratio schedules was studied as a function of two durations of food presentation. Latency of the first response after food presentation (post-reinforcement pause) was consistently shorter when food was presented for the longer duration. Only one of the four pigeons studied showed a consistently higher response rate, exclusive of post-reinforcement pause, as a function of the longer access to food. When ratio size was reduced, pause durations decreased, and the differences related to the two durations of food presentations became progressively smaller.     

The conditions for temporal tracking under interval schedules of reinforcement

On many cyclic-interval schedules, animals adjust their postreinforcement pause to follow the interval duration (temporal tracking). Six pigeons were trained on a series of square-wave (2-valued) interval schedules (e.g., 12 fixed-interval [FI] 60, 4 FI 180). Experiment 1 showed that pigeons track square-wave schedules, except those with a single long interval per cycle. Experiments 2 and 3 established that tracking and nontracking are learned and both can transfer from one cyclic schedule to another. Experiment 4 demonstrated that pigeons track a schedule with a single short interval per cycle, suggesting that a dual process--cuing and tracking--is necessary to explain behavior on these schedules. These findings suggest a potential explanation for earlier results that reported a failure to track square-wave schedules.     

Independence of concurrent responding maintained by interval schedules of reinforcement

A pigeon's responses were reinforced on a variable-interval schedule on one key; and, concurrently, either a multiple or a fixed-interval schedule of reinforcement was in effect on a second key. These concurrent schedules, conc VI 3 (mult VI 3 EXT) or conc VI 3 FI 6, were programmed with or without a changeover delay (COD). Because the COD provided that responses on one key could not be followed by reinforced responses on the other key, responding on one key was not likely to accidentally come under the control of the reinforcement schedule on the other. When the COD was used, the performances on each key were comparable to the performances maintained when these interval schedules are programmed separately. The VI schedule maintained a relatively constant rate of responding, even though the rate of responding on the second key varied in a manner appropriate to the schedule on the second key. The mult VI 3 EXT schedule maintained two separate rates of responding: a relatively high rate during the VI 3 component, and almost no responding during the EXT component. The FI schedule maintained the gradually increasing rate of responding within each interval that is characteristic of the performance maintained by this schedule. The concurrent performances, however, did include certain interactions involving the local characteristics of responding and the over-all rates of responding maintained by the various schedules. The relevance of the present findings to an inter-response time analysis of VI responding, a chaining account of FI responding, and the concept of the reflex reserve was discussed.     

Performances on ratio and interval schedules of reinforcement: Data and theory

Two differences between ratio and interval performance are well known: (a) Higher rates occur on ratio schedules, and (b) ratio schedules are unable to maintain responding at low rates of reinforcement (ratio “strain”). A third phenomenon, a downturn in response rate at the highest rates of reinforcement, is well documented for ratio schedules and is predicted for interval schedules. Pigeons were exposed to multiple variable-ratio variable-interval schedules in which the intervals generated in the variable-ratio component were programmed in the variable-interval component, thereby “yoking” or approximately matching reinforcement in the two components. The full range of ratio performances was studied, from strained to continuous reinforcement. In addition to the expected phenomena, a new phenomenon was observed: an upturn in variable-interval response rate in the midrange of rates of reinforcement that brought response rates on the two schedules to equality before the downturn at the highest rates of reinforcement. When the average response rate was corrected by eliminating pausing after reinforcement, the downturn in response rate vanished, leaving a strictly monotonic performance curve. This apparent functional independence of the postreinforcement pause and the qualitative shift in response implied by the upturn in variable-interval response rate suggest that theoretical accounts will require thinking of behavior as partitioned among at least three categories, and probably four: postreinforcement activity, other unprogrammed activity, ratio-typical operant behavior, and interval-typical operant behavior.     

Relative rate of response and relative magnitude of reinforcement in multiple schedules

Pigeons were trained on a multiple schedule in which the duration of access to grain reinforcement was varied independently in the two components. The relative response rate in one component was an increasing function of the relative duration of reinforcement in that component. The similarity of this interaction to that found in multiple schedules of different reinforcement frequency is discussed. Extinction data were also similar to those obtained after training on multiple schedules of different reinforcement frequency.     

A comparison of ratio and interval reinforcement schedules with comparable interreinforcement times

Pigeons were trained to peck keys on fixed-ratio and fixed-interval schedules of food reinforcement. Both schedules produced a pattern of behavior characterized as pause and run, but the relation of pausing to time between reinforcers differed for the two schedules even when mean time between reinforcers was the same. Pausing in the fixed ratio occupied less of the time between reinforcers for shorter interreinforcer times. For two of three birds, the relation was reversed at longer interreinforcer times. As an interreinforcer time elapsed, there was an increasing tendency to return to responding for the fixed interval, but a roughly constant tendency to return to responding for the fixed-ratio schedule. In Experiment 1 these observations were made for both single-reinforcement schedules and multiple schedules of fixed-ratio and fixed-interval reinforcement. In Experiment 2 the observations were extended to a comparison of fixed-ratio versus variable-interval reinforcement schedules, where the distribution of interreinforcement times in the variable interval approximated that for the fixed ratio.     

A quantitative analysis of the responding maintained by interval schedules of reinforcement

Interval schedules of reinforcement maintained pigeons' key-pecking in six experiments. Each schedule was specified in terms of mean interval, which determined the maximum rate of reinforcement possible, and distribution of intervals, which ranged from many-valued (variable-interval) to single-valued (fixed-interval). In Exp. 1, the relative durations of a sequence of intervals from an arithmetic progression were held constant while the mean interval was varied. Rate of responding was a monotonically increasing, negatively accelerated function of rate of reinforcement over a range from 8.4 to 300 reinforcements per hour. The rate of responding also increased as time passed within the individual intervals of a given schedule. In Exp. 2 and 3, several variable-interval schedules made up of different sequences of intervals were examined. In each schedule, the rate of responding at a particular time within an interval was shown to depend at least in part on the local rate of reinforcement at that time, derived from a measure of the probability of reinforcement at that time and the proximity of potential reinforcements at other times. The functional relationship between rate of responding and rate of reinforcement at different times within the intervals of a single schedule was similar to that obtained across different schedules in Exp. 1. Experiments 4, 5, and 6 examined fixed-interval and two-valued (mixed fixed-interval fixed-interval) schedules, and demonstrated that reinforcement at one time in an interval had substantial effects on responding maintained at other times. It was concluded that the rate of responding maintained by a given interval schedule depends not on the overall rate of reinforcement provided but rather on the summation of different local effects of reinforcement at different times within intervals.     

Effects of variations in local reinforcement rate on local response rate in variable interval schedules

Rats trained to lever press for sucrose were exposed to variable-interval schedules in which (i) the probability of reinforcement in each unit of time was a constant, (ii) the probability was high in the first ten seconds after reinforcement and low thereafter, (iii) the probability was low for ten seconds and high thereafter, (iv) the probability increased with time since reinforcement, or (v) the probability was initially zero and then increased with time since reinforcement. All schedules generated similar overall reinforcement rates. A peak in local response rate occurred several seconds after reinforcement under those schedules where reinforcement rate at this time was moderate or high ([i], [ii], and [iv]). Later in the inter-reinforcement interval, local response rate was roughly constant under those schedules with a constant local reinforcement rate ([i], [ii], and [iii]), but increased steadily when local reinforcement rate increased with time since reinforcement ([iv] and [v]). Postreinforcement pauses occurred on all schedules, but were much longer when local reinforcement rate was very low in the ten seconds after reinforcement ([iii]). The interresponse time distribution was highly correlated with the distribution of reinforced interresponse times, and the distribution of postreinforcement pauses was highly correlated with the distribution of reinforced postreinforcement pauses on some schedules. However, there was no direct evidence that these correlations resulted from selective reinforcement of classes of interresponse times and pauses.     

Schedule-induced defecation by rats during ratio and interval schedules of food reinforcement.

Lever pressing in rats was maintained by continuous and intermittent schedules of food while defecation was monitored. In Experiment 1, reinforcement densities were matched across variable-ratio and variable-interval schedules for three pairs of rats. Defecation occurred in all 3 rats on the variable-ratio schedule and in all 3 rats on the yoked variable-interval schedule. In Experiment 2, fixed-ratio and fixed-interval schedules with similar reinforcement densities maintained lever pressing. Defecation occurred in 3 of 4 rats on the fixed-ratio schedule and in 4 of 4 rats on the fixed-interval schedule. Almost no defecation occurred during continuous reinforcement in either experiment. These results demonstrate that defecation may occur during both ratio and interval schedules and that the inter-reinforcement interval is more important than the behavioral requirements of the schedule in generating schedule-induced defecation.     

A comparison of two models for performance under fixed interval schedules of reinforcement

Two models are proposed for responding under fixed-interval schedules of reinforcement. The first model is a Poisson model and seems suitable for situations in which responding produces a classical “FI scallop”. A second model is then developed to describe “break and run” performance, which is also known to occur under some Fixed Interval schedules. The models do not however give any indication of the circumstances under which a particular mode of responding should arise. A comparison of the models to a small set of data collected from rats performing under an FI 60 sec schedule indicates that for the data considered, the second model (a State model) produced by far the best fit.     

Reinforcement magnitude and pausing on progressive-ratio schedules

Rats responded under progressive-ratio schedules for sweetened milk reinforcers; each session ended when responding ceased for 10 min. Experiment 1 varied the concentration of milk and the duration of postreinforcement timeouts. Postreinforcement pausing increased as a positively accelerated function of the size of the ratio, and the rate of increase was reduced as a function of concentration and by timeouts of 10 s or longer. Experiment 2 varied reinforcement magnitude within sessions (number of dipper operations per reinforcer) in conjunction with stimuli correlated with the upcoming magnitude. In the absence of discriminative stimuli, pausing was longer following a large reinforcer than following a small one. Pauses were reduced by a stimulus signaling a large upcoming reinforcer, particularly at the highest ratios, and the animals tended to quit responding when the past reinforcer was large and the stimulus signaled that the next one would be small. Results of both experiments revealed parallels between responding under progressive-ratio schedules and other schedules containing ratio contingencies. Relationships between pausing and magnitude suggest that ratio pausing is under the joint control of inhibitory properties of the past reinforcer and excitatory properties of stimuli correlated with the upcoming reinforcer, rather than under the exclusive control of either factor alone.