A comparison of ratio and interval reinforcement schedules with comparable interreinforcement times

Pigeons were trained to peck keys on fixed-ratio and fixed-interval schedules of food reinforcement. Both schedules produced a pattern of behavior characterized as pause and run, but the relation of pausing to time between reinforcers differed for the two schedules even when mean time between reinforcers was the same. Pausing in the fixed ratio occupied less of the time between reinforcers for shorter interreinforcer times. For two of three birds, the relation was reversed at longer interreinforcer times. As an interreinforcer time elapsed, there was an increasing tendency to return to responding for the fixed interval, but a roughly constant tendency to return to responding for the fixed-ratio schedule. In Experiment 1 these observations were made for both single-reinforcement schedules and multiple schedules of fixed-ratio and fixed-interval reinforcement. In Experiment 2 the observations were extended to a comparison of fixed-ratio versus variable-interval reinforcement schedules, where the distribution of interreinforcement times in the variable interval approximated that for the fixed ratio.     

Modification of verbal behavior through variable interval reinforcement in a quasi-therapy situation

Principles from the operant conditioning laboratory were applied to the verbal behavior of female college students. An experimental setting was designed to replicate a psychotherapist's office, and subjects were seen daily in a study using a functional research design. Variable interval reinforcement was employed to modify the rate of emission of positive self-reference statements. Results demonstrated that the verbal, scheduled reinforcement exerted a definite, marked effect on the verbal behavior of the subjects. All subjects conditioned to the schedules. Sequence of scheduling did not appear to be a significant variable. The feasibility of employing such techniques in actual psychotherapy is discussed.     

Resistance to extinction in the goldfish following schedules of continuous and variable interval reinforcement

Four goldfish (Carassius auratus, comet) were reinforced with food when they intercepted a beam of light. Each fish performed on both CRF and VI 50-sec schedules. In subsequent extinctions, the CRF schedule produced more responses on the first day than did VI, but after the first day the post-CRF rate dropped much more rapidly. These results agree with those previously obtained in birds and mammals.     

The distribution of interresponse times in the pigeon during variable-interval reinforcement

Three pigeons' pecks were reinforced on 1- and 2-min variable-interval schedules, and frequency distributions of their interresponse times (IRTs) were recorded. The conditional probability that a response would fall into any IRT category was estimated by the interresponse-times-per-opportunity transformation (IRTs/op). The resulting functions were notable chiefly for the relatively low probability of IRTs in the 0.2- to 0.3-sec range; in other respects they varied within and between subjects. The overall level of the curves generally rose over the course of 32 experimental hours, but their shapes changed unsystematically. The shape of the IRT distribution was much the same for VI 1-min and VI 2-min. The variability of these distributions supports the notion that the VI schedule only loosely controls response rate, permitting wide latitude to adventitious effects. There was no systematic evidence that curves changed over sessions to conform to the distribution of reinforcements by IRT.     

Relative and absolute reinforcement frequency as determinants of choice in concurrent variable interval schedules

Rats were trained under concurrent schedules consisting of two equal variable interval component shedules providing sucrose solutions of different concentrations (0.6 M, 0.2 M; 50 μl in each case) as the reinforcers. The mean interreinforcement interval specified by the schedules was varied from 10 to 640 sec. Absolute response rate in each component was an increasing hyperbolic function of reinforcement frequency. Relative response rate and relative time allocation revealed a consistent preference for the more concentrated solution; neither measure of preference was systematically related to reinforcement frequency. The results are consistent with Baum and Rachlin's (1969) extension of the matching law, and with a derivation of the matching law from the hyperbolic relation between absolute response rate and reinforcement frequency in variable interval schedules (Herrnstein, 1970).     

Frequency distribution of interresponse times during VI and VR reinforcement

A detailed analysis was made of the interresponse times (IRTs) of two rats under both a VI 40-sec and a VR 15-sec schedule. Except for the latency of the first response after a reinforcement, the mean IRTs of all further responses differed little. Similarly, the frequency distributions of the successive IRTs did not vary greatly, but were of no simple form. Sequential dependencies between successive IRTs were small, never accounting for more than 1% of the variance.     

A quantitative analysis of the responding maintained by interval schedules of reinforcement

Interval schedules of reinforcement maintained pigeons' key-pecking in six experiments. Each schedule was specified in terms of mean interval, which determined the maximum rate of reinforcement possible, and distribution of intervals, which ranged from many-valued (variable-interval) to single-valued (fixed-interval). In Exp. 1, the relative durations of a sequence of intervals from an arithmetic progression were held constant while the mean interval was varied. Rate of responding was a monotonically increasing, negatively accelerated function of rate of reinforcement over a range from 8.4 to 300 reinforcements per hour. The rate of responding also increased as time passed within the individual intervals of a given schedule. In Exp. 2 and 3, several variable-interval schedules made up of different sequences of intervals were examined. In each schedule, the rate of responding at a particular time within an interval was shown to depend at least in part on the local rate of reinforcement at that time, derived from a measure of the probability of reinforcement at that time and the proximity of potential reinforcements at other times. The functional relationship between rate of responding and rate of reinforcement at different times within the intervals of a single schedule was similar to that obtained across different schedules in Exp. 1. Experiments 4, 5, and 6 examined fixed-interval and two-valued (mixed fixed-interval fixed-interval) schedules, and demonstrated that reinforcement at one time in an interval had substantial effects on responding maintained at other times. It was concluded that the rate of responding maintained by a given interval schedule depends not on the overall rate of reinforcement provided but rather on the summation of different local effects of reinforcement at different times within intervals.     

Effects of variations in local reinforcement rate on local response rate in variable interval schedules

Rats trained to lever press for sucrose were exposed to variable-interval schedules in which (i) the probability of reinforcement in each unit of time was a constant, (ii) the probability was high in the first ten seconds after reinforcement and low thereafter, (iii) the probability was low for ten seconds and high thereafter, (iv) the probability increased with time since reinforcement, or (v) the probability was initially zero and then increased with time since reinforcement. All schedules generated similar overall reinforcement rates. A peak in local response rate occurred several seconds after reinforcement under those schedules where reinforcement rate at this time was moderate or high ([i], [ii], and [iv]). Later in the inter-reinforcement interval, local response rate was roughly constant under those schedules with a constant local reinforcement rate ([i], [ii], and [iii]), but increased steadily when local reinforcement rate increased with time since reinforcement ([iv] and [v]). Postreinforcement pauses occurred on all schedules, but were much longer when local reinforcement rate was very low in the ten seconds after reinforcement ([iii]). The interresponse time distribution was highly correlated with the distribution of reinforced interresponse times, and the distribution of postreinforcement pauses was highly correlated with the distribution of reinforced postreinforcement pauses on some schedules. However, there was no direct evidence that these correlations resulted from selective reinforcement of classes of interresponse times and pauses.     

Random interval schedules of reinforcement

A method for generating a reinforcement schedule that closely approximates idealized VI schedules in which reinforcement assignments occur randomly in time (RI schedules) is described. Response rates of pigeons exposed for 20 sessions to this schedule appeared very similar to response rates characteristic of arithmetic series VIs. The distribution function describing these schedules was derived and its relations to other VI distributions, as well as to FI and random ratio (RR) were shown.     

Responding maintained by the opportunity to attack during an interval food reinforcement schedule

Pigeons responded in a two-key situation. Responses on the right key (food key) were reinforced with food presentation on a response-initiated fixed-interval schedule, (i.e., first response after a fixed period of time was reinforced); responses on the left key (target key) were reinforced on a fixed-ratio schedule (i.e., every nth response was reinforced) with the presentation of a target bird that could be attacked. When the interval value of the food reinforcement schedule was varied from 1 min to 5 min, both the rate of attack responding on the target bird and the rate of responding on the target key were a function of the interval value. Responding on the target key was not maintained by the stimulus change associated with target availability, and was successively extinguished and reconditioned by removing and returning the target bird to the restraining box. When food was delivered independently of behavior, responding on the target key either remained unchanged or decreased, but was not eliminated. Responding on the target key was not maintained in the absence of an intermittent schedule of food presentation.     

The reinforcement of least-frequent interresponse times

A new schedule of reinforcement was used to maintain key-pecking by pigeons. The schedule reinforced only pecks terminating interresponse times which occurred least often relative to the exponential distribution of interresponse times to be expected from an ideal random generator. Two schedule parameters were varied: (1) the rate constant of the controlling exponential distribution and (2) the probability that a response would be reinforced, given that it met the interresponse-time contingency. Response rate changed quickly and markedly with changes in the rate constant; it changed only slightly with a fourfold change in the reinforcement probability. The schedule produced stable rates and high intra- and inter-subject reliability, yet interresponse time distributions were approximately exponential. Such local interresponse time variability in the context of good overall control suggests that the schedule may be used to generate stable, predictable, yet sensitive baseline rates. Implications for the measurement of rate are discussed.     

The reinforcement of short interresponse times

Five contingencies were superimposed successively on a variable-interval schedule of reinforcement. In each of the resulting conditions, a different short, interresponse time was reinforced and an interresponse-time distribution was obtained from each of three pigeons. The lower bound of the reinforced interresponse times ranged from 0.3 to 2.4 sec. The resulting distributions were combined, according to a rationale based upon concurrent operants, induction, and a property of variable-interval schedules, to describe the interresponse-time distributions from a variable-interval schedule.     

Response rates track the history of reinforcement times

When conditioning involves a consistent temporal relationship between the conditioned stimulus (CS) and unconditioned stimulus (US), the expression of conditioned responses within a trial peaks at the usual time of the US relative to the CS. Here we examine the temporal profile of responses during conditioning with variable CS-US intervals. We conditioned stimuli with either uniformly distributed or exponentially distributed random CS-US intervals. In the former case, the frequency of each CS-US interval within a specified range is uniform but the momentary probability of the US (the hazard function) increases as time elapses during the trial; with the latter distribution, short CS-US intervals are more frequent than longer intervals, but the momentary probability of the US is constant across time within the trial. We report that, in a magazine approach paradigm, rats' response rates remained stable as time elapses during the CS when the CS-US intervals were uniformly distributed, whereas their response rates declined when the CS-US intervals were exponentially distributed. In other words, the profile of responding during the CS matched the frequency distribution of the US times, not the momentary probability of the US during the CS. These results are inconsistent with real-time associative models, which predict that associative strength tracks the momentary probability of the US, but may provide support for timing models of conditioning in which conditioned responding is tied to remembered times of reinforcement.     

Schedule-induced defecation by rats during ratio and interval schedules of food reinforcement.

Lever pressing in rats was maintained by continuous and intermittent schedules of food while defecation was monitored. In Experiment 1, reinforcement densities were matched across variable-ratio and variable-interval schedules for three pairs of rats. Defecation occurred in all 3 rats on the variable-ratio schedule and in all 3 rats on the yoked variable-interval schedule. In Experiment 2, fixed-ratio and fixed-interval schedules with similar reinforcement densities maintained lever pressing. Defecation occurred in 3 of 4 rats on the fixed-ratio schedule and in 4 of 4 rats on the fixed-interval schedule. Almost no defecation occurred during continuous reinforcement in either experiment. These results demonstrate that defecation may occur during both ratio and interval schedules and that the inter-reinforcement interval is more important than the behavioral requirements of the schedule in generating schedule-induced defecation.     

Positive reinforcement and the elimination of reinforced responses

Key pecking was maintained on a fixed-interval schedule while either a differential-reinforcement-of-not-responding or a fixed-time schedule was imposed simultaneously. The lower the time parameter of the not-responding schedule, the lower was the response rate. Similar effects occurred with the fixed-time schedule, if the pigeons had experience with reinforcement for not responding. Otherwise the effects were less orderly, to the extent that rate could reach maximum with the lowest-valued fixed-time schedule. The not-responding and the response-independent schedules had similar effects on rate in experienced pigeons only when the time parameter or nominal frequency of food presentation was considered. When considered in terms of obtained frequency of food presentation, reinforcement of not responding produced larger decrements in rate than did the fixed-time schedule.     

Fixed interval schedules and delay of reinforcement

In a fixed interval schedule of reinforcement the only responses to be reinforced are those made when a certain time interval has elapsed since the previous reinforcement. The behaviour of three rats on such a schedule was compared with their behaviour on a schedule where a response made at any time during the interval was reinforced by setting up a reward which was delivered when the interval had elapsed. Response rates were higher in the ordinary fixed interval schedule than in its modified version, and it is argued that this rules out attempts to explain the maintenance of fixed interval performance by delayed reinforcement. Despite the clear difference in response rates, there was considerable similarity between the post-reinforcement pauses developed in the two schedules, and this suggests that pausing is influenced more by temporal than by response contingencies.