Effects of reinforcement magnitude on interval and ratio schedules

Rats' lever pressing was studied on three schedules of reinforcement: fixed interval, response-initiated fixed interval, and fixed ratio. In testing, concentration of the milk reinforcer was varied within each session. On all schedules, duration of the postreinforcement pause was an increasing function of the concentration of the preceding reinforcer. The running rate (response rate calculated by excluding the postreinforcement pauses) increased linearly as a function of the preceding magnitude of reinforcement on fixed interval, showed slight increases for two of the three animals on response-initiated fixed interval, and did not change systematically on fixed ratio. In all cases, the overall response rate either declined or showed no effect of concentration. The major effect of increasing the reinforcement magnitude was in determining the duration of the following postreinforcement pause, and changes in the response rate reflected this main effect.     

Effects of magnitude of food reinforcement on free-operant response rates

In Experiment 1 rats were trained to press a lever on a variable-ratio schedule of food presentation and were then exposed to progressively increasing magnitudes of food reinforcement. Response running rates (rates exclusive of the postreinforcement pause) were found to increase as a function of increasing reinforcement magnitudes. The effect of reinforcement magnitude on response rates inclusive of the postreinforcement pause, however, was less pronounced. Increases in the magnitude of reinforcement were also found to increase the length of the postreinforcement pause. Rats in Experiment 2 were trained to respond on a chained differential-reinforcement-of-low-rate variable-ratio schedule, and were exposed to increasing magnitudes of reinforcement as in Experiment 1. Response running rates increased in the variable-ratio component but decreased in the other component of the schedule. The results are discussed with reference to incentive accounts of reinforcement and the action of reinforcement on the response units generated by the operative contingencies.     

Temporal control of behavior: schedule interactions

In Experiment I the response that terminated the postreinforcement pauses occurring under a fixed-interval 60-second schedule was reinforced, if the pause duration exceeded 30 seconds. The percentage of such pauses, rather than increasing, decreased. There were complex effects on the discriminative control of the pause by the reinforcer terminating the previous fixed interval, depending on whether the fixed interval and the added reinforcer were the same or different. In Experiments II(a) and II(b), each reinforcement initiated an alternative fixed-interval interresponse-time-greater-than-t-sec schedule, the schedule values being systematically varied. When the response following a pause exceeding a given duration was reinforced, fewer such pauses occurred than when they were not reinforced, i.e., on the comparable simple fixed-interval schedule. There was no systematic relationship between mean interrinforcement interval and duration of the postreinforcement pause. The pause duration initiated by reinforcement was directly related to the dependency controlling the shortest pause at that time, regardless of changes in mean interreinforcement interval.     

Species differences in temporal control of behavior II: human performance

Human subjects responded on two panels. A differential-reinforcement-of-low-rate schedule with a limited-hold contingency operated on Panel A. In Condition 1, responses on Panel B produced a stimulus on the panel that signalled whether reinforcement was available on Panel A. In Condition 2, responses on Panel B briefly illuminated a digital clock. In both conditions, performance on Panel A was very efficient; with few exceptions, Panel A was pressed only when reinforcement was available. Thus, in effect, a fixed-interval schedule operated on Panel B. In Condition 1, a “break-and-run” response pattern occurred on Panel B; with increasing temporal parameters, the duration of the postreinforcement pause on Panel B increased linearly while overall response rate and running rate (calculated by excluding the postreinforcement pauses) remained approximately constant. In Condition 2, the response pattern on Panel B was scalloped; the postreinforcement pause was a negatively accelerated increasing function of schedule value, while overall response rate and running rate were negatively accelerated decreasing functions of schedule value. The performance of subjects in Condition 2, but not in Condition 1, was highly sensitive to the contingencies in operation, and resembled that of other species on the fixed-interval schedule.     

Inhibiting function of reinforcement: magnitude effects on variable-interval schedules

In two experiments, the performance of rats under constant-probability and arithmetic variable-interval schedules respectively was compared when the concentration of a liquid reinforcer was varied within sessions; in other sessions, half of the reinforcers were randomly omitted. When the discriminative function of the reinforcer as a signal for a decrease in the probability of reinforcement was attenuated (the constant-probability schedule) the postreinforcement pause duration was nevertheless an increasing function of reinforcer magnitude. This relationship was also present, but more marked, when the temporal discriminative function of the reinforcer was enhanced (the arithmetic schedule). These results suggested that reinforcement has an unconditioned suppressive effect on the reinforced response distinct from any discriminative function it may acquire. The reinforcement-omission effect, where response rate accelerates following omission, was observed when the reinforcer functioned as an effective temporal discriminative stimulus, but not when such temporal control was absent.     

Temporal control of behavior and the power law

The performance of rats and pigeons under fixed-interval schedules was studied in two experiments. The duration of postreinforcement pause was a declining proportion of fixed-interval duration. For pigeons this was true both when the duration of the reinforcer was fixed and when it was increased in direct proportion to increases in fixed-interval duration; the longer reinforcer durations did, however, lengthen the postreinforcement pause at higher schedule values. A quantitative analysis of data from Experiments 1 and 2 and from other studies showed that fractional exponent power functions described the relationship between postreinforcement pause and fixed-interval value; similar functions have previously been observed in studies of temporal differentiation. It was concluded that power functions reflect a direct causal, rather than artifactual, relationship between performance and the temporal requirements of reinforcement schedules.     

Food deliveries during the pause on fixed-interval schedules

Pigeons were trained on fixed-interval schedules of food delivery. In Experiments I and II, the fixed interval was initiated by the previous fixed-interval reinforcer; in Experiment III, the fixed interval was initiated by the first key peck following the preceding fixed-interval reinforcer (a chain fixed-ratio one, fixed-interval schedule). During the postreinforcement pause, variable-time schedules delivered food independent of any specific response. Rate of food delivery during the pause had only small effects on pause duration in Experiments I and II. In Experiment III, however, pause duration increased systematically with the rate of food delivery during the pause. These data suggest that the momentary proximity to reinforcement delivered via the fixed-interval schedule exerts potent control over pause termination. Additional analysis revealed that pause termination was unaffected by the intermittent delivery of food during the pause. Such data suggest that the temporal control by fixed-interval schedules is highly resistant to interference.     

Effects Of Fixed-interval Schedule And Reinforcer Duration On Responding Reinforced By The Opportunity To Run

Two experiments investigated the effects of schedule value and reinforcer duration on responding for the opportunity to run on fixed-interval (FI) schedules in rats. In the first experiment, 8 male Wistar rats were exposed to FI 15-s, 30-s, and 60-s schedules of wheel-running reinforcement. The operant was lever pressing, and the consequence was the opportunity to run for 60 s. In the second experiment, 8 male Long-Evans rats were exposed to reinforcer durations of 15 s, 30 s, and 90 s. The schedule of reinforcement was an FI 60-s schedule. Results showed that postreinforcement pause and wheel-running rates varied systematically with reinforcer duration but not schedule value. Local lever-pressing rates decreased with reinforcer duration. Overall lever-pressing rates decreased with reinforcer duration but increased with schedule value. Although the reinforcer-duration effect is consistent with previous research, the lack a schedule effect appears to be the result of long post-reinforcement pauses following wheel-running reinforcement that render the manipulation of the interval requirement ineffective.     

Responding for sucrose and wheel-running reinforcement: effects of sucrose concentration and wheel-running reinforcer duration

Six male albino rats were placed in running wheels and exposed to a fixed-interval 30-s schedule of lever pressing that produced either a drop of sucrose solution or the opportunity to run for a fixed duration as reinforcers. Each reinforcer type was signaled by a different stimulus. In Experiment 1, the duration of running was held constant at 15 s while the concentration of sucrose solution was varied across values of 0, 2.5. 5, 10, and 15%. As concentration decreased, postreinforcement pause duration increased and local rates decreased in the presence of the stimulus signaling sucrose. Consequently, the difference between responding in the presence of stimuli signaling wheel-running and sucrose reinforcers diminished, and at 2.5%, response functions for the two reinforcers were similar. In Experiment 2, the concentration of sucrose solution was held constant at 15% while the duration of the opportunity to run was first varied across values of 15, 45, and 90 s then subsequently across values of 5, 10, and 15 s. As run duration increased, postreinforcement pause duration in the presence of the wheel-running stimulus increased and local rates increased then decreased. In summary, inhibitory aftereffects of previous reinforcers occurred when both sucrose concentration and run duration varied; changes in responding were attributable to changes in the excitatory value of the stimuli signaling the two reinforcers.     

Observing behavior: effects of rate and magnitude of primary reinforcement

Four experiments examined the free-operant observing behavior of rats. In Experiment 1, observing was a bitonic function of random-ratio schedule requirements for the primary reinforcer. In Experiment 2, decreases in the magnitude of the primary reinforcer decreased observing. Experiment 3 examined observing when a random-ratio schedule or a yoked random-time schedule of primary reinforcement was in effect across conditions. Removing the response requirement for the primary reinforcer increased observing, suggesting that the effects of the random-ratio schedule in Experiment 1 likely were due to an interaction between observing and responding for the primary reinforcer. In Experiment 4, decreasing the rate of primary reinforcement by increasing the duration of a random-time schedule decreased observing monotonically. Overall, these results suggest that observing decreases with decreases in the rate or magnitude of the primary reinforcer, but that behavior related to the primary reinforcer can affect observing and potentially affect measurement of conditioned reinforcing value.     

Varying wheel-running reinforcer duration within a session: effect on the revolution-postreinforcement pause relation

Previous investigations of wheel-running reinforcement that manipulated reinforcer duration across conditions showed a strong relation between wheel-running rate and average postreinforcement pause (PRP) duration. To determine if the basis of this relation across conditions was a local effect of fatigue or satiation, the correlation between revolutions run and the duration of the immediately following PRP was investigated under conditions in which reinforcer duration was either constant or variable within a session. Seven male Wistar rats pressed a lever on a fixed-interval 60-s reinforcement schedule with the opportunity to run for 60 s as the reinforcing consequence. In the constant-duration condition, the duration of the reinforcer was always 60 s. In the variable-duration condition, the duration of the reinforcer varied between 2 and 240 s with a mean of 60 s. Mean correlations between revolutions run and the next PRP duration for constant, variable, and constant conditions were -.07, .20, and -.07, respectively. Although the positive correlation in the variable-duration condition is consistent with an effect of momentary fatigue or satiation, little of the variance in PRP duration appears to be attributable to these factors.     

The role of verbal behavior in human learning: infant performance on fixed-interval schedules

The performances of two infants less than one year old were investigated on fixed-interval schedules. When the infants touched a cylinder either music or food was presented according to fixed-interval schedules ranging in value from 10 to 50 seconds. With respect to two principal criteria, namely, pattern of responding and sensitivity to the schedule parameter, the subjects' behavior closely resembled that of animals but differed markedly from that of older children and adults. Negatively accelerated responding in the course of the fixed interval in the early sessions gave way to a scalloped pattern, consisting of a pause after reinforcement followed by an accelerated response rate. This scalloped pattern was the final form of responding on all schedule values. Analysis of data after performance had stabilized showed that postreinforcement pause was a negatively accelerated increasing function, and running rate (calculated after excluding the postreinforcement pause) was a declining function, of schedule value. On each schedule, the durations of mean successive interresponse times declined in the course of the fixed interval and were directly related to schedule value. The results supported Lowe's (1979) suggestion that verbal behavior may be responsible for major differences in the schedule performance of older humans and animals.     

EFFECT OF REINFORCER MAGNITUDE ON PERFORMANCE MAINTAINED BY PROGRESSIVE‐RATIO SCHEDULES

This experiment examined the relationship between reinforcer magnitude and quantitative measures of performance on progressive‐ratio schedules. Fifteen rats were trained under a progressive‐ratio schedule in seven phases of the experiment in which the volume of a 0.6‐M sucrose solution reinforcer was varied within the range 6–300 μl. Overall response rates in successive ratios conformed to a bitonic equation derived from Killeen's (1994) Mathematical Principles of Reinforcement. The “specific activation” parameter, a, which is presumed to reflect the incentive value of the reinforcer, was a monotonically increasing function of reinforcer volume; the “response time” parameter, δ, which defines the minimum response time, increased as a function of reinforcer volume; the “currency” parameter, b, which is presumed to reflect the coupling of responses to the reinforcer, declined as a function of volume. Running response rate (response rate calculated after exclusion of the postreinforcement pause) decayed monotonically as a function of ratio size; the index of curvature of this function increased as a function of reinforcer volume. Postreinforcement pause increased as a function of ratio size. Estimates of a derived from overall response rates and postreinforcement pauses showed a modest positive correlation across conditions and between animals. Implications of the results for the quantification of reinforcer value and for the use of progressive‐ratio schedules in behavioral neuroscience are discussed.     

Effects of pre-operant running and sucrose concentration on operant wheel running on a fixed interval schedule of reinforcement

Prior research proposed that temporal control over the pattern of operant wheel running on a fixed interval (FI) schedule of sucrose reinforcement is a function of automatic reinforcement generated by wheel running and the experimentally arranged sucrose reinforcement. Two experiments were conducted to assess this prediction. In the first experiment, rats ran for different durations (0, 30, 60, and 180 min) prior to a session of operant wheel running on a FI 120-s schedule. In the second experiment, the concentration of sucrose reinforcement on a FI 180-s schedule was varied across values of 0, 5, 15, and 25%. In Experiment 1, as the duration of pre-operant running increased, the postreinforcement pause before initiation of running lengthened while wheel revolutions in the latter part of the FI interval increased. In Experiment 2, wheel revolutions markedly increased then decreased to a plateau early in the FI interval. Neither manipulation increased temporal control of the pattern of wheel running. Instead, results indicate that operant wheel running is regulated by automatic reinforcement generated by wheel activity and an adjunctive pattern of running induced by the temporal presentation of sucrose. Furthermore, the findings question whether the sucrose contingency regulates wheel running as a reinforcing consequence.     

The role of intermittent food in the induction of attack in pigeons

The present experiments evaluated whether transitions in reinforcer probability are necessary to induce attack in pigeons. In Experiment I, three of six pigeons exposed to response-contingent constant-probability food schedules and a photograph of a conspecific as a target exhibited sustained postreinforcement attack on the target. The postreinforcement pattern of attack developed over the course of the experiment and was accompanied by a reduction in the rate of postreinforcement key pecking and an increase in the postreinforcement pause in key pecking. These effects on key pecking resulted in unprogrammed variations in the probability of reinforcement which may have been responsible for the induction of attack. In Experiment II, the attack-inducing properties of a constant-probability response-independent food schedule were compared to a periodic food schedule matched for overall rate of food delivery and to a no-food condition. In addition to attack, the spatial location of the subjects was monitored during each interfood interval. The periodic and aperiodic food schedules generated very different patterns of spatial location. Postfood attack was induced by both food schedules, although the constant-probability schedule induced attack in fewer birds. The no-food condition was not effective in inducing attack in any birds. These experiments indicate that intermittent food schedules without reductions in reinforcer probability are sufficient to induce attack in some pigeons, although not as effective as schedules with transitions in reinforcer probability.     

A comparison of response patterns on fixed-, variable-, and random-ratio schedules

The behavior of individual pigeons on fixed-ratio, variable-ratio, and random-ratio schedules was examined. Within each type of ratio schedule the size of the ratio was varied in an irregular sequence. At various ratio sizes (5, 10, 40, 80) no differences were found among overall response rates (postreinforcement pause plus running response rate) as a function of ratio type. This similarity in overall response rates held despite noticeable differences in the microstructure of performance both within and across subjects; the primary performance difference on the three types of ratio schedules was the relatively longer postreinforcement pause duration on the fixed-ratio schedule. We concluded that the gross temporal characteristics of performance determined by the relative weightings of the postreinforcement pause and running response rate were primarily controlled by the type of ratio schedule (fixed, variable, or random), whereas the overall rate of responding was controlled by the size of the ratio.     

Determinants of pausing under variable-ratio schedules: Reinforcer magnitude, ratio size, and schedule configuration

Pigeons pecked a key under two-component multiple variable-ratio schedules that offered 8-s or 2-s access to grain. Phase 1 assessed the effects of differences in reinforcer magnitude on postreinforcement pausing, as a function of ratio size. In Phase 2, postreinforcement pausing and the first five interresponse times in each ratio were measured as a function of differences in reinforcer magnitude under equal variable-ratio schedules consisting of different configurations of individual ratios. Rates were also calculated exclusive of postreinforcement pause times in both phases. The results from Phase 1 showed that as ratio size increased, the differences in pausing educed by unequal reinforcer magnitudes also increased. The results of Phase 2 showed that the effects of reinforcer magnitude on pausing and IRT durations were a function of schedule configuration. Under one configuration, in which the smallest ratio was a fixed-ratio 1, pauses were unaffected by magnitude but the first five interresponse times were affected. Under the other configuration, in which the smallest ratio was a fixed-ratio 7, pauses were affected by reinforcer magnitude but the first five interresponse times were not. The effect of each configuration seemed to be determined by the value of the smallest individual ratio. Rates calculated exclusive of postreinforcement pause times were, in general, directly related to reinforcer magnitude, and the relation was shown to be a function of schedule configuration.     

Species differences in temporal control of behavior

Temporal control of rats' and pigeons' responding was analyzed and compared in detail on fixed-interval and fixed-time schedules with parameters of 30, 60, and 120 seconds. On fixed-time schedules, rats' responding decreased greatly or ceased, whereas pigeons continued to respond, especially on low schedule values. The running rate of responses (calculated by excluding the postreinforcement pause) was related to the duration of the preceding postreinforcement pause for rats but not for pigeons. Changes in response rate in successive segments of the interval were best described by normal curves. The relationship between midpoints of the normal curves and schedule value was a power function, with an exponent of less than one for pigeons but greater than one for rats. These differences could be explained in terms of a basic difference between the key-peck and lever-press responses, the two being differently affected by the response-eliciting properties of food.     

Within-session changes in responding during several simple schedules

Pigeons' key pecking was reinforced by food delivered by several fixed-interval, variable-ratio, and differential-reinforcement-of-low-rate schedules. Rate of responding, number of responses per reinforcer, length of postreinforcement pause, running response rate, and the time required to collect an available reinforcer changed systematically within sessions when the schedules provided high rates of reinforcement, but usually not when they provided low rates. These results suggest that the factors that produce within-session changes in responding are generally similar for different schedules of reinforcement. However, a separate factor may also contribute during variable-ratio schedules. The results question explanations for within-session changes that are related solely to the passage of time, to responding, and to one interpretation of attention. They support the idea that one or more factors related to reinforcement play a role.     

Alternative fixed-ratio fixed-interval schedules of reinforcement

Five rats were trained under alternative fixed-ratio fixed-interval schedules, in which food reinforcement was provided for the completion of either a fixed-ratio or a fixed-interval requirement, whichever was met first. Overall response rate and running rate (the rate of responding after the postreinforcement pause) decreased for all subjects as the fixed-ratio value increased. As the proportion of reinforcements obtained from the fixed-ratio component increased and the alternative schedule approached a simple fixed ratio, overall response rate and running rate both increased; conversely, as the proportion of reinforcements obtained from the fixed-interval component increased and the alternative schedule approached a simple fixed interval, response rates decreased. Postreinforcement pause length increased linearly as the average time between reinforcements increased, regardless of the schedule parameters. A break-run pattern of responding was predominant at low- and medium-valued fixed ratios. All subjects displayed at least occasional positively accelerated responding within interreinforcement intervals at higher fixed-ratio values.