Response persistence under ratio and interval reinforcement schedules

In Experiment 1, rats were exposed to progressive-ratio schedules of food reinforcement while other rats were exposed simultaneously to yoked-interval schedules that arranged equivalent interreinforcer intervals but required only a single response at the end of the interval for food delivery. In Experiment 2, a within-subject yoked-control procedure was employed in which pigeons were exposed to alternating sessions (one per day) of progressive-ratio schedules and yoked-interval schedules as described above. In both experiments, responding under the yoked-interval schedule persisted beyond the point at which responding under the progressive-ratio schedule had ceased. The progressive-ratio schedules controlled break-and-run distributions, and the yoked-interval schedules controlled more even distributions of responses in time. Response rates decreased and postreinforcement pauses increased over time within individual sessions under both schedules. The results suggest that responding maintained by interval schedules is more persistent than that maintained by ratio schedules. The limitations and implications of this conclusion are discussed in the context of other investigations of response strength and behavioral momentum.     

Effects of ratio reinforcement schedules on discrimination performance by Japanese monkeys

In Experiment 1, Japanese monkeys were trained on three conditional position-discrimination problems with colors as the conditional cues. Within each session, each problem was presented for two blocks of ten reinforcements; correct responses were reinforced under continuous-reinforcement, fixed-ratio 5, and variable-ratio 5 schedules, each assigned to one of the three problems. The assignment of schedules to problems was rotated a total of three times (15 sessions per assignment) after 30 sessions of acquisition training. Accuracy of discrimination increased to a moderate level with fewer trials under CRF than under ratio schedules. In contrast, the two ratio schedules, fixed and variable, were more effective in maintaining accurate discrimination than was CRF. With further training, as asymptotes were reached, accuracy was less affected by the schedule differences. These results demonstrated an interaction between the effects of reinforcement schedules and the level of acquisition. In Experiment 2, ratio sizes were gradually increased to 30. Discrimination accuracy was maintained until the ratio reached 20; ratio 30 strained the performance. Under FR conditions, accuracy increased as correct choice responses cumulated after reinforcement.     

Schedule-induced defecation by rats during ratio and interval schedules of food reinforcement.

Lever pressing in rats was maintained by continuous and intermittent schedules of food while defecation was monitored. In Experiment 1, reinforcement densities were matched across variable-ratio and variable-interval schedules for three pairs of rats. Defecation occurred in all 3 rats on the variable-ratio schedule and in all 3 rats on the yoked variable-interval schedule. In Experiment 2, fixed-ratio and fixed-interval schedules with similar reinforcement densities maintained lever pressing. Defecation occurred in 3 of 4 rats on the fixed-ratio schedule and in 4 of 4 rats on the fixed-interval schedule. Almost no defecation occurred during continuous reinforcement in either experiment. These results demonstrate that defecation may occur during both ratio and interval schedules and that the inter-reinforcement interval is more important than the behavioral requirements of the schedule in generating schedule-induced defecation.     

Septal lesions lower responding under fixed-ratio schedules of reinforcement.

Albino Sprague-Dawley rats with complete septal lesions and rats with control operations were studied under fixed-ratio (FR) schedules of reinforcement. Both groups were trained for 10 sessions each under FR 10, 20, 40, 60, 80, and 100. In contrast to findings from progressive FR studies and some simple FR studies, septal lesions resulted in lower overall and local response rates along with longer postreinforcement pauses. These effects were especially evident during the FR 100 schedule of reinforcement. A comparison of reinforcement rate as a function of FR size within the context of behavioral economics (i.e., a demand function) indicated that septal lesions did not alter the reward value of food. These findings suggest that responding on FR schedules of reinforcement can be altered by the various procedures used to train rats to reach the terminal value of a reinforcement schedule.     

Second-order schedules of token reinforcement with pigeons: effects of fixed- and variable-ratio exchange schedules

Pigeons' key pecks produced food under second-order schedules of token reinforcement, with light-emitting diodes serving as token reinforcers. In Experiment 1, tokens were earned according to a fixed-ratio 50 schedule and were exchanged for food according to either fixed-ratio or variable-ratio exchange schedules, with schedule type varied across conditions. In Experiment 2, schedule type was varied within sessions using a multiple schedule. In one component, tokens were earned according to a fixed-ratio 50 schedule and exchanged according to a variable-ratio schedule. In the other component, tokens were earned according to a variable-ratio 50 schedule and exchanged according to a fixed-ratio schedule. In both experiments, the number of responses per exchange was varied parametrically across conditions, ranging from 50 to 400 responses. Response rates decreased systematically with increases in the fixed-ratio exchange schedules, but were much less affected by changes in the variable-ratio exchange schedules. Response rates were consistently higher under variable-ratio exchange schedules than tinder comparable fixed-ratio exchange schedules, especially at higher exchange ratios. These response-rate differences were due both to greater pre-ratio pausing and to lower local rates tinder the fixed-ratio exchange schedules. Local response rates increased with proximity to food under the higher fixed-ratio exchange schedules, indicative of discriminative control by the tokens.     

The effect of punishment shock intensity upon responding under multiple schedules

In the first of two experiments, responses of two pigeons were maintained by multiple variable-interval, variable-ratio schedules of food reinforcement. Concurrent punishment was introduced, which consisted of a brief electric shock after each tenth response. The initial punishment intensities had no lasting effect upon responding. Then, as shock intensity increased, variable-ratio response rates were suppressed more quickly than variable-interval response rates. When shock intensity decreased, variable-interval responding recovered more quickly, but the rates under both schedules eventually returned to their pre-punishment levels. In the second experiment, the following conditions were studied in three additional pigeons: (1) With each shock intensity in effect for a number of sessions, punishment shock intensity was gradually increased and decreased and responding was maintained by multiple variable-ratio, fixed-ratio schedules of food reinforcement; (2) Changes in punishment shock intensity as described above with responding maintained by either a variable-ratio or a fixed-ratio schedule, which were presented on alternate days; (3) Session-to-session changes in shock intensity with responding maintained by multiple variable-ratio, fixed-ratio schedules. Responding under the two schedules was suppressed to approximately the same extent by a particular shock intensity. Also, post-reinforcement pauses under the fixed-ratio schedule increased as response suppression increased.     

Changing the response unit from a single peck to a fixed number of pecks in fixed-interval schedules

Each of three pigeons was studied first under a standard fixed-interval schedule. With the fixed interval held constant, the schedule was changed to a second-order schedule in which the response unit was the behavior on a small fixed-ratio schedule (first a fixed-ratio 10 and then a fixed-ratio 20 schedule). That is, every completion of the fixed-ratio schedule produced a 0.7-sec darkening of the key and reset the response count to zero for the next ratio. The first fixed-ratio completed after the fixed-interval schedule elapsed produced the 0.7-sec blackout followed immediately by food. These manipulations were carried out under two different fixed-interval durations for each bird ranging from 3 min to 12 min. The standard fixed-interval schedules produced the typical pause after reinforcement followed by responding at a moderate rate until the next reinforcement. The second-order schedules also engendered a pause after reinforcement, but responding occurred in bursts separated by brief pauses after each blackout. For a particular fixed-interval duration, post-reinforcement pauses increased slightly as the number of pecks in the response unit increased despite large differences in the rate and pattern of key pecking. Post-reinforcement pause increased with the fixed-interval duration under all response units. These data confirm that the allocation of time between pausing and responding is relatively independent of the rate and topography of responding after the pause.     

A comparison of ratio and interval reinforcement schedules with comparable interreinforcement times

Pigeons were trained to peck keys on fixed-ratio and fixed-interval schedules of food reinforcement. Both schedules produced a pattern of behavior characterized as pause and run, but the relation of pausing to time between reinforcers differed for the two schedules even when mean time between reinforcers was the same. Pausing in the fixed ratio occupied less of the time between reinforcers for shorter interreinforcer times. For two of three birds, the relation was reversed at longer interreinforcer times. As an interreinforcer time elapsed, there was an increasing tendency to return to responding for the fixed interval, but a roughly constant tendency to return to responding for the fixed-ratio schedule. In Experiment 1 these observations were made for both single-reinforcement schedules and multiple schedules of fixed-ratio and fixed-interval reinforcement. In Experiment 2 the observations were extended to a comparison of fixed-ratio versus variable-interval reinforcement schedules, where the distribution of interreinforcement times in the variable interval approximated that for the fixed ratio.     

Percentage reinforcement of fixed-ratio and variable-interval performances

Twenty to seventy per cent of the reinforcements scheduled for pigeons' fixed-ratio 80 performances were replaced by a 4-sec timeout. Pauses after reinforced ratios were unchanged at 80% reinforcement, but were lengthened at lower reinforcement percentages. Pauses after nonreinforced ratios were shorter than post-reinforcement pauses. When 50% of the reinforcements arranged by a variable-interval 60-sec schedule were replaced by a 4-sec timeout, pauses after reinforcement omission increased. Both frustrative nonreward and reinforcement aftereffects notions can explain the fixed-ratio results; neither easily explains the variable-interval data.     

Alternative fixed-ratio fixed-interval schedules of reinforcement

Five rats were trained under alternative fixed-ratio fixed-interval schedules, in which food reinforcement was provided for the completion of either a fixed-ratio or a fixed-interval requirement, whichever was met first. Overall response rate and running rate (the rate of responding after the postreinforcement pause) decreased for all subjects as the fixed-ratio value increased. As the proportion of reinforcements obtained from the fixed-ratio component increased and the alternative schedule approached a simple fixed ratio, overall response rate and running rate both increased; conversely, as the proportion of reinforcements obtained from the fixed-interval component increased and the alternative schedule approached a simple fixed interval, response rates decreased. Postreinforcement pause length increased linearly as the average time between reinforcements increased, regardless of the schedule parameters. A break-run pattern of responding was predominant at low- and medium-valued fixed ratios. All subjects displayed at least occasional positively accelerated responding within interreinforcement intervals at higher fixed-ratio values.     

Choice for aperiodic versus periodic ratio schedules: A comparison of concurrent and concurrent-chain procedures

Choice between mixed-ratio schedules, consisting of equiprobable ratios of 1 and 99 responses per reinforcement, and fixed-ratio schedules of food reinforcement was assessed by two commonly used procedures: concurrent schedules and concurrent-chains schedules. Rats were trained under concurrent fixed-ratio mixed-ratio schedules, in which both ratio schedules were simultaneously available, and under a concurrent-chains schedule, in which access to one of the mutually exclusive ratio schedules comprising the terminal links was contingent on a single “choice” response. The distribution of responses between the two ratio schedules was taken as the choice proportion under the concurrent procedure, and the distribution of “choice” responses was taken as the choice proportion under the concurrent-chains procedure. Seven of eight rats displayed systematic choice; of those, each displayed nearly exclusive choice for fixed-ratio 35 to the mixed-ratio schedule under the concurrent procedure, but each displayed nearly exclusive choice for the mixed-ratio schedule to fixed-ratio 35 under the concurrent-chains procedure. Thus, preference for a fixed or a mixed schedule of reinforcement depended on the procedure used to assess preference.     

Second-order schedules of token reinforcement: comparisons of performance under fixed-ratio and variable-ratio exchange schedules

Rats' lever pressing produced tokens according to a 20-response fixed-ratio schedule. Sequences of token schedules were reinforced under a second-order schedule by presentation of periods when tokens could be exchanged for food pellets. When the exchange period schedule was a six-response fixed ratio, patterns of completing the component token schedules were bivalued, with relatively long and frequent pauses marking the initiation of each new sequence. Altering the exchange period schedule to a six-response variable ratio resulted in sharp reductions in the frequency and duration of these initial pauses, and increases in overall rates of lever pressing. These results are comparable to those ordinarily obtained under simple fixed-ratio and variable-ratio schedules.     

Fixed interval schedules and delay of reinforcement

In a fixed interval schedule of reinforcement the only responses to be reinforced are those made when a certain time interval has elapsed since the previous reinforcement. The behaviour of three rats on such a schedule was compared with their behaviour on a schedule where a response made at any time during the interval was reinforced by setting up a reward which was delivered when the interval had elapsed. Response rates were higher in the ordinary fixed interval schedule than in its modified version, and it is argued that this rules out attempts to explain the maintenance of fixed interval performance by delayed reinforcement. Despite the clear difference in response rates, there was considerable similarity between the post-reinforcement pauses developed in the two schedules, and this suggests that pausing is influenced more by temporal than by response contingencies.     

Responding under chained and tandem fixed-ratio schedules

The role of stimuli in chained fixed-ratio schedules of reinforcement was examined. At various ratio values, responding on schedules consisting of three or five equal components, with a different colored light in each component (“block counter”) was compared with responding on tandem or simple fixed-ratio schedules having the same color present throughout the entire ratio. At all ratio values except the smallest, the chain stimuli resulted in longer pauses after reinforcement. The magnitude of this effect became greater as the size of the ratio was increased. Post-reinforcement pause durations were longer under five-component schedules than under three-component schedules. Running rates in the first component were lower on the chained schedules than on the tandem schedules; on both kinds of schedule, rates were lower in the first component than in the rest of the ratio. When the sequence of stimuli was reversed, the duration of the post-reinforcement pause dropped markedly and the running rate in the initial component increased, but these effects gradually disappeared after the first reversal session. When the final chain stimulus was substituted for the first component stimulus but continued to appear in the final chain component as well, the pause duration dropped and remained at this lower level during subsequent sessions.     

Second-order schedules with fixed-ratio components: variation of component size

Key pecking by pigeons was reinforced with food under second-order schedules with fixed-ratio units. A constant total number of key pecks was required for reinforcement under each condition, but the size and, inversely, number of fixed-ratio components were varied. The total response requirement of 256 pecks was divided into fixed-ratio units of 128, 64, 32, 8, and 2 responses. A brief stimulus, which always preceded food reinforcement, was presented upon completion of each fixed-ratio unit. Under most conditions, the pattern of within-unit responding was typical of that under simple fixed-ratio schedules. Overall response rate was an inverted U-shaped function of component size. That is, response rates were highest under moderate sized units (fixed ratio 128 and 64). This relationship is consistent with previous determinations of rate as a function of fixed-ratio value for simple fixed-ratio schedules.     

The effects of number of responses on pause length with temporal variables controlled

A change in the size of a fixed-ratio schedule involves a simultaneous change in number of responses, in time to complete the ratio (work time), and in the interval between successive reinforcements (interreinforcement interval). Previous studies have suggested the importance of work time and the interreinforcement interval in controlling the length of the post-reinforcement pause. The present study sought to determine whether number of responses is also a significant factor. Pigeons were trained on a multiple fixed-ratio x fixed-ratio 2 plus timeout schedule in which the size of the fixed-ratio x was manipulated. When the work times (Experiment I) or interreinforcement intervals (Experiment II) were equated for the two components, the pause before the fixed-ratio x was longer than the pause before the fixed-ratio 2 plus timeout. As fixed-ratio x size increased, the relative difference in the lengths of the two types of pauses also increased. Because the fixed-ratio x component contained a larger number of responses than the fixed-ratio 2 plus timeout component, the relatively longer pause preceding the fixed-ratio x indicates that number of responses played a significant role in determining the length of the post-reinforcement pause.     

Multiple determinants of the effects of reinforcement magnitude on free-operant response rates

Four experiments examined the effects of increasing the number of food pellets given to hungry rats for a lever-press response. On a simple variable-interval 60-s schedule, increased number of pellets depressed response rates (Experiment 1). In Experiment 2, the decrease in response rate as a function of increased reinforcement magnitude was demonstrated on a variable-interval 30-s schedule, but enhanced rates of response were obtained with the same increase in reinforcement magnitude on a variable-ratio 30 schedule. In Experiment 3, higher rates of responding were maintained by the component of a concurrent variable-interval 60-s variable-interval 60-s schedule associated with a higher reinforcement magnitude. In Experiment 4, higher rates of response were produced in the component of a multiple variable-interval 60-s variable-interval 60-s schedule associated with the higher reinforcement magnitude. It is suggested that on simple schedules greater reinforcer magnitudes shape the reinforced pattern of responding more effectively than do smaller reinforcement magnitudes. This effect is, however, overridden by another process, such a contrast, when two magnitudes are presented within a single session on two-component schedules.     

Post-reinforcement pauses and response rate of monkeys on a two-hand fixed-ratio schedule

Fixed-ratio behavior of monkeys was analyzed separately for two hands. While one hand responded on the fixed-ratio schedule the other performed a holding response and the function of the hands changed in alternate ratio runs. After performance was stable on the fixed ratio (70 responses, two monkeys; 100 responses, two monkeys, 120 responses, two monkeys) 90 sessions of further training equalized post-reinforcement pauses and the mean interresponse time of the two hands. Hand preference in reaching for food remained unchanged. Then, the fixed-ratio requirement was changed (a) in small sequential steps, (b) in two large steps, and, (c) within sessions alternating two runs at a high ratio with two runs at a low ratio. The mean duration of post-reinforcement pauses was correlated with a fixed ratio maintained throughout a session but single pauses were neither controlled by the immediately preceding nor by the following ratio run when a cue to its length was available. The mean interresponse time was insensitive to changes in fixed ratio. The fixed-ratio performance was generally similar to that of pigeons and rats.     

Cooperative responding in rats: II. Performance on fixed-ratio schedules of mutual reinforcement

Coordinated responses of 5 dyads of rats were investigated under fixed-ratio (FR) schedules of mutual water reinforcement. Coordinated responding was defined as 2 consecutive lever-presses, 1 from each of 2 rats, occurring <.5 s apart. In the FR schedules, each coordinated episode was defined as 1 response in the FR sequence. The size of FR schedules was parametrically manipulated assuming the values of FR 1, 6, 12, 18, 24, 30, 50, and 9, in this order. Each FR remained in effect until responding reached stability. Under all conditions, pairs of rats received access to water simultaneously (mutual reinforcement). Rates and proportions of coordinated responding showed a bitonic inverted U-shaped function of ratio size. Postreinforcement pauses increased systematically as the interreinforcement interval increased. Local rates and proportions increased as a function of response location within ratios. Results of a control condition with relaxed temporal constraints for mutual reinforcement showed decreases in rates and proportion of coordinated responses, suggesting that the coordinated responses were controlled by the mutual reinforcement contingencies. The present experiment showed that coordinated responding is quantitatively affected by 3 properties of FR schedules: response requirement, reinforcement rates, and proximity to reinforcement.     

Fixed-ratio reinforcement of spaced responding

Responses by rats were reinforced with food under a second-order schedule involving fixed-ratio reinforcement of temporally spaced responses. Requirements of 20, 8, and 3 responses were examined. The typical characteristic of spaced responding was maintained under the ratio schedules: interresponse time distributions were similar to those typically seen, and were not noticeably affected by the ratio value. Comparison of total response rate, correct response rate, and accuracy showed correct response rate to be the most consistently affected by changes in the ratio value. Substantial evidence of schedule control was seen only for correct responses. Incorrect response records were erratic, but rates generally declined as reinforcement was approached. Correct response records were characterized by increasing rate as reinforcement was approached. It was suggested that the pattern of fixed-ratio performance revealed may be affected by the behavioral unit examined.