Separating the effects of interreinforcement time and number of interreinforcement responses

The relative importance of interreinforcement time and interreinforcement responses was evaluated by varying each independently. To do this, a blackout was presented after each nonreinforced response under both fixed-ratio and fixed-interval schedules of reinforcement. Manipulating the blackout duration under the fixed-ratio schedule caused interreinforcement time to vary without affecting the number of interreinforcement responses. Pigeons' post-reinforcement and post-blackout response latencies were found to increase linearly with interreinforcement time. Under the fixed-interval schedule, the same blackout manipulations changed the number of interreinforcement responses without affecting interreinforcement time. Post-reinforcement and post-blackout response latencies under this condition were approximately constant. These results suggest that responding is controlled by interreinforcement time and is not influenced by the number of responses emitted between reinforcements.     

Conjunctive schedules of reinforcement: III. A fixed-interval adjusting fixed-ratio schedule

Key pecking of three pigeons was studied under a conjunctive schedule that specified both a fixed-interval and an adjusting fixed-ratio requirement. The fixed-interval schedule was 6 min for one pigeon and 3 min for the other two. The size of the ratio requirement was determined within each cycle of the fixed interval by the duration of the pause before responding began. The fixed-ratio value was at maximum at the start of each fixed interval and decreased linearly until the first response occurred (adjusting fixed-ratio schedule). A peck produced food when the number of responses remaining on the fixed-ratio schedule was completed and when the fixed interval had elapsed. If no response occurred during the interval, the fixed-ratio requirement decreased to one and a single response after the interval elapsed produced food. The initial value of the adjusting fixed-ratio schedule was studied over a range of 0 to 900. Increases in the adjusting fixed-ratio schedule to about 300 responses increased both pause duration and running response rate and also modified the pattern of responding from that obtained under the fixed-interval schedule. Higher values of the adjusting fixed ratio generally decreased pause duration and running response rate and also disrupted responding. Interreinforcement time under the conjunctive schedule was increased substantially when the adjusting fixed-ratio size exceeded 300 responses.     

A comparison of ratio and interval reinforcement schedules with comparable interreinforcement times

Pigeons were trained to peck keys on fixed-ratio and fixed-interval schedules of food reinforcement. Both schedules produced a pattern of behavior characterized as pause and run, but the relation of pausing to time between reinforcers differed for the two schedules even when mean time between reinforcers was the same. Pausing in the fixed ratio occupied less of the time between reinforcers for shorter interreinforcer times. For two of three birds, the relation was reversed at longer interreinforcer times. As an interreinforcer time elapsed, there was an increasing tendency to return to responding for the fixed interval, but a roughly constant tendency to return to responding for the fixed-ratio schedule. In Experiment 1 these observations were made for both single-reinforcement schedules and multiple schedules of fixed-ratio and fixed-interval reinforcement. In Experiment 2 the observations were extended to a comparison of fixed-ratio versus variable-interval reinforcement schedules, where the distribution of interreinforcement times in the variable interval approximated that for the fixed ratio.     

Sequential patterns in post-reinforcement pauses on fixed-interval schedules of food

Responding by one pigeon was reinforced with food on fixed-interval schedules of 30, 60, and 300 sec duration. A second pigeon was studied under fixed-interval durations of 60 and 300 sec. For both birds, the average post-reinforcement pause was one-half the duration of the fixed interval. Autocorrelation coefficients revealed first-order sequential dependencies in series of post-reinforcement pauses. On the 300-sec fixed-interval schedule, successive post-reinforcement tended to alternate between long and short durations. At the shorter fixed-interval durations there was less evidence of alternation sequences. A second experiment was conducted to determine if the time intervals between the first response after reinforcement and the next reinforcement (the work periods) were responsible for the alternation patterns in the series of post-reinforcement pauses. To evaluate the role of the work period, several procedures were used to modify the work period from that obtained on the fixed-interval 300-sec schedule. Adding a schedule to the fixed-interval schedule that set the minimum amount of time that could elapse between the first response after reinforcement and the next reinforcement eliminated the alternation pattern. Control schedules indicated that the elimination of alternation patterns resulted from constraints on the work period per se and not from confounded changes in the interreinforcement intervals.     

Adjusting fixed-ratio schedules in the squirrel monkey

On an adjusting schedule of reinforcement, a parameter of the schedule is varied as a function of some characteristic of the animal's performance. In Experiment I, the fixed-ratio response requirement was varied as a function of the time that elapsed before the animal started responding in each fixed-ratio (initial pause). When initial pauses were shorter than a specified duration, the response requirement was increased; when they were longer than the specified duration, the response requirement was decreased. Specified durations of 1, 2, 4, 8, and 15 min were studied. The average response requirement maintained by each monkey was directly related to the length of the specified duration of initial pause. In Experiment II, the fixed-ratio response requirement was constant, but reinforcement occurred only when the initial pause was longer than a specified duration. The average durations of initial pauses were directly related to the length of the specified duration and to the response requirement. Meprobamate consistently decreased the average durations of initial pauses.     

Changing the response unit from a single peck to a fixed number of pecks in fixed-interval schedules

Each of three pigeons was studied first under a standard fixed-interval schedule. With the fixed interval held constant, the schedule was changed to a second-order schedule in which the response unit was the behavior on a small fixed-ratio schedule (first a fixed-ratio 10 and then a fixed-ratio 20 schedule). That is, every completion of the fixed-ratio schedule produced a 0.7-sec darkening of the key and reset the response count to zero for the next ratio. The first fixed-ratio completed after the fixed-interval schedule elapsed produced the 0.7-sec blackout followed immediately by food. These manipulations were carried out under two different fixed-interval durations for each bird ranging from 3 min to 12 min. The standard fixed-interval schedules produced the typical pause after reinforcement followed by responding at a moderate rate until the next reinforcement. The second-order schedules also engendered a pause after reinforcement, but responding occurred in bursts separated by brief pauses after each blackout. For a particular fixed-interval duration, post-reinforcement pauses increased slightly as the number of pecks in the response unit increased despite large differences in the rate and pattern of key pecking. Post-reinforcement pause increased with the fixed-interval duration under all response units. These data confirm that the allocation of time between pausing and responding is relatively independent of the rate and topography of responding after the pause.     

The effects of number of responses on the postreinforcement pause in fixed-interval schedules

The present study manipulated the number of responses in a modified fixed-interval schedule by imposing a blackout after each unreinforced response during the interval. The blackout duration was varied, and the duration of the fixed interval was held constant. The subjects were initially exposed to a fixed-interval 300-sec schedule. Blackout durations of 0, 10, and 50 sec were used. Following this, a fixed-interval 30-sec schedule was used with blackout durations of 0, 1, and 5 sec. Under the fixed-interval 300-sec schedule, the number of interreinforcement responses varied over a wider range than occurred under the fixed-interval 30-sec schedule. The duration of the postreinforcement pause decreased as blackout durations were increased and number of responses decreased on the fixed-interval 300-sec schedule, but pause length did not vary with changes in blackout duration and number of responses for the fixed-interval 30-sec schedule. The differences in the effects of blackout duration and response manipulation on the two fixed-interval schedules were attributed to relatively greater changes in the number of interreinforcement responses for the fixed-interval 300-sec schedule.     

Pause relationships in multiple and chained fixed-ratio schedules

On a multiple fixed-ratio 10 fixed-ratio 100 schedule, pigeons pause for relatively long periods of time before the fixed-ratio 100 schedule. Only a short pause occurs before the fixed-ratio 10 schedule. A chain fixed-ratio 10 fixed-ratio 100 schedule produces the reverse pattern, i.e., a short pause before the fixed-ratio 100 schedule and a long pause before the fixed-ratio 10 schedule. Procedurally, the only difference between the two schedules is that the fixed-ratio 10 component is always terminated by some unconditioned reinforcer in the multiple schedule but never in the chained schedule. In the present experiment, the percentage of fixed-ratio 10 components which included reinforcement was gradually decreased for birds on the multiple schedule and gradually increased for birds on the chained schedule. It was found that percentage reinforcement within the fixed-ratio 10 component was inversely related to the duration of the pause before the fixed-ratio 10 component and directly related to the duration of the pause before the fixed-ratio 100 component. Thus, the relative rate of reinforcement paired with a particular stimulus was seen to be an important factor in determining response latency to that stimulus.     

Mirror pecking and timeout under a multiple fixed-ratio schedule of food delivery

Pigeons were trained to peck a key under a multiple fixed-ratio 25 fixed-ratio 175 schedule of food presentation. In the first condition, either a mirror or the opportunity to produce a 30-second timeout were available. In a second condition, mirror and timeout availability were reversed for the two groups. Following a return to the initial condition, mirror and timeout keys were presented together for all birds. Mirror and timeout responses occurred predominantly in the pause in the larger fixed-ratio component, regardless of whether the opportunities for the two responses were available singly or together. Mirror responding occurred in a greater proportion of the pauses than did timeouts. When the opportunities for both mirror pecking and timeout were available concurrently, they occurred with probabilities similar to those under the single conditions. Within the pause itself, mirror responses most frequently occurred immediately after reinforcement. Timeouts occurred most frequently toward the end of the pause, and some timeouts occurred in the early part of the run. Longer preratio pausing occurred in the larger fixed-ratio component in the conditions in which the mirror was present, whether or not any mirror pecks were recorded.     

Responding under fixed-ratio and multiple fixed-interval fixed-ratio schedules of electric shock presentation

Squirrel monkeys, initially trained under a schedule of electric shock postponement and then under fixed-interval schedules of electric shock presentation, were studied under multiple fixed-interval fixed-ratio and under fixed-ratio schedules of shock presentation. Under the fixed-interval (10-min) component of the multiple schedule, a pause was followed by a gradual increase in responding to a rate maintained until shock presentation; under the fixed-ratio (3-, 10-, or 30-response) component of the multiple schedule, a brief pause was typically followed by a relatively high and uniform rate of responding until shock was presented. When the 60-sec timeout periods, which usually followed shock presentation, were eliminated from the multiple schedule for one monkey, responding was only transiently affected. In the one monkey studied, responding was maintained under a fixed-ratio schedule alone (with timeout periods), but rates of responding were lower than under the fixed-ratio component of the multiple schedule. Characteristic patterns of responding, similar to those engendered under schedules of food presentation or shock termination, can be maintained under fixed-ratio schedules of shock presentation; further, patterns of responding can be controlled by discriminative stimuli in multiple schedules.     

A comparison of measures of responding under fixed-interval schedules

Average response rate, post-reinforcement pause, elapsed time to the fourth response, average quarter-life, and running rate were examined to see how they reflected changes in fixed-interval performance. Rats were exposed to a mixed schedule of water presentation comprising fixed-interval schedules of two durations. Changes in responding were produced by varying the duration of the shorter component. The five measures were derived only from the longer schedule component. Post-reinforcement pause, elapsed time to the fourth response in the interval, and quarter-life all showed high, positive inter-correlations (0.78<r<0.99). Running rate and post-reinforcement pause were not as highly correlated. Quarter-life reliably reflected changes in fixed-interval performance but changes in the quarter-life value did not necessarily result from similar changes in fixed-interval response pattern. The two measures that adequately described changes in response patterning were post-reinforcement pause and running rate. These two measures also had the advantage of being simple both computationally and in terms of the instrumentation involved in their recording.     

The effect of reinforcement magnitude upon responding under fixed-ratio schedules

Responding under fixed-ratio schedules was studied as a function of two durations of food presentation. Latency of the first response after food presentation (post-reinforcement pause) was consistently shorter when food was presented for the longer duration. Only one of the four pigeons studied showed a consistently higher response rate, exclusive of post-reinforcement pause, as a function of the longer access to food. When ratio size was reduced, pause durations decreased, and the differences related to the two durations of food presentations became progressively smaller.     

Responding under chained and tandem fixed-ratio schedules

The role of stimuli in chained fixed-ratio schedules of reinforcement was examined. At various ratio values, responding on schedules consisting of three or five equal components, with a different colored light in each component (“block counter”) was compared with responding on tandem or simple fixed-ratio schedules having the same color present throughout the entire ratio. At all ratio values except the smallest, the chain stimuli resulted in longer pauses after reinforcement. The magnitude of this effect became greater as the size of the ratio was increased. Post-reinforcement pause durations were longer under five-component schedules than under three-component schedules. Running rates in the first component were lower on the chained schedules than on the tandem schedules; on both kinds of schedule, rates were lower in the first component than in the rest of the ratio. When the sequence of stimuli was reversed, the duration of the post-reinforcement pause dropped markedly and the running rate in the initial component increased, but these effects gradually disappeared after the first reversal session. When the final chain stimulus was substituted for the first component stimulus but continued to appear in the final chain component as well, the pause duration dropped and remained at this lower level during subsequent sessions.     

Alternative fixed-ratio fixed-interval schedules of reinforcement

Five rats were trained under alternative fixed-ratio fixed-interval schedules, in which food reinforcement was provided for the completion of either a fixed-ratio or a fixed-interval requirement, whichever was met first. Overall response rate and running rate (the rate of responding after the postreinforcement pause) decreased for all subjects as the fixed-ratio value increased. As the proportion of reinforcements obtained from the fixed-ratio component increased and the alternative schedule approached a simple fixed ratio, overall response rate and running rate both increased; conversely, as the proportion of reinforcements obtained from the fixed-interval component increased and the alternative schedule approached a simple fixed interval, response rates decreased. Postreinforcement pause length increased linearly as the average time between reinforcements increased, regardless of the schedule parameters. A break-run pattern of responding was predominant at low- and medium-valued fixed ratios. All subjects displayed at least occasional positively accelerated responding within interreinforcement intervals at higher fixed-ratio values.     

Self-imposed timeouts under increasing response requirements

Self-imposed timeouts by pigeons working under a progressive-ratio food schedule were studied under different conditions. The main findings were (1) continued production of timeouts over an extended series of sessions, (2) more frequent responding on the key with the timeout consequence than on a key having no consequence, (3) an inverse relationship between number of timeouts and level of body weight, (4) production of timeouts when the timeout duration was brief, lengthy, or controlled by the pigeon, and (5) dependence of self-imposed timeouts on variables controlling responding under the progressive-ratio schedule. Under all experimental conditions, with the exception of performances at the high body weight, timeouts were more frequent during the longer progressive-ratio steps and usually were localized in the post-reinforcement pause or the early part of the step. The timeout behavior could be interpreted as either an escape from aversive stimuli generated by the progressive-ratio schedule or as a response reinforced by the consequent stimulus change.     

Response-rate differences in variable-interval and variable-ratio schedules: An old problem revisited

In Experiment 1, a variable-ratio 10 schedule became, successively, a variable-interval schedule with only the minimum interreinforcement intervals yoked to the variable ratio, or a variable-interval schedule with both interreinforcement intervals and reinforced interresponse times yoked to the variable ratio. Response rates in the variable-interval schedule with both interreinforcement interval and reinforced interresponse time yoking fell between the higher rates maintained by the variable-ratio schedule and the lower rates maintained by the variable-interval schedule with only interreinforcement interval yoking. In Experiment 2, a tandem variable-interval 15-s variable-ratio 5 schedule became a yoked tandem variable-ratio 5 variable-interval x-s schedule, and a tandem variable-interval 30-s variable-ratio 10 schedule became a yoked tandem variable-ratio 10 variable-interval x-s schedule. In the yoked tandem schedules, the minimum interreinforcement intervals in the variable-interval components were those that equated overall interreinforcement times in the two phases. Response rates did not decline in the yoked schedules even when the reinforced interresponse times became longer. Experiment 1 suggests that both reinforced interresponse times and response rate–reinforcement rate correlations determine response-rate differences in variable-ratio 10 and yoked variable-interval schedules in rats. Experiment 2 suggests a minimal role for the reinforced interresponse time in determining response rates on tandem variable-interval 30-s variable-ratio 10 and yoked tandem variable-ratio 10 variable-interval x-s schedules in rats.     

Effects of reinforcement magnitude and ratio values on behaviour maintained by a cyclic ratio schedule of reinforcement

In Experiments 1 and 2, lever pressing by rats was reinforced on a cyclic ratio schedule of food reinforcement, comprising a repeated sequence of fixed-ratio component schedules. Reinforcement magnitude was varied, on occasional sessions in Experiment 1 and across blocks of sessions in Experiment 2, from one to two or three 45-mg food pellets. In the one-pellet condition, post-reinforcement pauses increased with component schedule value. At higher magnitudes, post-reinforcement pauses increased, and overall response rates declined. Response rate on component schedules was a decreasing linear function of the obtained rate of reinforcement in all conditions. Plotted against component schedule value, response rate increased exponentially to an asymptote that decreased when reinforcement magnitude increased. These findings are consistent with regulatory accounts of food reinforced behaviour. In Experiment 3, rats were trained under a cyclic ratio schedule comprising fixed-ratio components including higher values, and some inverted U-shaped response functions were obtained. Those rats that did not showthis relationship were trained on cyclic ratios with even higher values, and all showed inverted U-shaped response functions. This suggests that behaviour on cyclic ratio schedules can reflect activating of reinforcement as well as the satiating effects seen in Experiments 1 and 2.     

Conjunctive schedules of reinforcement II: response requirements and stimulus effects

Responding of three pigeons was maintained under conjunctive fixed-ratio, fixed-interval schedules where a key peck produced food after both schedule requirements were completed. The individual schedule requirements were then successively removed and reinstated with responding maintained under the following conditions: conjunctive fixed-ratio, fixed-time; fixed-time; and fixed-interval schedules. Patterns of responding changed in accord with the successive removal of the schedule requirements. Compared to the conjunctive fixed-ratio, fixed-interval schedule, pause duration increased and response rate decreased under conjunctive fixed-ratio, fixed-time schedules and under fixed-time schedules alone. Overall mean rates of responding were highest and pause duration lowest under fixed-interval schedules. When changes in the keylight colors were correlated with completion of the fixed-ratio, the end of the fixed-interval, or both of these conditions, the pattern of responding was modified and indicated a greater degree of control by the individual schedules. Although two birds showed large increases in interreinforcement time when they were initially exposed to the conjunctive schedule, when responding stabilized this measure was largely invariant for all birds across most schedule conditions.     

Temporal control of behavior: schedule interactions

In Experiment I the response that terminated the postreinforcement pauses occurring under a fixed-interval 60-second schedule was reinforced, if the pause duration exceeded 30 seconds. The percentage of such pauses, rather than increasing, decreased. There were complex effects on the discriminative control of the pause by the reinforcer terminating the previous fixed interval, depending on whether the fixed interval and the added reinforcer were the same or different. In Experiments II(a) and II(b), each reinforcement initiated an alternative fixed-interval interresponse-time-greater-than-t-sec schedule, the schedule values being systematically varied. When the response following a pause exceeding a given duration was reinforced, fewer such pauses occurred than when they were not reinforced, i.e., on the comparable simple fixed-interval schedule. There was no systematic relationship between mean interrinforcement interval and duration of the postreinforcement pause. The pause duration initiated by reinforcement was directly related to the dependency controlling the shortest pause at that time, regardless of changes in mean interreinforcement interval.     

Increased reinforcement when timeout from avoidance includes access to a safe place

Three experiments investigated the reinforcing value of access to a safe place during timeout from an avoidance schedule. Rats were trained on conjoint schedules in which responding both postponed shock on a free-operant avoidance schedule and produced periods of timeout on fixed-ratio schedules. In some conditions, a shelf was inserted into the operant chamber during timeout, enabling subjects to get off the grid floor. The combination of timeout and shelf maintained substantially higher response rates than the baseline avoidance schedule with ratio requirements as high as 90 (Experiment I). Adding the shelf to timeouts in one component of multiple fixed-ratio schedules of timeout resulted in higher response rates in the component where the shelf was included (Experiment II). When timeouts with and without the shelf were arranged on concurrent schedules, the shelf-timeout combination was preferred, even when of shorter duration than timeout alone (Experiment III). In all three experiments, subjects climbed on the shelf, although all shocks were cancelled during timeout periods. The results could not be accounted for solely in terms of the reinforcing properties of changes in shock rates, but required an interpretation that ascribed conditioned reinforcing value to stimuli associated with such changes.