The post-reinforcement pause

Measurements of the post-reinforcement pause and response rate were obtained from four birds on a range of fixed ratio schedules from 25 to 150. The results indicated a consistent increase in the length of the pause as the ratio was increased. Response rate tended to decrease, but these data were less consistent and some reversals were apparent.     

The effect of reinforcement magnitude upon responding under fixed-ratio schedules

Responding under fixed-ratio schedules was studied as a function of two durations of food presentation. Latency of the first response after food presentation (post-reinforcement pause) was consistently shorter when food was presented for the longer duration. Only one of the four pigeons studied showed a consistently higher response rate, exclusive of post-reinforcement pause, as a function of the longer access to food. When ratio size was reduced, pause durations decreased, and the differences related to the two durations of food presentations became progressively smaller.     

After-effects of reinforcement magnitude: Dependence upon context

On a fixed-interval schedule with rat subjects the duration of the post-reinforcement pause was found to be an increasing function of the magnitude of the preceding reinforcer. This relationship was observed when two magnitudes were contrasted closely in time, but not when the subjects were trained on each magnitude until the establishment of stable responding. After the behaviour was stable, the effect of the magnitude of reinforcement re-emerged when 50% of the scheduled reinforcers were omitted. Thus, the positive relationship between the magnitude of reinforcement and the duration of the post-reinforcement pause depended on the context of presentation of a given magnitude.     

Responding under chained and tandem fixed-ratio schedules

The role of stimuli in chained fixed-ratio schedules of reinforcement was examined. At various ratio values, responding on schedules consisting of three or five equal components, with a different colored light in each component (“block counter”) was compared with responding on tandem or simple fixed-ratio schedules having the same color present throughout the entire ratio. At all ratio values except the smallest, the chain stimuli resulted in longer pauses after reinforcement. The magnitude of this effect became greater as the size of the ratio was increased. Post-reinforcement pause durations were longer under five-component schedules than under three-component schedules. Running rates in the first component were lower on the chained schedules than on the tandem schedules; on both kinds of schedule, rates were lower in the first component than in the rest of the ratio. When the sequence of stimuli was reversed, the duration of the post-reinforcement pause dropped markedly and the running rate in the initial component increased, but these effects gradually disappeared after the first reversal session. When the final chain stimulus was substituted for the first component stimulus but continued to appear in the final chain component as well, the pause duration dropped and remained at this lower level during subsequent sessions.     

The effects of number of responses on pause length with temporal variables controlled

A change in the size of a fixed-ratio schedule involves a simultaneous change in number of responses, in time to complete the ratio (work time), and in the interval between successive reinforcements (interreinforcement interval). Previous studies have suggested the importance of work time and the interreinforcement interval in controlling the length of the post-reinforcement pause. The present study sought to determine whether number of responses is also a significant factor. Pigeons were trained on a multiple fixed-ratio x fixed-ratio 2 plus timeout schedule in which the size of the fixed-ratio x was manipulated. When the work times (Experiment I) or interreinforcement intervals (Experiment II) were equated for the two components, the pause before the fixed-ratio x was longer than the pause before the fixed-ratio 2 plus timeout. As fixed-ratio x size increased, the relative difference in the lengths of the two types of pauses also increased. Because the fixed-ratio x component contained a larger number of responses than the fixed-ratio 2 plus timeout component, the relatively longer pause preceding the fixed-ratio x indicates that number of responses played a significant role in determining the length of the post-reinforcement pause.     

A comparison of measures of responding under fixed-interval schedules

Average response rate, post-reinforcement pause, elapsed time to the fourth response, average quarter-life, and running rate were examined to see how they reflected changes in fixed-interval performance. Rats were exposed to a mixed schedule of water presentation comprising fixed-interval schedules of two durations. Changes in responding were produced by varying the duration of the shorter component. The five measures were derived only from the longer schedule component. Post-reinforcement pause, elapsed time to the fourth response in the interval, and quarter-life all showed high, positive inter-correlations (0.78<r<0.99). Running rate and post-reinforcement pause were not as highly correlated. Quarter-life reliably reflected changes in fixed-interval performance but changes in the quarter-life value did not necessarily result from similar changes in fixed-interval response pattern. The two measures that adequately described changes in response patterning were post-reinforcement pause and running rate. These two measures also had the advantage of being simple both computationally and in terms of the instrumentation involved in their recording.     

Changing the response unit from a single peck to a fixed number of pecks in fixed-interval schedules

Each of three pigeons was studied first under a standard fixed-interval schedule. With the fixed interval held constant, the schedule was changed to a second-order schedule in which the response unit was the behavior on a small fixed-ratio schedule (first a fixed-ratio 10 and then a fixed-ratio 20 schedule). That is, every completion of the fixed-ratio schedule produced a 0.7-sec darkening of the key and reset the response count to zero for the next ratio. The first fixed-ratio completed after the fixed-interval schedule elapsed produced the 0.7-sec blackout followed immediately by food. These manipulations were carried out under two different fixed-interval durations for each bird ranging from 3 min to 12 min. The standard fixed-interval schedules produced the typical pause after reinforcement followed by responding at a moderate rate until the next reinforcement. The second-order schedules also engendered a pause after reinforcement, but responding occurred in bursts separated by brief pauses after each blackout. For a particular fixed-interval duration, post-reinforcement pauses increased slightly as the number of pecks in the response unit increased despite large differences in the rate and pattern of key pecking. Post-reinforcement pause increased with the fixed-interval duration under all response units. These data confirm that the allocation of time between pausing and responding is relatively independent of the rate and topography of responding after the pause.     

Effects of concurrent schedules on human fixed-interval performance

Young adults performed a lever-pressing task for money on two schedules of reinforcement: concurrent fixed-interval 1 min—differential-reinforcement-of-low-rate 20-sec, and concurrent fixed-interval 1-min—fixed ratio 100 responses. All subjects were trained on both schedules. Fixed-interval performance concurrent with the differential reinforcement procedure was characterized by high constant rates with no post-reinforcement pauses. Fixed-interval performance concurrent with fixed ratio was characterized by low rates and lengthy post-reinforcement pauses. These results differ from those obtained in prior studies on the effects of conditioning history upon subsequent fixed-interval performance. The prior work, using non-concurrent procedures, had shown that fixed-interval performance following differential reinforcement of low rates was characterized by post-reinforcement pauses and low rates, while fixed-interval performance following fixed ratio exhibited high constant rates and no post-reinforcement pause. The present results suggest that alternative concurrent contingencies are another major determinant of human fixed-interval performance.     

Sequential patterns in post-reinforcement pauses on fixed-interval schedules of food

Responding by one pigeon was reinforced with food on fixed-interval schedules of 30, 60, and 300 sec duration. A second pigeon was studied under fixed-interval durations of 60 and 300 sec. For both birds, the average post-reinforcement pause was one-half the duration of the fixed interval. Autocorrelation coefficients revealed first-order sequential dependencies in series of post-reinforcement pauses. On the 300-sec fixed-interval schedule, successive post-reinforcement tended to alternate between long and short durations. At the shorter fixed-interval durations there was less evidence of alternation sequences. A second experiment was conducted to determine if the time intervals between the first response after reinforcement and the next reinforcement (the work periods) were responsible for the alternation patterns in the series of post-reinforcement pauses. To evaluate the role of the work period, several procedures were used to modify the work period from that obtained on the fixed-interval 300-sec schedule. Adding a schedule to the fixed-interval schedule that set the minimum amount of time that could elapse between the first response after reinforcement and the next reinforcement eliminated the alternation pattern. Control schedules indicated that the elimination of alternation patterns resulted from constraints on the work period per se and not from confounded changes in the interreinforcement intervals.     

Aversive aspects of a fixed-interval schedule of food reinforcement

The key pecking of pigeons was reinforced according to a fixed-interval schedule of reinforcement. The pigeons were also given the opportunity to attack a restrained target pigeon. The attack rates during the sessions of fixed-interval reinforcement were higher than during the operant level sessions in four of the five pigeons. Most attack occurred during the post-reinforcement pause in key pecking. It was suggested that a fixed-interval schedule of positive reinforcement possesses aversive properties, the most aversive of which are located during the post-reinforcement pause.     

Concurrent fixed-ratio fixed-interval performances in adult human subjects

Two undergraduate males worked for money on a button-pressing task associated with concurrent fixed-ratio fixed-interval schedules of reinforcement. Manipulations of the fixed-ratio requirement produced an interaction between the various fixed-ratio and fixed-interval performances. When the fixed ratio was small, more fixed-interval responding occurred per interval than when the fixed ratio was large. In general, the data were similar to those obtained with lower organisms except that no post-reinforcement pause or ratio strain was seen.     

Fixed-ratio performance under conditions of delayed reinforcement

Four rats were trained on a schedule in which completion of a fixed number of lever presses initiated a signalled delay period, at the end of which food was delivered. Lever presses made during the delay had no scheduled consequences. Delays of 12, 3, and 0.75 sec were used, and it was found that the latency of the first response after food (the post-reinforcement pause) increased with length of delay. There was, on the other hand, no consistent effect of delay upon rates of responding after the post-reinforcement pause.     

Effects of reinforcement magnitude and ratio values on behaviour maintained by a cyclic ratio schedule of reinforcement

In Experiments 1 and 2, lever pressing by rats was reinforced on a cyclic ratio schedule of food reinforcement, comprising a repeated sequence of fixed-ratio component schedules. Reinforcement magnitude was varied, on occasional sessions in Experiment 1 and across blocks of sessions in Experiment 2, from one to two or three 45-mg food pellets. In the one-pellet condition, post-reinforcement pauses increased with component schedule value. At higher magnitudes, post-reinforcement pauses increased, and overall response rates declined. Response rate on component schedules was a decreasing linear function of the obtained rate of reinforcement in all conditions. Plotted against component schedule value, response rate increased exponentially to an asymptote that decreased when reinforcement magnitude increased. These findings are consistent with regulatory accounts of food reinforced behaviour. In Experiment 3, rats were trained under a cyclic ratio schedule comprising fixed-ratio components including higher values, and some inverted U-shaped response functions were obtained. Those rats that did not showthis relationship were trained on cyclic ratios with even higher values, and all showed inverted U-shaped response functions. This suggests that behaviour on cyclic ratio schedules can reflect activating of reinforcement as well as the satiating effects seen in Experiments 1 and 2.     

Controlling human fixed-interval performance

Both high and relatively constant rates of responding without post-reinforcement pauses and lower rates with pauses after reinforcement are produced by human subjects under fixed-interval (FI) schedules. Such FI rates and patterns may be controlled when subjects are provided with different histories of conditioning and different conditions of response cost (reinforcement penalties per response). Subjects with a conditioning history under ratio schedules typically produce high and relatively constant rates of responding under FI schedules; this responding does not change systematically with changes in FI value. In contrast, subjects with a history under schedules which produce little or no responding between reforcements [such as differential-reinforcement-of-low-rate (DRL) schedules] tend to pause after reinforcement and respond at low rates under FI schedules, whether or not they also have ratio conditioning histories; cost increases the likelihood of this type of performance. For DRL-history subjects, post-reinforcement pauses increase and response rates decrease as FI values increase.     

Time limits for completing fixed ratios

Two experiments investigated how the addition of time limits affected fixed-ratio behavior. In Exp. 1, pigeons obtained food only if they completed the ratio within a specified time after the end of the preceding ratio. In Exp. 2, they obtained food only if they took longer than a specified time. Failures to meet the time criteria produced brief timeouts. The times taken depended on the requirements in both experiments. In Exp. 1, progressively briefer time criteria resulted in faster ratios, and in Exp. 2, longer time criteria increased the time taken in each ratio. The pigeon's sensitivity to the temporal variable, a property of the entire period extending from the first opportunity to respond to the end of the ratio, indicated that performance involved a behavioral unit encompassing both the post-reinforcement pause and the responses comprising the ratio.     

Properties of behavior under random interval reinforcement schedules

In a temporally defined system of reinforcement schedules, the fixed interval case is defined when reinforcement probability, P, is equal to unity for the first response in any cycle length, T; when P is less than 1.0, random interval schedules emerge wherein T/P specifies the expected interval between reinforcements. Key-pecking rates were found to be: (a) inversely related to T/P; (b) higher at T=1.0 second than at other T parameter values; (c) low and linear at several T and T/P values. The mean post-reinforcement pause, if initially small, increased, and if initially large, decreased, as T/P increased.     

Fixed-interval performance: The dynamics of behavior and the interval length

Postreinforcement pauses from successive intervals under various fixed-interval schedules (ranging from 15 seconds to 480 seconds in length) were subjected to lag-1 autocorrelation analysis. Results from both rats and pigeons suggested that there was a consistent tendency for pause values in successive intervals to be weakly positively related. This tendency did not appear to change systematically with interval length and was exhibited both when the reinforcer magnitude was constant and when it was variable at different interval values. The findings do not support suggestions that the dynamic properties of performance under fixed-interval schedules vary systematically with interval length, and are in the opposite direction from some previous findings suggesting that measures of behavior (such as post-reinforcement pause length or number of responses) in successive intervals are inversely related.     

Conjunctive schedules of reinforcement II: response requirements and stimulus effects

Responding of three pigeons was maintained under conjunctive fixed-ratio, fixed-interval schedules where a key peck produced food after both schedule requirements were completed. The individual schedule requirements were then successively removed and reinstated with responding maintained under the following conditions: conjunctive fixed-ratio, fixed-time; fixed-time; and fixed-interval schedules. Patterns of responding changed in accord with the successive removal of the schedule requirements. Compared to the conjunctive fixed-ratio, fixed-interval schedule, pause duration increased and response rate decreased under conjunctive fixed-ratio, fixed-time schedules and under fixed-time schedules alone. Overall mean rates of responding were highest and pause duration lowest under fixed-interval schedules. When changes in the keylight colors were correlated with completion of the fixed-ratio, the end of the fixed-interval, or both of these conditions, the pattern of responding was modified and indicated a greater degree of control by the individual schedules. Although two birds showed large increases in interreinforcement time when they were initially exposed to the conjunctive schedule, when responding stabilized this measure was largely invariant for all birds across most schedule conditions.     

Escape from SD associated with fixed-ratio reinforcement

Throughout ascending and descending fixed-ratio (FR) sequences, rats were allowed to terminate the FR stimulus control by pressing a time-out (TO) lever. To minimize chance or accidental responses on this second lever, three presses were required to produce the 30-sec S^Δ period. As FR performance became more “strained,” there was an increased predisposition to escape from the time-in stimulus complex. The generality of this finding was extended by obtaining recoverability (independent of the direction of stimulus change) of the FR-TO function in the descending series. Typically, escapes were produced only during the post-reinforcement pause; however, under a mixed FR FR schedule, their occurrence shifted to a point within the inter-reinforcement interval corresponding to the unreinforced completion of the lower ratio component. It appears that the point where the rat can discriminate the size of the ratio requirement will be the place where TOs are imposed. This inference was supported by a substantial increase in TO frequency accompanying a shift from CRF to extinction on the FR lever. Finally, the escape lever was placed on a progressively increasing FR schedule and later extinguished to demonstrate that the TO condition was in fact reinforcing.     

Selective punishment early and late in fixed-ratio schedules of food reinforcement

Pigeons key pecked for grain on a fixed-ratio 100 schedule; electric shocks occurred intermittently at the fifteenth or eighty-fifth response in the ratio. In Experiment I, shock was at the fifteenth response for two birds, and at the eighty-fifth response for two others, in every sixth, twelfth, or eighteenth ratio. Rate of responding decreased as frequency of shock increased, and the pattern of responding included an increased initial pause and low rates or pause-run sequences that extended further into the ratio when shock was at the fifteenth response than when it was at the eighty-fifth response. Shock early in the ratio engendered longer initial pauses than shock late in the ratio. In Experiment II, four birds responded on a two-component multiple schedule in which shock occurred at the fifteenth response of the third ratio in the presence of a white keylight and at the eighty-fifth response of the third ratio in the presence of a green keylight. The overall rates of responding decreased as shock intensity increased. All four birds responded differentially to the white and green keylights, but with a pattern that varied between birds. In general, punishment reduced the probability of responses that preceded it, regardless of the ordinal position of those responses. Both studies confirm that the probability of responding is reduced less by aversive stimuli produced late in a fixed-ratio than by aversive stimuli produced early in a fixed-ratio.