Temporal control of periodic schedules: signal properties of reinforcement and blackout

Pigeons were exposed to periodic food-reinforcement schedules in which intervals ended with equal probability in either reinforcement or brief blackout. The effects on the pattern of key pecking of sequential probability of reinforcement, interval duration, and time to reinforcement opportunity were investigated in three experiments. The major results were: (1) at short absolute interval durations, time to reinforcement opportunity determined both postreinforcement and postblackout pause (time to first key peck within an interval); (2) at long intervals, postblackout pause was consistently shorter than postreinforcement pause, even if both events signalled the same time to the next reinforcement opportunity (omission effect); (3) when reinforcement and blackout signalled different times to the next reinforcement opportunity, within the same experiment, there was some evidence for interactions analogous to behavioral contrast.     

Dynamics of time discrimination

Pigeons tracked sinusoidal sequences of interfood intervals (IFIs) by pausing in each interval for a time proportional to the preceding interval. Schedules with either long (30-90 s) or short (5-15 s) values, with variable numbers of cycles and starting phase each day, were tracked about equally well. Tracking was apparently immediate and did not improve across sessions. Experiment 2, in which long and short series were presented on alternate days, showed that tracking on long was more impaired than on short. Experiment 3 showed that occasional presentation of a short IFI in a series of fixed, longer IFIs caused a reduction in waiting time in the next IFI. These effects are evidence for a fast-acting timing mechanism in which waiting time in the IFI N + 1 is strongly determined by the preceding IFI, N. Earlier IFIs have some cumulative effect, but the details remain to be elucidated.     

Periodicities within a fixed-interval session

Within-session periodicities in number of responses per interval and postreinforcement pauses were investigated on fixed-interval schedules of 1, 2, and 3 minutes with rats. Postreinforcement pause values and the number of responses in successive intervals were not systematically related. The direction of change of these variables from one pair of intervals to the next revealed periodicities in that the direction of change varied more than would be expected by chance. A response prevention technique used to manipulate the length of time spent responding in an interval had little effect on the postreinforcement pause value of the next interval except when only a single response was permitted in an interval. This procedure tended to reduce the postreinforcement pause value of the next interval to an abnormally low level.     

Dynamics of time discrimination: II. The effects of multiple impulses.

According to a diffusion generalization model, time discrimination is determined by the frequency and recency of preceding intervals of time. A procedure for studying rapid timing was used to investigate whether pigeons' wait-time responses were sensitive to these factors. In Experiment 1 the number (two or eight) and spacing (consecutive or far apart) of 5-s interfood intervals (called impulses) intercalated in a series of 15-s interfood intervals (nonimpulses) were studied. Experiment 2 was identical to the first but the interfood intervals were increased by a factor of three. Overall, impulses shortened wait times in the next interfood interval. However, several impulses occurring in succession extended the localized effect of an impulse: Wait times following a set of eight-close impulses were slow to recover to preimpulse levels. The results show that linear waiting is only an approximation to the dynamic process, and a process that is sensitive to events in an animal's remote past, such as the diffusion generalization model, provides a better account of rapid timing effects.     

The effects of interval duration on temporal tracking and alternation learning

On cyclic-interval reinforcement schedules, animals typically show a postreinforcement pause that is a function of the immediately preceding time interval (temporal tracking). Animals, however, do not track single-alternation schedules-when two different intervals are presented in strict alternation on successive trials. In this experiment, pigeons were first trained with a cyclic schedule consisting of alternating blocks of 12 short intervals (5 s or 30 s) and 12 long intervals (180 s), followed by three different single-alternation interval schedules: (a) 30 s and 180 s, (b) 5 s and 180 s, and (c) 5 s and 30 s. Pigeons tracked both schedules with alternating blocks of 12 intervals. With the single-alternation schedules, when the short interval duration was 5 s, regardless of the duration of the longer interval, pigeons learned the alternation pattern, and their pause anticipated the upcoming interval. When the shorter interval was 30 s, even when the ratio of short to long intervals was kept at 6:1, pigeons did not initially show anticipatory pausing-a violation of the principle of timescale invariance.     

Temporal control of behavior: schedule interactions

In Experiment I the response that terminated the postreinforcement pauses occurring under a fixed-interval 60-second schedule was reinforced, if the pause duration exceeded 30 seconds. The percentage of such pauses, rather than increasing, decreased. There were complex effects on the discriminative control of the pause by the reinforcer terminating the previous fixed interval, depending on whether the fixed interval and the added reinforcer were the same or different. In Experiments II(a) and II(b), each reinforcement initiated an alternative fixed-interval interresponse-time-greater-than-t-sec schedule, the schedule values being systematically varied. When the response following a pause exceeding a given duration was reinforced, fewer such pauses occurred than when they were not reinforced, i.e., on the comparable simple fixed-interval schedule. There was no systematic relationship between mean interrinforcement interval and duration of the postreinforcement pause. The pause duration initiated by reinforcement was directly related to the dependency controlling the shortest pause at that time, regardless of changes in mean interreinforcement interval.     

Effects Of Fixed-interval Schedule And Reinforcer Duration On Responding Reinforced By The Opportunity To Run

Two experiments investigated the effects of schedule value and reinforcer duration on responding for the opportunity to run on fixed-interval (FI) schedules in rats. In the first experiment, 8 male Wistar rats were exposed to FI 15-s, 30-s, and 60-s schedules of wheel-running reinforcement. The operant was lever pressing, and the consequence was the opportunity to run for 60 s. In the second experiment, 8 male Long-Evans rats were exposed to reinforcer durations of 15 s, 30 s, and 90 s. The schedule of reinforcement was an FI 60-s schedule. Results showed that postreinforcement pause and wheel-running rates varied systematically with reinforcer duration but not schedule value. Local lever-pressing rates decreased with reinforcer duration. Overall lever-pressing rates decreased with reinforcer duration but increased with schedule value. Although the reinforcer-duration effect is consistent with previous research, the lack a schedule effect appears to be the result of long post-reinforcement pauses following wheel-running reinforcement that render the manipulation of the interval requirement ineffective.     

TEMPORAL CONTROL ON INTERVAL SCHEDULES: WHAT DETERMINES THE POSTREINFORCEMENT PAUSE?

On fixed-interval or response-initiated delay schedules of reinforcement, the average pause following food presentation is proportional to the interfood interval. Moreover, when a number of intervals of different durations occur in a programmed cyclic series, postreinforcement pauses track the changes in interval value. What controls the duration of postreinforcement pauses under these conditions? Staddon, Wynne, and Higa (1991), in their linear waiting model, propose control by the preceding interfood interval. Another possibility is that delay to reinforcement, signaled by a key peck and/or stimulus change, determines the subsequent pause. The experiments reported here examined the role of these two possible time markers by studying the performance of pigeons under a chained cyclic fixed-interval procedure. The data support the linear waiting model, but suggest that more than the immediately preceding interfood interval plays a role in temporal control.     

Effects of reinforcement magnitude on interval and ratio schedules

Rats' lever pressing was studied on three schedules of reinforcement: fixed interval, response-initiated fixed interval, and fixed ratio. In testing, concentration of the milk reinforcer was varied within each session. On all schedules, duration of the postreinforcement pause was an increasing function of the concentration of the preceding reinforcer. The running rate (response rate calculated by excluding the postreinforcement pauses) increased linearly as a function of the preceding magnitude of reinforcement on fixed interval, showed slight increases for two of the three animals on response-initiated fixed interval, and did not change systematically on fixed ratio. In all cases, the overall response rate either declined or showed no effect of concentration. The major effect of increasing the reinforcement magnitude was in determining the duration of the following postreinforcement pause, and changes in the response rate reflected this main effect.     

Responding for sucrose and wheel-running reinforcement: effects of sucrose concentration and wheel-running reinforcer duration

Six male albino rats were placed in running wheels and exposed to a fixed-interval 30-s schedule of lever pressing that produced either a drop of sucrose solution or the opportunity to run for a fixed duration as reinforcers. Each reinforcer type was signaled by a different stimulus. In Experiment 1, the duration of running was held constant at 15 s while the concentration of sucrose solution was varied across values of 0, 2.5. 5, 10, and 15%. As concentration decreased, postreinforcement pause duration increased and local rates decreased in the presence of the stimulus signaling sucrose. Consequently, the difference between responding in the presence of stimuli signaling wheel-running and sucrose reinforcers diminished, and at 2.5%, response functions for the two reinforcers were similar. In Experiment 2, the concentration of sucrose solution was held constant at 15% while the duration of the opportunity to run was first varied across values of 15, 45, and 90 s then subsequently across values of 5, 10, and 15 s. As run duration increased, postreinforcement pause duration in the presence of the wheel-running stimulus increased and local rates increased then decreased. In summary, inhibitory aftereffects of previous reinforcers occurred when both sucrose concentration and run duration varied; changes in responding were attributable to changes in the excitatory value of the stimuli signaling the two reinforcers.     

Determinants of pausing under variable-ratio schedules: Reinforcer magnitude, ratio size, and schedule configuration

Pigeons pecked a key under two-component multiple variable-ratio schedules that offered 8-s or 2-s access to grain. Phase 1 assessed the effects of differences in reinforcer magnitude on postreinforcement pausing, as a function of ratio size. In Phase 2, postreinforcement pausing and the first five interresponse times in each ratio were measured as a function of differences in reinforcer magnitude under equal variable-ratio schedules consisting of different configurations of individual ratios. Rates were also calculated exclusive of postreinforcement pause times in both phases. The results from Phase 1 showed that as ratio size increased, the differences in pausing educed by unequal reinforcer magnitudes also increased. The results of Phase 2 showed that the effects of reinforcer magnitude on pausing and IRT durations were a function of schedule configuration. Under one configuration, in which the smallest ratio was a fixed-ratio 1, pauses were unaffected by magnitude but the first five interresponse times were affected. Under the other configuration, in which the smallest ratio was a fixed-ratio 7, pauses were affected by reinforcer magnitude but the first five interresponse times were not. The effect of each configuration seemed to be determined by the value of the smallest individual ratio. Rates calculated exclusive of postreinforcement pause times were, in general, directly related to reinforcer magnitude, and the relation was shown to be a function of schedule configuration.     

Fixed-interval performance and self-control in children.

Operant responses of 16 children (mean age 6 years and 1 month) were reinforced according to different fixed-interval schedules (with interreinforcer intervals of 20, 30, or 40 s) in which the reinforcers were either 20-s or 40-s presentations of a cartoon. In another procedure, they received training on a self-control paradigm in which both reinforcer delay (0.5 s or 40 s) and reinforcer duration (20 s or 40 s of cartoons) varied, and subjects were offered a choice between various combinations of delay and duration. Individual differences in behavior under the self-control procedure were precisely mirrored by individual differences under the fixed-interval schedule. Children who chose the smaller immediate reinforcer on the self-control procedure (impulsive) produced short postreinforcement pauses and high response rates in the fixed-interval conditions, and both measures changed little with changes in fixed-interval value. Conversely, children who chose the larger delayed reinforcer in the self-control condition (the self-controlled subjects) exhibited lower response rates and long postreinforcement pauses, which changed systematically with changes in the interval, in their fixed-interval performances.     

Temporal tracking on cyclic-interval reinforcement schedules

Pigeons were exposed to four cycles per session of a schedule in which the duration of successive interreinforcement intervals differed by t-sec. A cycle was composed of seven increasing and seven decreasing intervals, from 2t to 8t sec in length. In Exp. 1, postreinforcement pause tracked interval duration on five cyclic schedules, with values of t ranging from 2 to 40 sec. Tracking was better at shorter t values, and when discriminative stimuli signalled increasing and decreasing parts of the cycle. Pooled data for the whole experiment showed postreinforcement pause to bear a power function relationship to interval length, with a smaller exponent than the comparable function for fixed-interval schedules. Tests in a second experiment showed that pigeons trained on an arithmetic progression could also track schedules in which successive intervals followed either a logarithmic or a geometric progression, although tracking was more stable in the logarithmic case.     

Molecular analyses of the effects of d-amphetamine on fixed-interval schedule performances of rats.

A series of doses (0.5 to 2.0 mg/kg) of d-amphetamine was administered to rats whose lever pressing was maintained by fixed-interval 30-s, 60-s, or 120-s schedules of reinforcement by sucrose delivery. Under both saline and d-amphetamine conditions, molecular features of responding were reliably described in terms of the distribution of postreinforcement pauses and local response rate following the onset of responding. Postreinforcement pause always varied from interval to interval but, on average, shortened under the drug. Local response rate (response rate exclusive of pause time) tended to decrease under the drug, and where acceleration occurred within runs of responses, it was reduced by the drug. All of these effects were dose-related. These findings suggest that fixed-interval behavior can be analyzed effectively at a molecular level, and that the effects of d-amphetamine are best described as disruption of temporal discrimination.     

Aftereffects of reinforcement on variable-ratio schedules

On each of variable-ratio 10, 40, and 80 schedules of reinforcement, when rats' lever-pressing rates were stable, the concentration of a liquid reinforcer was varied within sessions. The duration of the postreinforcement pause was an increasing function of the reinforcer concentration, this effect being more marked the higher the schedule parameter. The running rate, calculated by excluding the postreinforcement pause, was unaffected by concentration. The duration of the postreinforcement pause increased with the schedule parameter, but the proportion of the interreinforcement interval taken up by the pause decreased. Consequently, the overall response rate was an increasing function of the schedule parameter; i.e., it was inversely related to reinforcement frequency, contrary to the law of effect. The running rate, however, decreased with the reinforcement frequency, in accord with the law of effect. When 50% of reinforcements were randomly omitted, the postomission pause was shorter than the postreinforcement pause, but the running rate of responses was not affected.     

Sequential patterns in post-reinforcement pauses on fixed-interval schedules of food

Responding by one pigeon was reinforced with food on fixed-interval schedules of 30, 60, and 300 sec duration. A second pigeon was studied under fixed-interval durations of 60 and 300 sec. For both birds, the average post-reinforcement pause was one-half the duration of the fixed interval. Autocorrelation coefficients revealed first-order sequential dependencies in series of post-reinforcement pauses. On the 300-sec fixed-interval schedule, successive post-reinforcement tended to alternate between long and short durations. At the shorter fixed-interval durations there was less evidence of alternation sequences. A second experiment was conducted to determine if the time intervals between the first response after reinforcement and the next reinforcement (the work periods) were responsible for the alternation patterns in the series of post-reinforcement pauses. To evaluate the role of the work period, several procedures were used to modify the work period from that obtained on the fixed-interval 300-sec schedule. Adding a schedule to the fixed-interval schedule that set the minimum amount of time that could elapse between the first response after reinforcement and the next reinforcement eliminated the alternation pattern. Control schedules indicated that the elimination of alternation patterns resulted from constraints on the work period per se and not from confounded changes in the interreinforcement intervals.     

Varying wheel-running reinforcer duration within a session: effect on the revolution-postreinforcement pause relation

Previous investigations of wheel-running reinforcement that manipulated reinforcer duration across conditions showed a strong relation between wheel-running rate and average postreinforcement pause (PRP) duration. To determine if the basis of this relation across conditions was a local effect of fatigue or satiation, the correlation between revolutions run and the duration of the immediately following PRP was investigated under conditions in which reinforcer duration was either constant or variable within a session. Seven male Wistar rats pressed a lever on a fixed-interval 60-s reinforcement schedule with the opportunity to run for 60 s as the reinforcing consequence. In the constant-duration condition, the duration of the reinforcer was always 60 s. In the variable-duration condition, the duration of the reinforcer varied between 2 and 240 s with a mean of 60 s. Mean correlations between revolutions run and the next PRP duration for constant, variable, and constant conditions were -.07, .20, and -.07, respectively. Although the positive correlation in the variable-duration condition is consistent with an effect of momentary fatigue or satiation, little of the variance in PRP duration appears to be attributable to these factors.     

Effects of pre-operant running and sucrose concentration on operant wheel running on a fixed interval schedule of reinforcement

Prior research proposed that temporal control over the pattern of operant wheel running on a fixed interval (FI) schedule of sucrose reinforcement is a function of automatic reinforcement generated by wheel running and the experimentally arranged sucrose reinforcement. Two experiments were conducted to assess this prediction. In the first experiment, rats ran for different durations (0, 30, 60, and 180 min) prior to a session of operant wheel running on a FI 120-s schedule. In the second experiment, the concentration of sucrose reinforcement on a FI 180-s schedule was varied across values of 0, 5, 15, and 25%. In Experiment 1, as the duration of pre-operant running increased, the postreinforcement pause before initiation of running lengthened while wheel revolutions in the latter part of the FI interval increased. In Experiment 2, wheel revolutions markedly increased then decreased to a plateau early in the FI interval. Neither manipulation increased temporal control of the pattern of wheel running. Instead, results indicate that operant wheel running is regulated by automatic reinforcement generated by wheel activity and an adjunctive pattern of running induced by the temporal presentation of sucrose. Furthermore, the findings question whether the sucrose contingency regulates wheel running as a reinforcing consequence.     

Species differences in temporal control of behavior

Temporal control of rats' and pigeons' responding was analyzed and compared in detail on fixed-interval and fixed-time schedules with parameters of 30, 60, and 120 seconds. On fixed-time schedules, rats' responding decreased greatly or ceased, whereas pigeons continued to respond, especially on low schedule values. The running rate of responses (calculated by excluding the postreinforcement pause) was related to the duration of the preceding postreinforcement pause for rats but not for pigeons. Changes in response rate in successive segments of the interval were best described by normal curves. The relationship between midpoints of the normal curves and schedule value was a power function, with an exponent of less than one for pigeons but greater than one for rats. These differences could be explained in terms of a basic difference between the key-peck and lever-press responses, the two being differently affected by the response-eliciting properties of food.