Temporal control by progressive-interval schedules of reinforcement.

Progressive-interval performances are described using measures that have proven to be successful in the analysis of fixed-interval responding. Five rats were trained with schedules in which the durations of consecutive intervals increased arithmetically as each interval was completed (either 6-s or 12-s steps for different subjects). The response patterns that emerged with extended training (90 sessions) indicated that performances had come under temporal control. Postreinforcement pausing increased as a function of the interval duration, the pauses were proportional to the prevailing duration, and the likelihood of the first response within an interval increased as the interval elapsed. To assess the resistance of these patterns to disruption, subjects were trained with a schedule that generated high response rates and short pauses (variable ratio). When the progressive-interval schedule was reinstated, pausing was attenuated and rates were elevated, but performances reverted to earlier patterns with continued exposure. The results indicated that temporal control by progressive-interval schedules, although slow to develop, is similar in many respects to that for fixed-interval schedules.     

Molecular analyses of the effects of d-amphetamine on fixed-interval schedule performances of rats.

A series of doses (0.5 to 2.0 mg/kg) of d-amphetamine was administered to rats whose lever pressing was maintained by fixed-interval 30-s, 60-s, or 120-s schedules of reinforcement by sucrose delivery. Under both saline and d-amphetamine conditions, molecular features of responding were reliably described in terms of the distribution of postreinforcement pauses and local response rate following the onset of responding. Postreinforcement pause always varied from interval to interval but, on average, shortened under the drug. Local response rate (response rate exclusive of pause time) tended to decrease under the drug, and where acceleration occurred within runs of responses, it was reduced by the drug. All of these effects were dose-related. These findings suggest that fixed-interval behavior can be analyzed effectively at a molecular level, and that the effects of d-amphetamine are best described as disruption of temporal discrimination.     

Temporal control of behavior: schedule interactions

In Experiment I the response that terminated the postreinforcement pauses occurring under a fixed-interval 60-second schedule was reinforced, if the pause duration exceeded 30 seconds. The percentage of such pauses, rather than increasing, decreased. There were complex effects on the discriminative control of the pause by the reinforcer terminating the previous fixed interval, depending on whether the fixed interval and the added reinforcer were the same or different. In Experiments II(a) and II(b), each reinforcement initiated an alternative fixed-interval interresponse-time-greater-than-t-sec schedule, the schedule values being systematically varied. When the response following a pause exceeding a given duration was reinforced, fewer such pauses occurred than when they were not reinforced, i.e., on the comparable simple fixed-interval schedule. There was no systematic relationship between mean interrinforcement interval and duration of the postreinforcement pause. The pause duration initiated by reinforcement was directly related to the dependency controlling the shortest pause at that time, regardless of changes in mean interreinforcement interval.     

The effects of d-amphetamine on the temporal control of operant responding in rats during a preshock stimulus.

The operant behavior of six rats was maintained by a random-interval schedule of reinforcement. Three-minute periods of noise were superimposed on this behavior, each period ending with the delivery of an unavoidable shock. Overall rates of responding were generally lower during the periods of noise than in its absence (conditioned suppression). These suppressed response rates also exhibited temporal patterning, with responding becoming less frequent as each noise period progressed. The effects of d-amphetamine on this behavioral baseline were then assessed. In four animals the relative response rates during the noise and in its absence suggested that the drug produced a dose-related decrease in the amount of conditioned suppression. However, this effect was often due to a decrease in the rates of responding in the absence of the preshock stimulus, rather than to an increase in response rates during the stimulus. Temporal patterning in response rates during the preshock stimulus was abolished, an effect that was interpreted in terms of rate-dependent effect of d-amphetamine. This study thus extends rate-dependent analyses of the effects of amphetamines to the patterns of operant behavior that occur during a preshock stimulus, and which have been discussed in terms of the disrupting effects of anxiety on operant behavior.     

Temporal control by signals of interval duration within variable-interval schedules

Rats' lever pressing produced sucrose reinforcers on a variable-interval schedule where, in different conditions, the duration of a stimulus presented immediately after reinforcement was either correlated or uncorrelated with the duration of the current interreinforcement interval. Under the baseline schedule, in which no stimulus was presented, the minimum interreinforcement interval was 8 s and the mean postreinforcement pause of each subject approximated this value. Response rates increased slowly over the first 10 to 15 s and then remained roughly constant throughout the remainder of the interval. In both the correlated and uncorrelated conditions, the added stimulus resulted in the postreinforcement pauses lengthening to values in excess of the duration of the preceding stimulus. This resulted in a poststimulus pause which was, in most cases, roughly constant irrespective of the duration of the preceding stimulus, or of the reinforcement contingencies prevailing immediately after stimulus offset. Local response-rate patterns in the uncorrelated conditions were similar to those obtained under the baseline schedule in which no stimulus was presented. However, in the correlated condition local response rates increased across the remainder of the interreinforcer interval. Further, the rate of acceleration was inversely related to the duration of the preceding stimulus. These results show that a correlation between stimulus duration and the ensuing time to reinforcement can control behavior—a type of temporal control not previously reported.     

Sequential effects of interval duration on fixed-interval performance

The bar pressing of rats was reinforced on a multiple fixed-interval schedule. The schedule intervals were 1 and 5 min long, and the sequence was such that intervals of either duration were equally likely to be followed by intervals of the same or of the other duration. Rates were higher during 1-min and after 5-min intervals. Best fit equations for cumulative responses during the 5-min intervals produced very similar exponents regardless of preceding duration. It was concluded that preceding duration may have affected the subjects' performances through direct effects on temporal discrimination.     

The role of verbal behavior in human learning: III. Instructional effects in children

Lever pressing of children from three age groups (2½ to 4, 5 to 6½, and 7½ to 9 years) could produce reinforcers according to a fixed-interval 40-s schedule: (1) Some were instructed to respond at a high rate, others at a low rate, and (2) they were subsequently taught to provide their own spoken self-instructions consonant with the earlier, experimenter-supplied instructions. All subjects who received high-rate instructions responded at a steady, high rate, which was maintained following self-instructional training. The effects of low-rate instructions were directly related to the age of the children. The two older groups produced low-rate patterns, with the oldest children responding at very low rates; effects were least noticeable in the youngest age group. Following self-instructional training, all three groups showed adult-like low-rate behavior and the oldest children showed an improved ability to estimate the interval length. The results provide further evidence of the importance of language as a determinant of human behavior.     

Varying wheel-running reinforcer duration within a session: effect on the revolution-postreinforcement pause relation

Previous investigations of wheel-running reinforcement that manipulated reinforcer duration across conditions showed a strong relation between wheel-running rate and average postreinforcement pause (PRP) duration. To determine if the basis of this relation across conditions was a local effect of fatigue or satiation, the correlation between revolutions run and the duration of the immediately following PRP was investigated under conditions in which reinforcer duration was either constant or variable within a session. Seven male Wistar rats pressed a lever on a fixed-interval 60-s reinforcement schedule with the opportunity to run for 60 s as the reinforcing consequence. In the constant-duration condition, the duration of the reinforcer was always 60 s. In the variable-duration condition, the duration of the reinforcer varied between 2 and 240 s with a mean of 60 s. Mean correlations between revolutions run and the next PRP duration for constant, variable, and constant conditions were -.07, .20, and -.07, respectively. Although the positive correlation in the variable-duration condition is consistent with an effect of momentary fatigue or satiation, little of the variance in PRP duration appears to be attributable to these factors.     

Inhibiting function of reinforcement: magnitude effects on variable-interval schedules

In two experiments, the performance of rats under constant-probability and arithmetic variable-interval schedules respectively was compared when the concentration of a liquid reinforcer was varied within sessions; in other sessions, half of the reinforcers were randomly omitted. When the discriminative function of the reinforcer as a signal for a decrease in the probability of reinforcement was attenuated (the constant-probability schedule) the postreinforcement pause duration was nevertheless an increasing function of reinforcer magnitude. This relationship was also present, but more marked, when the temporal discriminative function of the reinforcer was enhanced (the arithmetic schedule). These results suggested that reinforcement has an unconditioned suppressive effect on the reinforced response distinct from any discriminative function it may acquire. The reinforcement-omission effect, where response rate accelerates following omission, was observed when the reinforcer functioned as an effective temporal discriminative stimulus, but not when such temporal control was absent.     

Species differences in temporal control of behavior

Temporal control of rats' and pigeons' responding was analyzed and compared in detail on fixed-interval and fixed-time schedules with parameters of 30, 60, and 120 seconds. On fixed-time schedules, rats' responding decreased greatly or ceased, whereas pigeons continued to respond, especially on low schedule values. The running rate of responses (calculated by excluding the postreinforcement pause) was related to the duration of the preceding postreinforcement pause for rats but not for pigeons. Changes in response rate in successive segments of the interval were best described by normal curves. The relationship between midpoints of the normal curves and schedule value was a power function, with an exponent of less than one for pigeons but greater than one for rats. These differences could be explained in terms of a basic difference between the key-peck and lever-press responses, the two being differently affected by the response-eliciting properties of food.     

Alternative fixed-ratio fixed-interval schedules of reinforcement

Five rats were trained under alternative fixed-ratio fixed-interval schedules, in which food reinforcement was provided for the completion of either a fixed-ratio or a fixed-interval requirement, whichever was met first. Overall response rate and running rate (the rate of responding after the postreinforcement pause) decreased for all subjects as the fixed-ratio value increased. As the proportion of reinforcements obtained from the fixed-ratio component increased and the alternative schedule approached a simple fixed ratio, overall response rate and running rate both increased; conversely, as the proportion of reinforcements obtained from the fixed-interval component increased and the alternative schedule approached a simple fixed interval, response rates decreased. Postreinforcement pause length increased linearly as the average time between reinforcements increased, regardless of the schedule parameters. A break-run pattern of responding was predominant at low- and medium-valued fixed ratios. All subjects displayed at least occasional positively accelerated responding within interreinforcement intervals at higher fixed-ratio values.     

Effects of reinforcement history on response rate and response pattern in periodic reinforcement

Several researchers have suggested that conditioning history may have long-term effects on fixed-interval performances of rats. To test this idea and to identify possible factors involved in temporal control development, groups of rats initially were exposed to different reinforcement schedules: continuous, fixed-time, and random-interval. Afterwards, half of the rats in each group were studied on a fixed-interval 30-s schedule of reinforcement and the other half on a fixed-interval 90-s schedule of reinforcement. No evidence of long-term effects attributable to conditioning history on either response output or response patterning was found; history effects were transitory. Different tendencies in trajectory across sessions were observed for measures of early and late responding within the interreinforcer interval, suggesting that temporal control is the result of two separate processes: one involved in response output and the other in time allocation of responding and not responding.     

Progressive-ratio schedules: effects of later schedule requirements on earlier performances

Four rats were studied with variants of a progressive-ratio schedule with a step size of 6 in which different terminal components followed completion of the 20th ratio: (a) a reversal of the progression, (b) a fixed-ratio 6 schedule, or (c) extinction. Responding in the progressive-ratio components of these schedules was compared to performances under conventional progressive-ratio baselines. Under baseline conditions, postreinforcement pauses increased exponentially as a function of increasing ratio size, whereas running rates showed modest declines. The procedure of linking the progressive-ratio schedule to the reversed progression or to the fixed-ratio component resulted in decreased pausing. Linking the progressive-ratio schedule to the extinction component had the opposite effect, that of producing weakened progressive-ratio performances as evidenced by increased pausing. Subjects whose responses were reinforced on half of the ratios also showed exponential increases; however, pauses were substantially shorter following ratios on which the reinforcer was omitted. The results suggested that progressive-ratio pausing reflects the influence of remote as well as local contingencies.     

The role of verbal behavior in human learning: II. Developmental differences

When children in four different age ranges operated a response device, reinforcers were presented according to fixed-interval schedules ranging in value from 10 to 70 seconds. Only the behavior of the subjects in the youngest of the four groups, the preverbal infants, resembled that of other animal species. The children in age ranges 5 to 6½ and 7½ to 9 years exhibited either the low-rate or high-rate response patterns typical of human adults. Those who showed the low-rate pattern reported a time-based formulation of the contingencies and some of them were observed to occasionally count out the interval before responding. The performance of children aged 2½ to 4 years differed from that of both infants and older children, though containing some patterning elements similar to those produced by the older and younger subjects. The predominant response pattern of the infants consisted of a pause after reinforcement followed by an accelerated rate of responding that terminated when the next reinforcer was delivered. Analysis of postreinforcement-pause duration and response rate showed that infant performance, but not that of the older children, consistently exhibited the same kinds of schedule sensitivity observed in animal behavior. The evidence supports the suggestion that the development of verbal behavior greatly alters human operant performance and may account for many of the differences found between human and animal learning.     

An animal model of excessive eating: schedule-induced hyperphagia in food-satiated rats.

Nineteen rats were maintained throughout the experiment on ad libitum wet mash and water and were trained to press a lever on fixed-interval or fixed-ratio schedules of reinforcement with electrical brain stimulation. Fourteen rats ate at least 150% more mash during intermittent reinforcement sessions than during baseline, massed reinforcement control, and/or extinction sessions. In a 3-hr session, 11 of those 14 consumed more than 22 g of wet mash (13 g dry weight), the equivalent of nearly half an animal's daily food intake. In subsequent control sessions, the electrodes did not support stimulus-bound eating despite attempts to make stimulation parameters optimal. These results indicate that the eating was schedule induced or adjunctive, and suggest that the procedure may provide an animal model of excessive nonregulatory eating that contributes to obesity in humans.     

Behavior of rats under fixed consecutive number schedules: effects of drugs of abuse.

Four rats responded under a simple fixed consecutive number schedule in which eight or more consecutive responses on the run lever, followed by a single response on the reinforcement lever, produced the food reinforcer. Under this simple schedule, dose-response curves were determined for diazepam, morphine, pentobarbital, and phencyclidine. The rats were then trained to respond under a multiple fixed consecutive number schedule in which a discriminative stimulus signaled when the response requirement on the run lever had been completed in one of the two fixed consecutive number component schedules. Under control conditions, the percentage of reinforced runs under the multiple-schedule component with the discriminative stimulus added was much higher than the percentage of reinforced runs under the multiple-schedule component without the discriminative stimulus. All of the drugs decreased the percentage of reinforced runs under each of the fixed consecutive number schedules by increasing the conditional probability of short run lengths. This effect was most consistently produced by morphine. The drugs produced few differences in responding between the multiple fixed consecutive number components. Responding under the simple fixed consecutive number schedule, however, was affected at lower doses of the drugs than was responding under the same fixed consecutive number schedule when it was a component of the multiple schedule. This result may be due to the difference in schedule context or, perhaps, to the order of the experiments.     

Species differences in temporal control of behavior II: human performance

Human subjects responded on two panels. A differential-reinforcement-of-low-rate schedule with a limited-hold contingency operated on Panel A. In Condition 1, responses on Panel B produced a stimulus on the panel that signalled whether reinforcement was available on Panel A. In Condition 2, responses on Panel B briefly illuminated a digital clock. In both conditions, performance on Panel A was very efficient; with few exceptions, Panel A was pressed only when reinforcement was available. Thus, in effect, a fixed-interval schedule operated on Panel B. In Condition 1, a “break-and-run” response pattern occurred on Panel B; with increasing temporal parameters, the duration of the postreinforcement pause on Panel B increased linearly while overall response rate and running rate (calculated by excluding the postreinforcement pauses) remained approximately constant. In Condition 2, the response pattern on Panel B was scalloped; the postreinforcement pause was a negatively accelerated increasing function of schedule value, while overall response rate and running rate were negatively accelerated decreasing functions of schedule value. The performance of subjects in Condition 2, but not in Condition 1, was highly sensitive to the contingencies in operation, and resembled that of other species on the fixed-interval schedule.     

Fixed-interval performance: The dynamics of behavior and the interval length

Postreinforcement pauses from successive intervals under various fixed-interval schedules (ranging from 15 seconds to 480 seconds in length) were subjected to lag-1 autocorrelation analysis. Results from both rats and pigeons suggested that there was a consistent tendency for pause values in successive intervals to be weakly positively related. This tendency did not appear to change systematically with interval length and was exhibited both when the reinforcer magnitude was constant and when it was variable at different interval values. The findings do not support suggestions that the dynamic properties of performance under fixed-interval schedules vary systematically with interval length, and are in the opposite direction from some previous findings suggesting that measures of behavior (such as post-reinforcement pause length or number of responses) in successive intervals are inversely related.     

Time horizons in rats: the effect of operant control of access to future food.

The primary goal of this experiment was to determine whether the addition of an operant requirement for access to a less costly (continuous reinforcement) patch of future food increased the time horizon over which that future patch decreased intake in a currently available depleting (progressive-ratio) patch. Three groups of 4 rats were tested. Each member of the earned-time group was required to cumulate a fixed-time outside the progressive-ratio patch to obtain access to food in the less costly patch; the fixed-time requirement ranged from 2 to 64 min. Rats in the matched-time group received response-independent access to less costly food at the average delay shown by the earned-time group. Rats in the matched-time no-food group were removed from the chamber at the same average delay without receiving access to less costly food. Two of the earned-time rats showed an increased time horizon relative to that shown by the matched-time rats (approaching 40 min for 1 rat). The other 2 earned-time rats markedly increased instrumental responding but showed suppression of intake only when food was less than 20 min away. The matched-time group showed less suppression of intake over a similar range of delay intervals. Surprisingly, the matched-time no-food animals also showed suppression of intake concentrated at the end of the session, possibly reflecting the receipt of their entire daily ration 30 min after the session. The potential importance of time horizons to the foraging process is clear, but experimenters are still working out paradigms for investigation of these horizons.