Elimination of reinforced behavior: intermittent schedules of not-responding

Pigeons' key pecking resulted in food according to either a variable-ratio or a variable-interval schedule. At the same time, food was available for not pecking for a specified time. The required time of not-pecking was segmented into not-responding units, and these units were followed by food according to a fixed-ratio schedule. Both unit duration and the number required were varied. In general, the shorter the time unit or the smaller the ratio, the lower was response rate. When total required not-responding time was constant, but changes in unit duration and the number required altered how the total was achieved, shorter units produced lower rates. Other conditions involved substitution of food delivered independent of responding for the not-responding schedule. With low and moderate total times to food presentation, the not-responding schedule produced lower rates; with the longest times, the response-independent schedule generated less responding. When considered in terms of relative frequency of food presentation available from a source other than pecking, the not-responding schedule reduced rate more effectively than did the response-independent schedule. Comparisons with other research suggested that food presented dependent on not responding compared favorably with punishment as a procedure for reducing response rate. Transient effects differed. Although punishment temporarily depresses rate when first imposed and temporarily enhances it when first removed, food given for not responding quickly generated steady-state rates.     

Selective punishment early and late in fixed-ratio schedules of food reinforcement

Pigeons key pecked for grain on a fixed-ratio 100 schedule; electric shocks occurred intermittently at the fifteenth or eighty-fifth response in the ratio. In Experiment I, shock was at the fifteenth response for two birds, and at the eighty-fifth response for two others, in every sixth, twelfth, or eighteenth ratio. Rate of responding decreased as frequency of shock increased, and the pattern of responding included an increased initial pause and low rates or pause-run sequences that extended further into the ratio when shock was at the fifteenth response than when it was at the eighty-fifth response. Shock early in the ratio engendered longer initial pauses than shock late in the ratio. In Experiment II, four birds responded on a two-component multiple schedule in which shock occurred at the fifteenth response of the third ratio in the presence of a white keylight and at the eighty-fifth response of the third ratio in the presence of a green keylight. The overall rates of responding decreased as shock intensity increased. All four birds responded differentially to the white and green keylights, but with a pattern that varied between birds. In general, punishment reduced the probability of responses that preceded it, regardless of the ordinal position of those responses. Both studies confirm that the probability of responding is reduced less by aversive stimuli produced late in a fixed-ratio than by aversive stimuli produced early in a fixed-ratio.     

Induced attack during fixed-ratio and matched-time schedules of food presentation

Adjunctive or induced behavior is generated during a variety of schedules of reinforcement. Several theoretical conceptualizations suggest that rate of reinforcement is the primary variable controlling the strength or levels of induced behavior. The operant response requirement within the schedule context has not been extensively studied as a determinant of induced responding. In the present study, levels of induced attack by food-deprived pigeons against restrained conspecifics were compared during response-dependent and response-independent schedules of food presentation equated or yoked interval-by-interval for reinforcement frequency. Experiment 1 compared levels of attack induced by fixed-ratio schedules of key pecking and yoked "matched-time" schedules. Experiment 2 similarly compared chained fixed-ratio 1 fixed-ratio 74 and yoked chained matched-time matched-time schedules. In both experiments, the response-dependent schedules generated greater levels (amount and probability) of induced attack than the response-independent time-based schedules. Thus, the ratio response requirement may be an important determinant of levels of induced responding, and the lower levels of attack observed during the response-independent condition may not be due to the absence of stimuli predicting food presentations. It is concluded that rate of reinforcement is not the sole variable determining levels of induced responding and that response-based and time-based schedules differ in their generation of induced responding.     

Comparison of drug effects on fixed-ratio performance and chain performance maintained under a second-order fixed-ratio schedule.

In one component of a multiple schedule, pigeons were required to complete the same four-response chain each session by responding sequentially on three identically lighted keys in the presence of four successively presented colors (chain performance). Food presentation occurred after five completions of the chain (i.e., after 20 correct responses). Errors, such as responding on the center or right key when the left was designated correct, produced a brief timeout but did not reset the chain. In the other component, responding on a single key (lighted white) was maintained by food presentation under a fixed-ratio 20 schedule. In general, phencyclidine and d-amphetamine produced dose-dependent decreases in the overall response rates in both components. With pentobarbital, overall rate in each component generally increased at intermediate doses and decreased at higher doses. All three drugs produced dose-dependent disruptive effects on chain-performance accuracy. Phencyclidine and pentobarbital increased percent errors at doses that had little or no rate-decreasing effects, whereas d-amphetamine generally increased percent errors only at doses that substantially decreased overall rate. At high doses, all three drugs produced greater disruption of chain performance than of fixed-ratio performance, as indicated by a slower return to control responding, although the effects of d-amphetamine were less selective than those of phencyclidine or pentobarbital.     

Drug discrimination under a concurrent fixed-ratio fixed-ratio schedule.

Pigeons were trained to discriminate 5.0 mg/kg pentobarbital from saline under a two-key concurrent fixed-ratio 10 fixed-ratio 40 schedule of food presentation, in which the fixed-ratio component with the lower response requirement was programmed to reinforce responding on one key after drug administration (pentobarbital-biased key) and on the other key after saline administration (saline-biased key). After responding stabilized, pigeons averaged 98% of their responses on the pentobarbital-biased key during training sessions preceded by pentobarbital, and they averaged 90% of their responses on the saline-biased key during training sessions preceded by saline. In test sessions preceded by doses of pentobarbital, chlordiazepoxide, or ethanol, pigeons switched from responding on the saline-biased key at low doses to responding on the pentobarbital-biased key at higher doses (the dose-response curve was quantal). High doses of phencyclidine produced responding on both keys, whereas pigeons responded almost exclusively on the saline-biased key after all doses of methamphetamine. These and previous experiments using concurrent reinforcement schedules to study drug discrimination illustrate that the schedule of reinforcement is an important determinant of the shape of dose-effect curves in drug-discrimination experiments.     

Choice between sequences of fixed-ratio schedules: effects of ratio values and probability of food delivery.

Pigeons were exposed to schedules of food delivery that consisted of two sequential fixed ratios. When alternative sequences provided two food deliveries per 50 responses, the schedule with the shorter initial fixed-ratio value was consistently preferred. Progressively reducing from 1.0 to .25 the probability of food delivery following completion of the second fixed ratio of the sequence with the shorter initial fixed ratio did not reduce preference for this sequence. Moreover, the sequence with the shorter initial fixed ratio also was preferred when the probability of food delivery following completion of the initial ratio in that sequence was progressively reduced from 1.0 to .5, although preference shifted to the alternative when the probability was reduced to 0. These findings suggest that the length of the initial fixed ratio was a primary determinant of choice. Subsequent manipulations demonstrated, however, that when the initial fixed ratios of the two alternatives were equal, changes in the ratio value and probability of food delivery following completion of the second fixed ratio lawfully affected choice.     

The contribution of an added counter to a fixed-ratio schedule

Although previous research showed that a visual counter increased the rate of responding on a large fixed-ratio schedule, a theoretical analysis of the factors responsible for fixed-ratio performance suggests that the primary control by number of responses since reinforcement is to weaken the performance. The present experiment employed a multiple schedule in which the same fixed-ratio value alternated with and without an added counter. It tested the hypothesis that the differential reinforcement of high-rate responding masked the attenuation of the fixed-ratio performance from the unoptimal discriminative control produced by the fixed relation between number of responses and reinforcement. In the present experiment the postreinforcement pause was consistently longer in the components with the added counter, while running rates remained comparable between the components of the multiple schedule. Both components of the multiple schedule involved differential reinforcement of high-rate responding while only the components with the added counter amplified the discriminative control by number of pecks since reinforcement.     

Positive reinforcement and the elimination of reinforced responses

Key pecking was maintained on a fixed-interval schedule while either a differential-reinforcement-of-not-responding or a fixed-time schedule was imposed simultaneously. The lower the time parameter of the not-responding schedule, the lower was the response rate. Similar effects occurred with the fixed-time schedule, if the pigeons had experience with reinforcement for not responding. Otherwise the effects were less orderly, to the extent that rate could reach maximum with the lowest-valued fixed-time schedule. The not-responding and the response-independent schedules had similar effects on rate in experienced pigeons only when the time parameter or nominal frequency of food presentation was considered. When considered in terms of obtained frequency of food presentation, reinforcement of not responding produced larger decrements in rate than did the fixed-time schedule.     

Cocaine and food deprivation: effects on food-reinforced fixed-ratio performance in pigeons.

Key pecking by 6 pigeons was maintained by a fixed-ratio 30 schedule of food presentation while body weights were 80% of free-feeding weights. Acute administration of cocaine (0.3 to 13.0 mg/kg, i.m.) dose-dependently decreased response rates. Dose-effect curves were shifted to the right when 3 of the 6 pigeons were maintained at 70% of free-feeding weights and were shifted to the left when the other 3 pigeons were maintained at 90% of free-feeding weights. Then a dose of cocaine that initially decreased response rates by more than 95% of control rates was administered before each daily session. Comparable degrees of tolerance to these rate-decreasing effects developed in the two groups. The rate at which responding recovered was relatively rapid for pigeons in the 70% free-feeding-weight group and was slower for 2 of the 3 pigeons in the 90% free-feeding-weight group. When body weights were then increased from 70% to 80% or were decreased from 90% to 80% of free-feeding weight, performance was disrupted initially only for pigeons whose weight went from 70% to 80% of free feeding. In the present experiment the degree of deprivation may have indirectly influenced the degree of tolerance that developed to cocaine's response rate-decreasing effects because it directly influenced the dose chosen to be administered chronically. The degree of deprivation appeared to have a more direct influence on the rate at which tolerance developed.     

Neurochemical changes correlated with behavior maintained under fixed-interval and fixed-ratio schedules of reinforcement.

Key pecking of 4 pigeons was maintained under a multiple 3-min fixed-interval, 30-response fixed-ratio schedule of food presentation. Only one schedule was in effect during an experimental session, and each was correlated with a different keylight stimulus and location (left vs. right). The different schedule components alternated across days or weeks. Cerebrospinal fluid was collected from chronically implanted intracerebroventricular cannulae following sessions with the different schedules, as well as following sessions in which reinforcement was withheld (extinction), when response-independent food was delivered, and when the experimental chamber was dark and there were no scheduled events. Metabolites of the neurotransmitters serotonin, norepinephrine, and dopamine were assayed in cerebrospinal fluid using high-performance liquid chromatography with electrochemical detection. Compared to the fixed-ratio condition, responding maintained under the fixed-interval schedule resulted in consistently higher levels of the serotonin metabolite 5-hydroxyindoleacetic acid and of the dopamine metabolite homovanillic acid in all pigeons. Levels of 3-methoxy-4-hydroxyphenylethylene glycol, a metabolite of norepinephrine, and dihydroxyphenylacetic acid, another dopamine metabolite, were also higher in 3 of the 4 pigeons following exposure to the fixed-interval schedules when compared to levels of these metabolites after exposure to the fixed-ratio schedule. Extinction of fixed-ratio responding resulted in large increases in 5-hydroxyindoleacetic acid compared to levels of this metabolite under the fixed-ratio schedule, whereas this serotonin metabolite decreased during extinction of responding under the fixed-interval schedule. Control procedures suggested that the neurochemical changes were not related to the rate of responding but were a function of the specific experimental conditions. Distinctive neurochemical changes that accompany schedule-controlled responding show the sensitivity of the neurochemical environment to behavioral contingencies and demonstrate further the profound impact that such contingencies have on biobehavioral processes.     

Preference in concurrent variable-interval fixed-ratio schedules

Five pigeons were trained on concurrent variable-interval fixed-ratio schedules in three experiments. Experiment 1 used two variable-interval schedules and one fixed-ratio schedule, and the ratio requirement was varied. Using the generalized matching law, sensitivity to reinforcement was close to 1.0, but performance was biased toward the variable-interval schedule with the lower reinforcement rate. In Experiment 2, which used one variable-interval and one fixed-ratio schedule, the interval schedule was varied. All birds showed sensitivities to reinforcement of less than 1.0 and of less than the values obtained in Experiment 1. The performance was also biased toward the fixed-ratio schedule. Because the generalized matching law could not account for the differences in the data from Experiments 1 and 2, an extension of this law was suggested and successfully tested in Experiment 3. The proposed dual-sensitivity model was also shown to clarify some previously reported results.     

The effects of brief-stimulus presentations in fixed-ratio second-order schedules

Pigeons' responses were reinforced according to a three-component multiple schedule. In Component 1, key pecks produced food according to a fixed-ratio second-order schedule with fixed-ratio units. Here, a fixed number of fixed-ratio units produced food, and the brief stimulus terminating each unit also accompanied food. Responses in Component 2 produced food on an identical schedule except that the brief stimulus was not paired with food. Component 3 contained a simple fixed-ratio schedule whose response requirement equaled that of Components 1 and 2. Across conditions the size of the fixed-ratio unit (five, ten, twenty, forty, and eighty responses) and the total number of responses per reinforcement were parametrically manipulated. The highest response rates and shortest preratio pauses were observed in Component 3 (no brief stimulus). The lowest rates and longest pauses were found in the component with paired brief-stimulus presentations, indicating that the food-paired brief stimulus suppressed responding. The suppressive effects were greatest when the fixed-ratio units were small (e.g., fixed-ratio 5) and the total fixed-ratio requirement was large (e.g., fixed-ratio 160). Under no conditions did the paired brief stimulus facilitate responding. The nonpaired brief stimulus also suppressed responding but to a lesser extent. The suppressive effects of nonpaired brief stimuli were greatest when the fixed-ratio units were small and the total response requirement was large. These data suggest that the suppressive effects of the brief stimuli may have masked the conditioned-reinforcing effects reported in other studies, and that conditions that maximize suppression in second-order schedules involve the use of fixed-ratio schedule units and the presentation of many brief stimuli per reinforcer.     

Choice performance in several concurrent key-peck treadle-press reinforcement schedules

Five pigeons were exposed to several concurrent variable-interval food reinforcement schedules. For three subjects, one component of the schedule required a key-pecking response, the other a treadle-pressing response. For the other two subjects, both schedule components required treadle-pressing responses. The relative probability of reinforcement associated with the manipulanda was varied from 0 to 1.0 in 13 experimental conditions for the Key-Treadle subjects and nine conditions for the Treadle-Treadle subjects. The results indicated that the logarithms of relative time spent responding, and the logarithms of relative number of responses emitted on a manipulandum, approximated direct linear functions of logarithms of the relative frequencies of reinforcement associated with that manipulandum. No systematic bias in favor of time spent key pecking over time spent treadle pressing was apparent for the Key-Treadle subjects. All subjects exhibited undermatching, in that the ratios of time and response allocation at the alternatives systematically differed from the ratios of reinforcers obtained from the alternatives in the direction of indifference. Key pecking appeared to have no special link to food beyond treadle pressing or what would be expected on the basis of the reinforcement dependencies alone.     

Preference in pigeons given a choice between sequences of fixed-ratio schedules: Effects of ratio values and duration of food delivery

Pigeons were exposed to schedules that consisted of two sequential fixed ratios, the completion of each followed by food delivery. When each alternative provided two food deliveries per 100 responses, the schedule with the shorter initial fixed-ratio value was consistently preferred. Subsequent attempts were made to shift this established preference by (1) increasing the ratio requirement in the second fixed ratio of the preferred schedule; (2) increasing the duration of food delivery in the second fixed ratio of the nonpreferred schedule; (3) decreasing the duration of food delivery in the first fixed ratio of the preferred schedule; and (4) shortening the second fixed ratio of the nonpreferred schedule. Preference shifted from the schedule with the shorter initial fixed ratio only when the duration of food delivery associated with the first fixed ratio of that schedule was too brief to allow eating. Under all other conditions, pigeons strongly preferred the schedule with the shorter initial fixed ratio even when, overall, that schedule yielded briefer access to food or required more responses to obtain equivalent access.     

Tolerance to effects of cocaine on behavior under a response-initiated fixed-interval schedule

Tolerance to effects of cocaine can be modulated by schedules of reinforcement. With multiple ratio schedules, research has shown an inverse relationship between ratio requirement and amount of tolerance that resulted from daily administration of the drug. In contrast, tolerance to the effects of cocaine on behavior under multiple interval schedules generally has developed regardless of interval value. Under interval schedules reinforcement depends on the animal making one response following a time interval. Thus, as time to respond increases, the time to reinforcement decreases. On the other hand, fixed ratio schedules require a specified number of responses to be made prior to reinforcement. Therefore, delaying the initiation of responding does not coincide with a significant decrease in the time to reinforcement. In the current experiment, 6 pigeons were trained to respond under a three-component multiple schedule, with a different tandem fixed-ratio 1 fixed-interval schedule in each component. The multiple schedule required one response, which was followed by one of three fixed-interval values (5, 15, or 60 s). Thus, the multiple schedule was interval-like because after the fixed-ratio 1, only one more response was required for reinforcement, but it was also ratio-like because the length of the pause at the beginning of each interreinforcer interval affected the time until the next reinforcer. Acute administration of cocaine generally resulted in dose-dependent decreases in responding. Chronic (i.e., daily) administration of a rate-decreasing dose resulted in tolerance patterns similar to those usually obtained with multiple ratio schedules. That is, the magnitude of tolerance was related inversely to schedule size. These results suggest that delay to reinforcement from the initial response may play a role in the development of schedule-parameter-related tolerance.     

A quantitative analysis of chain-schedule performance

Six pigeons were trained with a chain variable-interval variable-interval schedule on the left key and with reinforcers available on the right key on a single variable-interval schedule arranged concurrently with both links of the chain. All three schedules were separately and systematically varied over a wide range of mean intervals. During these manipulations, the obtained reinforcer rates on constant arranged schedules also frequently changed systematically. Increasing reinforcer rates in Link 2 of the chain increased response rates in both links and decreased response rates in the variable-interval schedule concurrently available with Link 2. Increasing Link-1 reinforcer rates increased Link-1 response rates and decreased Link-2 response rates. Increasing reinforcer rates on the right-key schedule decreased response rates in Link 1 of the chain but did not affect the rate in Link 2. The results extend and amplify previous analyses of chain-schedule performance and help define the effects that a quantitative model must describe. However, the complexity of the results, and the fact that constant arranged reinforcer schedules did not necessarily lead to constant obtained reinforcer rates, precluded a quantitative analysis.     

Unsignalled delay of reinforcement in variable-interval schedules

Three pigeons responded on several tandem variable-interval fixed-time schedules in which the value of the fixed-time component was varied to assess the effects of different unsignalled delays of reinforcement. Actual (obtained) delays between the last key peck in an interval and reinforcement were consistently shorter than the nominal (programmed) delay. When nominal delays were relatively short, response rates were higher during the delay condition than during the corresponding nondelay condition. At longer nominal delay intervals, response rates decreased monotonically with increasing delays. The results were consistent with those obtained from delay-of-reinforcement procedures that impose either a stimulus change (signal) or a no-response requirement during the delay interval.     

Conjunctive schedules of reinforcement: III. A fixed-interval adjusting fixed-ratio schedule

Key pecking of three pigeons was studied under a conjunctive schedule that specified both a fixed-interval and an adjusting fixed-ratio requirement. The fixed-interval schedule was 6 min for one pigeon and 3 min for the other two. The size of the ratio requirement was determined within each cycle of the fixed interval by the duration of the pause before responding began. The fixed-ratio value was at maximum at the start of each fixed interval and decreased linearly until the first response occurred (adjusting fixed-ratio schedule). A peck produced food when the number of responses remaining on the fixed-ratio schedule was completed and when the fixed interval had elapsed. If no response occurred during the interval, the fixed-ratio requirement decreased to one and a single response after the interval elapsed produced food. The initial value of the adjusting fixed-ratio schedule was studied over a range of 0 to 900. Increases in the adjusting fixed-ratio schedule to about 300 responses increased both pause duration and running response rate and also modified the pattern of responding from that obtained under the fixed-interval schedule. Higher values of the adjusting fixed ratio generally decreased pause duration and running response rate and also disrupted responding. Interreinforcement time under the conjunctive schedule was increased substantially when the adjusting fixed-ratio size exceeded 300 responses.     

Discriminability of fixed-ratio schedules for pigeons: effects of payoff values

Three pigeons, previously trained to discriminate different numbers of responses (fixed ratios), were tested under different reinforcement contingencies (payoff matrices) at two levels of sensitivity. For one subject, relative reinforcement magnitude was varied—at first, across sessions and then, at midsession by reversing values—without exteroceptive cues. For another, relative reinforcement magnitude and/or probability was varied every 50 trials with cues by correlating different payoff matrices with different key colors. For the third subject, relative reinforcement probability was varied more frequently with cues—in the limit, at random—to demonstrate stimulus control of response bias on a trial-by-trial basis. A signal-detection analysis showed that bias changed with payoffs, for as many as seven different matrices, while sensitivity remained unchanged. The obtained functions (receiver operating characteristics) were similar under different payoff conditions, which suggests that a single mechanism controls bias. However, they differed enough in slope to require a relatively complex account (e.g., the general Gaussian model of detection theory).     

Effects of methadone on alternative fixed-ratio fixed-interval performance: latent influences on schedule-controlled responding.

Effects of methadone on pigeons' key pecking were examined under four conditions selected to analyze the control of behavior under alternative fixed-ratio fixed-interval schedules. In Condition 1, pigeons pecked under one of three different alternative schedules (alternative fixed-ratio 50 fixed-interval 90 s, alternative fixed-ratio 75 fixed-interval 90 s and alternative fixed-ratio 200 fixed-interval 90 s) each week. In Condition 2, fixed-ratio 50 or fixed-ratio 75 schedules were in effect during baseline sessions, and alternative fixed-ratio 50 fixed-interval 90-s or alternative fixed-ratio 75 fixed-interval 90-s schedules were in effect during sessions in which methadone was administered. In Condition 3, effects of methadone on key pecking maintained under fixed-ratio 50 and fixed-ratio 75 schedules were examined, whereas in Condition 4 the effects of methadone on key pecking under a fixed-interval 90-s schedule as well as fixed-ratio 50 and fixed-ratio 75 schedules were investigated. Control by the fixed-interval contingency was assessed by computing the proportion of total session reinforcers delivered under the fixed-interval schedule. Methadone administration (0.5-4.0 mg/kg) shifted the predominant source of schedule control under the alternative schedule from the fixed-ratio schedule to the fixed-interval contingency. This shift was dependent on methadone dose and fixed-ratio size. Control by the fixed-interval contingency was greatest following extensive exposure to the interval component embedded within the alternative schedule (Condition 1), but was apparent to a lesser degree with even very limited exposure to the alternative fixed-ratio fixed-interval schedule (Condition 2). Interreinforcement intervals comparable to those under fixed-interval schedule were not observed under the fixed-ratio schedules presented alone (Condition 3). Repeated exposure to the fixed-interval contingency outside the context of the alternative fixed-ratio fixed-interval schedule did not engender performance changes under a fixed-ratio schedule which would mimic those of increased fixed-interval contingency control (Condition 4). These data suggest that drug administration can be used to unmask the influence of contingencies that are latent under baseline conditions and reveal influences of both past and present environmental variables.