Tests of behavior momentum in simple and multiple schedules with rats and pigeons.

Four experiments examined the relationship between rate of reinforcement and resistance to change in rats' and pigeons' responses under simple and multiple schedules of reinforcement. In Experiment 1, 28 rats responded under either simple fixed-ratio, variable-ratio, fixed-interval, or variable-interval schedules; in Experiment 2, 3 pigeons responded under simple fixed-ratio schedules. Under each schedule, rate of reinforcement varied across four successive conditions. In Experiment 3, 14 rats responded under either a multiple fixed-ratio schedule or a multiple fixed-interval schedule, each with two components that differed in rate of reinforcement. In Experiment 4, 7 pigeons responded under either a multiple fixed-ratio or a multiple fixed-interval schedule, each with three components that also differed in rate of reinforcement. Under each condition of each experiment, resistance to change was studied by measuring schedule-controlled performance under conditions with prefeeding, response-independent food during the schedule or during timeouts that separated components of the multiple schedules, and by measuring behavior under extinction. There were no consistent differences between rats and pigeons. There was no direct relationship between rates of reinforcement and resistance to change when rates of reinforcement varied across successive conditions in the simple schedules. By comparison, in the multiple schedules there was a direct relationship between rates of reinforcement and resistance to change during most tests of resistance to change. The major exception was delivering response-independent food during the schedule; this disrupted responding, but there was no direct relationship between rates of reinforcement and resistance to change in simple- or multiple-schedule contexts. The data suggest that rate of reinforcement determines resistance to change in multiple schedules, but that this relationship does not hold under simple schedules.     

Response-independent Events In The Behavior Stream

The metaphor of the behavior stream provides a framework for studying the effects of response-independent food presentations intruded into an environment in which operant responding of pigeons was maintained by variable-interval schedules. In the first two experiments, response rates were reduced when response-independent food was intruded during the variable-interval schedule according to a concomitantly present fixed-time schedule. These reductions were not always an orderly function of the percentage of response-dependent food. Negatively accelerated patterns of key pecking across the fixed-time period occurred in Experiment 1 under the concomitant fixed-time variable-interval schedules. In Experiment 2, positively and negatively accelerated and linear response patterns occurred even though the schedules were similar to those used in Experiment 1. The variable findings in the first two experiments led to three subsequent experiments that were designed to further illuminate the controlling variables of the effects of intruded response-independent events. When the fixed and variable schedules were correlated with distinct operanda by employing a concurrent fixed-interval variable-interval schedule (Experiment 3) or with distinct discriminative stimuli (Experiments 4 and 5), negatively accelerated response patterns were obtained. Even in these latter cases, however, the response patterns were a joint function of the physical separation of the two schedules and the ratio of fixed-time or fixed-interval to variable-interval schedule food presentations. The results of the five experiments are discussed in terms of the contributions of both reinforcement variables and discriminative stimuli in determining the effects of intruding response-independent food into a stream of operant behavior.     

Effects of concurrent response-independent reinforcement on fixed-interval schedule performance

In three experiments, behavior maintained by fixed-interval schedules changed when response-independent reinforcement was delivered concurrently according to fixed- or variable-time schedules. In Experiment I, a pattern of positively accelerated responding during fixed interval was changed to a linear pattern when response-independent reinforcement occurred under a variable-time schedule. Overall response rates (total responses/total time) decreased as the frequency of response-independent reinforcement increased. Experiment II showed that the response-rate changes in the first experiment were controlled by the response-reinforcer relation, but the changes in patterns of responding were similar whether concurrently available reinforcement at varying times was response-dependent or response-independent. In the final experiment, the addition of response-independent reinforcement at fixed times to a fixed-interval schedule resulted in changes in both local and overall response rates and in the occurrence of positively accelerated responding between reinforcements. These results suggest that the temporal distribution of reinforcers determines response patterns and that both the response-reinforcement dependency and the schedule of reinforcement determine overall response rates during concurrently scheduled response-dependent and response-independent reinforcement.     

The role of intermittent food in the induction of attack in pigeons

The present experiments evaluated whether transitions in reinforcer probability are necessary to induce attack in pigeons. In Experiment I, three of six pigeons exposed to response-contingent constant-probability food schedules and a photograph of a conspecific as a target exhibited sustained postreinforcement attack on the target. The postreinforcement pattern of attack developed over the course of the experiment and was accompanied by a reduction in the rate of postreinforcement key pecking and an increase in the postreinforcement pause in key pecking. These effects on key pecking resulted in unprogrammed variations in the probability of reinforcement which may have been responsible for the induction of attack. In Experiment II, the attack-inducing properties of a constant-probability response-independent food schedule were compared to a periodic food schedule matched for overall rate of food delivery and to a no-food condition. In addition to attack, the spatial location of the subjects was monitored during each interfood interval. The periodic and aperiodic food schedules generated very different patterns of spatial location. Postfood attack was induced by both food schedules, although the constant-probability schedule induced attack in fewer birds. The no-food condition was not effective in inducing attack in any birds. These experiments indicate that intermittent food schedules without reductions in reinforcer probability are sufficient to induce attack in some pigeons, although not as effective as schedules with transitions in reinforcer probability.     

Interlocking schedules: the relationship between response and time requirements.

Rats were exposed to an interlocking fixed-ratio 150 fixed-interval 5-minute schedule of food reinforcement and then to yoked variable-ratio schedules in which individual ratios corresponded exactly to the ratios of responses to reinforcement obtained on the interlocking schedule. After additional training with the interlocking schedule, the rats were exposed to yoked variable-interval schedules in which intervals corresponded to the intervals between successive reinforcements obtained on the second interlocking schedule. Response rates were highest in the yoked VR condition and lowest in the yoked VI, while intermediate rates characterized the interlocking schedule. Break-run patterns of responding were generated by the interlocking schedule for all subjects, while both the yoked VR and VI schedules produced comparatively stable local rates of responding. These results indicate that responding is sensitive to the interlocking schedule's inverse relationship between reinforcement frequency and responses per reinforcement.     

Neurochemical changes correlated with behavior maintained under fixed-interval and fixed-ratio schedules of reinforcement.

Key pecking of 4 pigeons was maintained under a multiple 3-min fixed-interval, 30-response fixed-ratio schedule of food presentation. Only one schedule was in effect during an experimental session, and each was correlated with a different keylight stimulus and location (left vs. right). The different schedule components alternated across days or weeks. Cerebrospinal fluid was collected from chronically implanted intracerebroventricular cannulae following sessions with the different schedules, as well as following sessions in which reinforcement was withheld (extinction), when response-independent food was delivered, and when the experimental chamber was dark and there were no scheduled events. Metabolites of the neurotransmitters serotonin, norepinephrine, and dopamine were assayed in cerebrospinal fluid using high-performance liquid chromatography with electrochemical detection. Compared to the fixed-ratio condition, responding maintained under the fixed-interval schedule resulted in consistently higher levels of the serotonin metabolite 5-hydroxyindoleacetic acid and of the dopamine metabolite homovanillic acid in all pigeons. Levels of 3-methoxy-4-hydroxyphenylethylene glycol, a metabolite of norepinephrine, and dihydroxyphenylacetic acid, another dopamine metabolite, were also higher in 3 of the 4 pigeons following exposure to the fixed-interval schedules when compared to levels of these metabolites after exposure to the fixed-ratio schedule. Extinction of fixed-ratio responding resulted in large increases in 5-hydroxyindoleacetic acid compared to levels of this metabolite under the fixed-ratio schedule, whereas this serotonin metabolite decreased during extinction of responding under the fixed-interval schedule. Control procedures suggested that the neurochemical changes were not related to the rate of responding but were a function of the specific experimental conditions. Distinctive neurochemical changes that accompany schedule-controlled responding show the sensitivity of the neurochemical environment to behavioral contingencies and demonstrate further the profound impact that such contingencies have on biobehavioral processes.     

Effects of reinforcement amount on attack induced under a fixed-interval schedule in pigeons

Key pecking by pigeons was maintained on a chained fixed-interval 4-min (12-min for 1 subject) fixed-ratio 1 schedule of food presentation. Attacks toward a restrained and protected conspecific were recorded. In the first experiment, the amount of food presented per interval was manipulated across phases by varying the number of fixed ratios required in the terminal link of the chain. Measures of attack for all pigeons during the fixed-interval component increased monotonically as a function of food amount. In the second experiment, two different food amounts alternated within each experimental session under a multiple schedule. For both pigeons in this experiment, measures of attack were higher during the component that delivered the larger food amount per interval. The differences in levels of attack induced by the two food amounts in Experiment 2, however, were not as great as in Experiment 1; apparently this was because attack during the first interval of each component was controlled in part (P-5626) or entirely (P-7848) by the reinforcement amount delivered at the end of the previous component. Attack was also a function of the location of the interfood interval within the session. For both pigeons, attack tended to decrease throughout the session. The results of both experiments suggest that attack is an increasing function of reinforcement amount under fixed-interval schedules, but that this function may be influenced by the manner in which reinforcement amount is manipulated, by the duration of the interfood interval, and by the location of the interfood interval within the experimental session. In general, these results are compatible with theories of induced attack and other schedule-induced behavior that emphasize aversive after-effects of reinforcement presentation.     

A comparison of ratio and interval reinforcement schedules with comparable interreinforcement times

Pigeons were trained to peck keys on fixed-ratio and fixed-interval schedules of food reinforcement. Both schedules produced a pattern of behavior characterized as pause and run, but the relation of pausing to time between reinforcers differed for the two schedules even when mean time between reinforcers was the same. Pausing in the fixed ratio occupied less of the time between reinforcers for shorter interreinforcer times. For two of three birds, the relation was reversed at longer interreinforcer times. As an interreinforcer time elapsed, there was an increasing tendency to return to responding for the fixed interval, but a roughly constant tendency to return to responding for the fixed-ratio schedule. In Experiment 1 these observations were made for both single-reinforcement schedules and multiple schedules of fixed-ratio and fixed-interval reinforcement. In Experiment 2 the observations were extended to a comparison of fixed-ratio versus variable-interval reinforcement schedules, where the distribution of interreinforcement times in the variable interval approximated that for the fixed ratio.     

Pentobarbital and d-amphetamine effects on concurrent performances.

The effects of pentobarbital and d-amphetamine were studied in pigeons responding under several concurrent fixed-ratio variable-interval and concurrent fixed-ratio fixed-interval schedules of food presentation. Drug effects were compared with different fixed ratios, fixed and variable intervals, changeover delays, and with the schedules operating singly. Doses of d-amphetamine that increased or did not affect responding under the interval schedules decreased responding under the fixed-ratio schedule, whereas doses of pentobarbital that increased responding under the fixed-ratio schedule decreased or eliminated responding under the interval schedules. These effects depended both on the schedule of food delivery and the parameters of schedules arranged concurrently. Pentobarbital increased responding under the fixed-ratio schedule with 4-minute and 10-minute interval schedules arranged concurrently, but not with 1.5-minute schedules. d-Amphetamine decreased concurrent ratio and interval responding with the 1.5-minute interval schedules, but either increased or did not affect responding with the longer intervals. Changes in the parameter of one schedule altered responding controlled by that schedule and also other concurrent performances. As a consequence, the effects of drugs on each behavior were altered.     

Choice dynamics in concurrent ratio schedules of reinforcement

The generalized matching law predicts performance on concurrent schedules when variable-interval schedules are programmed but is trivially applicable when independent ratio schedules are used. Responding usually is exclusive to the schedule with the lowest response requirement. Determining a method to program concurrent ratio schedules such that matching analyses can be usefully employed would extend the generality of matching research and lead to new avenues of research. In the present experiments, ratio schedules were programmed dependently such that responses to either of the two options progressed the requirement on both schedules. Responding is not exclusive because the probability of reinforcement increases on both schedules as responses are allocated to either schedule. In Experiment 1, performance on concurrent variable-ratio schedules was assessed, and reinforcer ratios were varied across conditions to investigate changes in sensitivity. Additionally, the length of a changeover delay was manipulated. In Experiment 2, performance was compared under concurrently available, dependently programmed variable-ratio and fixed-ratio schedules. Performance was well described by the generalized matching law. Increases in the changeover delay decreased sensitivity, whereas sensitivity was higher when variable-ratio schedules were employed, compared with fixed-ratio schedules. Concurrent ratio schedules can be a viable approach to studying functional differences between ratio and interval schedules.     

Drug discrimination under a concurrent fixed-ratio fixed-ratio schedule.

Pigeons were trained to discriminate 5.0 mg/kg pentobarbital from saline under a two-key concurrent fixed-ratio 10 fixed-ratio 40 schedule of food presentation, in which the fixed-ratio component with the lower response requirement was programmed to reinforce responding on one key after drug administration (pentobarbital-biased key) and on the other key after saline administration (saline-biased key). After responding stabilized, pigeons averaged 98% of their responses on the pentobarbital-biased key during training sessions preceded by pentobarbital, and they averaged 90% of their responses on the saline-biased key during training sessions preceded by saline. In test sessions preceded by doses of pentobarbital, chlordiazepoxide, or ethanol, pigeons switched from responding on the saline-biased key at low doses to responding on the pentobarbital-biased key at higher doses (the dose-response curve was quantal). High doses of phencyclidine produced responding on both keys, whereas pigeons responded almost exclusively on the saline-biased key after all doses of methamphetamine. These and previous experiments using concurrent reinforcement schedules to study drug discrimination illustrate that the schedule of reinforcement is an important determinant of the shape of dose-effect curves in drug-discrimination experiments.     

Preference in concurrent variable-interval fixed-ratio schedules

Five pigeons were trained on concurrent variable-interval fixed-ratio schedules in three experiments. Experiment 1 used two variable-interval schedules and one fixed-ratio schedule, and the ratio requirement was varied. Using the generalized matching law, sensitivity to reinforcement was close to 1.0, but performance was biased toward the variable-interval schedule with the lower reinforcement rate. In Experiment 2, which used one variable-interval and one fixed-ratio schedule, the interval schedule was varied. All birds showed sensitivities to reinforcement of less than 1.0 and of less than the values obtained in Experiment 1. The performance was also biased toward the fixed-ratio schedule. Because the generalized matching law could not account for the differences in the data from Experiments 1 and 2, an extension of this law was suggested and successfully tested in Experiment 3. The proposed dual-sensitivity model was also shown to clarify some previously reported results.     

Schedule-induced defecation by rats during ratio and interval schedules of food reinforcement.

Lever pressing in rats was maintained by continuous and intermittent schedules of food while defecation was monitored. In Experiment 1, reinforcement densities were matched across variable-ratio and variable-interval schedules for three pairs of rats. Defecation occurred in all 3 rats on the variable-ratio schedule and in all 3 rats on the yoked variable-interval schedule. In Experiment 2, fixed-ratio and fixed-interval schedules with similar reinforcement densities maintained lever pressing. Defecation occurred in 3 of 4 rats on the fixed-ratio schedule and in 4 of 4 rats on the fixed-interval schedule. Almost no defecation occurred during continuous reinforcement in either experiment. These results demonstrate that defecation may occur during both ratio and interval schedules and that the inter-reinforcement interval is more important than the behavioral requirements of the schedule in generating schedule-induced defecation.     

Cooperative responding in rats maintained by fixed‐ and variable‐ratio schedules

The present study investigated the effects of fixed‐ratio (FR) and variable‐ratio (VR) reinforcement schedules on patterns of cooperative responding in pairs of rats. Experiment 1 arranged FR 1, FR 10, and VR 10 schedules to establish cooperative responding (water delivery depended on the joint responding of two rats). Cooperative response rates and proportions were higher under intermittent schedules than under continuous reinforcement. The FR 10 schedule generated a break‐and‐run pattern, whereas the VR 10 schedule generated a relatively high and constant rate pattern. Experiment 2 evaluated the effects of parametric manipulations of FR and VR schedules on cooperative responding. Rates and proportions of cooperative responding generally increased between ratio sizes of 1 and 5 but showed no consistent trend as the ratio increased from 5 to 10. Experiment 3 contrasted cooperative responding between an FR6 schedule and a yoked control schedule. Coordinated behavior occurred at a higher rate under the former schedule. The present study showed that external consequences and the schedules under which the delivery of these consequences are based, select patterns of coordinated behavior.     

Elimination of reinforced behavior: intermittent schedules of not-responding

Pigeons' key pecking resulted in food according to either a variable-ratio or a variable-interval schedule. At the same time, food was available for not pecking for a specified time. The required time of not-pecking was segmented into not-responding units, and these units were followed by food according to a fixed-ratio schedule. Both unit duration and the number required were varied. In general, the shorter the time unit or the smaller the ratio, the lower was response rate. When total required not-responding time was constant, but changes in unit duration and the number required altered how the total was achieved, shorter units produced lower rates. Other conditions involved substitution of food delivered independent of responding for the not-responding schedule. With low and moderate total times to food presentation, the not-responding schedule produced lower rates; with the longest times, the response-independent schedule generated less responding. When considered in terms of relative frequency of food presentation available from a source other than pecking, the not-responding schedule reduced rate more effectively than did the response-independent schedule. Comparisons with other research suggested that food presented dependent on not responding compared favorably with punishment as a procedure for reducing response rate. Transient effects differed. Although punishment temporarily depresses rate when first imposed and temporarily enhances it when first removed, food given for not responding quickly generated steady-state rates.     

The effects of brief-stimulus presentations in fixed-ratio second-order schedules

Pigeons' responses were reinforced according to a three-component multiple schedule. In Component 1, key pecks produced food according to a fixed-ratio second-order schedule with fixed-ratio units. Here, a fixed number of fixed-ratio units produced food, and the brief stimulus terminating each unit also accompanied food. Responses in Component 2 produced food on an identical schedule except that the brief stimulus was not paired with food. Component 3 contained a simple fixed-ratio schedule whose response requirement equaled that of Components 1 and 2. Across conditions the size of the fixed-ratio unit (five, ten, twenty, forty, and eighty responses) and the total number of responses per reinforcement were parametrically manipulated. The highest response rates and shortest preratio pauses were observed in Component 3 (no brief stimulus). The lowest rates and longest pauses were found in the component with paired brief-stimulus presentations, indicating that the food-paired brief stimulus suppressed responding. The suppressive effects were greatest when the fixed-ratio units were small (e.g., fixed-ratio 5) and the total fixed-ratio requirement was large (e.g., fixed-ratio 160). Under no conditions did the paired brief stimulus facilitate responding. The nonpaired brief stimulus also suppressed responding but to a lesser extent. The suppressive effects of nonpaired brief stimuli were greatest when the fixed-ratio units were small and the total response requirement was large. These data suggest that the suppressive effects of the brief stimuli may have masked the conditioned-reinforcing effects reported in other studies, and that conditions that maximize suppression in second-order schedules involve the use of fixed-ratio schedule units and the presentation of many brief stimuli per reinforcer.     

Reinforcing the absence of fixed-ratio performance

Pigeons received food for key pecking according to a fixed-ratio schedule, while, at the same time, food also was available for not pecking for a specified time. With a fixed ratio of 60, responding was not affected by not-pecking times of 80 or 40 seconds, and was eliminated completely at 10 seconds. With ratios of 180, pecking stopped with not-pecking times of 80 seconds or less; with ratios of 300, it stopped at 120 seconds or less. Not-responding schedules produced steady-state performance immediately following contact with the schedule. With return to the fixed-ratio schedule alone, response rate sometimes was elevated temporarily. When response-independent food presentations replaced the not-pecking schedule, response rate often was enhanced, and the ratio pattern was lost. Only the highest densities of food delivery eliminated responding, even with a fixed ratio of 300. In general, the effects corresponded to those of punishment, except that contrast had appeared both during and after punishment, and now appeared only after the response elimination procedure was suspended.     

Reinforcement frequency and contingency as factors in fixed-ratio behavior

Two variables often confounded in fixed-ratio schedules are reinforcement frequency and response requirement. These variables were isolated by a technique that yoked the distributions of reinforcements in time for one group of pigeons to those of pigeons responding on various fixed-ratio schedules. The contingencies for the yoked birds were then manipulated by adding various tandem fixed-ratio requirements to their schedules. Post-reinforcement pause was approximately equal for the yoked and ratio pigeons, and was relatively insensitive to changes in the tandem requirement. Terminal response rate increased with increases in the tandem requirement, even though reinforcement rate was invariant. This increase was attributed to the progressive interference of the tandem requirement with the differential reinforcement of long interresponse times.     

Second-order schedules of token reinforcement with pigeons: effects of fixed- and variable-ratio exchange schedules

Pigeons' key pecks produced food under second-order schedules of token reinforcement, with light-emitting diodes serving as token reinforcers. In Experiment 1, tokens were earned according to a fixed-ratio 50 schedule and were exchanged for food according to either fixed-ratio or variable-ratio exchange schedules, with schedule type varied across conditions. In Experiment 2, schedule type was varied within sessions using a multiple schedule. In one component, tokens were earned according to a fixed-ratio 50 schedule and exchanged according to a variable-ratio schedule. In the other component, tokens were earned according to a variable-ratio 50 schedule and exchanged according to a fixed-ratio schedule. In both experiments, the number of responses per exchange was varied parametrically across conditions, ranging from 50 to 400 responses. Response rates decreased systematically with increases in the fixed-ratio exchange schedules, but were much less affected by changes in the variable-ratio exchange schedules. Response rates were consistently higher under variable-ratio exchange schedules than tinder comparable fixed-ratio exchange schedules, especially at higher exchange ratios. These response-rate differences were due both to greater pre-ratio pausing and to lower local rates tinder the fixed-ratio exchange schedules. Local response rates increased with proximity to food under the higher fixed-ratio exchange schedules, indicative of discriminative control by the tokens.     

Resistance to change of operant variation and repetition

A multiple chained schedule was used to compare the relative resistance to change of variable and fixed four-peck response sequences in pigeons. In one terminal link, a response sequence produced food only if it occurred infrequently relative to 15 other response sequences (vary). In the other terminal link, a single response sequence produced food (repeat). Identical variable-interval schedules operated in the initial links. During baseline, lower response rates generally occurred in the vary initial link, and similar response and reinforcement rates occurred in each terminal link. Resistance of responding to prefeeding and three rates of response-independent food delivered during the intercomponent intervals then was compared between components. During each disruption condition, initial- and terminal-link response rates generally were more resistant in the vary component than in the repeat component. During the response-independent food conditions, terminal-link response rates were more resistant than initial-link response rates in each component, but this did not occur during prefeeding. Variation (in vary) and repetition (in repeat) both decreased during the response-independent food conditions in the respective components, but with relatively greater disruption in repeat. These results extend earlier findings demonstrating that operant variation is more resistant to disruption than is operant repetition and suggest that theories of response strength, such as behavioral momentum theory, must consider factors other than reinforcement rate. The implications of the results for understanding operant response classes are discussed.