Elimination of reinforced behavior: intermittent schedules of not-responding

Pigeons' key pecking resulted in food according to either a variable-ratio or a variable-interval schedule. At the same time, food was available for not pecking for a specified time. The required time of not-pecking was segmented into not-responding units, and these units were followed by food according to a fixed-ratio schedule. Both unit duration and the number required were varied. In general, the shorter the time unit or the smaller the ratio, the lower was response rate. When total required not-responding time was constant, but changes in unit duration and the number required altered how the total was achieved, shorter units produced lower rates. Other conditions involved substitution of food delivered independent of responding for the not-responding schedule. With low and moderate total times to food presentation, the not-responding schedule produced lower rates; with the longest times, the response-independent schedule generated less responding. When considered in terms of relative frequency of food presentation available from a source other than pecking, the not-responding schedule reduced rate more effectively than did the response-independent schedule. Comparisons with other research suggested that food presented dependent on not responding compared favorably with punishment as a procedure for reducing response rate. Transient effects differed. Although punishment temporarily depresses rate when first imposed and temporarily enhances it when first removed, food given for not responding quickly generated steady-state rates.     

Dependency, temporal contiguity, and response-independent reinforcement

A comparison was made of the effects of variable-interval, variable-time, and tandem variable-interval fixed-time schedules on key-peck responding of pigeons. The variable-interval component of the tandem schedule retained the response-reinforcement dependency; the fixed-time component allowed the temporal proximity between responding and reinforcement to vary, constrained only by the duration of the fixed-time interval. Response rates were highest during the variable-interval and lowest during the variable-time schedule. Intermediate response rates occurred during the tandem schedule. The results of a yoked control condition showed that the effects of the tandem schedule were not due simply to changes in reinforcement distribution or frequency. The results suggest that substantial reductions in responding occur when reinforcement is response-dependent but not necessarily contiguous with the response required to produce reinforcement.     

Responding maintained under fixed-interval and fixed-time schedules of electric shock presentation

Following initial histories under a schedule of electric shock postponement, lever pressing in squirrel monkeys was maintained under fixed-interval and fixed-time schedules of electric shock presentation. No difference in either rate or pattern of responding was obtained when these schedules were presented as components of a multiple schedule. When they were presented singly for long periods of time, the fixed-interval schedule consistently maintained a higher response rate than the fixed-time schedule. The pattern of responding under both schedules was similar, typically consisting of a pause at the beginning of each interval followed by either a steady or a positively accelerating rate of responding. The results suggest that the response-shock dependency is of critical importance in the maintenance of high rates of responding under schedules of electric shock presentation, and support the general view that such responding may be conceptualized as operant behavior under control of many of the same variables that control responding under comparable schedules of food or water reinforcement.     

Induced attack during fixed-ratio and matched-time schedules of food presentation

Adjunctive or induced behavior is generated during a variety of schedules of reinforcement. Several theoretical conceptualizations suggest that rate of reinforcement is the primary variable controlling the strength or levels of induced behavior. The operant response requirement within the schedule context has not been extensively studied as a determinant of induced responding. In the present study, levels of induced attack by food-deprived pigeons against restrained conspecifics were compared during response-dependent and response-independent schedules of food presentation equated or yoked interval-by-interval for reinforcement frequency. Experiment 1 compared levels of attack induced by fixed-ratio schedules of key pecking and yoked "matched-time" schedules. Experiment 2 similarly compared chained fixed-ratio 1 fixed-ratio 74 and yoked chained matched-time matched-time schedules. In both experiments, the response-dependent schedules generated greater levels (amount and probability) of induced attack than the response-independent time-based schedules. Thus, the ratio response requirement may be an important determinant of levels of induced responding, and the lower levels of attack observed during the response-independent condition may not be due to the absence of stimuli predicting food presentations. It is concluded that rate of reinforcement is not the sole variable determining levels of induced responding and that response-based and time-based schedules differ in their generation of induced responding.     

Response-independent Events In The Behavior Stream

The metaphor of the behavior stream provides a framework for studying the effects of response-independent food presentations intruded into an environment in which operant responding of pigeons was maintained by variable-interval schedules. In the first two experiments, response rates were reduced when response-independent food was intruded during the variable-interval schedule according to a concomitantly present fixed-time schedule. These reductions were not always an orderly function of the percentage of response-dependent food. Negatively accelerated patterns of key pecking across the fixed-time period occurred in Experiment 1 under the concomitant fixed-time variable-interval schedules. In Experiment 2, positively and negatively accelerated and linear response patterns occurred even though the schedules were similar to those used in Experiment 1. The variable findings in the first two experiments led to three subsequent experiments that were designed to further illuminate the controlling variables of the effects of intruded response-independent events. When the fixed and variable schedules were correlated with distinct operanda by employing a concurrent fixed-interval variable-interval schedule (Experiment 3) or with distinct discriminative stimuli (Experiments 4 and 5), negatively accelerated response patterns were obtained. Even in these latter cases, however, the response patterns were a joint function of the physical separation of the two schedules and the ratio of fixed-time or fixed-interval to variable-interval schedule food presentations. The results of the five experiments are discussed in terms of the contributions of both reinforcement variables and discriminative stimuli in determining the effects of intruding response-independent food into a stream of operant behavior.     

Effects of reinforcement history on responding under progressive-ratio schedules of reinforcement.

The effects of experimental history on responding under a progressive-ratio schedule of reinforcement were examined. Sixteen pigeons were divided into four equal groups. Groups 1 to 3 were trained to peck a key for food under a fixed-ratio, variable-ratio, or differential-reinforcement-of-low-rate schedule of reinforcement. After training, these pigeons were shifted to a progressive-ratio schedule, later were shifted back to their original schedule (with decreased rates of reinforcement), and finally were returned to the progressive-ratio schedule. Pigeons in Group 4 (control) were maintained on the progressive-ratio schedule for the entire experiment. To test for potential "latent history" effects, pigeons responding under the progressive-ratio schedule were injected with d-amphetamine and given behavioral-momentum tests of prefeeding and extinction. Experimental histories affected responding in the immediate transition to the progressive-ratio schedule; response rates of pigeons with variable-ratio and fixed-ratio histories were higher than rates of pigeons with differential-reinforcement-of-low-rate and progressive-ratio-only histories. Pigeons with differential-reinforcement-of-low-rate histories, and to a lesser degree pigeons with variable-ratio and fixed-ratio histories, also had shorter postreinforcement pauses than pigeons with only a progressive-ratio history. No consistent long-term effects of prior contingencies on responding under the progressive-ratio schedule were evident. d-Amphetamine and resistance-to-change tests failed to reveal consistent latent history effects. The data suggest that history effects are sometimes transitory and not susceptible to latent influences.     

Choice between response units: The rate constancy model

In a conjoint schedule, reinforcement is available simultaneously on two or more schedules for the same response. The present experiments provided food for key pecking on both a random-interval and a differential-reinforcement-of-low-rate (DRL) schedule. Experiment 1 involved ordinary DRL schedules; Experiment 2 added an external stimulus to indicate when the required interresponse time had elapsed. In both experiments, the potential reinforcer frequency from each component was varied by means of a second-order fixed-ratio schedule, and the DRL time parameter was changed as well. Response rates were described by a model stating that time allocation to each component matches the relative frequency of reinforcement for that component. When spending time in a given component, the subject is assumed to respond at the rate characteristic of baseline performance. This model appeared preferable to the absolute-rate version of the matching law. The model was shown to be applicable to multiple-response concurrent schedules as well as to conjoint schedules, and it described some of the necessary conditions for response matching, undermatching, and bias. In addition, the pigeons did not optimize reinforcer frequency.     

Matching under concurrent fixed-ratio variable-interval schedules of food presentation

Four pigeons were exposed to concurrent fixed-ratio, variable-interval schedules of food presentation. The fixed-ratio requirement was either 25, 50, 75, or 100 responses, with the variable-interval schedule parameter held constant at 4 minutes. A delay time was imposed between a changeover from one schedule to the other and subsequent food availability. The delay time was varied at each ratio requirement over four values; no delay, 0-second delay, 1.5-second delay, and 5.0-second delay. As the fixed-ratio requirement or the delay time increased, a greater proportion of the total responses and time spent responding occurred under the variable-interval schedule relative to the proportion of food deliveries under that schedule. Neither relative overall response rate nor relative time spent responding equalled the relative frequency of food presentation, as would be predicted by a linear “matching” model. Rather, these data were described by power functions with slopes of approximately 1.0 and intercepts greater than 1.0. In the terms of Baum's (1974) analysis, these deviations from linear matching represent bias in favor of responding under the interval schedule. Bias, as reflected in the intercept of the power function, was greater for the ratio of time than the ratio of responses.     

Duration and rate of reinforcement as determinants of concurrent responding

The duration and frequency of food presentation were varied in concurrent variable-interval variable-interval schedules of reinforcement. In the first experiment, in which pigeons were exposed to a succession of eight different schedules, neither relative duration nor relative frequency of reinforcement had as great an effect on response distribution as they have when they are manipulated separately. These results supported those previously reported by Todorov (1973) and Schneider (1973). In a second experiment, each of seven pigeons was exposed to only one concurrent schedule in which the frequency and/or duration of reinforcement differed on the two keys. Under these conditions, each pigeon's relative rate of response closely matched the relative total access to food that each schedule provided. This result suggests that previous failures to obtain matching may be due to factors such as an insufficient length of exposure to each schedule or to the pigeons' repeated exposure to different concurrent schedules.     

Response persistence under variable-time schedules following immediate and unsignalled delayed reinforcement

Key pecking of three pigeons was maintained in separate components of a multiple schedule by either immediate reinforcement (i.e., tandem variable-time fixed-interval schedule) or unsignalled delayed reinforcement (i.e., tandem variable-interval fixed-time schedule). The relative rate of food delivery was equal across components, and this absolute rate differed across conditions. Immediate reinforcement always generated higher response rates than did unsignalled delayed reinforcement. Then, variable-time schedules of food delivery replaced the contingencies just described such that food was delivered at the same rate but independently of responding. In most cases, response rates decreased to near-zero levels. In addition, response persistence was not systematically different between multiple-schedule components across pigeons. The implications of the results for the concepts of response strength and the response-reinforcer relation are noted.     

The role of contingencies and "principles of behavioral variation" in pigeons' pecking

Staddon and Simmelhag's proposal that behavior is produced by “principles of behavioral variation” instead of contingencies of reinforcement was tested in two experiments. In the first experiment pigeons were exposed to either a fixed-interval schedule of response-contingent reinforcement, an autoshaping schedule of stimulus-contingent reinforcement, or a fixed-time schedule of noncontingent reinforcement. Pigeons exposed to contingent reinforcement came to peck more rapidly than those exposed to noncontingent reinforcement. Staddon and Simmelhag's “principles of behavioral variation” included the proposal that patterns (interim and terminal) were a function of momentary probability of reinforcement. In the second experiment pigeons were exposed to either a fixed-time or a random-time schedule of noncontingent reinforcement. Pecking showed a constant frequency of occurrence over postfood time on the random-time schedule. Most behavior showed patterns on the fixed-time schedule that differed in overall shape (i.e., interim versus terminal) from those shown on the random-time schedule. It was concluded that both the momentary probability of reinforcement and postfood time can affect patterning.     

Behavior-dependent reinforcer-rate changes in concurrent schedules: A further analysis

Six pigeons were trained on concurrent variable-interval schedules in three different procedures. The first procedure was a standard concurrent schedule, and the relative reinforcer frequency for responding was varied. The second was a schedule in which a relative left-key response rate (over a fixed period of time) exceeding .75 produced, in the next identical time period a higher reinforcer rate on the right key. If this criterion was not exceeded, equal reinforcer rates were arranged on the two keys in this period. This was the dependent procedure. In the third (independent) procedure, the periods of higher right-key reinforcer rates occurred with the same probability as in the second procedure, but occurred independently of behavior. In the second and third procedures, the fixed-time period (window) was varied from 5 s to 60 s, and to 240 s in the second procedure only. Performance on the two keys was similar in the concurrent and independent procedures. The procedure used in the dependent conditions generally affected performance when the windows were shorter than about 30 s. Models of performance that assume that subjects do not discriminate changes in local relative reinforcer rates cannot account for the data. Moreover, existing models are inherently unable to account for the effects of contingencies of reinforcement between responding on one alternative and gaining reinforcers on another that are arranged or that emerge as a result of time allocated to alternative schedules. Undermatching on concurrent variable-interval schedules may result from such emergent contingencies.     

Patterning with fixed-time schedules of response-independent reinforcement

Pigeons were first exposed to a schedule providing food when the time between successive key pecks (the interresponse time) exceeded a specified duration. When food then was presented at regular intervals independent of responding (fixed-time schedule), responses typically occurred at a steady rate in the periods between successive food presentations. Once the birds had been exposed to a fixed-ratio schedule, however, response rate under fixed-time schedules was positively accelerated. Variations in the sequence of conditions given different subjects indicated that the changes in patterning were due to the fixed-ratio schedule, rather than to the number of transitions from a response-dependent to the response-independent fixed-time schedule, to changed parameter values, or to prolonged experience with the fixed-time schedule. The effects of fixed-time schedules on patterning depended upon experimental history.     

Pentobarbital and d-amphetamine effects on concurrent performances.

The effects of pentobarbital and d-amphetamine were studied in pigeons responding under several concurrent fixed-ratio variable-interval and concurrent fixed-ratio fixed-interval schedules of food presentation. Drug effects were compared with different fixed ratios, fixed and variable intervals, changeover delays, and with the schedules operating singly. Doses of d-amphetamine that increased or did not affect responding under the interval schedules decreased responding under the fixed-ratio schedule, whereas doses of pentobarbital that increased responding under the fixed-ratio schedule decreased or eliminated responding under the interval schedules. These effects depended both on the schedule of food delivery and the parameters of schedules arranged concurrently. Pentobarbital increased responding under the fixed-ratio schedule with 4-minute and 10-minute interval schedules arranged concurrently, but not with 1.5-minute schedules. d-Amphetamine decreased concurrent ratio and interval responding with the 1.5-minute interval schedules, but either increased or did not affect responding with the longer intervals. Changes in the parameter of one schedule altered responding controlled by that schedule and also other concurrent performances. As a consequence, the effects of drugs on each behavior were altered.     

Resistance to reinforcement change in multiple and concurrent schedules assessed in transition and at steady state

Behavioral momentum theory relates resistance to change of responding in a multiple-schedule component to the total reinforcement obtained in that component, regardless of how the reinforcers are produced. Four pigeons responded in a series of multiple-schedule conditions in which a variable-interval 40-s schedule arranged reinforcers for pecking in one component and a variable-interval 360-s schedule arranged them in the other. In addition, responses on a second key were reinforced according to variable-interval schedules that were equal in the two components. In different parts of the experiment, responding was disrupted by changing the rate of reinforcement on the second key or by delivering response-independent food during a blackout separating the two components. Consistent with momentum theory, responding on the first key in Part 1 changed more in the component with the lower reinforcement total when it was disrupted by changes in the rate of reinforcement on the second key. However, responding on the second key changed more in the component with the higher reinforcement total. In Parts 2 and 3, responding was disrupted with free food presented during intercomponent blackouts, with extinction (Part 2) or variable-interval 80-s reinforcement (Part 3) arranged on the second key. Here, resistance to change was greater for the component with greater overall reinforcement. Failures of momentum theory to predict short-term differences in resistance to change occurred with disruptors that caused greater change between steady states for the richer component. Consistency of effects across disruptors may yet be found if short-term effects of disruptors are assessed relative to the extent of change observed after prolonged exposure.     

Choice and segmented interreinforcement intervals

Pigeons were trained on a two-key concurrent schedule, where food reinforcers on one key were arranged by a simple variable-interval schedule and on the other key by a chain variable-interval variable-interval schedule. When the initial link of the chain was in effect, the pigeons tended to respond more on the simple variable-interval schedule, and hence less on the chain, than would be expected from a comparison of both the local and overall rates of reinforcement of the two schedules. When the terminal link of the chain was in effect, the pigeons responded more on the chain than would be expected from a comparison of the rates of reinforcement of the schedules then in effect. Overall responding on the chain was not proportional to overall reinforcement on the chain but rather was a by-product of responding during initial- and terminal-link phases.     

Response strength in multiple periodic and aperiodic schedules

Responding in multiple periodic and aperiodic schedules of equal mean reinforcement rate was examined during extinction, satiation, and in the presence of various free-food schedules. In Experiments I and II, pigeons were trained on multiple variable-interval–fixed-interval schedules. Decreases in the rate of responding due to extinction, satiation, or food schedules were approximately equal regardless of the temporal pattern of reinforcer presentation. In Experiment III, pigeons responded on a two-component multiple schedule in which each component was a two-member homogeneous response chain terminating in a fixed-interval schedule during one component and in a variable-interval schedule during the other. The length of both terminal links was varied over a series of conditions. Initial-link responding in the fixed-interval component was reduced more by increasing terminal-link length than was initial-link responding in the variable-interval component. However, no differences in resistance to satiation and extinction were obtained across the fixed and variable components. If the relative decrease in responding produced by satiation and extinction is used as an index of the “value” of the conditions maintaining responding, then these data suggest that fixed and variable schedules of equal mean length are equally valued. This conclusion, however, is not consistent with findings of preference for variable over fixed schedules obtained in studies using concurrent-chain procedures.     

Neurochemical changes correlated with behavior maintained under fixed-interval and fixed-ratio schedules of reinforcement.

Key pecking of 4 pigeons was maintained under a multiple 3-min fixed-interval, 30-response fixed-ratio schedule of food presentation. Only one schedule was in effect during an experimental session, and each was correlated with a different keylight stimulus and location (left vs. right). The different schedule components alternated across days or weeks. Cerebrospinal fluid was collected from chronically implanted intracerebroventricular cannulae following sessions with the different schedules, as well as following sessions in which reinforcement was withheld (extinction), when response-independent food was delivered, and when the experimental chamber was dark and there were no scheduled events. Metabolites of the neurotransmitters serotonin, norepinephrine, and dopamine were assayed in cerebrospinal fluid using high-performance liquid chromatography with electrochemical detection. Compared to the fixed-ratio condition, responding maintained under the fixed-interval schedule resulted in consistently higher levels of the serotonin metabolite 5-hydroxyindoleacetic acid and of the dopamine metabolite homovanillic acid in all pigeons. Levels of 3-methoxy-4-hydroxyphenylethylene glycol, a metabolite of norepinephrine, and dihydroxyphenylacetic acid, another dopamine metabolite, were also higher in 3 of the 4 pigeons following exposure to the fixed-interval schedules when compared to levels of these metabolites after exposure to the fixed-ratio schedule. Extinction of fixed-ratio responding resulted in large increases in 5-hydroxyindoleacetic acid compared to levels of this metabolite under the fixed-ratio schedule, whereas this serotonin metabolite decreased during extinction of responding under the fixed-interval schedule. Control procedures suggested that the neurochemical changes were not related to the rate of responding but were a function of the specific experimental conditions. Distinctive neurochemical changes that accompany schedule-controlled responding show the sensitivity of the neurochemical environment to behavioral contingencies and demonstrate further the profound impact that such contingencies have on biobehavioral processes.     

Effects of fixed-time shocks and brief stimuli on food-maintained behavior of rats

When a fixed-time schedule of shocks was presented to rats lever pressing for food on a random-interval schedule, a pattern of behavior developed with a high rate of pressing after shock declining to near zero before the next shock was delivered. Once this pattern had stabilized, one-quarter of the shocks were replaced with brief auditory stimuli (tones) in a random sequence. Tone maintained behavior similar to shock, although tone was never paired with shock. Both tone and shocks elicited responding when presented at various times as probe stimuli, and responding was usually totally suppressed if neither stimulus occurred at the beginning of the fixed-time interval. When other stimuli were paired with tone and shock, only those paired with tone gained discriminative control and elicited responding. These findings suggest that stimuli that signal a shock-free, or safe, period will maintain the pattern of behavior generated by shock on a fixed-time schedule. There is a parallel between this phenomenon and the control of behavior on second-order schedules of positive reinforcement with nonpaired brief stimuli.     

Drug discrimination under a concurrent fixed-ratio fixed-ratio schedule.

Pigeons were trained to discriminate 5.0 mg/kg pentobarbital from saline under a two-key concurrent fixed-ratio 10 fixed-ratio 40 schedule of food presentation, in which the fixed-ratio component with the lower response requirement was programmed to reinforce responding on one key after drug administration (pentobarbital-biased key) and on the other key after saline administration (saline-biased key). After responding stabilized, pigeons averaged 98% of their responses on the pentobarbital-biased key during training sessions preceded by pentobarbital, and they averaged 90% of their responses on the saline-biased key during training sessions preceded by saline. In test sessions preceded by doses of pentobarbital, chlordiazepoxide, or ethanol, pigeons switched from responding on the saline-biased key at low doses to responding on the pentobarbital-biased key at higher doses (the dose-response curve was quantal). High doses of phencyclidine produced responding on both keys, whereas pigeons responded almost exclusively on the saline-biased key after all doses of methamphetamine. These and previous experiments using concurrent reinforcement schedules to study drug discrimination illustrate that the schedule of reinforcement is an important determinant of the shape of dose-effect curves in drug-discrimination experiments.