Elimination of reinforced behavior: intermittent schedules of not-responding

Pigeons' key pecking resulted in food according to either a variable-ratio or a variable-interval schedule. At the same time, food was available for not pecking for a specified time. The required time of not-pecking was segmented into not-responding units, and these units were followed by food according to a fixed-ratio schedule. Both unit duration and the number required were varied. In general, the shorter the time unit or the smaller the ratio, the lower was response rate. When total required not-responding time was constant, but changes in unit duration and the number required altered how the total was achieved, shorter units produced lower rates. Other conditions involved substitution of food delivered independent of responding for the not-responding schedule. With low and moderate total times to food presentation, the not-responding schedule produced lower rates; with the longest times, the response-independent schedule generated less responding. When considered in terms of relative frequency of food presentation available from a source other than pecking, the not-responding schedule reduced rate more effectively than did the response-independent schedule. Comparisons with other research suggested that food presented dependent on not responding compared favorably with punishment as a procedure for reducing response rate. Transient effects differed. Although punishment temporarily depresses rate when first imposed and temporarily enhances it when first removed, food given for not responding quickly generated steady-state rates.     

Concurrent schedules: a quantitative relation between changeover behavior and its consequences

Data from several published experiments on concurrent variable-interval schedules were analyzed with respect to the effects of changeover delay on the time spent responding on a schedule before changing to an alternate schedule: i.e., the interchangeover time. Interchangeover time increases as the duration of the changeover delay increases, and the present analysis shows that a power function describes the relation. The power relation applied in spite of numerous differences in the experiments: different variable-interval schedules for the concurrent pairs; equal or unequal reinforcement rates for the schedules of the concurrent pairs; different durations of the changeover delay; response-dependent or response-independent reinforcers; pigeons or rats as subjects; different reinforcers. A power function also described the data in experiments where the changeover incurred a timeout, where a fixed ratio was required to changeover, and also when asymmetrical changeover delays were used.     

Fixed-interval responding under second-order schedules of food presentation or cocaine injection.

Squirrel monkeys operated a key under second-order schedules in which every tenth completion of a 5-minute fixed interval resulted in either presentation of food or intravenous injection of cocaine. When a 2-second light was presented at the completion of the component fixed-interval schedules, positively accelerated responding developed and was maintained in each component. Over a tenfold range of doses of cocaine(30 to 300 microgram/kg/injection) and amounts of food (0.75 to 7.5 g/presentation); the second-order schedule of cocaine injection maintained higher average rates of responding than the second-order schedule of food presentation. Substituting saline for cocaine or eliminating food presentation decreased average rates of responding. When no stimulus change occurred at the completion of the first nine component fixed-interval schedules, but the 2-second light and food presentation or cocaine injection still occurred after the tenth component, only low and relatively constant rates of responding were maintained in each component. Patterns of responding characteristic of 5-minute fixed-interval schedules were maintained by the 2-second light paired with either cocaine injection or food presentation, though the maximum frequency of cocaine injection or food presentation was less than once per 50 minutes.     

Demand for food on fixed-ratio schedules as a function of the quality of concurrently available reinforcement

Six rats lever pressed for food on concurrent fixed-ratio schedules, in a two-compartment chamber. In one compartment, mixed diet pellets were delivered on fixed-ratio schedules of 1, 6, 11, and 16; in the other, either no food was delivered, or sucrose or mixed diet pellets were delivered on fixed-ratio 8. The number of pellets obtained in the first compartment declined as a function of fixed-ratio size in that compartment in all three conditions, but the decline was greatest overall with mixed diet pellets concurrently available in the other compartment, and least with no food concurrently available. The result is discussed in terms of economic demand theory, and is consistent with the prediction that elasticity of demand for a commodity (defined in operant terms as the ratio of the proportionate change in number of reinforcements per session to the proportionate change in fixed-ratio size) is greater the more substitutable for that commodity are any concurrently available commodities.     

Duration and rate of reinforcement as determinants of concurrent responding

The duration and frequency of food presentation were varied in concurrent variable-interval variable-interval schedules of reinforcement. In the first experiment, in which pigeons were exposed to a succession of eight different schedules, neither relative duration nor relative frequency of reinforcement had as great an effect on response distribution as they have when they are manipulated separately. These results supported those previously reported by Todorov (1973) and Schneider (1973). In a second experiment, each of seven pigeons was exposed to only one concurrent schedule in which the frequency and/or duration of reinforcement differed on the two keys. Under these conditions, each pigeon's relative rate of response closely matched the relative total access to food that each schedule provided. This result suggests that previous failures to obtain matching may be due to factors such as an insufficient length of exposure to each schedule or to the pigeons' repeated exposure to different concurrent schedules.     

On Herrnstein's equation and related forms

In 1970, Herrnstein proposed a simple equation to describe the relation between response and reinforcement rates on interval schedules. Its empirical basis is firm, but its theoretical foundation is still uncertain. Two approaches to the derivation of Herrnstein's equation are discussed. It can be derived as the equilibrium solution to a process model equivalent to familiar linear-operator learning models. Modifications of this approach yield competing power-function formulations. The equation can also be derived from the assumption that response strength is proportional to reinforcement rate, given that there is a ceiling on response rate. The proportional relation can, in turn, be derived from a threshold assumption equivalent to Shimp's “momentary maximizing”. This derivation implies that the two parameters of Herrnstein's equation should be correlated, and may explain its special utility in application to internal schedules.     

Positive contrast, negative induction, and inhibitory stimulus control in the rat

In Experiment I, 24 rats were trained on a multiple variable-interval variable-interval schedule with a doorlight and white noise serving as component cues. Two groups were then shifted to a multiple extinction variable-interval schedule, and a third group was maintained on the multiple variable-interval variable-interval schedule. The multiple extinction variable-interval condition produced positive contrast when either the light or noise signalled extinction, and both of these cues acquired inhibitory stimulus control as measured by a combined cue test. In Experiment II, the multiple variable-interval variable-interval condition was shifted to multiple extinction variable-interval for one group, to multiple variable-time variable-interval for a second group, and was unchanged for the third group. The two experimental conditions produced identical patterns of response-rate reduction in the altered component, but the multiple extinction variable-interval condition produced positive contrast, whereas the multiple variable-time variable-interval condition did not. Subsequent combined cue and resistance to reinforcement tests revealed that the cue signalling extinction acquired stronger inhibitory stimulus control than the cue signalling variable time.     

Choice, time allocation, and response rate during stimulus generalization

Six pigeons were trained to discriminate between two noise intensities using a procedure that assessed choice, time allocation, and response rate simultaneously and independently. Responses on the left or right key (R1 or R2) were respectively correct in the presence of two different intensities, S1 and S2. After a correct response, reinforcement became available for pecks on the center key. Reinforcement density for R1¦S1 relative to R2¦S2 was varied across experimental conditions. Generalization tests followed extensive training at each condition. As a function of stimulus intensity, proportions of initial choices of R2, of time spent in R2-initiated components, and of center-key responses emitted in R2-initiated components all yielded sigmoidal gradients of similar slope, which shifted slightly in location when relative reinforcement density changed. Changeovers were maximal where initial choice proportions approximated 0.5. Gradients relating the absolute number of center-key responses to stimulus intensity were also roughly sigmoidal, but were more sensitive to changes in reinforcement density. Gradients of momentary response rate also depended on reinforcement density. During training, large but transitory shifts in choice responding occurred when reinforcement density changed, while differences in momentary response rate developed slowly, suggesting separate control of choice and response rate by the contingencies of reinforcement.     

A comparison of frequency, interval, and time-sampling methods of data collection

Data representing high, medium, and low response rates in constant and nonconstant patterns were generated by electromechanical equipment to determine whether the same data collected by time-sampling, interval recording, and frequency recording would be represented similarly by each method. Results indicated: (1) that time-sampling provided an extremely inaccurate estimate of responding, and (2) that interval recording accurately represented responding of low and medium rates, but grossly underestimated high-rate responding.     

Clock control of human performance on avoidance and fixed-interval schedules

The avoidance and fixed-interval performances of human subjects were studied in two experiments. Addition of time-correlated stimuli (added clock) improved behavioral efficiency, since response rates decreased without decreases in reinforcement rates. Response-dependent display of the clock maintained a second, observing response and reductions in clock duration weakened such observing behavior. Generally, the reinforcing properties of the clock were more apparent with the avoidance than with the fixed-interval schedule, a finding attributed to temporal cues already provided by delivery of the fixed-interval reinforcers. Reduced rates of the main response when the clock was dependent on an observing response were more than offset by rates of the observing response in the majority of subjects. Thus, the results do not support an interpretation of the reinforcing properties of added clocks simply in terms of work reduction.