ALTERNATIVE REINFORCEMENT INCREASES RESISTANCE TO CHANGE: PAVLOVIAN OR OPERANT CONTINGENCIES?

Two multiple-schedule experiments with pigeons examined the effect of adding food reinforcement from an alternative source on the resistance of the reinforced response (target response) to the decremental effects of satiation and extinction. In Experiment 1, key pecks were reinforced by food in two components according to variable-interval schedules and, in some conditions, food was delivered according to variable-time schedules in one of the components. The rate of key pecking in a component was negatively related to the proportion of reinforcers from the alternative (variable-time) source. Resistance to satiation and extinction, in contrast, was positively related to the overall rate of reinforcement in the component. Experiment 2 was conceptually similar except that the alternative reinforcers were contingent on a specific concurrent response. Again, the rate of the target response varied as a function of its relative reinforcement, but its resistance to satiation and extinction varied directly with the overall rate of reinforcement in the component stimulus regardless of its relative reinforcement. Together the results of the two experiments suggest that the relative reinforcement of a response (the operant contingency) determines its rate, whereas the stimulus-reinforcement contingency (a Pavlovian contingency) determines its resistance to change.     

Interresponse-time shaping by variable-interval-like interresponse-time reinforcement contingencies

The interresponse-time reinforcement contingencies and distributions of interreinforcement intervals characteristic of certain variable-interval schedules were mimicked by reinforcing each key peck with a probability equal to the duration of the interresponse time it terminated, divided by the scheduled mean interreinforcement interval. The interresponse-time reinforcement contingency was then eliminated by basing the probability of reinforcement on the fifth interresponse time preceding the key peck. Even though distributions of interreinforcement intervals were unaffected by this manipulation, response rates consistently increased. A second experiment replicated this effect and showed it to combine additively with that of mean reinforcement rate. These results provide strong support for the contention that current analyses of variable-interval response rates that ignore the inherent interresponse-time reinforcement contingency may be seriously in error.     

The role of discriminative stimuli in concurrent performances

Key pecking in pigeons was examined under concurrent and parallel arrangements of two independent and simultaneously available variable-interval schedules. Pecks on the changeover key alternated the schedule of reinforcement for responses on the main key. Under concurrent schedules, discriminative stimuli were paired with the reinforcement schedule arranged in each component and changeover responses also alternated these stimuli. Under parallel schedules, changeover responses alternated the effective reinforcement schedule, but did not change the discriminative stimulus. On concurrent procedures, changeover response rate was inversely related to the difference in reinforcement rate between the two components, whereas on parallel schedules no consistent relationship was found. With both schedules, absolute response and reinforcement rates were positively related, although for a given set of reinforcement frequencies, rates were often higher on the concurrent schedules. On concurrent schedules, relative response rates and relative times were equal to relative reinforcement rates. On parallel schedules these ratios were positively related, but response and time ratios were much smaller than were obtained with comparable concurrent schedules. This inequality was most pronounced when absolute reinforcement frequencies were lowest.     

Behavior-dependent reinforcer-rate changes in concurrent schedules: A further analysis

Six pigeons were trained on concurrent variable-interval schedules in three different procedures. The first procedure was a standard concurrent schedule, and the relative reinforcer frequency for responding was varied. The second was a schedule in which a relative left-key response rate (over a fixed period of time) exceeding .75 produced, in the next identical time period a higher reinforcer rate on the right key. If this criterion was not exceeded, equal reinforcer rates were arranged on the two keys in this period. This was the dependent procedure. In the third (independent) procedure, the periods of higher right-key reinforcer rates occurred with the same probability as in the second procedure, but occurred independently of behavior. In the second and third procedures, the fixed-time period (window) was varied from 5 s to 60 s, and to 240 s in the second procedure only. Performance on the two keys was similar in the concurrent and independent procedures. The procedure used in the dependent conditions generally affected performance when the windows were shorter than about 30 s. Models of performance that assume that subjects do not discriminate changes in local relative reinforcer rates cannot account for the data. Moreover, existing models are inherently unable to account for the effects of contingencies of reinforcement between responding on one alternative and gaining reinforcers on another that are arranged or that emerge as a result of time allocated to alternative schedules. Undermatching on concurrent variable-interval schedules may result from such emergent contingencies.     

DOES CONTINGENT REINFORCEMENT STRENGTHEN OPERANT BEHAVIOR?

In Experiment 1, pigeons were trained to peck keys with equal food-reinforcement schedules in components that ended with either noncontingent or contingent transitions to a third component with a five-fold richer schedule. Response rates were higher in the initial component with contingent transitions, but resistance to prefeeding or extinction was not consistently greater. Experiment 2 also included noncontingent or contingent transitions to a signaled period of nonreinforcement. There was no effect of the contingency on transitions to nonreinforcement, but the difference in response rates maintained by contingent versus noncontingent transitions to the richer schedule was replicated. In addition, response rates were higher in components that preceded nonreinforcement than in components that preceded the richer schedule. However, resistance to extinction was greater for noncontingent transitions to the richer schedule than to nonreinforcement, implicating stimulus–reinforcer relations in the determination of resistance to change. Resistance to change was also somewhat greater for noncontingent than for contingent transitions to the richer schedule. The latter result, together with the results of Experiment 1 and related research, suggests that response-contingent reinforcement does not increase resistance to change.     

Competition between stimulus-reinforcer contingencies and anticipatory contrast.

Procedures used to study anticipatory contrast are conceptually similar to those used to study autoshaping, in that two target stimuli signal either higher or lower rates of reinforcement in the following components of the schedule. Despite this signal contingency, anticipatory contrast entails response rates that are higher to the target stimulus followed by the lower rate of reinforcement. To determine the relation between such effects and autoshaping, different variations of the procedure were used in which the signal contingency was presented in the absence of reinforcement in the target components themselves and in which the reinforcement schedules in the different following components were signaled by the same stimulus. Autoshaping effects of this signal contingency were demonstrated when no reinforcement was available during the target-component signals themselves. Intermediate patterns of behavior occurred when reinforcement was available during the target-component signals and when their different following schedules were correlated with the same stimulus. Attempts to isolate these signal and contrast effects functionally by using the signal-key procedure were unsuccessful. The results demonstrate that Pavlovian stimulus contingencies are in competition with the dynamics of anticipatory contrast, thus reducing its occurrence under some circumstances.     

Sensitivity to relative reinforcer rate in concurrent schedules: independence from relative and absolute reinforcer duration

Twelve pigeons responded on two keys under concurrent variable-interval (VI) schedules. Over several series of conditions, relative and absolute magnitudes of reinforcement were varied. Within each series, relative rate of reinforcement was varied and sensitivity of behavior ratios to reinforcer-rate ratios was assessed. When responding at both alternatives was maintained by equal-sized small reinforcers, sensitivity to variation in reinforcer-rate ratios was the same as when large reinforcers were used. This result was observed when the overall rate of reinforcement was constant over conditions, and also in another series of concurrent schedules in which one schedule was kept constant at VI ached 120 s. Similarly, reinforcer magnitude did not affect the rate at which response allocation approached asymptote within a condition. When reinforcer magnitudes differred between the two responses and reinforcer-rate ratios were varied, sensitivity of behavior allocation was unaffected although response bias favored the schedule that arranged the larger reinforcers. Analysis of absolute response rates ratio sensitivity to reinforcement occurrred on the two keys showed that this invariance of response despite changes in reinforcement interaction that were observed in absolute response rates on the constant VI 120-s schedule. Response rate on the constant VI 120-s schedule was inversely related to reinforcer rate on the varied key and the strength of this relation depended on the relative magnitude of reinforcers arranged on varied key. Independence of sensitivity to reinforcer-rate ratios from relative and absolute reinforcer magnitude is consistent with the relativity and independence assumtions of the matching law.     

STIMULUS EFFECTS ON LOCAL PREFERENCE: STIMULUS—RESPONSE CONTINGENCIES,STIMULUS—FOOD PAIRING,AND STIMULUS—FOOD CORRELATION

Four pigeons were trained in a procedure in which concurrent‐schedule food ratios changed unpredictably across seven unsignaled components after 10 food deliveries. Additional green‐key stimulus presentations also occurred on the two alternatives, sometimes in the same ratio as the component food ratio, and sometimes in the inverse ratio. In eight experimental conditions, we varied the contingencies surrounding these additional stimuli: In two conditions, stimulus onset and offset were noncontingent; in another two, stimulus onset was noncontingent, and offset was response contingent. In four conditions, both stimulus onset and offset were contingent, and in two of these conditions the stimulus was simultaneously paired with food delivery. Sensitivity to component food ratios was significantly higher when stimulus onset was response contingent compared to when it was noncontingent. Choice changes following food delivery were similar in all eight conditions. Choice changes following stimuli were smaller than those following food, and directionally were completely determined by the food‐ratio:stimulus‐ratio correlation, not by the stimulus contingency nor by whether the stimulus was paired with food or not. These results support the idea that conditional reinforcers may best be viewed as signals for next‐food location rather than as stimuli that have acquired hedonic value, at least when the signals are differential with respect to future conditions.     

Concurrent performances: rate constancies without changeover delays

Pigeons' pecks on two keys were maintained, without changeover delays, by independent variable-interval schedules of food reinforcement. Four regularly cycling 2-min components scheduled reinforcement respectively for both keys, left key only, both keys, and right key only. Initially, reinforcement scheduled for one key alone produced more responding on that key than reinforcement scheduled concurrently for both keys. Continued sessions reduced this difference; response rate on a given key approached constancy, or invariance with respect to the performance on and schedule for the other key. When extinction replaced the reinforcement schedule on either key, responding on that key decreased more during components that scheduled reinforcement for the other key than during those that did not. This demonstration that responses on one key were not supported by reinforcers on the other key suggested that the alternation of concurrent responding and either-key-alone responding prevented concurrent superstitions from developing.     

The control of choice by its consequences

Five pigeons were trained on concurrent variable-interval schedules in which equal rates of reinforcement were always arranged for left- and right-key responses, but different overall rates were signaled by key colors. Sessions began with both keys lit yellow for the instrumental phase. If, after 20 s of this phase, the relative number of responses that had been made to the left key equaled or exceeded .75, both keys changed red for the contingent phase. The contingent phase arranged another concurrent variable-interval schedule for a further 20 s before the instrumental phase was reinstated. However, if preference in the instrumental phase did not exceed .75, the instrumental phase continued for a further 20 s before preference was again compared with the criterion. In Part 1, the reinforcer rate arranged in the instrumental phase was held constant at 4.8 reinforcers per minute, while the reinforcer rate arranged in the contingent phase was varied across conditions from 0 to 19.2 over five steps. In Part 2, reinforcer rates in the contingent phase were kept constant at 36 per minute, while reinforcer rates in the instrumental phase were varied from 0 to 36 over seven steps. Part 3 replicated Part 2 but used reinforcer rates in both phases that were one third of those arranged in Part 2. Measures of choice obtained by summing responses across presentations of the instrumental phase became more extreme toward the left key as the reinforcer rate obtained in the contingent phase was increased (Part 1) and as the reinforcer rate obtained in the instrumental phase was decreased (Parts 2 and 3). Changes in these measures of choice were accompanied by systematic changes in the relative frequency with which the criterion was exceeded. Changes in both these measures were correlated with changes in the relative frequency with which subjects responded exclusively to the left key. These results are discussed with respect to the two choices that were concurrently available in this procedure and the response alternatives that might constitute the concurrent operants in each choice.     

Concurrent second-order schedules of reinforcement

Responses on one key (the main key) of a two-key chamber produced food according to a second-order variable-interval schedule with fixed-interval schedule components. A response on a second key (the changeover key) alternated colors on the main key and provided a second independent second-order variable-interval schedule with fixed-interval components. The fixed-interval component on one variable-interval schedule was held constant at 8 sec, while the fixed interval on the other variable-interval schedule was varied from 0 to 32 sec. Under some conditions, a brief stimulus terminated each fixed interval and generated fixed-interval patterns; in other conditions, the brief stimulus was omitted. Relative response rate and relative time deviated substantially from scheduled relative reinforcement rate and, to a lesser extent, from obtained relative reinforcement rate under both brief-stimulus and no-stimulus conditions. Matching was observed with equal components on both schedules; with unequal components, increasingly greater proportions of time and responses than the matching relation would predict were spent on the variable-interval schedule containing the shorter component. Preference for the shorter fixed interval was typically more extreme under brief-stimulus than under no-stimulus schedules. The results limit the extension of the matching relation typically observed under simple concurrent variable-interval schedules to concurrent second-order variable-interval schedules.     

Reinforcement of human observing behavior by a stimulue correlated with extinction or increased effort

Young men pulled a plunger on mixed and multiple schedules in which periods of variable-interval monetary reinforcement alternated irregularly with periods of extinction (Experiment 1), or in which reinforcement was contingent on different degrees of effort in the two alternating components (Experiment 2). In the baseline conditions, the pair of stimuli correlated with the schedule components could be obtained intermittently by pressing either of two observing keys. In the main conditions, pressing one of the keys continued to produce both discriminative stimuli as appropriate. Pressing the other key produced only the stimulus correlated with variable-interval reinforcement or reduced effort; presses on this key were ineffective during periods of extinction or increased effort. In both experiments, key presses producing both stimuli occurred at higher rates than key presses producing only one, demonstrating enhancement of observing behavior by a stimulus correlated with the less favorable of two contingencies. A control experiment showed that stimulus change alone was not an important factor in the maintenance of the behavior. These findings suggest that negative as well as positive stimuli may play a role in the conditioned reinforcement of human behavior.     

Drug discrimination under concurrent variable-ratio variable-ratio schedules

Pigeons were trained to discriminate 5 mg/kg pentobarbital from saline under concurrent variable-ratio (VR) VR schedules, in which responses on the pentobarbital-biased lever were reinforced under the VR schedule with the smaller response requirements when pentobarbital was given before the session, and responses on the saline-biased key were reinforced under the VR schedule with the larger response requirements. When saline was administered before the session, the reinforcement contingencies associated with the two response keys were reversed. When responding stabilized under concurrent VR 20 VR 30, concurrent VR 10 VR 40, or concurrent VR 5 VR 50 schedules, pigeons responded almost exclusively on the key on which fewer responses were required to produce the reinforcer. When other doses of pentobarbital and other drugs were substituted for the training dose, low doses of all drugs produced responding on the saline-biased key. Higher doses of pentobarbital and chlordiazepoxide produced responding only on the pentobarbital-biased key, whereas higher doses of ethanol and phencyclidine produced responding only on this key less often. d-Amphetamine produced responding primarily on the saline-biased key. When drugs generalized to pentobarbital, the shape of the generalization curve under concurrent VR VR schedules was more often graded than quantal in shape. Thus, drug discrimination can be established under concurrent VR VR schedules, but the shapes of drug-discrimination dose-response curves under concurrent VR VR schedules more closely resemble those seen under interval schedules than those seen under fixed-ratio schedules. Graded dose-response curves under concurrent VR VR schedules may relate to probability matching and difficulty in discriminating differences in reinforcement frequency.     

Matching and maximizing with concurrent ratio-interval schedules.

Animals exposed to standard concurrent variable-ratio variable-interval schedules could maximize overall reinforcement rate if, in responding, they showed a strong response bias toward the variable-ratio schedule. Tests with the standard schedules have failed to find such a bias and have been widely cited as evidence against maximization as an explanation of animal choice behavior. However, those experiments were confounded in that the value of leisure (behavior other than the instrumental response) partially offsets the value of reinforcement. The present experiment provides another such test using a concurrent procedure in which the confounding effects of leisure were mostly eliminated while the critical aspects of the concurrent variable-ratio variable-interval contingency were maintained: Responding in one component advanced only its ratio schedule while responding in the other component advanced both ratio schedules. The bias toward the latter component predicted by maximization theory was found.     

Response rate and changeover performance on concurrent variable-interval schedules

Six pigeons were exposed to variable-interval schedules arranged on one, two, three, and four response keys. The reinforcement rate was also varied across conditions. Numbers of responses, the time spent responding, the number of reinforcements, and the number of changeovers between keys were recorded. Response rates on each key were an increasing function of reinforcement rate on that key and a decreasing function of the reinforcement rate on other keys. Response and time-allocation ratios under-matched ratios of obtained reinforcements. Three sets of equations were developed to express changeover rate as a function of response rate, time allocation, and reinforcement rate respectively. These functions were then applied to a broad range of experiments in the literature in order to test their generality. Further expressions were developed to account for changeover rates reported in experiments where changeover delays were varied.     

Human Sensitivity To Concurrent Schedules Of Reinforcement: Effects Of Observing Schedule-correlated Stimuli

The determinants of human sensitivity to concurrent variable-interval variable-interval schedules of reinforcement have been difficult to identify, in part because of procedural differences separating published experiments. This experiment investigated vigilance to stimuli correlated with concurrent schedules. Across phases, 3 college students were provided with either no schedule-correlated stimuli, an observing response that provided brief access to the stimuli, or a contingency that required the subject to identify the stimulus correlated with the source of each obtained reinforcer. Sensitivity, as quantified by the generalized matching equation, was low when no stimuli were available. When the stimuli were response contingent, 1 subject observed them, and her behavior became more sensitive to the distribution of reinforcers across the concurrent schedules. When the procedure required discrimination of the stimulus correlated with each reinforcer, the other 2 subjects also observed the stimuli, and their schedule sensitivity was increased as well. These results implicate procedural differences, rather than inherent behavioral differences, as the source of differences in sensitivity to schedules of reinforcement between humans and nonhumans.     

Effects of response rate, reinforcement frequency, and the duration of a stimulus preceding response-independent food

Food-reinforced key pecking in the pigeon was maintained under a four-component multiple schedule. In two components, responding was maintained at high rates under a random-ratio schedule. In the other two components, responding was maintained at low rates under a schedule that specified a minimum interresponse time. For both high and low response rates, one of the schedule components was associated with a high reinforcement frequency and the other components with a lower reinforcement frequency. During performance under these schedules, a stimulus terminated by access to response-independent food was periodically presented. The duration of this pre-food stimulus was 5, 30, 60, or 120 sec. Changes in rate of key pecking during the pre-food stimulus were systematically related to baseline response rate and the duration of the stimulus. Both high and low response rates were increased during the 5-sec stimulus. At longer stimulus durations, low response rates were unaffected and high response rates were decreased during the stimulus. For two of three pigeons, high response rates maintained under a lower frequency of reinforcement tended to be decreased more than high response rates maintained under a higher reinforcement frequency. In general, the magnitude of decrease in high response rates was inversely related to the duration of the pre-food stimulus.     

Resistance To Change As A Function Of Stimulus-reinforcer And Location-reinforcer Contingencies

Pigeons responded on two keys in each component of a multiple concurrent schedule. In one series of conditions the distribution of reinforcers between keys within one component was varied so as to produce changes in ratios of reinforcer totals for key locations when summed across components. In a second series, reinforcer allocation between components was varied so as to produce changes in ratios of reinforcer totals for components, summed across key locations. In each condition, resistance to change was assessed by presenting response-independent reinforcers during intercomponent blackouts and (for the first series) by extinction of responding on both keys in both components. Resistance to change for response totals within a component was always greater for the component with the larger total reinforcer rate. However, resistance to change for response totals at a key location was not a positive function of total reinforcement for pecking that key; indeed, relative resistance to extinction for the two locations showed a weak negative relation to ratios of reinforcer totals for key location. These results confirm the determination of resistance to change by stimulus—reinforcer contingencies.     

Behavior simultaneously maintained by both presentation and termination of noxious stimuli

Lever pressing by two squirrel monkeys was maintained under a 3-minute variable-interval schedule of response-produced electric-shock presentation. At the same time, responding on a second lever was maintained under a 3-minute fixed-interval schedule of termination of the shock-presentation schedule and shock-correlated stimuli. Under the termination schedule, the first response after a 3-minute period produced a 1-minute timeout, during which no events occurred and responding had no scheduled consequence. Relatively high and constant rates of responding were maintained on the lever where responding produced shock. Lower rates and positively accelerated patterns of responding occurred on the lever where responding terminated the shock schedule. Thus, responding was simultaneously maintained by presentation of an event and by termination of a stimulus associated with that event. Rates and patterns of responding on each lever were reversed when the schedules arranged on each lever were reversed on two occasions. When shock intensity was increased from 0 to 10 mA, responding maintained both by presentation of shock and by termination of the shock schedule increased, but responding maintained by shock presentation increased to a greater extent. Positive and negative reinforcement, usually regarded as separate behavioral processes involving different events, can coexist when behavior is controlled by different contingencies involving the same event.     

Preference for mixed-interval versus fixed-interval schedules

Pigeons were trained on a two-link concurrent chain schedule in which responses on two keys were reinforced according to independent variable-interval schedules by the production of a change in key color. Further responses on the key on which the stimulus change had been produced gave a single food reinforcement and a return to concurrent variable-interval conditions. On one key the terminal link was a two-valued mixed-interval schedule, while on the other, the terminal link was a fixed-interval schedule. When the mixed-interval values were kept constant and the fixed-interval values varied, relative response rates in the initial concurrent links matched relative reinforcement rates in the terminal links when these were computed from cubic transformations of the reciprocals of the intervals comprising the terminal link schedules.