Contrast and reallocation of extraneous reinforcers between multiple-schedule components

Four pigeons responded in components of multiple schedules in which two responses were available and reinforced with food. Pecks on the left key (“main” key) were reinforced at a constant rate in one component and at a rate that varied over conditions in the other component. When reinforcer rate was varied, behavioral contrast occurred in the constant component. On the right key (“extra” key), five variable-interval schedules and one variable-ratio schedule, presented conjointly, arranged reinforcers for responses in all conditions. These conjoint schedules were common to both multiple-schedule components—rather than unique to particular components—and reinforcers from these schedules could therefore be arranged in one component and obtained during the other component. In this way, the additional reinforcers were analogous to the “extraneous” reinforcers thought to maintain behavior other than pecking in conventional multiple schedules. Response rate on the extra key did not change systematically over conditions in the constant component, and in the varied component extra responding was inversely related to main-key reinforcement. All subjects obtained more extra-key reinforcers in whichever component arranged fewer main-key reinforcers. Consistent with the theory that reallocation of extraneous reinforcers may cause behavioral contrast, absolute reinforcer rate for the extra key in the constant component was low in conditions that produced positive contrast on the main key and high in those that produced negative contrast. Also consistent with this theory, behavioral contrast was reduced in two conditions that canceled extra-key reinforcers that had been arranged but not obtained at the end of components. Thus, a constraint on reallocation markedly reduced the extent of contrast.     

Positive behavioral contrast when pigeons press treadles during multiple schedules

Pigeons were placed on multiple variable-interval 15-second variable-interval 15-second and on multiple variable-interval 15-second extinction schedules in which treadle presses produced food reinforcers. Positive behavioral contrast occurred. That is, rates of responding were higher during the variable-interval 15-second component when the other component was extinction than when it was another variable-interval 15-second schedule. These results contradict the findings of other studies, which failed to find positive contrast when pigeons pressed treadles for food reinforcers. They may also question the additive theories of behavioral contrast that predict that contrast should not occur in this situation.     

Negative behavioral contrast on multiple treadle-press schedules

Eight pigeons pressed treadles for food reinforcers delivered by several multiple variable-interval schedules. The rate of reinforcement for responding during one component schedule was held constant at 30 reinforcers per hour. The rate of reinforcement for responding during the other component varied from 0 to 120 or 240 reinforcers per hour. The schedules were presented in different orders for different subjects. The rate of responding emitted during the variable component schedule varied directly with the rate of reinforcement it provided. The rate of responding during the constant component did not increase consistently when the rate of reinforcement obtained from the variable component decreased from 30 to 0 reinforcers per hr. The rate of responding emitted during the constant component decreased when the rate of reinforcement obtained from the variable component increased from 30 reinforcers per hour to a higher rate. That is, negative but not positive behavioral contrast occurred. The failure to find positive contrast is consistent with one of the predictions of the additive theories of behavioral contrast. Finding negative contrast has ambiguous implications for the additive theories.     

Behavioral contrast in rats with different reinforcers and different response topographies

Experiment I demonstrated positive behavioral contrast in rats when one of two qualitatively different reinforcers (milk and pellets) was removed from a component of a multiple schedule. The contrast effect was larger and more enduring when milk was removed. Experiment II showed that the rats spent more time on the side of a shuttle-box on which milk was freely available than on the side on which pellets were freely available. Experiment III, a partial replication of Experiment I, failed to demonstrate the contrast effect of Experiment I. Experiment IV demonstrated contrast when two topographically distinct responses, nose-key poking and lever pressing, were required in different components of a multiple schedule. These results extend the conditions that generate behavioral contrast in rats.     

Contrast and undermatching as a function of reinforcer duration and quality during multiple schedules

Eight pigeons pecked keys under multiple variable-interval two-minute variable-interval two-minute schedules. In Experiment 1, the reinforcers were 2, 4, or 8 seconds access to a food magazine. In Experiments 2 and 3, the reinforcers were grains that had been determined to be most-, moderately-, or non-preferred. Both positive and negative behavioral contrast occurred when the reinforcers in one component were held constant and the duration or type of reinforcer obtained in the other component varied. Undermatching occurred when the relative rate of responding during a component was plotted as a function of the relative duration of the reinforcers in that component.     

Positive and negative contrast as a function of component duration for key pecking and treadle pressing

Pigeons responded on several multiple schedules for food reinforcers. The duration of the components varied from four seconds to 16 minutes. The absolute size of positive (Experiment 1) and negative (Experiment 2) behavioral contrast varied inversely with component duration when key pecks produced the reinforcers. The absolute size of negative contrast varied directly with component duration, when treadle presses produced the reinforcers (Experiment 3). These results conform to theories that suggest that positive and negative contrast are symmetrical when pigeons peck keys. They also conform to theories that suggest that the same principles do not govern contrast when pigeons peck keys as when they press treadles. Finally, the results support the measurement of behavioral contrast by the differences between baseline rates of responding and the rates emitted when contrast is present.     

The economics of daily consumption controlling food- and water-reinforced responding

In the first experiment, two rhesus monkeys earned their entire ration of food and water during daily sessions with no provisions to ensure constant daily intakes. Two variable-interval schedules of food presentations were concurrent with one variable-interval schedule of water presentations; the maximum rate of food presentations arranged by one food schedule was varied. As the rate of food presentations was increased, the absolute level of responding on the two food schedules combined decreased, while responding on the water schedule increased. The preference for the variable food schedule compared to the other food schedule approximately matched the proportion of reinforcers obtained from it. The preference for the variable food schedule compared to the water schedule did not match, but greatly decreased, as the proportion of reinforcers from the food schedule increased. When Experiment I was replicated, with provisions to ensure constant daily intakes of food and water (Experiment II), the absolute response rates under the two food schedules combined and under the water schedule no longer changed with increases in the rate of food during the sessions. On the other hand, choice between the two food schedules remained proportional to the distribution of obtained food pellets. These results were interpreted as indicating that behavior to obtain nonsubstitutable commodities, such as food and water, is strongly controlled by the economic conditions of daily consumption, while choice between substitutable commodities is independent of these factors.     

Signalled reinforcement and multiple schedules

The responses of four pigeons were first reinforced in the presence of two different wave-lengths (green and red) on a two-ply multiple schedule with identical variable-interval 3-min schedules of reinforcement associated with each component. While the constant-component reinforcement schedule remained unchanged during the experiment, the schedule associated with the variable component was changed to (1) signalled variable time, (2) unsignalled variable time, or (3) signalled variable interval. The probability with which the availability of the reinforcer was signalled in the variable-interval schedules was either 0.5 or 1.0. Positive contrast occurred in both signalled variable-interval and variable-time schedules, but only when the availability of all the variable-component reinforcers was signalled. Signalling the availability of only 50% of the reinforcers in signalled variable-interval schedules resulted in negative induction. The present data suggest that positive behavioral contrast resulting from signalled reinforcer availability is due to the presence of an extinction-correlated stimulus.     

Within-session changes in key and lever pressing for water during several multiple variable-interval schedules

Rats pressed keys or levers for water reinforcers delivered by several multiple variable-interval schedules. The programmed rate of reinforcement varied from 15 to 240 reinforcers per hour in different conditions. Responding usually increased and then decreased within experimental sessions. As for food reinforcers, the within-session changes in both lever and key pressing were smaller, peaked later, and were more symmetrical around the middle of the session for lower than for higher rates of reinforcement. When schedules provided high rates of reinforcement, some quantitative differences appeared in the within-session changes for lever and key pressing and for food and water. These results imply that basically similar factors produce within-session changes in responding for lever and key pressing and for food and water. The nature of the reinforcer and the choice of response can also influence the quantitative properties of within-session changes at high rates of reinforcement. Finally, the results show that the application of Herrnstein's (1970) equation to rates of responding averaged over the session requires careful consideration.     

Resistance to reinforcement change in multiple and concurrent schedules assessed in transition and at steady state

Behavioral momentum theory relates resistance to change of responding in a multiple-schedule component to the total reinforcement obtained in that component, regardless of how the reinforcers are produced. Four pigeons responded in a series of multiple-schedule conditions in which a variable-interval 40-s schedule arranged reinforcers for pecking in one component and a variable-interval 360-s schedule arranged them in the other. In addition, responses on a second key were reinforced according to variable-interval schedules that were equal in the two components. In different parts of the experiment, responding was disrupted by changing the rate of reinforcement on the second key or by delivering response-independent food during a blackout separating the two components. Consistent with momentum theory, responding on the first key in Part 1 changed more in the component with the lower reinforcement total when it was disrupted by changes in the rate of reinforcement on the second key. However, responding on the second key changed more in the component with the higher reinforcement total. In Parts 2 and 3, responding was disrupted with free food presented during intercomponent blackouts, with extinction (Part 2) or variable-interval 80-s reinforcement (Part 3) arranged on the second key. Here, resistance to change was greater for the component with greater overall reinforcement. Failures of momentum theory to predict short-term differences in resistance to change occurred with disruptors that caused greater change between steady states for the richer component. Consistency of effects across disruptors may yet be found if short-term effects of disruptors are assessed relative to the extent of change observed after prolonged exposure.     

Ratio requirement and reinforcer effects in concurrent fixed-interval fixed-ratio schedules

The fixed-ratio requirement was varied in concurrent fixed-interval fixed-ratio schedules. Fixed-interval responding was reinforced by food. In different phases, fixed-ratio responding was reinforced by food or water. There was a direct relation between the ratio requirement and interval response rates when both responses were reinforced with food, but essentially no relation when the reinforcers were different. The role of reinforcers in concurrent schedules merits detailed study.     

Successive discrimination training with equated reinforcement frequencies: failure to obtain behavioral contrast

In two experiments, pigeons were trained on two-component multiple schedules in which responding in one component (S₁) was always maintained by a variable-interval schedule. In Experiment I, low response rates were reinforced in the second (S₂) component for six master subjects. This schedule was adjusted to equate reinforcement frequencies in the two components. These subjects were compared to yoked partners, for which reinforcement in the S₂ component was made available on a variable-interval schedule whose value was determined by the master subjects. A similar procedure was used in Experiment II, where the S₂ schedule for master subjects made reinforcers contingent on the absence of responding. No evidence was found in either experiment for a behavioral contrast effect in the S₁ component attributable to response reduction in the S₂ component. A reliable contrast effect was obtained from a group of pigeons given extinction conditions in the S₂ component, which was compared to a group maintained throughout on a multiple variable-interval schedule. The results suggest that previous indications of behavioral contrast in similar situations were probably caused by uneven reinforcement distributions or reflect uncontrolled fluctuations in response rates.     

Control of responding during stimuli that precede transitions in reinforcement frequency

Pigeons' responses were reinforced on a variant of a mixed variable-interval extinction schedule of reinforcement in which the transition to the higher reinforcement rate was signaled by a trace stimulus projected on the response key prior to the onset of the component correlated with food delivery. In the first of two experiments, the duration of the trace stimulus preceding the component correlated with food delivery was varied from 1.5 to 50.0 s and in the second experiment, the reinforcement frequency in the same component was varied from 10 to 60 reinforcers per hour. Pigeons pecked at the trace stimulus preceding the onset of the component correlated with food delivery even though responding was not reinforced in its presence and only one of the changes in reinforcement rate (i.e., from extinction to reinforcement) was signaled. The rate of pecking during the trace stimulus was a function of its duration but not of the reinforcement frequency in the following component. Higher rates generally occurred at the shorter trace-stimulus durations. Component responding following the offset of the trace stimulus was under discriminative control of the trace stimulus whether or not responding occurred in the presence of the trace stimulus.     

Food-paired stimuli as conditioned reinforcers: effects of d-amphetamine.

Seven pigeons were studied in two experiments in which key pecks were reinforced under a second-order schedule wherein satisfaction of variable-interval schedule requirements produced food or a brief stimulus. In the second part of each session, responses produced only the brief stimulus according to a variable-interval schedule (food extinction). For the 4 pigeons in Experiment 1, the response key was red throughout the session. In separate phases, the brief stimulus was either paired with food, not paired with food, or not presented during extinction. d-Amphetamine (0.3 to 10.0 mg/kg) dose-dependently reduced food-maintained responding during the first part of the session and, at intermediate dosages, increased responding during the extinction portion of the session. The magnitude of these increases, however, did not consistently depend on whether the brief stimulus was paired, not paired, or not presented. It was also true that under nondrug conditions, response rates during extinction did not differ reliably depending on pairing operations for the brief stimulus. In Experiment 2, 3 different pigeons responded under a procedure wherein the key was red in the component with food presentations and blue in the extinction component (i.e., multiple schedule). Again, d-amphetamine produced dose-related decreases in responding during the first part of a session and increases in responding in the second part of the session. These increases, however, were related to the pairing operations; larger increases were observed when the brief stimulus was paired with food than when it was not or when it was not presented at all. Under nondrug conditions, the paired brief stimulus controlled higher response rates during extinction than did a nonpaired stimulus or no stimulus. These findings suggest that d-amphetamine can enhance the efficacy of conditioned reinforcers, and that this effect may be more robust if conditioned reinforcers occur in the context of a signaled period of extinction.     

Spaced food but not electrical brain stimulation induces polydipsia and air licking.

An attempt was made to induce polydipsia in rats whose lever pressing was reinforced with food pellets or electrical brain stimulation. Nine food-deprived, water-sated rats drank water excessively during sessions in which food pellets were delivered. When brain stimulation was substituted for food, drinking immediately ceased. Delivering brain stimulation according to a variety of schedules, pairing brain stimulation with food reinforcement, and substituting an air stream for water, each failed to produce polydipsic licking. These results show that polydipsia is not induced by all reinforcers.     

Alternative reinforcement effects on fixed-interval performance

Pigeons' key pecks were reinforced with food on a fixed-interval schedule. Food also was available at variable time periods either independently of responding or for not key pecking (a differential-reinforcement-of-other-behavior schedule). The latter condition arranged reinforcement following the first pause of t seconds after it became available according to a variable-time schedule. This schedule allowed separation of the effects of pause requirements ≤ five-seconds and reinforcement frequency. The time spent pausing increased as the duration of the pause required for reinforcement increased from 0 to 30 seconds and as the frequency of reinforcement for pausing increased from 0 to 2 reinforcers per minute. Key pecking was more evenly distributed within each fixed interval with shorter required pauses and with more frequent reinforcement for pausing. The results complement those obtained with other concurrent schedules in which the same operant response was reinforced in both components.     

Assessing reinforcers under progressive schedule requirements.

Recent research findings suggest that reinforcing stimuli may be differentially effective as response requirements increase. We extended this line of research by evaluating responding under increasing schedule requirements via progressive‐ratio schedules and behavioral economic analyses. The differential effectiveness of preferred stimuli in treating destructive behavior maintained by automatic reinforcement also was examined. Results showed that one of two stimuli was associated with more responding under increasing schedule requirements for the 4 participants. Furthermore, stimuli associated with more responding under increasing schedule requirements generally were more effective in treating destructive behavior than stimuli associated with less responding. These data suggest that progressive‐ratio schedules and behavioral economic analyses may be useful for developing a new technology for reinforcer identification. From a clinical perspective, these results suggest that two reinforcers may be similarly effective for low‐effort tasks and differentially effective for high‐effort tasks.     

Preference for reinforcers under varying schedule arrangements: A behavioral economic analysis

The field of behavioral economics combines concepts from economics and operant conditioning to examine the influence of schedules or price on preference for reinforcers. Three case studies are reported in which behavioral economic analyses were used to assess relative preference for reinforcers shown by people with intellectual disabilities when schedule requirements varied. The studies examined (a) preference for different reinforcers, (b) substitutability of reinforcers, and (c) changes in preference as a function of schedule requirements.     

Stimulus-reinforcer contingencies and local behavioral contrast

Four pigeons were exposed to a series of multiple schedules of variable-interval reinforcement in which pecks were required on one key (operant key) and components were signalled on a second key (signal key). Four additional pigeons experienced identical conditions, except that a yoking procedure delivered food on variable-time schedules, with no key pecks required. One of the components of the multiple schedule was constant throughout the experiment as a variable-interval (or variable-time) 30-second schedule. Operant-key responding during the constant component was uniform throughout the component, uninfluenced by changes in the duration of the variable component, and only slightly influenced by changes in reinforcement frequency correlated with the variable component. By comparison, signal-key response rate during the constant component was highest at the onset of the component, was higher when the variable component was 60-sec long than when it was 1-sec long, and was higher when no reinforcement occurred in the variable component than when reinforcement was scheduled in the variable component. These characteristics of signal-key pecking matched characteristics of local positive behavioral contrast. These data are taken to support the “additivity theory” of behavioral contrast and to suggest that Pavlovian stimulus-reinforcer relations contribute primarily to the phenomenon of local positive contrast.