Local contrast in behavior allocation during multiple-schedule components

Allocation of responses between two keys was studied during two alternating multiple-schedule components. Responses were recorded in successive quarters of each component. Variable-interval reinforcer schedules on the two keys were constant throughout the experiment for one (constant) component and were varied over conditions on one key for the other, producing changes in reinforcer ratios for the varied component. Behavior allocation for the first quarter of the constant component was inversely related to varied-component reinforcer ratios, a form of local contrast, but this relationship was not observed later in the component. During the first quarter of the varied component, slopes of matching lines were high and decreased later in the component. It is argued that this form of local contrast cannot be explained in terms of reallocation of extraneous reinforcers between components, and that the matching law for concurrent operants does not capture some sources of control over behavior allocation. A simple extension of the matching law is offered that adequately describes behavior changes during both components. A version of this formulation can predict contrast effects in absolute response rates.     

Dynamic changes in reinforcer effectiveness: satiation and habituation have different implications for theory and practice

Reinforcers lose their effectiveness when they are presented repeatedly. Early researchers labeled this loss of effectiveness as satiation without conducting an experimental analysis. When such an analysis is conducted, habituation provides a more precise and empirically accurate label for the changes in reinforcer effectiveness. This paper reviews some of the data that suggest that habituation occurs to repeatedly presented reinforcers. It also argues that habituation has surprisingly different implications than satiation for theory and practice in behavior analysis. For example, postulating that habituation occurs to repeatedly presented reinforcers suggests ways for maintaining the strength of an existing reinforcer and for weakening the strength of a problematic reinforcer that differ from those implied by an account in terms of satiation. An habituation account may also lead to different ways of conceptualizing the regulation of behavior. For example, habituation may be a single-process contributor to the termination of behaviors that are usually attributed to satiation (e.g., ingestive behaviors such as eating and drinking), fatigue (e.g., energetic behaviors such as running), the waning of attention (e.g., cognitive behaviors such as studying), and pharmacodynamic factors (e.g., drug taking).     

Contrast and reallocation of extraneous reinforcers between multiple-schedule components

Four pigeons responded in components of multiple schedules in which two responses were available and reinforced with food. Pecks on the left key (“main” key) were reinforced at a constant rate in one component and at a rate that varied over conditions in the other component. When reinforcer rate was varied, behavioral contrast occurred in the constant component. On the right key (“extra” key), five variable-interval schedules and one variable-ratio schedule, presented conjointly, arranged reinforcers for responses in all conditions. These conjoint schedules were common to both multiple-schedule components—rather than unique to particular components—and reinforcers from these schedules could therefore be arranged in one component and obtained during the other component. In this way, the additional reinforcers were analogous to the “extraneous” reinforcers thought to maintain behavior other than pecking in conventional multiple schedules. Response rate on the extra key did not change systematically over conditions in the constant component, and in the varied component extra responding was inversely related to main-key reinforcement. All subjects obtained more extra-key reinforcers in whichever component arranged fewer main-key reinforcers. Consistent with the theory that reallocation of extraneous reinforcers may cause behavioral contrast, absolute reinforcer rate for the extra key in the constant component was low in conditions that produced positive contrast on the main key and high in those that produced negative contrast. Also consistent with this theory, behavioral contrast was reduced in two conditions that canceled extra-key reinforcers that had been arranged but not obtained at the end of components. Thus, a constraint on reallocation markedly reduced the extent of contrast.     

Dishabituation produces interactions during multiple schedules

McSweeney and Weatherly (1998) argued that differential habituation to the reinforcer contributes to the behavioral interactions observed during multiple schedules. The present experiment confirmed that introducing dishabituators into one component of a multiple schedule increases response rate in the other, constant, component. During baseline, pigeons and rats responded on multiple variable interval 30-s variable interval 30-s schedules. During experimental conditions, subjects responded on the same schedule except that a dishabituating stimulus (manipulation of a light) was also presented randomly during one of the components. Constant-component responding was faster during the experimental than during the baseline conditions. This difference in responding grew larger across the session. The within-session pattern of responding was similar for the two components of each multiple schedule. Qualitatively similar results were observed for rats and pigeons. These results suggest that behavioral interactions sometimes arise from a change in reinforcer effectiveness between the baseline and experimental phases of the experiment, rather than from an assessment of reinforcer relativity (a comparison of reinforcers delivered during the two components in the experimental phase). Behavioral contrast and induction are sometimes produced by similar factors.     

Contrast and undermatching as a function of reinforcer duration and quality during multiple schedules

Eight pigeons pecked keys under multiple variable-interval two-minute variable-interval two-minute schedules. In Experiment 1, the reinforcers were 2, 4, or 8 seconds access to a food magazine. In Experiments 2 and 3, the reinforcers were grains that had been determined to be most-, moderately-, or non-preferred. Both positive and negative behavioral contrast occurred when the reinforcers in one component were held constant and the duration or type of reinforcer obtained in the other component varied. Undermatching occurred when the relative rate of responding during a component was plotted as a function of the relative duration of the reinforcers in that component.     

Resistance To Change As A Function Of Stimulus-reinforcer And Location-reinforcer Contingencies

Pigeons responded on two keys in each component of a multiple concurrent schedule. In one series of conditions the distribution of reinforcers between keys within one component was varied so as to produce changes in ratios of reinforcer totals for key locations when summed across components. In a second series, reinforcer allocation between components was varied so as to produce changes in ratios of reinforcer totals for components, summed across key locations. In each condition, resistance to change was assessed by presenting response-independent reinforcers during intercomponent blackouts and (for the first series) by extinction of responding on both keys in both components. Resistance to change for response totals within a component was always greater for the component with the larger total reinforcer rate. However, resistance to change for response totals at a key location was not a positive function of total reinforcement for pecking that key; indeed, relative resistance to extinction for the two locations showed a weak negative relation to ratios of reinforcer totals for key location. These results confirm the determination of resistance to change by stimulus—reinforcer contingencies.     

Signalled reinforcement and multiple schedules

The responses of four pigeons were first reinforced in the presence of two different wave-lengths (green and red) on a two-ply multiple schedule with identical variable-interval 3-min schedules of reinforcement associated with each component. While the constant-component reinforcement schedule remained unchanged during the experiment, the schedule associated with the variable component was changed to (1) signalled variable time, (2) unsignalled variable time, or (3) signalled variable interval. The probability with which the availability of the reinforcer was signalled in the variable-interval schedules was either 0.5 or 1.0. Positive contrast occurred in both signalled variable-interval and variable-time schedules, but only when the availability of all the variable-component reinforcers was signalled. Signalling the availability of only 50% of the reinforcers in signalled variable-interval schedules resulted in negative induction. The present data suggest that positive behavioral contrast resulting from signalled reinforcer availability is due to the presence of an extinction-correlated stimulus.     

Altering Reinforcer Variety or Intensity Changes the Within-Session Decrease in Responding

Operant responding often changes systematically within experimental sessions. McSweeney, Hinson, and Cannon (1996) argued that sensitization and habituation produce within-session changes in responding. The present study tested two predictions of the sensitization–habituation explanation. In two experiments, rats pressed a lever for reinforcers delivered by a multiple variable interval 15-s variable interval 15-s schedule. In Experiment 1, the variety of reinforcers delivered during the session was manipulated by varying the percentage of programmed reinforcers replaced with qualitatively different reinforcers from 0 to 75%, in five different conditions. In Experiment 2, the intensity of the reinforcer was manipulated by varying the concentration of sucrose in the sucrose and water solution used as the reinforcer from 0 to 30%, in five different conditions. Increasing the variety or the intensity of the reinforcers slowed the within-session decrease in responding. The results are consistent with the predictions of a sensitization–habituation explanation of within-session changes in responding.     

Sensitization and habituation regulate reinforcer effectiveness

We argue that sensitization and habituation occur to the sensory properties of reinforcers when those reinforcers are presented repeatedly or for a prolonged time. Sensitization increases, and habituation decreases, the ability of a reinforcer to control behavior. Supporting this argument, the rate of operant responding changes systematically within experimental sessions even when the programmed rate of reinforcement is held constant across the session. These within-session changes in operant responding are produced by repeated delivery of the reinforcer, and their empirical characteristics correspond to the characteristics of behavior undergoing sensitization and habituation. Two characteristics of habituation (dishabituation, stimulus specificity) are particularly useful in separating habituation from alternative explanations. Arguing that habituation occurs to reinforcers expands the domain of habituation. The argument implies that habituation occurs to biologically important, not just to neutral, stimuli. The argument also implies that habituation may be observed in “voluntary” (operant), not just in reflexive, behavior. Expanding the domain of habituation has important implications for understanding operant and classical conditioning. Habituation may also contribute to the regulation of motivated behaviors. Habituation provides a more accurate and a less cumbersome explanation for motivated behaviors than homeostasis. Habituation also has some surprising, and easily testable, implications for the control of motivated behaviors.     

Contrast and reallocation of extraneous reinforcers as a function of component duration and baseline rate of reinforcement

Four pigeons responded on multiple schedules arranged on a “main” key in a two-key experimental chamber. A constant schedule component was alternated with another component that was varied over conditions. On an extra response key, conjoint schedules of reinforcement that operated in both components were arranged concurrently with the multiple schedule on the main key. On the main key, changes in reinforcement rate in the varied component were inversely related to changes in response rates in the constant component (behavioral contrast). On the extra key, some reinforcers were reallocated between components, depending on the schedules in effect on the main key in the varied component. In the varied component, the obtained rates of reinforcement on the extra key were inversely related to main-key reinforcement rate. In the constant component, extra-key reinforcer rates were positively related to main-key reinforcer rates obtained in the varied component, and were not a function of response rates on the extra key. In two comparisons, the rate at which components alternated and the value of the main-key schedule in the constant component were varied. Consistent with earlier work, long components reduced the extent of contrast. Reductions in contrast as a function of component duration were accompanied by similar reductions in the extent of reinforcer reallocation on the extra key. In the second comparison, lowering the rate of reinforcement in the constant component increased the rate at which extra-key reinforcers were obtained, reduced the extent of reinforcer reallocation, and reduced contrast. Overall, the results are consistent with the suggestion that some contrast effects are due to the changes in extraneous reinforcement during the constant component, and that manipulations of component duration, and manipulations of the rate of reinforcement in the constant component, affect contrast because they influence the extent of extraneous reinforcer real-location.     

Fixed-ratio pausing: Joint effects of past reinforcer magnitude and stimuli correlated with upcoming magnitude

Pigeons responded on fixed-ratio schedules ending in small or large reinforcers (grain presentations of different duration) interspersed within each session. In mixed-schedule conditions, the response key was lit with a single color throughout the session, and pausing was directly related to the past reinforcer (longer pauses after large reinforcers than after small ones). In multiple-schedule conditions, different colors accompanied the ratios ending in small and large reinforcers, and pausing was affected by the upcoming reinforcer as well as the past one. Pauses were shorter before large reinforcers than before small ones, but they continued to be longer after large reinforcers than after small ones. The influence of the past reinforcer was modulated by the magnitude of the upcoming reinforcer; in the presence of the stimulus before the small reinforcer, the effect of the past reinforcer was enhanced relative to its effect in the stimulus before the large reinforcer. These results show that pausing between ratios is jointly determined by two competing factors: past conditions of reinforcement and stimuli correlated with upcoming conditions.     

Choice in a variable environment: every reinforcer counts

Six pigeons were trained in sessions composed of seven components, each arranged with a different concurrent-schedule reinforcer ratio. These components occurred in an irregular order with equal frequency, separated by 10-s blackouts. No signals differentiated the different reinforcer ratios. Conditions lasted 50 sessions, and data were collected from the last 35 sessions. In Part 1, the arranged overall reinforcer rate was 2.22 reinforcers per minute. Over conditions, number of reinforcers per component was varied from 4 to 12. In Part 2, the overall reinforcer rate was six per minute, with both 4 and 12 reinforcers per component. Within components, log response-allocation ratios adjusted rapidly as more reinforcers were delivered in the component, and the slope of the choice relation (sensitivity) leveled off at moderately high levels after only about eight reinforcers. When the carryover from previous components was taken into account, the number of reinforcers in the components appeared to have no systematic effect on the speed at which behavior changed after a component started. Consequently, sensitivity values at each reinforcer delivery were superimposable. However, adjustment to changing reinforcer ratios was faster, and reached greater sensitivity values, when overall reinforcer rate was higher. Within a component, each successive reinforcer from the same alternative ("confirming") had a smaller effect than the one before, but single reinforcers from the other alternative ("disconfirming") always had a large effect. Choice in the prior component carried over into the next component, and its effects could be discerned even after five or six reinforcement and nonreinforcement is suggested.     

Non‐generalized and generalized conditioned reinforcers: establishment and validation

Conditioned reinforcers, or tokens, are frequently used in clinical settings; however, little applied research has focused on establishing and validating their effectiveness. Following pairing of a novel token with an edible reinforcer in the present study, preference and reinforcer assessments verified that the tokens functioned as conditioned reinforcers. Results indicated that the tokens were established as conditioned reinforcers that matched the reinforcer value of the primary reinforcers with which they had been paired. The effects of motivating operations (MOs) on the effectiveness of these conditioned reinforcers were then evaluated. Conditioned reinforcer effectiveness decreased during satiation conditions. Subsequently it was found that increasing the number of backup reinforcers with which the token was paired resulted in the effectiveness maintaining during satiation conditions. Copyright © 2008 John Wiley & Sons, Ltd.     

Negative behavioral contrast on multiple treadle-press schedules

Eight pigeons pressed treadles for food reinforcers delivered by several multiple variable-interval schedules. The rate of reinforcement for responding during one component schedule was held constant at 30 reinforcers per hour. The rate of reinforcement for responding during the other component varied from 0 to 120 or 240 reinforcers per hour. The schedules were presented in different orders for different subjects. The rate of responding emitted during the variable component schedule varied directly with the rate of reinforcement it provided. The rate of responding during the constant component did not increase consistently when the rate of reinforcement obtained from the variable component decreased from 30 to 0 reinforcers per hr. The rate of responding emitted during the constant component decreased when the rate of reinforcement obtained from the variable component increased from 30 reinforcers per hour to a higher rate. That is, negative but not positive behavioral contrast occurred. The failure to find positive contrast is consistent with one of the predictions of the additive theories of behavioral contrast. Finding negative contrast has ambiguous implications for the additive theories.     

Temporal constraint on choice: Sensitivity and bias in multiple schedules

In multiple schedules of reinforcement, ratios of responses in successive components are relatively insensitive to ratios of obtained reinforcers. An analysis is proposed that attributes changes in absolute response rates to concurrent interactions between programmed reinforcement and extraneous reinforcement in other components. The analysis predicts that ratios of responses in successive components vary with reinforcer ratios, qualified by a term describing the reinforcement context, that is, programmed and extraneous reinforcers. Two main predictions from the analysis were confirmed in an experiment in which pigeons' responses were reinforced in the components of a multiple schedule and analog extraneous reinforcement was scheduled for an alternative response in each component. Sensitivity of response and time ratios to reinforcer ratios in the multiple schedules varied as a function of the rate of extraneous reinforcers. Bias towards responding in one component of the multiple schedule varied as an inverse function of the ratios of extraneous reinforcer rate in the two components. The data from this and previous studies of multiple-concurrent performance were accurately predicted by our analysis and supported our contention that the allocation of behavior in multiple-schedule components depends on the relative values of concurrently-available reinforcers within each component.     

Reinforcer-ratio variation and its effects on rate of adaptation

Six pigeons were trained in sessions that consisted of six or seven concurrent-schedule components, each of which could have a different reinforcer ratio arranged in it. The components were unsignaled and occurred in a random order separated by 10-s blackouts. The overall reinforcer rate arranged in each component was 2.22 reinforcers per minute. In Experiment 1, the range of reinforcer ratios in the seven components was varied from a condition in which the ratios were always 1:1, to a condition in which the ratios varied between concurrent variable-interval 27 s extinction (EXT) and concurrent extinction variable-interval 27 s (ratios of 1:EXT, 9:1, 3:1, 1:1, 1:3, 1:9, EXT:1). In Experiment 2, the range of reinforcer ratios was always 27:1 to 1:27, and the presence and absence of the intermediate reinforcer ratios used in Experiment 1 (9:1, 3:1, 1:1, 1:3, 1:9) were investigated. Log response-allocation ratios in components changed rapidly with increasing numbers of reinforcers in components, and Experiment 1 showed that sensitivity to reinforcement was usually higher when the range of reinforcer ratios was greater. When the range of reinforcer ratios was kept constant in Experiment 2, the presence or absence of less extreme reinforcer ratios had no clear effect on sensitivity. At a local level, individual reinforcers had predictable quantitative effects on response ratios: Successive same-alternative reinforcers in a component had rapidly diminishing effects in both experiments. Reinforcers obtained on the opposite alternative to one or more prior reinforcers always had large effects on preference, and these changes were greater when the range of reinforcer ratios was greater. The effects of such reinforcers in changing preference were enhanced, and produced clear preference reversals, when intermediate reinforcer ratios were absent in Experiment 2. Two processes, one local to reinforcers and one with a longer time course, may be necessary to account for these results.     

Every reinforcer counts: reinforcer magnitude and local preference

Six pigeons were trained on concurrent variable-interval schedules. Sessions consisted of seven components, each lasting 10 reinforcers, with the conditions of reinforcement differing between components. The component sequence was randomly selected without replacement. In Experiment 1, the concurrent-schedule reinforcer ratios in components were all equal to 1.0, but across components reinforcer-magnitude ratios varied from 1:7 through 7:1. Three different overall reinforcer rates were arranged across conditions. In Experiment 2, the reinforcer-rate ratios varied across components from 27:1 to 1:27, and the reinforcer-magnitude ratios for each alternative were changed across conditions from 1:7 to 7:1. The results of Experiment 1 replicated the results for changing reinforcer-rate ratios across components reported by Davison and Baum (2000, 2002): Sensitivity to reinforcer-magnitude ratios increased with increasing numbers of reinforcers in components. Sensitivity to magnitude ratio, however, fell short of sensitivity to reinforcer-rate ratio. The degree of carryover from component to component depended on the reinforcer rate. Larger reinforcers produced larger and longer postreinforcer preference pulses than did smaller reinforcers. Similar results were found in Experiment 2, except that sensitivity to reinforcer magnitude was considerably higher and was greater for magnitudes that differed more from one another. Visit durations following reinforcers measured either as number of responses emitted or time spent responding before a changeover were longer following larger than following smaller reinforcers, and were longer following sequences of same reinforcers than following other sequences. The results add to the growing body of research that informs model building at local levels.     

Contrast and undermatching with regular or irregular alternation of components

Behavioral contrast and response-ratio sensitivity to reinforcement were compared in multiple schedules in which components alternated strictly or according to a pseudorandom sequence. Average component durations in the two regimes were always 60 s, and order of presentation of component alternation regimes was counterbalanced across subjects. In Part 1, the reinforcer rate in one component was reduced from 60 per hour to zero, while that in the other component was unchanged. Positive behavioral contrast occurred in the constant component in that response rates increased, but neither the reliability nor the magnitude of contrast was affected by the manner in which components alternated. Part 2 was similar, except that a number of different reinforcer rates were used in the varied component. Neither contrast nor sensitivity of response ratios to changes in reinforcer ratios depended on the regime of component alternation. Thus, the predictability in time of future reinforcement conditions, which is a feature of regular multiple scheduling, does not appear to be a determinant of multiple-schedule performance.     

The momentum of human behavior in a natural setting

Adults with mental retardation in a group home received popcorn or coffee reinforcers for sorting plastic dinnerware. In Part 1 of the experiment, reinforcers were dispensed according to a variable-interval 60-s schedule for sorting dinnerware of one color and according to a variable-interval 240-s schedule for sorting dinnerware of a different color in successive components of a multiple schedule. Sorting rates were similar in baseline, but when a video program was shown concurrently, sorting of dinnerware was more resistant to distraction when correlated with a higher rate of reinforcement. In Part 2 of the experiment, popcorn or coffee reinforcers were contingent upon sorting both colors of dinnerware according to variable-interval 60-s schedules, but additional reinforcers were given independently of sorting according to a variable-time 30-s schedule during one dinnerware-color component. Baseline sorting rate was lower but resistance to distraction by the video program was greater in the component with additional variable-time reinforcers. These results demonstrate that resistance to distraction depends on the rate of reinforcers obtained in the presence of component stimuli but is independent of baseline response rates and response–reinforcer contingencies. Moreover, these results are similar to those obtained in laboratory studies with pigeons, demonstrating that the determination of resistance to change by stimulus–reinforcer relations is not confined to controlled laboratory settings or unique to the pigeon.