Contrast and undermatching with regular or irregular alternation of components

Behavioral contrast and response-ratio sensitivity to reinforcement were compared in multiple schedules in which components alternated strictly or according to a pseudorandom sequence. Average component durations in the two regimes were always 60 s, and order of presentation of component alternation regimes was counterbalanced across subjects. In Part 1, the reinforcer rate in one component was reduced from 60 per hour to zero, while that in the other component was unchanged. Positive behavioral contrast occurred in the constant component in that response rates increased, but neither the reliability nor the magnitude of contrast was affected by the manner in which components alternated. Part 2 was similar, except that a number of different reinforcer rates were used in the varied component. Neither contrast nor sensitivity of response ratios to changes in reinforcer ratios depended on the regime of component alternation. Thus, the predictability in time of future reinforcement conditions, which is a feature of regular multiple scheduling, does not appear to be a determinant of multiple-schedule performance.     

Undermatching: a reappraisal of performance on concurrent variable-interval schedules of reinforcement

The extant data for pigeons' performance on concurrent variable-interval schedules were examined in detail. Least-squares lines relating relative pecks and time to the corresponding relative reinforcements were obtained for four studies. The between-study group slopes for time and pecks and five of seven within-study group slopes from individual studies were less than 1.00. This suggested the generality that pigeons respond less to the richer reinforcement schedule than predicted by matching. For pecks, a nonparametric test for distribution of points also supported this concept of undermatching (to the richer reinforcement schedule). In addition, using mean squared error as the criterion, a cubic curve fit the peck proportion data better than any line or other polynomial. This indicates that the relation between peck and reinforcement proportions may be nonlinear.     

Signalled reinforcement and multiple schedules

The responses of four pigeons were first reinforced in the presence of two different wave-lengths (green and red) on a two-ply multiple schedule with identical variable-interval 3-min schedules of reinforcement associated with each component. While the constant-component reinforcement schedule remained unchanged during the experiment, the schedule associated with the variable component was changed to (1) signalled variable time, (2) unsignalled variable time, or (3) signalled variable interval. The probability with which the availability of the reinforcer was signalled in the variable-interval schedules was either 0.5 or 1.0. Positive contrast occurred in both signalled variable-interval and variable-time schedules, but only when the availability of all the variable-component reinforcers was signalled. Signalling the availability of only 50% of the reinforcers in signalled variable-interval schedules resulted in negative induction. The present data suggest that positive behavioral contrast resulting from signalled reinforcer availability is due to the presence of an extinction-correlated stimulus.     

Contrast and undermatching as a function of reinforcer duration and quality during multiple schedules

Eight pigeons pecked keys under multiple variable-interval two-minute variable-interval two-minute schedules. In Experiment 1, the reinforcers were 2, 4, or 8 seconds access to a food magazine. In Experiments 2 and 3, the reinforcers were grains that had been determined to be most-, moderately-, or non-preferred. Both positive and negative behavioral contrast occurred when the reinforcers in one component were held constant and the duration or type of reinforcer obtained in the other component varied. Undermatching occurred when the relative rate of responding during a component was plotted as a function of the relative duration of the reinforcers in that component.     

Contrast and induction in multiple schedules of discrete-trial concurrent reinforcement

Three pigeons were exposed to two-key discrete-trial concurrent schedules of reinforcement. Red and white key colors alternated irregularly and the assignment of reinforcers depended on key color. The red-key schedules were held constant, with the scheduled relative frequency of reinforcement for left-key pecks set at 0.75, while the white-key schedules varied. When the location of white-key reinforcement was changed from one side to the other, while its overall frequency was constant, red-key choices shifted in the same direction as white-key choices, an induction effect. When the overall frequency of white-key reinforcement was changed while its location remained constant, red key choices shifted in a direction opposite to white-key choices, a contrast effect. Both induction and contrast effects were clearer when the overall frequency of red-key reinforcement was reduced. These data demonstrate that the allocation of responding may exhibit schedule interaction effects similar to those commonly reported for response rate.     

A re-examination of local contrast in multiple schedules

Pigeons were presented with multiple schedules of alternating 90-sec components. When components in which grain was never presented alternated with components in which grain was presented on a variable-interval schedule, the average rate of responding in the variable-interval components increased, showing overall positive behavioral contrast. Unlike previous reports, this study found that the response rates for all birds increased toward the end of the variable-interval components as training proceeded. This increase in local response rate disappeared when the multiple schedule was composed solely of variable-interval components and reappeared when the variable-interval components were again alternated with extinction. This finding cannot be predicted or explained by recent theories of behavioral contrast based on autoshaping, and thus questions their sufficiency. We suggest that this local response-rate increase results from the predictable change from high to low density of reinforcement at the end of the fixed-duration component. Thus, the present effect apparently illustrates a different type of interaction between components of a multiple schedule than that described by previous theories of contrast. In a given procedure, either or both types of interaction may occur; neither provides a complete account of behavioral contrast.     

Effects of component length and of the transitions among components in multiple schedules

Pigeons received equal variable-interval reinforcement during presentations of two line-orientation stimuli while five other orientations appeared in extinction. Component duration was 30 seconds for all orientations and the sequence was arranged so that each orientation preceded itself and each other orientation equally often. The duration of one component (0°) was shortened to 10 seconds and the other (90°) was lengthened to 50 seconds. All animals showed large increases in response rate in the shortened component and this increase was recoverable after an interpolated condition in which all components were again 30 seconds in duration. This effect was replicated in a second experiment in which component duration was changed from 150 seconds to 50 seconds and 250 seconds. An examination of local contrast effects during the first experiment showed that the shortened component produced local contrast during subsequent presentations of the lengthened component, just as would a component associated with more frequent reinforcement. When the presentation sequence was changed so that the lengthened component was always followed by the shortened component, response rates generally increased during the lengthened component. When the sequence was arranged so that the shortened component always preceded the longer component, response rate decreased in the former. These effects, as well as the increases in response rate following change in component length, seem not to be the product of local contrast effects among components.     

Inelastic supply: An economic approach to simple interval schedules

Economic theory predicts an inverse relationship between the quantity of a commodity supplied to the marketplace and the equilibrium market price of that commodity. This prediction was tested in three experiments. Pigeons responded on simple variable-interval schedules, and quantity of reinforcement supplied was varied in a different way in each experiment. In Experiment 1, quantity supplied was varied by manipulating reinforcement rate while keeping session length constant. In Experiment 2, quantity supplied was varied by manipulating reinforcement rate while keeping reinforcers per session constant. In Experiment 3, quantity supplied was varied by manipulating reinforcer magnitude while keeping number of reinforcers constant. As predicted by economic theory, the obtained behavioral cost (responses per reinforcer) increased as supply decreased. The results could not be explained by simple artifacts such as satiation and time available to respond. In addition, the function relating response rate to reinforcement rate was bitonic in 7 of 9 animals in Experiments 1 and 2, which supports economic and regulatory theories over more traditional reinforcement theories.     

Local contrast in behavior allocation during multiple-schedule components

Allocation of responses between two keys was studied during two alternating multiple-schedule components. Responses were recorded in successive quarters of each component. Variable-interval reinforcer schedules on the two keys were constant throughout the experiment for one (constant) component and were varied over conditions on one key for the other, producing changes in reinforcer ratios for the varied component. Behavior allocation for the first quarter of the constant component was inversely related to varied-component reinforcer ratios, a form of local contrast, but this relationship was not observed later in the component. During the first quarter of the varied component, slopes of matching lines were high and decreased later in the component. It is argued that this form of local contrast cannot be explained in terms of reallocation of extraneous reinforcers between components, and that the matching law for concurrent operants does not capture some sources of control over behavior allocation. A simple extension of the matching law is offered that adequately describes behavior changes during both components. A version of this formulation can predict contrast effects in absolute response rates.     

Negative behavioral contrast on multiple treadle-press schedules

Eight pigeons pressed treadles for food reinforcers delivered by several multiple variable-interval schedules. The rate of reinforcement for responding during one component schedule was held constant at 30 reinforcers per hour. The rate of reinforcement for responding during the other component varied from 0 to 120 or 240 reinforcers per hour. The schedules were presented in different orders for different subjects. The rate of responding emitted during the variable component schedule varied directly with the rate of reinforcement it provided. The rate of responding during the constant component did not increase consistently when the rate of reinforcement obtained from the variable component decreased from 30 to 0 reinforcers per hr. The rate of responding emitted during the constant component decreased when the rate of reinforcement obtained from the variable component increased from 30 reinforcers per hour to a higher rate. That is, negative but not positive behavioral contrast occurred. The failure to find positive contrast is consistent with one of the predictions of the additive theories of behavioral contrast. Finding negative contrast has ambiguous implications for the additive theories.     

Behavioral contrast for key pecking as a function of component duration when only one component varies

Pigeons pecked keys for food reinforcers delivered by multiple variable-interval 2-min variable-interval 2-min schedules. Positive behavioral contrast was created by changing one component to extinction; negative contrast was achieved by changing one component to a variable-interval 15-s schedule. The duration of each component was varied independently of the other from 5 to 960 s. The size of positive contrast was greatest when the extinction component was 30 or 60 s long. It did not change significantly with changes in the duration of the variable-interval 2-min component. The absolute size of negative contrast decreased with increases in the duration of the variable-interval 2-min component. It did not change significantly with changes in the duration of the variable-interval 15-s component. These results show that the size of contrast is determined primarily by the duration of the component that provides the less favorable conditions of reinforcement. These results are not predicted by current theories.     

Behavioral contrast: Pavlovian effects and anticipatory contrast.

Two sources of behavioral contrast have been identified previously: Pavlovian stimulus-reinforcer relations and component sequence effects (anticipatory contrast). This study sought to isolate these sources of control procedurally in a four-ply multiple schedule composed of two fixed two-component sequences. Different cues were associated with the first component of each sequence, and contrast effects were studied in these target components. In Experiment 1, differential cuing of Component 2 between sequences and availability of reinforcement during target components were varied across three groups of pigeons; the stimulus-reinforcer relation between target-component cues and schedule of reinforcement in Component 2 was varied within subjects. Control by the Pavlovian relation was demonstrated under all conditions, and anticipatory contrast was not observed. In Experiment 2, target-component duration was systematically varied in the three groups of Experiment 1. Control by the Pavlovian relation was reliably obtained only when target-component behavior was unreinforced, and diminished with increases in component duration. Anticipatory contrast emerged in the two groups for which target-component reinforcement was available. These and other data indicate that Pavlovian effects in multiple schedules may be obscured when the requisite conditions for anticipatory contrast are present.     

Undermatching on concurrent variable-interval schedules and the power law.

The phenomenon of undermatching on concurrent variable-interval schedules is shown to be derivable by transforming the individual interreinforcement intervals of each variable-interval schedule and averaging the transformed values to produce an "estimate" of the rate of reinforcement the schedules deliver. If the transformation is based on a power function with a fractional exponent, such as is found in many studies of temporal control in animals, matching response rations to the ratios of these estimated rates of reinforcement yields undermatching. If the concurrent variable-interval schedules are arranged such that the individual intervals in each schedule have a constant proportionality (a procedure found in many commonly used variable-interval schedules) the slope of the line relating logarithms of response ratios and of programmed reinforcement ratios is identical to the exponent of the power transformation applied to the individual time intervals in the variable-interval schedules. In other cases this simple relation does not hold but the degree of undermatching is greater the lower the value of the exponent of the power function. This account of undermatching predicts values similar to those typically observed.     

Performance in continuously available multiple schedules

Three pigeons were given continuous access in their home cages to food reinforcement on two-component multiple variable-interval variable-interval schedules. The reinforcer rates in the two components were varied over seven experimental conditions, and a partial replication over five conditions was arranged one year later. When component reinforcer rates were unequal, ratios of component response rates were more extreme than ratios of obtained component reinforcer rates, a result which in a generalized-matching analysis is termed overmatching. This finding contrasts sharply with results obtained when multiple schedules are arranged in shorter sessions, in which performance is characterized by undermatching when subjects are deprived of food, and by matching, or equality between component response- and reinforcer-rate ratios, when deprivation is minimal. More molecular data obtained in two subsequent conditions suggested that this finding did not reflect local fluctuations or asymmetries in deprivation. Theories of multiple-schedule performance that predict that matching cannot be exceeded are disconfirmed by the present results.     

Local contrast in multiple schedules: the effect of stimulus discriminability.

A three-ply multiple schedule assessed responding in a standard component as a function of the just-preceding schedule. The principal experimental condition was the difference among the wavelengths signaling the schedule components. Only the pigeons working in a narrow wavelength range showed persistent positive local contrast; that is, response rate during the standard component was higher when that component followed extinction than when it followed itself. Birds in both narrow- and medium-range groups showed persistent negative local contrast; that is, rate was lower following a relatively rich component. The dissipation of positive contrast appeared to be most clearly related to the establishment of differential responding. Negative contrast was inversely related to wavelength differences. Theories pertaining to contrast must account for the role of discrimination in both positive and negative types.     

Interactions in multiple schedules: the role of the stimulus-reinforcer contingency

In Experiments I and II, pigeons were exposed to single-key multiple schedules of response-independent and -dependent food presentation. Components were correlated with different keylights. When the rate of food presentation in the first component exceeded that in the second component, the local rate of key pecking was relatively high at onset of the first component. Overall rate in that component varied inversely with component duration and the rate of food presentation in the second component. When responding was maintained in the second component, the local rate of key pecking was relatively low at onset of that component. Overall rate in the second component varied directly with component duration and the rate of food presentation in that component. In Experiment III, pigeons were exposed to a two-key multiple schedule. Pecks on a constantly illuminated key produced food. Components were correlated with the color of a second key on which pecks had no scheduled consequences. The effects of component duration and rate of food presentation under the single-key response-dependent schedule were synthesized by combining response rates on each concurrently available key under the two-key procedure. The results support an account of multiple-schedule interactions in terms of the joint influence on responding of stimulus-reinforcer and response-reinforcer contingencies.     

Contrast and reallocation of extraneous reinforcers between multiple-schedule components

Four pigeons responded in components of multiple schedules in which two responses were available and reinforced with food. Pecks on the left key (“main” key) were reinforced at a constant rate in one component and at a rate that varied over conditions in the other component. When reinforcer rate was varied, behavioral contrast occurred in the constant component. On the right key (“extra” key), five variable-interval schedules and one variable-ratio schedule, presented conjointly, arranged reinforcers for responses in all conditions. These conjoint schedules were common to both multiple-schedule components—rather than unique to particular components—and reinforcers from these schedules could therefore be arranged in one component and obtained during the other component. In this way, the additional reinforcers were analogous to the “extraneous” reinforcers thought to maintain behavior other than pecking in conventional multiple schedules. Response rate on the extra key did not change systematically over conditions in the constant component, and in the varied component extra responding was inversely related to main-key reinforcement. All subjects obtained more extra-key reinforcers in whichever component arranged fewer main-key reinforcers. Consistent with the theory that reallocation of extraneous reinforcers may cause behavioral contrast, absolute reinforcer rate for the extra key in the constant component was low in conditions that produced positive contrast on the main key and high in those that produced negative contrast. Also consistent with this theory, behavioral contrast was reduced in two conditions that canceled extra-key reinforcers that had been arranged but not obtained at the end of components. Thus, a constraint on reallocation markedly reduced the extent of contrast.     

Successive independence and behavioral contrast in a closed economy.

Two pigeons had access to multiple concurrent schedules of reinforcement for 24 hours per day in their home cages. The variable-interval schedules comprising the multiple concurrent schedules were varied across 16 conditions. In three sets of conditions, one schedule was varied while its concurrent alternative and the concurrent schedules in the other component were held constant. Behavioral contrast was observed; that is, as the rate of reinforcement arranged by the varied schedule decreased, response rates on the constant schedules typically increased. These conditions formed part of two larger sets of conditions in which the concurrent schedules in one multiple-schedule component remained constant while the concurrent schedules in the other component were varied. Successive independence was found, in that behavior allocation during the constant component did not vary as a function of the reinforcer ratios in the varied component. Successive independence between components in multiple concurrent schedules is a robust result that occurs in closed economies and under conditions that promote behavioral contrast.     

Competition between stimulus-reinforcer contingencies and anticipatory contrast.

Procedures used to study anticipatory contrast are conceptually similar to those used to study autoshaping, in that two target stimuli signal either higher or lower rates of reinforcement in the following components of the schedule. Despite this signal contingency, anticipatory contrast entails response rates that are higher to the target stimulus followed by the lower rate of reinforcement. To determine the relation between such effects and autoshaping, different variations of the procedure were used in which the signal contingency was presented in the absence of reinforcement in the target components themselves and in which the reinforcement schedules in the different following components were signaled by the same stimulus. Autoshaping effects of this signal contingency were demonstrated when no reinforcement was available during the target-component signals themselves. Intermediate patterns of behavior occurred when reinforcement was available during the target-component signals and when their different following schedules were correlated with the same stimulus. Attempts to isolate these signal and contrast effects functionally by using the signal-key procedure were unsuccessful. The results demonstrate that Pavlovian stimulus contingencies are in competition with the dynamics of anticipatory contrast, thus reducing its occurrence under some circumstances.     

A molecular analysis of multiple schedule interactions: negative contrast

The present experiments investigated the relationship between changes in the relative reinforced interresponse-time distributions and the occurrence of positive and negative contrast in multiple variable-interval—variable-interval and multiple variable-interval—extinction schedules of reinforcement. Experiment I demonstrated that changes in the interresponse-time distributions were consistently correlated with response-rate changes referred to as positive and negative contrast. Corresponding changes in the reinforced interresponse-time distributions suggested that negative contrast resulted as an inductive effect of selectively reinforcing long interresponse times in the altered component at the moment the baseline schedule was reintroduced. Experiment II demonstrated that the magnitude of the negative-contrast effect could be significantly decreased if the altered component schedule was modified in order to prevent the reinforcement of these interresponse times during the first few sessions of baseline recovery. The results supported a proposal that interresponse time—reinforcer relations may act as amplifiers or attenuators of negative contrast.