On the effects of food deprivation and component reinforcer rates on multiple-schedule performance

Six pigeons were used to investigate the effects of varying body weight and component reinforcer rates in two-component multiple variable-interval variable-interval schedules. In Parts 1 and 3 of the experiment, unequal component reinforcer rates were arranged, and body weights were respectively increased and decreased. At 80% ad lib weight, response-rate ratios were closer to unity than reinforcer-rate ratios, but at 100% or more of ad lib weight, response-rate ratios generally equaled reinforcer-rate ratios. In Part 2, component reinforcer-rate ratios were varied over five conditions with the subjects maintained at 100% or more of their ad lib weights, and response-rate ratios matched reinforcer-rate ratios. The data thus support the empirical finding that response allocation in multiple schedules is a function of deprivation. Although this qualitative result is predicted by three models of multiple-schedule performance, only a model that assumes no direct component interaction adequately describes the data.     

Variable-ratio versus variable-interval schedules: response rate, resistance to change, and preference

Two experiments asked whether resistance to change depended on variable-ratio as opposed to variable-interval contingencies of reinforcement and the different response rates they establish. In Experiment 1, pigeons were trained on multiple random-ratio random-interval schedules with equated reinforcer rates. Baseline response rates were disrupted by intercomponent food, extinction, and prefeeding. Resistance to change relative to baseline was greater in the interval component, and the difference was correlated with the extent to which baseline response rates were higher in the ratio component. In Experiment 2, pigeons were trained on multiple variable-ratio variable-interval schedules in one half of each session and on concurrent chains in the other half in which the terminal links corresponded to the multiple-schedule components. The schedules were varied over six conditions, including two with equated reinforcer rates. In concurrent chains, preference strongly overmatched the ratio of obtained reinforcer rates. In multiple schedules, relative resistance to response-independent food during intercomponent intervals, extinction, and intercomponent food plus extinction depended on the ratio of obtained reinforcer rates but was less sensitive than was preference. When reinforcer rates were similar, both preference and relative resistance were greater for the variable-interval schedule, and the differences were correlated with the extent to which baseline response rates were higher on the variable-ratio schedule, confirming the results of Experiment 1. These results demonstrate that resistance to change and preference depend in part on response rate as well as obtained reinforcer rate, and challenge the independence of resistance to change and preference with respect to response rate proposed by behavioral momentum theory.     

A quantitative analysis of extreme choice

Six homing pigeons were trained on a variety of concurrent variable-interval schedules in a switching-key procedure. Unlike previous work, reinforcer ratios of up to 160 to 1 and concurrent extinction variable-interval schedules were arranged in order to investigate choice when reinforcer-frequency outcomes were extremely different. The data obtained over 11 conditions were initially analyzed according to the generalized matching law, which fitted the data well. The generalized matching law was then fitted only to conditions in which the reinforcer ratios were between 1 to 10 and 10 to 1. The deviations of choice measures from the other four more extreme reinforcer-ratio conditions were significantly more towards equal choice than predicted by this second generalized matching fit. A contingency-discriminability model, which predicts such deviations, described the data more effectively than did the generalized matching law, and also correctly predicted the maintenance of responding on both alternatives when one was associated with extinction.     

A quantitative analysis of chain-schedule performance

Six pigeons were trained with a chain variable-interval variable-interval schedule on the left key and with reinforcers available on the right key on a single variable-interval schedule arranged concurrently with both links of the chain. All three schedules were separately and systematically varied over a wide range of mean intervals. During these manipulations, the obtained reinforcer rates on constant arranged schedules also frequently changed systematically. Increasing reinforcer rates in Link 2 of the chain increased response rates in both links and decreased response rates in the variable-interval schedule concurrently available with Link 2. Increasing Link-1 reinforcer rates increased Link-1 response rates and decreased Link-2 response rates. Increasing reinforcer rates on the right-key schedule decreased response rates in Link 1 of the chain but did not affect the rate in Link 2. The results extend and amplify previous analyses of chain-schedule performance and help define the effects that a quantitative model must describe. However, the complexity of the results, and the fact that constant arranged reinforcer schedules did not necessarily lead to constant obtained reinforcer rates, precluded a quantitative analysis.     

Successive independence of multiple-schedule component performances

In three experiments, pigeons' responses were reinforced on two keys in each component of a series of multiple-schedule conditions. In each series, concurrent variable-interval schedules were constant in one component and were varied over conditions in the other component. In the first experiment both components arranged the same, constant total number of reinforcers, in the second the two components arranged constant but different totals, and in the third experiment the total was varied in one component and remained constant in the other. Relative reinforcer rate during the varied component was manipulated over conditions in all three experiments. In all these experiments, response and time allocation in the constant component were invariant when reinforcer ratios varied in the other component, demonstrating independence of behavior allocation in a multiple-schedule component from the relative reinforcer rate for the same alternatives in another component. In the two experiments which maintained constant reinforcer totals in components, sensitivity to reinforcement in the multiple schedules was the same as that in the concurrent schedules arranged during the varied component, with multiple-schedule bias in the experiment in which the totals were unequal.     

Sensitivity of time allocation to an overall reinforcer rate feedback function in concurrent interval schedules

Six pigeons were trained on concurrent variable-interval schedules in which feedback functions arranged that the overall reinforcer rate either (a) was independent of preference, (b) decreased with increasing absolute preference, or (c) increased with increasing absolute preference. In Experiment 1, the reinforcer rate in an interreinforcement interval was determined by the absolute time-allocation ratio in the previous interval. When arranged reinforcer ratios were varied, there was no evidence of control over preference by overall reinforcer rate. In Experiment 2, the feedback function arranged that reinforcer rates were an inverse function of absolute preference, and window durations were fixed times. In Phase 1, using schedules that provided a four-to-one reinforcer ratio, the window duration was decreased from 20 s to 5 s over four conditions. Then, in Phases 2 and 3, the arranged reinforcer ratios were varied. In Phase 2, the reinforcer rate in the current 5-s time window was determined by preference in the previous 5-s window, and in Phase 3, the window durations were 20 s. Again, there was no indication of control by obtained overall reinforcer rate. These data call into question theories that suggest that the process underlying matching is one of maximizing overall reinforcer rates, or that preference in concurrent aperiodic schedules is controlled to any extent by overall reinforcer rate. They also question the notion that concurrent-schedule preference is controlled by molecular maximizing.     

Successive independence and behavioral contrast in a closed economy.

Two pigeons had access to multiple concurrent schedules of reinforcement for 24 hours per day in their home cages. The variable-interval schedules comprising the multiple concurrent schedules were varied across 16 conditions. In three sets of conditions, one schedule was varied while its concurrent alternative and the concurrent schedules in the other component were held constant. Behavioral contrast was observed; that is, as the rate of reinforcement arranged by the varied schedule decreased, response rates on the constant schedules typically increased. These conditions formed part of two larger sets of conditions in which the concurrent schedules in one multiple-schedule component remained constant while the concurrent schedules in the other component were varied. Successive independence was found, in that behavior allocation during the constant component did not vary as a function of the reinforcer ratios in the varied component. Successive independence between components in multiple concurrent schedules is a robust result that occurs in closed economies and under conditions that promote behavioral contrast.     

Nonstable concurrent choice in pigeons

Six pigeons were trained on concurrent variable-interval schedules in which the arranged reinforcer ratios changed from session to session according to a 31-step pseudorandom binary sequence. This procedure allows a quantitative analysis of the degree to which performance in an experimental session is affected by conditions in previous sessions. Two experiments were carried out. In each, the size of the reinforcer ratios arranged between the two concurrent schedules was varied between 31-step conditions. In Experiment 1, the concurrent schedules were arranged independently, and in Experiment 2 they were arranged nonindependently. An extended form of the generalized matching law described the relative contribution of past and present events to present-session behavior. Total performance in sessions was mostly determined by the reinforcer ratio in that session and partially by reinforcers that had been obtained in previous sessions. However, the initial exposure to the random sequence produced a lower sensitivity to current-session reinforcers but no difference in overall sensitivity to reinforcement. There was no evidence that the size of the reinforcer ratios available on the concurrent schedules affected either overall sensitivity to reinforcement or the sensitivity to reinforcement in the current session. There was also no evidence of any different performance between independent and nonindependent scheduling. Because of these invariances, this experiment validates the use of the pseudorandom sequence for the fast determination of sensitivity to reinforcement.     

Multiple determinants of the effects of reinforcement magnitude on free-operant response rates

Four experiments examined the effects of increasing the number of food pellets given to hungry rats for a lever-press response. On a simple variable-interval 60-s schedule, increased number of pellets depressed response rates (Experiment 1). In Experiment 2, the decrease in response rate as a function of increased reinforcement magnitude was demonstrated on a variable-interval 30-s schedule, but enhanced rates of response were obtained with the same increase in reinforcement magnitude on a variable-ratio 30 schedule. In Experiment 3, higher rates of responding were maintained by the component of a concurrent variable-interval 60-s variable-interval 60-s schedule associated with a higher reinforcement magnitude. In Experiment 4, higher rates of response were produced in the component of a multiple variable-interval 60-s variable-interval 60-s schedule associated with the higher reinforcement magnitude. It is suggested that on simple schedules greater reinforcer magnitudes shape the reinforced pattern of responding more effectively than do smaller reinforcement magnitudes. This effect is, however, overridden by another process, such a contrast, when two magnitudes are presented within a single session on two-component schedules.     

Multiple and concurrent schedule performance: independence from concurrent and successive schedule contexts

Six pigeons were trained on multiple variable-interval schedules and performance was measured in the presence or absence of another variable-interval schedule (the common schedule) arranged concurrently with both components. Manipulations included varying the rate of reinforcement on the common schedule, leaving the common schedule unchanged while the components of the multiple schedule were varied, varying the multiple schedule components in the absence of the common schedule, and varying one component of the multiple schedule while the other component and the common schedule were unchanged. The normal rate-increasing and rate-decreasing effects of reinforcement rate increase were found, except that changing one multiple schedule component did not affect the response rate in the successively available common schedule component. Both concurrent and multiple schedule performance undermatched obtained reinforcement-rate ratios, but the degree of undermatching in multiple schedules was reliably greater. Allocation of responses between multiple schedule components was unaffected by the concurrent availability of reinforcement, and allocation of responses between concurrent schedules was unaffected by the successive availability of different reinforcement rates.     

Response strength in extreme multiple schedules

Four pigeons were trained in a series of two-component multiple schedules. Reinforcers were scheduled with random-interval schedules. The ratio of arranged reinforcer rates in the two components was varied over 4 log units, a much wider range than previously studied. When performance appeared stable, prefeeding tests were conducted to assess resistance to change. Contrary to the generalized matching law, logarithms of response ratios in the two components were not a linear function of log reinforcer ratios, implying a failure of parameter invariance. Over a 2 log unit range, the function appeared linear and indicated undermatching, but in conditions with more extreme reinforcer ratios, approximate matching was observed. A model suggested by McLean (1991), originally for local contrast, predicts these changes in sensitivity to reinforcer ratios somewhat better than models by Herrnstein (1970) and by Williams and Wixted (1986). Prefeeding tests of resistance to change were conducted at each reinforcer ratio, and relative resistance to change was also a nonlinear function of log reinforcer ratios, again contrary to conclusions from previous work. Instead, the function suggests that resistance to change in a component may be determined partly by the rate of reinforcement and partly by the ratio of reinforcers to responses.     

Effects of response-allocation constraints on multiple-schedule performance

Four pigeons were trained on multiple variable-interval schedules in which components alternated after a fixed number of responses had been emitted. In Part 1, each component change occurred after 20 responses; in Part 2, the number was 40; and in Part 3, the number of responses before change was 10. Component reinforcer rates were varied over five experimental conditions in each of Parts 1 to 3. Component response rates decreased as the specified number of responses per component was increased. However, the relation between component response-rate ratios and component reinforcer-rate ratios was independent of the specified number of responses per component, and was similar to that found when components alternate after fixed time periods. In the fourth part of the experiment, the results from Parts 1 to 3 were systematically replicated by keeping the component reinforcer rates constant, but different, while the number of responses that produced component alternation was varied from 5 to 60 responses. The results showed that multiple-schedule performance under component-response-number constraint is similar to that under conventional component-duration constraint. They further suggest that multiple-schedule response rates are controlled by component reinforcer rates and not by principles of maximizing overall reinforcer rates or meliorating component reinforcer rates.     

Preference and resistance to change with constant-duration schedule components

Previous research on preference between variable-interval terminal links in concurrent chains has most often used variable-duration terminal links ending with a single reinforcer. By contrast, most research on resistance to change in multiple schedules has used constant-duration components that include variable numbers of reinforcers in each presentation. Grace and Nevin (1997) examined both preference and resistance in variable-duration components; here, preference and resistance were examined in constant-duration components. Reinforcer rates were varied across eight conditions, and a generalized-matching-law analysis showed that initial-link preference strongly over-matched terminal-link reinforcer ratios. In multiple schedules, baseline response rates were unaffected by reinforcer rates, but resistance to intercomponent food, to extinction, and to intercomponent food plus extinction was greater in the richer component. The between-component difference in resistance to change exhibited additive effects for the three resistance tests, and was systematically related to reinforcer ratios. However, resistance was less sensitive to reinforcer ratios than was preference. Resistance to intercomponent food and to intercomponent food plus extinction was more sensitive to reinforcer ratios in the present study than in Grace and Nevin (1997). Thus, relative to variable-duration components, constant-duration components increased the sensitivity of both preference and relative resistance, supporting the proposition that these are independent and convergent measures of the effects of a history of reinforcement.     

Concurrent schedules: reinforcer magnitude effects

Five pigeons were trained on pairs of concurrent variable-interval schedules in a switching-key procedure. The arranged overall rate of reinforcement was constant in all conditions, and the reinforcer-magnitude ratios obtained from the two alternatives were varied over five levels. Each condition remained in effect for 65 sessions and the last 50 sessions of data from each condition were analyzed. At a molar level of analysis, preference was described well by a version of the generalized matching law, consistent with previous reports. More local analyses showed that recently obtained reinforcers had small measurable effects on current preference, with the most recently obtained reinforcer having a substantially larger effect. Larger reinforcers resulted in larger and longer preference pulses, and a small preference was maintained for the larger-magnitude alternative even after long inter-reinforcer intervals. These results are consistent with the notion that the variables controlling choice have both short- and long-term effects. Moreover, they suggest that control by reinforcer magnitude is exerted in a manner similar to control by reinforcer frequency. Lower sensitivities when reinforcer magnitude is varied are likely to be due to equal frequencies of different sized preference pulses, whereas higher sensitivities when reinforcer rates are varied might result from changes in the frequencies of different sized preference pulses.     

Behavior-dependent reinforcer-rate changes in concurrent schedules: A further analysis

Six pigeons were trained on concurrent variable-interval schedules in three different procedures. The first procedure was a standard concurrent schedule, and the relative reinforcer frequency for responding was varied. The second was a schedule in which a relative left-key response rate (over a fixed period of time) exceeding .75 produced, in the next identical time period a higher reinforcer rate on the right key. If this criterion was not exceeded, equal reinforcer rates were arranged on the two keys in this period. This was the dependent procedure. In the third (independent) procedure, the periods of higher right-key reinforcer rates occurred with the same probability as in the second procedure, but occurred independently of behavior. In the second and third procedures, the fixed-time period (window) was varied from 5 s to 60 s, and to 240 s in the second procedure only. Performance on the two keys was similar in the concurrent and independent procedures. The procedure used in the dependent conditions generally affected performance when the windows were shorter than about 30 s. Models of performance that assume that subjects do not discriminate changes in local relative reinforcer rates cannot account for the data. Moreover, existing models are inherently unable to account for the effects of contingencies of reinforcement between responding on one alternative and gaining reinforcers on another that are arranged or that emerge as a result of time allocated to alternative schedules. Undermatching on concurrent variable-interval schedules may result from such emergent contingencies.     

Contrast and reallocation of extraneous reinforcers between multiple-schedule components

Four pigeons responded in components of multiple schedules in which two responses were available and reinforced with food. Pecks on the left key (“main” key) were reinforced at a constant rate in one component and at a rate that varied over conditions in the other component. When reinforcer rate was varied, behavioral contrast occurred in the constant component. On the right key (“extra” key), five variable-interval schedules and one variable-ratio schedule, presented conjointly, arranged reinforcers for responses in all conditions. These conjoint schedules were common to both multiple-schedule components—rather than unique to particular components—and reinforcers from these schedules could therefore be arranged in one component and obtained during the other component. In this way, the additional reinforcers were analogous to the “extraneous” reinforcers thought to maintain behavior other than pecking in conventional multiple schedules. Response rate on the extra key did not change systematically over conditions in the constant component, and in the varied component extra responding was inversely related to main-key reinforcement. All subjects obtained more extra-key reinforcers in whichever component arranged fewer main-key reinforcers. Consistent with the theory that reallocation of extraneous reinforcers may cause behavioral contrast, absolute reinforcer rate for the extra key in the constant component was low in conditions that produced positive contrast on the main key and high in those that produced negative contrast. Also consistent with this theory, behavioral contrast was reduced in two conditions that canceled extra-key reinforcers that had been arranged but not obtained at the end of components. Thus, a constraint on reallocation markedly reduced the extent of contrast.     

Closed-economy multiple-schedule performance: Effects of deprivation and session duration

Three pigeons responded for food reinforcement on multiple variable-interval schedules in which the total consumption of food was entirely determined by the subjects' interaction with the schedules (a closed economy). The finding of overmatching, where response allocation between components is more extreme than the distribution of reinforcers, was reconfirmed. Generalized-matching sensitivity decreased from overmatching to undermatching values typical of conventional multiple schedules when food deprivation was increased by decreasing session duration, but not when deprivation was increased by decreasing overall reinforcer rate. Sensitivity also increased from undermatching to overmatching as session duration increased from 100 min to 24 hr, while deprivation was held constant by decreasing overall reinforcer rate. These results can be understood in terms of increases in the value of extraneous reinforcers relative to food reinforcers as deprivation decreases or as the economy for extraneous reinforcers becomes more closed. However, no published quantitative expression of the effects of extraneous reinforcers is entirely consistent with the results.     

Response strength in multiple periodic and aperiodic schedules

Responding in multiple periodic and aperiodic schedules of equal mean reinforcement rate was examined during extinction, satiation, and in the presence of various free-food schedules. In Experiments I and II, pigeons were trained on multiple variable-interval–fixed-interval schedules. Decreases in the rate of responding due to extinction, satiation, or food schedules were approximately equal regardless of the temporal pattern of reinforcer presentation. In Experiment III, pigeons responded on a two-component multiple schedule in which each component was a two-member homogeneous response chain terminating in a fixed-interval schedule during one component and in a variable-interval schedule during the other. The length of both terminal links was varied over a series of conditions. Initial-link responding in the fixed-interval component was reduced more by increasing terminal-link length than was initial-link responding in the variable-interval component. However, no differences in resistance to satiation and extinction were obtained across the fixed and variable components. If the relative decrease in responding produced by satiation and extinction is used as an index of the “value” of the conditions maintaining responding, then these data suggest that fixed and variable schedules of equal mean length are equally valued. This conclusion, however, is not consistent with findings of preference for variable over fixed schedules obtained in studies using concurrent-chain procedures.     

Choice dynamics in concurrent ratio schedules of reinforcement

The generalized matching law predicts performance on concurrent schedules when variable-interval schedules are programmed but is trivially applicable when independent ratio schedules are used. Responding usually is exclusive to the schedule with the lowest response requirement. Determining a method to program concurrent ratio schedules such that matching analyses can be usefully employed would extend the generality of matching research and lead to new avenues of research. In the present experiments, ratio schedules were programmed dependently such that responses to either of the two options progressed the requirement on both schedules. Responding is not exclusive because the probability of reinforcement increases on both schedules as responses are allocated to either schedule. In Experiment 1, performance on concurrent variable-ratio schedules was assessed, and reinforcer ratios were varied across conditions to investigate changes in sensitivity. Additionally, the length of a changeover delay was manipulated. In Experiment 2, performance was compared under concurrently available, dependently programmed variable-ratio and fixed-ratio schedules. Performance was well described by the generalized matching law. Increases in the changeover delay decreased sensitivity, whereas sensitivity was higher when variable-ratio schedules were employed, compared with fixed-ratio schedules. Concurrent ratio schedules can be a viable approach to studying functional differences between ratio and interval schedules.