On the measurement of time allocation on multiple variable-interval schedules

Six pigeons were trained on a modified multiple-schedule procedure. In a three-key chamber, the center key was lighted red or green, depending upon which component schedule was in effect. A response on this key transferred this color to each of two side keys, and responses on one of those keys produced reinforcers according to the component schedule. After 2 s, the side-key lights were extinguished, the center key was reilluminated, and a further center-key response was required to give access, as before, to the component schedules. Components alternated every 3 min. This limited-access procedure allowed both times spent switched into the side keys and time spent not switched in to be measured in the two components. Component reinforcer rates were varied over eight experimental conditions. Both component response rate and component time allocation were increasing functions of relative component reinforcer rate, and these functions were not significantly different. This finding implies that local response rates (responses divided by time switched in) were unaffected by changing component reinforcer rates on multiple schedules. Because a similar result was recently obtained for concurrent schedules, models of multiple and concurrent-schedule performance may need to consider only the time allocation of behavior emitted at equal tempo in the component schedules.     

Contrast and undermatching with regular or irregular alternation of components

Behavioral contrast and response-ratio sensitivity to reinforcement were compared in multiple schedules in which components alternated strictly or according to a pseudorandom sequence. Average component durations in the two regimes were always 60 s, and order of presentation of component alternation regimes was counterbalanced across subjects. In Part 1, the reinforcer rate in one component was reduced from 60 per hour to zero, while that in the other component was unchanged. Positive behavioral contrast occurred in the constant component in that response rates increased, but neither the reliability nor the magnitude of contrast was affected by the manner in which components alternated. Part 2 was similar, except that a number of different reinforcer rates were used in the varied component. Neither contrast nor sensitivity of response ratios to changes in reinforcer ratios depended on the regime of component alternation. Thus, the predictability in time of future reinforcement conditions, which is a feature of regular multiple scheduling, does not appear to be a determinant of multiple-schedule performance.     

On the effects of food deprivation and component reinforcer rates on multiple-schedule performance

Six pigeons were used to investigate the effects of varying body weight and component reinforcer rates in two-component multiple variable-interval variable-interval schedules. In Parts 1 and 3 of the experiment, unequal component reinforcer rates were arranged, and body weights were respectively increased and decreased. At 80% ad lib weight, response-rate ratios were closer to unity than reinforcer-rate ratios, but at 100% or more of ad lib weight, response-rate ratios generally equaled reinforcer-rate ratios. In Part 2, component reinforcer-rate ratios were varied over five conditions with the subjects maintained at 100% or more of their ad lib weights, and response-rate ratios matched reinforcer-rate ratios. The data thus support the empirical finding that response allocation in multiple schedules is a function of deprivation. Although this qualitative result is predicted by three models of multiple-schedule performance, only a model that assumes no direct component interaction adequately describes the data.     

Successive independence of multiple-schedule component performances

In three experiments, pigeons' responses were reinforced on two keys in each component of a series of multiple-schedule conditions. In each series, concurrent variable-interval schedules were constant in one component and were varied over conditions in the other component. In the first experiment both components arranged the same, constant total number of reinforcers, in the second the two components arranged constant but different totals, and in the third experiment the total was varied in one component and remained constant in the other. Relative reinforcer rate during the varied component was manipulated over conditions in all three experiments. In all these experiments, response and time allocation in the constant component were invariant when reinforcer ratios varied in the other component, demonstrating independence of behavior allocation in a multiple-schedule component from the relative reinforcer rate for the same alternatives in another component. In the two experiments which maintained constant reinforcer totals in components, sensitivity to reinforcement in the multiple schedules was the same as that in the concurrent schedules arranged during the varied component, with multiple-schedule bias in the experiment in which the totals were unequal.     

Temporal constraint on choice: Sensitivity and bias in multiple schedules

In multiple schedules of reinforcement, ratios of responses in successive components are relatively insensitive to ratios of obtained reinforcers. An analysis is proposed that attributes changes in absolute response rates to concurrent interactions between programmed reinforcement and extraneous reinforcement in other components. The analysis predicts that ratios of responses in successive components vary with reinforcer ratios, qualified by a term describing the reinforcement context, that is, programmed and extraneous reinforcers. Two main predictions from the analysis were confirmed in an experiment in which pigeons' responses were reinforced in the components of a multiple schedule and analog extraneous reinforcement was scheduled for an alternative response in each component. Sensitivity of response and time ratios to reinforcer ratios in the multiple schedules varied as a function of the rate of extraneous reinforcers. Bias towards responding in one component of the multiple schedule varied as an inverse function of the ratios of extraneous reinforcer rate in the two components. The data from this and previous studies of multiple-concurrent performance were accurately predicted by our analysis and supported our contention that the allocation of behavior in multiple-schedule components depends on the relative values of concurrently-available reinforcers within each component.     

Local contrast in behavior allocation during multiple-schedule components

Allocation of responses between two keys was studied during two alternating multiple-schedule components. Responses were recorded in successive quarters of each component. Variable-interval reinforcer schedules on the two keys were constant throughout the experiment for one (constant) component and were varied over conditions on one key for the other, producing changes in reinforcer ratios for the varied component. Behavior allocation for the first quarter of the constant component was inversely related to varied-component reinforcer ratios, a form of local contrast, but this relationship was not observed later in the component. During the first quarter of the varied component, slopes of matching lines were high and decreased later in the component. It is argued that this form of local contrast cannot be explained in terms of reallocation of extraneous reinforcers between components, and that the matching law for concurrent operants does not capture some sources of control over behavior allocation. A simple extension of the matching law is offered that adequately describes behavior changes during both components. A version of this formulation can predict contrast effects in absolute response rates.     

Choice in a variable environment: every reinforcer counts

Six pigeons were trained in sessions composed of seven components, each arranged with a different concurrent-schedule reinforcer ratio. These components occurred in an irregular order with equal frequency, separated by 10-s blackouts. No signals differentiated the different reinforcer ratios. Conditions lasted 50 sessions, and data were collected from the last 35 sessions. In Part 1, the arranged overall reinforcer rate was 2.22 reinforcers per minute. Over conditions, number of reinforcers per component was varied from 4 to 12. In Part 2, the overall reinforcer rate was six per minute, with both 4 and 12 reinforcers per component. Within components, log response-allocation ratios adjusted rapidly as more reinforcers were delivered in the component, and the slope of the choice relation (sensitivity) leveled off at moderately high levels after only about eight reinforcers. When the carryover from previous components was taken into account, the number of reinforcers in the components appeared to have no systematic effect on the speed at which behavior changed after a component started. Consequently, sensitivity values at each reinforcer delivery were superimposable. However, adjustment to changing reinforcer ratios was faster, and reached greater sensitivity values, when overall reinforcer rate was higher. Within a component, each successive reinforcer from the same alternative ("confirming") had a smaller effect than the one before, but single reinforcers from the other alternative ("disconfirming") always had a large effect. Choice in the prior component carried over into the next component, and its effects could be discerned even after five or six reinforcement and nonreinforcement is suggested.     

The control of choice by its consequences

Five pigeons were trained on concurrent variable-interval schedules in which equal rates of reinforcement were always arranged for left- and right-key responses, but different overall rates were signaled by key colors. Sessions began with both keys lit yellow for the instrumental phase. If, after 20 s of this phase, the relative number of responses that had been made to the left key equaled or exceeded .75, both keys changed red for the contingent phase. The contingent phase arranged another concurrent variable-interval schedule for a further 20 s before the instrumental phase was reinstated. However, if preference in the instrumental phase did not exceed .75, the instrumental phase continued for a further 20 s before preference was again compared with the criterion. In Part 1, the reinforcer rate arranged in the instrumental phase was held constant at 4.8 reinforcers per minute, while the reinforcer rate arranged in the contingent phase was varied across conditions from 0 to 19.2 over five steps. In Part 2, reinforcer rates in the contingent phase were kept constant at 36 per minute, while reinforcer rates in the instrumental phase were varied from 0 to 36 over seven steps. Part 3 replicated Part 2 but used reinforcer rates in both phases that were one third of those arranged in Part 2. Measures of choice obtained by summing responses across presentations of the instrumental phase became more extreme toward the left key as the reinforcer rate obtained in the contingent phase was increased (Part 1) and as the reinforcer rate obtained in the instrumental phase was decreased (Parts 2 and 3). Changes in these measures of choice were accompanied by systematic changes in the relative frequency with which the criterion was exceeded. Changes in both these measures were correlated with changes in the relative frequency with which subjects responded exclusively to the left key. These results are discussed with respect to the two choices that were concurrently available in this procedure and the response alternatives that might constitute the concurrent operants in each choice.     

Performance in continuously available multiple schedules

Three pigeons were given continuous access in their home cages to food reinforcement on two-component multiple variable-interval variable-interval schedules. The reinforcer rates in the two components were varied over seven experimental conditions, and a partial replication over five conditions was arranged one year later. When component reinforcer rates were unequal, ratios of component response rates were more extreme than ratios of obtained component reinforcer rates, a result which in a generalized-matching analysis is termed overmatching. This finding contrasts sharply with results obtained when multiple schedules are arranged in shorter sessions, in which performance is characterized by undermatching when subjects are deprived of food, and by matching, or equality between component response- and reinforcer-rate ratios, when deprivation is minimal. More molecular data obtained in two subsequent conditions suggested that this finding did not reflect local fluctuations or asymmetries in deprivation. Theories of multiple-schedule performance that predict that matching cannot be exceeded are disconfirmed by the present results.     

Concurrent schedules: Effects of time- and response-allocation constraints

Five pigeons were trained on concurrent variable-interval schedules arranged on two keys. In Part 1 of the experiment, the subjects responded under no constraints, and the ratios of reinforcers obtainable were varied over five levels. In Part 2, the conditions of the experiment were changed such that the time spent responding on the left key before a subsequent changeover to the right key determined the minimum time that must be spent responding on the right key before a changeover to the left key could occur. When the left key provided a higher reinforcer rate than the right key, this procedure ensured that the time allocated to the two keys was approximately equal. The data showed that such a time-allocation constraint only marginally constrained response allocation. In Part 3, the numbers of responses emitted on the left key before a changeover to the right key determined the minimum number of responses that had to be emitted on the right key before a changeover to the left key could occur. This response constraint completely constrained time allocation. These data are consistent with the view that response allocation is a fundamental process (and time allocation a derivative process), or that response and time allocation are independently controlled, in concurrent-schedule performance.     

Successive independence and behavioral contrast in a closed economy.

Two pigeons had access to multiple concurrent schedules of reinforcement for 24 hours per day in their home cages. The variable-interval schedules comprising the multiple concurrent schedules were varied across 16 conditions. In three sets of conditions, one schedule was varied while its concurrent alternative and the concurrent schedules in the other component were held constant. Behavioral contrast was observed; that is, as the rate of reinforcement arranged by the varied schedule decreased, response rates on the constant schedules typically increased. These conditions formed part of two larger sets of conditions in which the concurrent schedules in one multiple-schedule component remained constant while the concurrent schedules in the other component were varied. Successive independence was found, in that behavior allocation during the constant component did not vary as a function of the reinforcer ratios in the varied component. Successive independence between components in multiple concurrent schedules is a robust result that occurs in closed economies and under conditions that promote behavioral contrast.     

Leaving patches: An investigation of a laboratory analogue

Five pigeons were trained on a procedure that has been used as a laboratory analogue to natural patch residence. Trials commenced with two responses available. One of these might provide a reinforcer if the patch was a prey patch; the other ended the residence time in the patch and, after a fixed travel time in blackout, produced another patch that might or might not provide a reinforcer. Patch residence also ended, and was followed by the same travel time, after a reinforcer was obtained or after a fixed maximum time was spent in the patch. The dependent variable was patch residence time, from the commencement of the patch to the time at which the subject emitted a response to exit from the patch or until the maximum patch residence time had elapsed. In Parts 1 to 3, the duration of the imposed travel time was varied from 0.25 to 16 s at three different probabilities (.05, .1, and .2) of food per second (λ) in prey patches. As reported in previous research, both increasing travel time and decreasing probabilities of reinforcers per second increased patch residence time. In Parts 4 to 7, the probability of prey trials (ρ) was varied in an irregular order from .1, through .2, .5, and .7, to .9 for different combinations of λ and travel time. Respectively, these were in Part 4, .05 per second and 0.25 s; in Part 5, .05 per second and 16 s; in Part 6, .2 per second and 0.25 s; and in Part 7, .2 per second and 16 s. A previously offered model, based on optimization assumptions, substantially and consistently underpredicted patch residence time. However, a modification of that model, which assumes that the subjects could not accurately discriminate the residence time that provided the minimum interreinforcer interval, described the data well. The same model also described previously reported residence times in a different species with a uniform distribution of prey-arrival times.     

Response-independent food delivery and behavioral resistance to change

Response-independent food was delivered during a dark-key phase between two multiple-schedule components to explore its disruptive effects on responding. Responding in components was maintained by separate variable-interval 120-s schedules, with a 2-s reinforcer in Component 1 and a 6-s reinforcer in Component 2. Across conditions the rate and duration of response-independent food presentations were manipulated. The results showed that response rates in both components decreased as a function of the duration and the rate of response-independent food presentations; moreover, the decrease in response rate relative to the baseline level was larger in Component 1 than in Component 2. These findings were consistent with expectations from behavioral momentum theory, which predicts that if equal disruption (response-independent food in this case) is applied to responding in two components, then the ratio of response-rate change in Component 1 versus Component 2 should remain constant, irrespective of the magnitude of that disruption.     

Resistance to reinforcement change in multiple and concurrent schedules assessed in transition and at steady state

Behavioral momentum theory relates resistance to change of responding in a multiple-schedule component to the total reinforcement obtained in that component, regardless of how the reinforcers are produced. Four pigeons responded in a series of multiple-schedule conditions in which a variable-interval 40-s schedule arranged reinforcers for pecking in one component and a variable-interval 360-s schedule arranged them in the other. In addition, responses on a second key were reinforced according to variable-interval schedules that were equal in the two components. In different parts of the experiment, responding was disrupted by changing the rate of reinforcement on the second key or by delivering response-independent food during a blackout separating the two components. Consistent with momentum theory, responding on the first key in Part 1 changed more in the component with the lower reinforcement total when it was disrupted by changes in the rate of reinforcement on the second key. However, responding on the second key changed more in the component with the higher reinforcement total. In Parts 2 and 3, responding was disrupted with free food presented during intercomponent blackouts, with extinction (Part 2) or variable-interval 80-s reinforcement (Part 3) arranged on the second key. Here, resistance to change was greater for the component with greater overall reinforcement. Failures of momentum theory to predict short-term differences in resistance to change occurred with disruptors that caused greater change between steady states for the richer component. Consistency of effects across disruptors may yet be found if short-term effects of disruptors are assessed relative to the extent of change observed after prolonged exposure.     

Behavior of rats under fixed consecutive number schedules: effects of drugs of abuse.

Four rats responded under a simple fixed consecutive number schedule in which eight or more consecutive responses on the run lever, followed by a single response on the reinforcement lever, produced the food reinforcer. Under this simple schedule, dose-response curves were determined for diazepam, morphine, pentobarbital, and phencyclidine. The rats were then trained to respond under a multiple fixed consecutive number schedule in which a discriminative stimulus signaled when the response requirement on the run lever had been completed in one of the two fixed consecutive number component schedules. Under control conditions, the percentage of reinforced runs under the multiple-schedule component with the discriminative stimulus added was much higher than the percentage of reinforced runs under the multiple-schedule component without the discriminative stimulus. All of the drugs decreased the percentage of reinforced runs under each of the fixed consecutive number schedules by increasing the conditional probability of short run lengths. This effect was most consistently produced by morphine. The drugs produced few differences in responding between the multiple fixed consecutive number components. Responding under the simple fixed consecutive number schedule, however, was affected at lower doses of the drugs than was responding under the same fixed consecutive number schedule when it was a component of the multiple schedule. This result may be due to the difference in schedule context or, perhaps, to the order of the experiments.     

Response strength in extreme multiple schedules

Four pigeons were trained in a series of two-component multiple schedules. Reinforcers were scheduled with random-interval schedules. The ratio of arranged reinforcer rates in the two components was varied over 4 log units, a much wider range than previously studied. When performance appeared stable, prefeeding tests were conducted to assess resistance to change. Contrary to the generalized matching law, logarithms of response ratios in the two components were not a linear function of log reinforcer ratios, implying a failure of parameter invariance. Over a 2 log unit range, the function appeared linear and indicated undermatching, but in conditions with more extreme reinforcer ratios, approximate matching was observed. A model suggested by McLean (1991), originally for local contrast, predicts these changes in sensitivity to reinforcer ratios somewhat better than models by Herrnstein (1970) and by Williams and Wixted (1986). Prefeeding tests of resistance to change were conducted at each reinforcer ratio, and relative resistance to change was also a nonlinear function of log reinforcer ratios, again contrary to conclusions from previous work. Instead, the function suggests that resistance to change in a component may be determined partly by the rate of reinforcement and partly by the ratio of reinforcers to responses.     

The effects of number of sample stimuli and number of choices in a detection task on measures of discriminability

Six pigeons were trained on a conditional discrimination task involving the discrimination of various intensities of yellow light. The research asked whether stimulus—response discriminability measures between any pair of stimuli would remain constant when a third or fourth sample and reinforced response were added. The numbers of different sample stimuli presented and different responses reinforced were two (Part 1), three (Parts 2 and 4), and four (Part 3). Across conditions within parts, the ratios of reinforcers obtainable for correct responses were varied over at least five levels. In Part 5, the numbers of sample stimuli and reinforced responses were varied among two, three, and four, and the reinforcer ratio between consecutive remaining samples was constant at 2:1. It was found that once a particular response had been reinforced, subjects continued to emit that response when the conditional stimulus for that response was no longer presented. Data analysis using a generalization-based detection model indicated that this model was able to describe the data effectively. Four findings were in accord with the theory. First, estimates of stimulus—response discriminability usually decreased as the arranged physical disparity between the sample stimuli decreased. Second, stimulus—response discriminability measures were independent of response—reinforcer discriminability measures, preserving parameter invariance between these measures. Third, stimulus—response discriminability measures for constant pairs of conditional stimuli did not change systematically as conditional stimulus—response alternatives were added. Fourth, log stimulus—response discriminability values between physically adjacent conditional stimuli summed to values that were not significantly different from estimates of the discriminability values for conditional stimuli that were spaced further apart.     

Stimulus and reinforcer relativity in multiple schedules: Local and dimensional effects on sensitivity to reinforcement

Pigeons' responses in two successive components of multiple schedules were reinforced according to variable-interval schedules of reinforcement that varied over five different conditions. Within each session of all conditions, line orientations of 0°, 30°, or 45° in Component 1 alternated with orientations of 45°, 60°, or 90° in Component 2. Response rates were recorded in three successive subintervals of each component. Ratios were taken between the response rate in each Component 1 line orientation and the response rate in each Component 2 orientation. These ratios were found to be power functions of the corresponding ratios of obtained reinforcement rates. Sensitivity of response ratios to changes in reinforcer ratios, given by the value of the exponent of the power function, increased systematically with increasing disparity between the dimensional values of orientation stimuli. In addition, sensitivity decreased systematically over successive subintervals of components, that is, with increasing time since component alternation. Dimensional and local (subinterval) effects interacted in that sensitivity increased with stimulus disparity to a far greater extent in the first subinterval than later in components. The data could be described by a combination of rectangular hyperbolae which attributed the interaction between local and dimensional effects to limits set by local effects on the extent that stimulus differences could affect sensitivity.     

Every reinforcer counts: reinforcer magnitude and local preference

Six pigeons were trained on concurrent variable-interval schedules. Sessions consisted of seven components, each lasting 10 reinforcers, with the conditions of reinforcement differing between components. The component sequence was randomly selected without replacement. In Experiment 1, the concurrent-schedule reinforcer ratios in components were all equal to 1.0, but across components reinforcer-magnitude ratios varied from 1:7 through 7:1. Three different overall reinforcer rates were arranged across conditions. In Experiment 2, the reinforcer-rate ratios varied across components from 27:1 to 1:27, and the reinforcer-magnitude ratios for each alternative were changed across conditions from 1:7 to 7:1. The results of Experiment 1 replicated the results for changing reinforcer-rate ratios across components reported by Davison and Baum (2000, 2002): Sensitivity to reinforcer-magnitude ratios increased with increasing numbers of reinforcers in components. Sensitivity to magnitude ratio, however, fell short of sensitivity to reinforcer-rate ratio. The degree of carryover from component to component depended on the reinforcer rate. Larger reinforcers produced larger and longer postreinforcer preference pulses than did smaller reinforcers. Similar results were found in Experiment 2, except that sensitivity to reinforcer magnitude was considerably higher and was greater for magnitudes that differed more from one another. Visit durations following reinforcers measured either as number of responses emitted or time spent responding before a changeover were longer following larger than following smaller reinforcers, and were longer following sequences of same reinforcers than following other sequences. The results add to the growing body of research that informs model building at local levels.     

Variable-ratio versus variable-interval schedules: response rate, resistance to change, and preference

Two experiments asked whether resistance to change depended on variable-ratio as opposed to variable-interval contingencies of reinforcement and the different response rates they establish. In Experiment 1, pigeons were trained on multiple random-ratio random-interval schedules with equated reinforcer rates. Baseline response rates were disrupted by intercomponent food, extinction, and prefeeding. Resistance to change relative to baseline was greater in the interval component, and the difference was correlated with the extent to which baseline response rates were higher in the ratio component. In Experiment 2, pigeons were trained on multiple variable-ratio variable-interval schedules in one half of each session and on concurrent chains in the other half in which the terminal links corresponded to the multiple-schedule components. The schedules were varied over six conditions, including two with equated reinforcer rates. In concurrent chains, preference strongly overmatched the ratio of obtained reinforcer rates. In multiple schedules, relative resistance to response-independent food during intercomponent intervals, extinction, and intercomponent food plus extinction depended on the ratio of obtained reinforcer rates but was less sensitive than was preference. When reinforcer rates were similar, both preference and relative resistance were greater for the variable-interval schedule, and the differences were correlated with the extent to which baseline response rates were higher on the variable-ratio schedule, confirming the results of Experiment 1. These results demonstrate that resistance to change and preference depend in part on response rate as well as obtained reinforcer rate, and challenge the independence of resistance to change and preference with respect to response rate proposed by behavioral momentum theory.