Concurrent schedules: Interaction of reinforcer frequency and reinforcer duration

Six pigeons were trained on concurrent variable-interval schedules with unequal reinforcer durations for the two responses. The schedules arranged on the two keys were kept equal while they were varied in absolute size. As the overall reinforcer rate was increased, both response-allocation and time-allocation measures of choice showed a trend toward indifference, and measures of sensitivity to reinforcer-duration ratios significantly decreased. Recent reports have shown that the generalized matching law cannot describe the changes in behavior allocation under constant delay-, duration-, or rate-ratios when changes are made in the absolute levels of each of these variables. The present results complement these findings by demonstrating that the concatenated generalized matching law cannot describe the interactions of two reinforcer variables on behavior allocation.     

Arousal, changeover responses, and preference in concurrent schedules

Pigeons were trained on multiple schedules that provided concurrent reinforcement in each of two components. In Experiment 1, one component consisted of a variable-interval (VI) 40-s schedule presented with a VI 20-s schedule, and the other a VI 40-s schedule presented with a VI 80-s schedule. After extended training, probe tests measured preference between the stimuli associated with the two 40-s schedules. Probe tests replicated the results of Belke (1992) that showed preference for the 40-s schedule that had been paired with the 80-s schedule. In a second condition, the overall reinforcer rate provided by the two components was equated by adding a signaled VI schedule to the component with the lower reinforcer rate. Probe results were unchanged. In Experiment 2, pigeons were trained on alternating concurrent VI 30-s VI 60-s schedules. One schedule provided 2-s access to food and the other provided 6-s access. The larger reinforcer magnitude produced higher response rates and was preferred on probe trials. Rate of changeover responding, however, did not differ as a function of reinforcer magnitude. The present results demonstrate that preference on probe trials is not a simple reflection of the pattern of changeover behavior established during training.     

Choice, foraging, and reinforcer duration.

Pigeons were exposed to a foraging schedule characterized by three different states, beginning with a search state in which completion of a variable interval on a white key led to a choice state. In the choice state the subject could, by appropriate responding on a fixed ratio of three, either accept or reject the schedule offered. If the subject accepted the schedule, it entered a handling state in which the appropriate reinforcer amount was presented according to a variable-interval schedule. In Experiment 1 the shorter duration reinforcer was more likely to be accepted the longer the duration of the search state and the shorter the equal durations of the handling states. In Experiment 2 the shorter duration reinforcer was more likely to be accepted the longer the handling time preceding the longer duration reinforcer. All of the results were in qualitative--and some were in quantitative--agreement with those predicted by the delay-reduction hypothesis and the optimal-diet model.     

Choice in a variable environment: effects of unequal reinforcer distributions

Six pigeons were trained in a procedure in which sessions included seven unsignaled components, each offering two pecking keys, and each providing a potentially different reinforcer ratio between the two keys. Across conditions, various combinations of reinforcer ratios and reinforcer-magnitude ratios were used to create unequal reinforcer distributions between the two alternatives when averaged across a session. The results extended previous research using the same basic procedure that had included only reinforcer distributions symmetrical around 1:1. Data analyses suggested that the variables controlling choice operated at a number of levels: First, individual reinforcers had local effects on choice; second, sequences of successive reinforcers obtained at the same alternative (continuations) had cumulative effects; and, third, when these sequences themselves occurred with greater frequency, their effects further cumulated. A reinforcer obtained at the other alternative following a sequence of continuations (a discontinuation) had a large effect and apparently reset choice to levels approximating the sessional reinforcer ratio.     

Effects of reinforcer duration on responding in two-link chained interval schedules

Each of five pigeons was exposed to two or more durations of access to mixed grains on two-link, chained, interval schedules in which both links were identical fixed-interval or variable-interval schedules. Response rates were an increasing function of reinforcer duration for both initial and terminal links. For both types of interval schedules, initial-link response rates were more sensitive to reinforcer duration than were terminal-link response rates. The present results, together with prior ones, suggest that chaining and choice procedures are each sufficient for demonstrating substantial sensitivity of responding to changes in reinforcer duration.     

Concurrent schedules: reinforcer magnitude effects

Five pigeons were trained on pairs of concurrent variable-interval schedules in a switching-key procedure. The arranged overall rate of reinforcement was constant in all conditions, and the reinforcer-magnitude ratios obtained from the two alternatives were varied over five levels. Each condition remained in effect for 65 sessions and the last 50 sessions of data from each condition were analyzed. At a molar level of analysis, preference was described well by a version of the generalized matching law, consistent with previous reports. More local analyses showed that recently obtained reinforcers had small measurable effects on current preference, with the most recently obtained reinforcer having a substantially larger effect. Larger reinforcers resulted in larger and longer preference pulses, and a small preference was maintained for the larger-magnitude alternative even after long inter-reinforcer intervals. These results are consistent with the notion that the variables controlling choice have both short- and long-term effects. Moreover, they suggest that control by reinforcer magnitude is exerted in a manner similar to control by reinforcer frequency. Lower sensitivities when reinforcer magnitude is varied are likely to be due to equal frequencies of different sized preference pulses, whereas higher sensitivities when reinforcer rates are varied might result from changes in the frequencies of different sized preference pulses.     

Effects of varying stimulus disparity and the reinforcer ratio in concurrent-schedule and signal-detection procedures.

The present study measured the effects of stimulus and reinforcer variations on pigeons' behavior in two different choice procedures. Two intensities of white light were presented as the stimuli on the main key in a switching-key concurrent schedule and as the sample stimuli in a signal-detection procedure. Under both procedures, the scheduled rate of reinforcement was varied across conditions to produce various ratios of obtained reinforcement. These ratios were obtained for seven pairs of light intensities. In the concurrent schedules, the effects of reinforcer-ratio variations were positively correlated with the physical disparity between the two light intensities. In the signal-detection procedure, changes in the reinforcer ratio produced greater effects on performance when stimulus disparity was very low or very high compared to those found at intermediate levels of stimulus disparity. This discrepancy creates a dilemma for existing behavioral models of signal-detection performance.     

Concurrent-schedule performance in transition: changeover delays and signaled reinforcer ratios

Six pigeons were trained in experimental sessions that arranged six or seven components with various concurrent-schedule reinforcer ratios associated with each. The order of the components was determined randomly without replacement. Components lasted until the pigeons had received 10 reinforcers, and were separated by 10-s blackout periods. The component reinforcer ratios arranged in most conditions were 27:1, 9:1, 3:1, 1:1, 1:3, 1:9 and 1:27; in others, there were only six components, three of 27:1 and three of 1:27. In some conditions, each reinforcement ratio was signaled by a different red-yellow flash frequency, with the frequency perfectly correlated with the reinforcer ratio. Additionally, a changeover delay was arranged in some conditions, and no changeover delay in others. When component reinforcer ratios were signaled, sensitivity to reinforcement values increased from around 0.40 before the first reinforcer in a component to around 0.80 before the 10th reinforcer. When reinforcer ratios were not signaled, sensitivities typically increased from zero to around 0.40. Sensitivity to reinforcement was around 0.20 lower in no-changeover-delay conditions than in changeover-delay conditions, but increased in the former after exposure to changeover delays. Local analyses showed that preference was extreme towards the reinforced alternative for the first 25 s after reinforcement in changeover-delay conditions regardless of whether components were signaled or not. In no-changeover-delay conditions, preference following reinforcers was either absent, or, following exposure to changeover delays, small. Reinforcers have both local and long-term effects on preference. The former, but not the latter, is strongly affected by the presence of a changeover delay. Stimulus control may be more closely associated with longer-term, more molar, reinforcer effects.     

Relative reinforcer rates and magnitudes do not control concurrent choice independently

One assumption of the matching approach to choice is that different independent variables control choice independently of each other. We tested this assumption for reinforcer rate and magnitude in an extensive parametric experiment. Five pigeons responded for food reinforcement on switching-key concurrent variable-interval variable-interval schedules. Across conditions, the ratios of reinforcer rates and of reinforcer magnitudes on the two alternatives were both manipulated. Control by each independent variable, as measured by generalized-matching sensitivity, changed significantly with the ratio of the other independent variable. Analyses taking the model-comparison approach, which weighs improvement in goodness-of-fit against increasing number of free parameters, were inconclusive. These analyses compared a model assuming constant sensitivity to magnitude across all reinforcer-rate ratios with two alternative models. One of those alternatives allowed sensitivity to magnitude to vary freely across reinforcer-rate ratios, and was less efficient than the common-sensitivity model for all pigeons, according to the Schwarz-Bayes information criterion. The second alternative model constrained sensitivity to magnitude to be equal for pairs of reinforcer-rate ratios that deviated from unity by proportionately equal amounts but in opposite directions. This model was more efficient than the common-magnitude-sensitivity model for 2 of the pigeons, but not for the other 3. An analysis of variance, carried out independently of the generalized-matching analysis, also showed a significant interaction between the effects of reinforcer rate and reinforcer magnitude on choice. On balance, these results suggest that the assumption of independence inherent in the matching approach cannot be maintained. Relative reinforcer rates and magnitudes do not control choice independently.     

Sensitivity to relative reinforcer rate in concurrent schedules: independence from relative and absolute reinforcer duration

Twelve pigeons responded on two keys under concurrent variable-interval (VI) schedules. Over several series of conditions, relative and absolute magnitudes of reinforcement were varied. Within each series, relative rate of reinforcement was varied and sensitivity of behavior ratios to reinforcer-rate ratios was assessed. When responding at both alternatives was maintained by equal-sized small reinforcers, sensitivity to variation in reinforcer-rate ratios was the same as when large reinforcers were used. This result was observed when the overall rate of reinforcement was constant over conditions, and also in another series of concurrent schedules in which one schedule was kept constant at VI ached 120 s. Similarly, reinforcer magnitude did not affect the rate at which response allocation approached asymptote within a condition. When reinforcer magnitudes differred between the two responses and reinforcer-rate ratios were varied, sensitivity of behavior allocation was unaffected although response bias favored the schedule that arranged the larger reinforcers. Analysis of absolute response rates ratio sensitivity to reinforcement occurrred on the two keys showed that this invariance of response despite changes in reinforcement interaction that were observed in absolute response rates on the constant VI 120-s schedule. Response rate on the constant VI 120-s schedule was inversely related to reinforcer rate on the varied key and the strength of this relation depended on the relative magnitude of reinforcers arranged on varied key. Independence of sensitivity to reinforcer-rate ratios from relative and absolute reinforcer magnitude is consistent with the relativity and independence assumtions of the matching law.     

Preference after training with differential changeover delays

Pigeons were trained on a multiple schedule in which each component consisted of concurrent variable-interval (VI) 30-s VI 60-s schedules. The two components of the multiple schedule differed only in terms of the changeover delays (COD): For one component short CODs were employed, and in the second component long CODs were used. After approximate matching was obtained in each component, probe tests involving new combinations of stimuli were presented (e.g., the VI 30-s schedule from each component) to determine how the different CODs affected preference. Despite shorter CODs producing higher changeover rates, the COD value had no systematic effect on preference on the probe trials. However, differences in reinforcement rate always produced preference for the schedule with the higher reinforcement rate. The results thus show that the the pattern of changeover behavior per se is not a critical determinant of choice in the probe-trial procedure.     

Choice, changing over, and reinforcement delays

In three experiments, pigeons were used to examine the independent effects of two normally confounded delays to reinforcement associated with changing between concurrently available variable-interval schedules of reinforcement. In Experiments 1 and 2, combinations of changeover-delay durations and fixed-interval travel requirements were arranged in a changeover-key procedure. The delay from a changeover-produced stimulus change to a reinforcer was varied while the delay between the last response on one alternative and a reinforcer on the other (the total obtained delay) was held constant. Changeover rates decreased as a negative power function of the total obtained delay. The delay between a changeover-produced stimulus change had a small and inconsistent effect on changeover rates. In Experiment 3, changeover delays and fixed-interval travel requirements were arranged independently. Changeover rates decreased as a negative power function of the total obtained delay despite variations in the delay from a change in stimulus conditions to a reinforcer. Periods of high-rate responding following a changeover, however, were higher near the end of the delay from a change in stimulus conditions to a reinforcer. The results of these experiments suggest that the effects of changeover delays and travel requirements primarily result from changes in the delay between a response at one alternative and a reinforcer at the other, but the pattern of responding immediately after a changeover depends on the delay from a changeover-produced change in stimulus conditions to a reinforcer.     

On the functions of the changeover delay

The function of changeover delays in producing matching was examined with pigeons responding on concurrent variable-interval variable-interval schedules. In Experiment 1, no changeover delay was compared to two different types of changeover delay. One type, designated generically as response-response but in the present example as peck-peck, was timed from the first response on the switched-to key; the other, designated generically as pause-response but in the present example as pause-peck, was timed from the last response on the switched-from key. High changeover rates occurred with no changeover delay. Peck-peck and pause-peck changeover delays produced low and intermediate changeover rates, respectively. In Experiment 2, pause-peck and peck-peck changeover delays were compared across a range of relative reinforcement rates. Similar matching relations developed despite differences in the changeover rates and local response patterns as a function of the type of changeover delay. In Experiment 3, both types of changeover delay yielded similar changeover rates when their obtained durations were equal via yoking. The results suggest that changeover delays function to separate responses on one key from reinforcers on the other or to delay reinforcement for changing over. In addition, the distribution of responding during and after the changeover delay may vary considerably without affecting matching.     

Local preference in concurrent schedules: the effects of reinforcer sequences

We investigated the effects that sequences of reinforcers obtained from the same response key have on local preference in concurrent variable-interval schedules with pigeons as subjects. With an overall reinforcer rate of one every 27 s, on average, reinforcers were scheduled dependently, and the probability that a reinforcer would be arranged on the same alternative as the previous reinforcer was manipulated. Throughout the experiment, the overall reinforcer ratio was 1:1, but across conditions we varied the average lengths of same-key reinforcer sequences by varying this conditional probability from 0 to 1. Thus, in some conditions, reinforcer locations changed frequently, whereas in others there tended to be very long sequences of same-key reinforcers. Although there was a general tendency to stay at the just-reinforced alternative, this tendency was considerably decreased in conditions where same-key reinforcer sequences were short. Some effects of reinforcers are at least partly to be accounted for by their signaling subsequent reinforcer locations.     

Choice, experience, and the generalized matching law

Five pigeons were exposed to different pairs of concurrent variable-interval, variable-interval schedules on nine experimental conditions of 30 sessions each. For every session, the parameters of the generalized matching equation were computed for the first five, six, seven, eight, and nine experimental conditions. The exponent a, both for response and time distribution, tended to decrease with increases in number of experimental conditions and to increase with number of sessions per condition, but values of k (bias) varied unsystematically. When the subjects were exposed to five new pairs of schedules, with 55 sessions per condition, the findings were confirmed. Data from the literature on the generalized matching law suggest that the variability of exponent values may be explained in part by the use of naive or experienced subjects in different investigations and by the variability in number of experimental conditions and in number of sessions per condition.     

Choice in a variable environment: every reinforcer counts

Six pigeons were trained in sessions composed of seven components, each arranged with a different concurrent-schedule reinforcer ratio. These components occurred in an irregular order with equal frequency, separated by 10-s blackouts. No signals differentiated the different reinforcer ratios. Conditions lasted 50 sessions, and data were collected from the last 35 sessions. In Part 1, the arranged overall reinforcer rate was 2.22 reinforcers per minute. Over conditions, number of reinforcers per component was varied from 4 to 12. In Part 2, the overall reinforcer rate was six per minute, with both 4 and 12 reinforcers per component. Within components, log response-allocation ratios adjusted rapidly as more reinforcers were delivered in the component, and the slope of the choice relation (sensitivity) leveled off at moderately high levels after only about eight reinforcers. When the carryover from previous components was taken into account, the number of reinforcers in the components appeared to have no systematic effect on the speed at which behavior changed after a component started. Consequently, sensitivity values at each reinforcer delivery were superimposable. However, adjustment to changing reinforcer ratios was faster, and reached greater sensitivity values, when overall reinforcer rate was higher. Within a component, each successive reinforcer from the same alternative ("confirming") had a smaller effect than the one before, but single reinforcers from the other alternative ("disconfirming") always had a large effect. Choice in the prior component carried over into the next component, and its effects could be discerned even after five or six reinforcement and nonreinforcement is suggested.     

Choice dynamics in concurrent ratio schedules of reinforcement

The generalized matching law predicts performance on concurrent schedules when variable-interval schedules are programmed but is trivially applicable when independent ratio schedules are used. Responding usually is exclusive to the schedule with the lowest response requirement. Determining a method to program concurrent ratio schedules such that matching analyses can be usefully employed would extend the generality of matching research and lead to new avenues of research. In the present experiments, ratio schedules were programmed dependently such that responses to either of the two options progressed the requirement on both schedules. Responding is not exclusive because the probability of reinforcement increases on both schedules as responses are allocated to either schedule. In Experiment 1, performance on concurrent variable-ratio schedules was assessed, and reinforcer ratios were varied across conditions to investigate changes in sensitivity. Additionally, the length of a changeover delay was manipulated. In Experiment 2, performance was compared under concurrently available, dependently programmed variable-ratio and fixed-ratio schedules. Performance was well described by the generalized matching law. Increases in the changeover delay decreased sensitivity, whereas sensitivity was higher when variable-ratio schedules were employed, compared with fixed-ratio schedules. Concurrent ratio schedules can be a viable approach to studying functional differences between ratio and interval schedules.     

Sensitivity to reinforcer duration in a self-control procedure

In a concurrent-chains procedure, pigeons' responses on left and right keys were followed by reinforcers of different durations at different delays following the choice responses. Three pairs of reinforcer delays were arranged in each session, and reinforcer durations were varied over conditions. In Experiment 1 reinforcer delays were unequal, and in Experiment 2 reinforcer delays were equal. In Experiment 1 preference reversal was demonstrated in that an immediate short reinforcer was chosen more frequently than a longer reinforcer delayed 6 s from the choice, whereas the longer reinforcer was chosen more frequently when delays to both reinforcers were lengthened. In both experiments, choice responding was more sensitive to variations in reinforcer duration at overall longer reinforcer delays than at overall shorter reinforcer delays, independently of whether fixed-interval or variable-interval schedules were arranged in the choice phase. We concluded that preference reversal results from a change in sensitivity of choice responding to ratios of reinforcer duration as the delays to both reinforcers are lengthened.