Arousal, changeover responses, and preference in concurrent schedules

Pigeons were trained on multiple schedules that provided concurrent reinforcement in each of two components. In Experiment 1, one component consisted of a variable-interval (VI) 40-s schedule presented with a VI 20-s schedule, and the other a VI 40-s schedule presented with a VI 80-s schedule. After extended training, probe tests measured preference between the stimuli associated with the two 40-s schedules. Probe tests replicated the results of Belke (1992) that showed preference for the 40-s schedule that had been paired with the 80-s schedule. In a second condition, the overall reinforcer rate provided by the two components was equated by adding a signaled VI schedule to the component with the lower reinforcer rate. Probe results were unchanged. In Experiment 2, pigeons were trained on alternating concurrent VI 30-s VI 60-s schedules. One schedule provided 2-s access to food and the other provided 6-s access. The larger reinforcer magnitude produced higher response rates and was preferred on probe trials. Rate of changeover responding, however, did not differ as a function of reinforcer magnitude. The present results demonstrate that preference on probe trials is not a simple reflection of the pattern of changeover behavior established during training.     

Comparing preference and resistance to change in constant- and variable-duration schedule components

Two experiments explored preference and resistance to change in concurrent chains in which the terminal links were variable-interval schedules that ended either after a single reinforcer had been delivered (variable duration) or after a fixed period of access to the schedule (constant duration). In Experiment 1, pigeons' preference between the same pair of terminal links overmatched relative reinforcement rate when the terminal links were of constant duration, but not when they were of variable duration. Responding during the richer terminal link decreased less, relative to baseline, when response-independent food was presented during the initial links according to a variable-time schedule. In Experiment 2, all subjects consistently preferred a terminal link that consisted of 20-s access to a variable-interval 20-s schedule over a terminal link that ended after one reinforcer had been delivered by the same schedule. Results of resistance-to-change tests corresponded to preference, as responding during the constant-duration terminal link decreased less, relative to baseline, when disrupted by both response-independent food during the initial links and prefeeding. Overall, these data extend the general covariation of preference and resistance to change seen in previous studies. However, they suggest that reinforcement numerosity, including variability in the number of reinforcers per terminal-link entry, may sometimes affect preference and resistance to change in ways that are difficult to explain in terms of current models.     

Variable-ratio versus variable-interval schedules: response rate, resistance to change, and preference

Two experiments asked whether resistance to change depended on variable-ratio as opposed to variable-interval contingencies of reinforcement and the different response rates they establish. In Experiment 1, pigeons were trained on multiple random-ratio random-interval schedules with equated reinforcer rates. Baseline response rates were disrupted by intercomponent food, extinction, and prefeeding. Resistance to change relative to baseline was greater in the interval component, and the difference was correlated with the extent to which baseline response rates were higher in the ratio component. In Experiment 2, pigeons were trained on multiple variable-ratio variable-interval schedules in one half of each session and on concurrent chains in the other half in which the terminal links corresponded to the multiple-schedule components. The schedules were varied over six conditions, including two with equated reinforcer rates. In concurrent chains, preference strongly overmatched the ratio of obtained reinforcer rates. In multiple schedules, relative resistance to response-independent food during intercomponent intervals, extinction, and intercomponent food plus extinction depended on the ratio of obtained reinforcer rates but was less sensitive than was preference. When reinforcer rates were similar, both preference and relative resistance were greater for the variable-interval schedule, and the differences were correlated with the extent to which baseline response rates were higher on the variable-ratio schedule, confirming the results of Experiment 1. These results demonstrate that resistance to change and preference depend in part on response rate as well as obtained reinforcer rate, and challenge the independence of resistance to change and preference with respect to response rate proposed by behavioral momentum theory.     

Multiple determinants of the effects of reinforcement magnitude on free-operant response rates

Four experiments examined the effects of increasing the number of food pellets given to hungry rats for a lever-press response. On a simple variable-interval 60-s schedule, increased number of pellets depressed response rates (Experiment 1). In Experiment 2, the decrease in response rate as a function of increased reinforcement magnitude was demonstrated on a variable-interval 30-s schedule, but enhanced rates of response were obtained with the same increase in reinforcement magnitude on a variable-ratio 30 schedule. In Experiment 3, higher rates of responding were maintained by the component of a concurrent variable-interval 60-s variable-interval 60-s schedule associated with a higher reinforcement magnitude. In Experiment 4, higher rates of response were produced in the component of a multiple variable-interval 60-s variable-interval 60-s schedule associated with the higher reinforcement magnitude. It is suggested that on simple schedules greater reinforcer magnitudes shape the reinforced pattern of responding more effectively than do smaller reinforcement magnitudes. This effect is, however, overridden by another process, such a contrast, when two magnitudes are presented within a single session on two-component schedules.     

Income maximizing on concurrent ratio-interval schedules of reinforcement

Three experiments examined the effect of food availability on pigeons' choice behavior under concurrent schedules of reinforcement. In Experiment 1, 3 pigeons earned their daily food ration by choosing, in 30-min sessions, between concurrent variable-ratio 30 variable-interval 40-s schedules. Food presentations during both schedules lasted 2 or 12 s, depending upon the condition. Relative variable-ratio response rate was inversely related to hopper duration. In Experiment 2, 4 pigeons received their daily feeding by responding on the same schedule pair as in Experiment 1 (with 4-s food presentations) in sessions that varied in length from 10 to 30 min, depending on the condition. The length of a vertical slit projected on a response key increased with time so that “passage of time” might be more easily discriminable. As session duration decreased, relative variable-ratio response rate increased. In Experiment 3, 4 pigeons chose between two variable-interval 40-s schedules. One schedule operated without regard to the schedule selected, whereas the other operated only when the subject responded in its presence (dependent). Although these schedules had the same feedback function, preference for the dependent variable interval increased as session duration decreased from 30 to 10 min. The preference changes in these studies reveal the operation of an income-maximizing process in choice.     

Matching under nonindependent variable-ratio schedules of drug reinforcement.

Response-contingent deliveries of oral pentobarbital maintained responding of 3 rhesus monkeys during daily 3-hr sessions. Deliveries of pentobarbital were arranged under nonindependent concurrent variable-ratio variable-ratio schedules. Responses to either schedule counted toward completion of both variable-ratio schedule requirements. This schedule is similar in some respects to conventional concurrent variable-interval variable-interval schedules, in which passage of time counts toward completion of the interval value on both schedules. Restricted nonindependent concurrent variable-ratio variable-ratio schedules were also studied. On that schedule, when a drug delivery was assigned to one spout, it had to be collected before responses on the opposite spout again counted toward completion of the schedule requirements. Relative reinforcer magnitude was varied by changing the drug concentration on one schedule while keeping the drug concentration constant on the other variable-ratio schedule. Under both types of concurrent variable-ratio schedules, the relative rate of responding corresponded to the relative drug intake. Unlike earlier studies of concurrent variable-interval variable-interval intravenous cocaine reinforcement, preference was proportionate to concentration, and exclusive preferences did not develop. The relationship between relative rate of responding and relative drug intake was well described by the generalized matching law.     

Sensitivity to relative reinforcer rate in concurrent schedules: independence from relative and absolute reinforcer duration

Twelve pigeons responded on two keys under concurrent variable-interval (VI) schedules. Over several series of conditions, relative and absolute magnitudes of reinforcement were varied. Within each series, relative rate of reinforcement was varied and sensitivity of behavior ratios to reinforcer-rate ratios was assessed. When responding at both alternatives was maintained by equal-sized small reinforcers, sensitivity to variation in reinforcer-rate ratios was the same as when large reinforcers were used. This result was observed when the overall rate of reinforcement was constant over conditions, and also in another series of concurrent schedules in which one schedule was kept constant at VI ached 120 s. Similarly, reinforcer magnitude did not affect the rate at which response allocation approached asymptote within a condition. When reinforcer magnitudes differred between the two responses and reinforcer-rate ratios were varied, sensitivity of behavior allocation was unaffected although response bias favored the schedule that arranged the larger reinforcers. Analysis of absolute response rates ratio sensitivity to reinforcement occurrred on the two keys showed that this invariance of response despite changes in reinforcement interaction that were observed in absolute response rates on the constant VI 120-s schedule. Response rate on the constant VI 120-s schedule was inversely related to reinforcer rate on the varied key and the strength of this relation depended on the relative magnitude of reinforcers arranged on varied key. Independence of sensitivity to reinforcer-rate ratios from relative and absolute reinforcer magnitude is consistent with the relativity and independence assumtions of the matching law.     

DOES SENSITIVITY TO MAGNITUDE DEPEND ON THE TEMPORAL DISTRIBUTION OF REINFORCEMENT?

Our research addressed the question of whether sensitivity to relative reinforcer magnitude in concurrent chains depends on the distribution of reinforcer delays when the terminal-link schedules are equal. In Experiment 1, 12 pigeons responded in a two-component procedure. In both components, the initial links were concurrent variable-interval 40-s variable-interval 40-s, and the terminal links were both 20-s interval schedules in which responses were reinforced by either 4-s of grain in one, or 2-s of grain in the other. The only difference between the components was whether the terminal-link schedules were fixed interval or variable intervals. For all subjects, the relative rate of responding in the initial links for the terminal link that produced the 4-s reinforcer was greater when the terminal links were fixed-interval schedules than when they were variable-interval schedules. This result is contrary to the prediction of Grace's (1994) contextual choice model, but is consistent with both Mazur's (2001) hyperbolic value-added model and Killeen's (1985) incentive theory. In Experiment 2, 4 pigeons responded in a concurrent-chains procedure in which 4-s or 2-s reinforcers were provided independently of responding according to equal fixed-time or mixed-time schedules. Preference for the 4-s reinforcer increased as the variability of the intervals comprising the mixed-time schedules was decreased. Generalized-matching sensitivity of initial-link response allocation to relative reinforcer magnitude was proportional to the geometric mean of the terminal-link delays.     

EFFECTS OF PROBLEM DIFFICULTY AND REINFORCER QUALITY ON TIME ALLOCATED TO CONCURRENT ARITHMETIC PROBLEMS

Students with learning difficulties participated in two studies that analyzed the effects of problem difficulty and reinforcer quality upon time allocated to two sets of arithmetic problems reinforced according to a concurrent variable-interval 30-s variable-interval 120-s schedule. In Study 1, high- and low-difficulty arithmetic problems were systematically combined with rich and lean concurrent schedules (nickels used as reinforcers) across conditions using a single-subject design. The pairing of the high-difficulty problems with the richer schedule failed to offset time allocated to that alternative. Study 2 investigated the interactive effects of problem difficulty and reinforcer quality (nickels vs. program money) upon time allocation to arithmetic problems maintained by the concurrent schedules of reinforcement. Unlike problem difficulty, the pairing of the lesser quality reinforcer (program money) with the richer schedule reduced the time allocated to that alternative. The magnitude of this effect was greatest when combined with the low-difficulty problems. These studies have important implications for a matching law analysis of asymmetrical reinforcement variables that influence time allocation.     

Effects of intertrial reinforcers on self-control choice.

In three experiments, pigeons chose between a small amount of food delivered after a short delay and a larger amount delivered after a longer delay. A discrete-trial adjusting-delay procedure was used to estimate indifference points--pairs of delay-amount combinations that were chosen about equally often. In Experiment 1, when additional reinforcers were available during intertrial intervals on a variable-interval schedule, preference for the smaller, more immediate reinforcer increased. Experiment 2 found that this shift in preference occurred partly because the variable-interval schedule started sooner after the smaller, more immediate reinforcer, but there was still a small shift in preference when the durations and temporal locations of the variable-interval schedules were identical for both alternatives. Experiment 3 found greater increases in preference for the smaller, more immediate reinforcer with a variable-interval 15-s schedule than with a variable-interval 90-s schedule. The results were generally consistent with a model that states that the impact of any event that follows a choice response declines according to a hyperbolic function with increasing time since the moment of choice.     

The role of discriminative stimuli in concurrent performances

Key pecking in pigeons was examined under concurrent and parallel arrangements of two independent and simultaneously available variable-interval schedules. Pecks on the changeover key alternated the schedule of reinforcement for responses on the main key. Under concurrent schedules, discriminative stimuli were paired with the reinforcement schedule arranged in each component and changeover responses also alternated these stimuli. Under parallel schedules, changeover responses alternated the effective reinforcement schedule, but did not change the discriminative stimulus. On concurrent procedures, changeover response rate was inversely related to the difference in reinforcement rate between the two components, whereas on parallel schedules no consistent relationship was found. With both schedules, absolute response and reinforcement rates were positively related, although for a given set of reinforcement frequencies, rates were often higher on the concurrent schedules. On concurrent schedules, relative response rates and relative times were equal to relative reinforcement rates. On parallel schedules these ratios were positively related, but response and time ratios were much smaller than were obtained with comparable concurrent schedules. This inequality was most pronounced when absolute reinforcement frequencies were lowest.     

Value of conditioned reinforcers as a function of temporal context

In two experiments, pigeons were trained on a multiple-chain schedule, in which the initial link for one chain was a variable-interval (VI) 100 s schedule and for the other chain a VI 10 s schedule. The terminal links were both fixed-time 30 s schedules signaled by differently colored stimuli. Following training, the pigeons had their preference for the terminal-link stimuli tested either by presenting these stimuli in concurrent probes or by presenting these stimuli as reinforcement for completing novel initial links. In Experiment 1, pigeons significantly preferred the terminal-link stimulus that followed the long initial link in three of the five conditions. This preference was observed across all three testing procedures (concurrent chains, concurrent chains probes, and concurrent probes). Experiment 2 was a replication of this effect in one of the conditions from Experiment 1. The results demonstrate that temporal context does impact the value of a conditioned reinforcer in a manner consistent with delay-reduction theory, and inconsistent with other choice theories, such as the contextual choice model and scalar expectancy theory.     

Choice behavior and the accessibility of the reinforcer

In Experiment 1, matching of relative response rates to relative rates of reinforcement was obtained in concurrent variable-interval schedules when the absolute values of the two concurrent variable-interval schedules varied from 6 sec and 12 sec to 600 sec and 1200 sec. Increases in the duration of the changeover delay, however, produced decreases in the relative response rates and, consequently, some deviation from matching. In Experiment 2, matching of relative response rates to the relative duration of the reinforcer failed to occur when the equal variable-interval schedules arranging access to the two different reinforcer durations (1.5 and 6 sec) were varied in size from concurrent variable-interval 10-sec schedules to concurrent variable-interval 600-sec schedules.     

Transfer tests of stimulus value in concurrent chains

We report two experiments that use transfer tests to investigate whether in concurrent chains the value of a terminal-link stimulus is affected by the alternate terminal link. In Experiment 1, two groups of pigeons were trained on multiple concurrent-chains schedules in which switching between the schedules was via pecking a changeover key. For one group, the terminal links were fixed-interval 8 s versus fixed-interval 16 s in one component and fixed-interval 16 s versus fixed-interval 32 s in the other component. For a second group, the terminal links were variable-interval 10 s versus variable-interval 20 s in one component and variable-interval 20 s versus variable-interval 40 s in the other. After sufficient baseline training had been given so that performances had stabilized, transfer tests were conducted in which the two chains with equal terminal-link schedules were presented together as a new concurrent pair. For 6 of the 7 subjects, initial-link responding changed fairly rapidly during the test in the manner predicted if the values of the terminal links were equal. In Experiment 2, pigeons were trained on multiple concurrent chains using a two-key procedure, and the terminal links were the same variable-interval schedules as in Experiment 1. After baseline training, transfer tests were conducted that assessed (a) the relative reinforcing strength of the terminal-link stimuli in a novel initial-link situation and (b) the relative ability of those stimuli to evoke responding. The data from the reinforcing strength test were consistent with those from Experiment 1, but those from the evocation strength test were not. Although this discrepancy shows that responding in transfer tests is not solely a function of stimulus value, the results from both experiments suggest, overall, that value is determined by the stimulus—reinforcer relation independently of the alternative terminal link.     

Concurrent schedules: Maximization versus reinforcement of changeover behaviour

Pigeons responded on concurrent variable-interval 180-sec variable-interval 36-sec schedules during Conditions 1 and 3 of Experiment 1. Condition 2 arranged variable-interval 60-sec schedules for both response alternatives. The schedule assigned to the alternative that was associated with the variable-interval 36-sec schedule in Conditions 1 and 3 operated only when the subject responded on that alternative. The proportion of time spent responding on the alternative with the conventional variable-interval 60-sec schedule increased during Condition 2, but exclusive choice of that alternative did not develop. This result is inconsistent with maximization of the overall reinforcement rate and with maximization of the momentary probability of reinforcement (momentary maximizing). Increasing time proportions were also found in Experiment 2, which arranged similar conditions, except that reinforcement was provided on a variable-time basis. The time proportions were close to the momentary maximizing prediction in Experiment 2. The results of both experiments can be explained if it is assumed that time allocation is controlled by delayed reinforcement of changeovers between alternatives.     

Effects of reinforcement rate and reinforcer magnitude on choice behavior of humans

Two experiments with human subjects investigated the effects of rate of reinforcement and reinforcer magnitude upon choice. In Experiment 1, each of five subjects responded on four concurrent variable-interval schedules. In contrast to previous studies using non-human organisms, relative response rate did not closely match relative rate of reinforcement. Discrepancies ranged from 0.03 to 0.43 (mean equal to 0.19). Similar discrepancies were found between relative amount of time spent responding on each schedule and the corresponding relative rates of reinforcement. In Experiment 2, in which reinforcer magnitude was varied for each of five subjects, similar discrepancies ranging from 0.05 to 0.50 (mean equal to 0.21), were found between relative response rate and relative proportion of reinforcers received. In both experiments, changeover rates were lower on the long-interval concurrent schedules than on the short-interval ones. The results suggest that simple application of previous generalizations regarding the effects of reinforcement rate and reinforcer magnitude on choice for variable-interval schedules does not accurately describe human behavior in a simple laboratory situation.     

Preference and resistance to change in concurrent variable-interval schedules

Pigeons were trained on a multiple schedule in which separate concurrent schedules were presented in the two components of the schedule. During one component, concurrent variable-interval 40-sec variable-interval 80-sec schedules operated. In the second component, concurrent variable-interval 40-sec variable-interval 20-sec schedules operated. After stable baseline performance was obtained in both components, extinction probe choice tests were presented to assess preference between the variable-interval 40-sec schedules from the two components. The variable-interval 40-sec schedule paired with the variable-interval 80-sec schedule was preferred over the variable-interval 40-sec schedule paired with the variable-interval 20-sec schedule. The subjects were also exposed to several resistance-to-change manipulations: (1) prefeeding prior to the experimental session, (2) a free-food schedule added to timeout periods separating components, and (3) extinction. The results indicated that preference and resistance to change do not necessarily covary.     

Effects of reinforcer rate and reinforcer quality on time allocation: Extensions of matching theory to educational settings

We examined how 3 special education students allocated their responding across two concurrently available tasks associated with unequal rates and equal versus unequal qualities of reinforcement. The students completed math problems from two alternative sets on concurrent variable-interval (VI) 30-s VI 120-s schedules of reinforcement. During the equal-quality reinforcer condition, high-quality (nickels) and low-quality items ("program money" in the school's token economy) were alternated across sessions as the reinforcer for both sets of problems. During the unequal-quality reinforcer condition, the low-quality reinforcer was used for the set of problems on the VI 30-s schedule, and the high-quality reinforcer was used for the set of problems on the VI 120-s schedule. Equal- and unequal-quality reinforcer conditions were alternated using a reversal design. Results showed that sensitivity to the features of the VI reinforcement schedules developed only after the reinforcement intervals were signaled through countdown timers. Thereafter, when reinforcer quality was equal, the time allocated to concurrent response alternatives was approximately proportional to obtained reinforcement, as predicted by the matching law. However the matching relation was disrupted when, as occurs in most natural choice situations, the quality of the reinforcers differed across the response options.     

Response strength in multiple periodic and aperiodic schedules

Responding in multiple periodic and aperiodic schedules of equal mean reinforcement rate was examined during extinction, satiation, and in the presence of various free-food schedules. In Experiments I and II, pigeons were trained on multiple variable-interval–fixed-interval schedules. Decreases in the rate of responding due to extinction, satiation, or food schedules were approximately equal regardless of the temporal pattern of reinforcer presentation. In Experiment III, pigeons responded on a two-component multiple schedule in which each component was a two-member homogeneous response chain terminating in a fixed-interval schedule during one component and in a variable-interval schedule during the other. The length of both terminal links was varied over a series of conditions. Initial-link responding in the fixed-interval component was reduced more by increasing terminal-link length than was initial-link responding in the variable-interval component. However, no differences in resistance to satiation and extinction were obtained across the fixed and variable components. If the relative decrease in responding produced by satiation and extinction is used as an index of the “value” of the conditions maintaining responding, then these data suggest that fixed and variable schedules of equal mean length are equally valued. This conclusion, however, is not consistent with findings of preference for variable over fixed schedules obtained in studies using concurrent-chain procedures.     

Sensitivity of time allocation to an overall reinforcer rate feedback function in concurrent interval schedules

Six pigeons were trained on concurrent variable-interval schedules in which feedback functions arranged that the overall reinforcer rate either (a) was independent of preference, (b) decreased with increasing absolute preference, or (c) increased with increasing absolute preference. In Experiment 1, the reinforcer rate in an interreinforcement interval was determined by the absolute time-allocation ratio in the previous interval. When arranged reinforcer ratios were varied, there was no evidence of control over preference by overall reinforcer rate. In Experiment 2, the feedback function arranged that reinforcer rates were an inverse function of absolute preference, and window durations were fixed times. In Phase 1, using schedules that provided a four-to-one reinforcer ratio, the window duration was decreased from 20 s to 5 s over four conditions. Then, in Phases 2 and 3, the arranged reinforcer ratios were varied. In Phase 2, the reinforcer rate in the current 5-s time window was determined by preference in the previous 5-s window, and in Phase 3, the window durations were 20 s. Again, there was no indication of control by obtained overall reinforcer rate. These data call into question theories that suggest that the process underlying matching is one of maximizing overall reinforcer rates, or that preference in concurrent aperiodic schedules is controlled to any extent by overall reinforcer rate. They also question the notion that concurrent-schedule preference is controlled by molecular maximizing.