Comparing preference and resistance to change in constant- and variable-duration schedule components

Two experiments explored preference and resistance to change in concurrent chains in which the terminal links were variable-interval schedules that ended either after a single reinforcer had been delivered (variable duration) or after a fixed period of access to the schedule (constant duration). In Experiment 1, pigeons' preference between the same pair of terminal links overmatched relative reinforcement rate when the terminal links were of constant duration, but not when they were of variable duration. Responding during the richer terminal link decreased less, relative to baseline, when response-independent food was presented during the initial links according to a variable-time schedule. In Experiment 2, all subjects consistently preferred a terminal link that consisted of 20-s access to a variable-interval 20-s schedule over a terminal link that ended after one reinforcer had been delivered by the same schedule. Results of resistance-to-change tests corresponded to preference, as responding during the constant-duration terminal link decreased less, relative to baseline, when disrupted by both response-independent food during the initial links and prefeeding. Overall, these data extend the general covariation of preference and resistance to change seen in previous studies. However, they suggest that reinforcement numerosity, including variability in the number of reinforcers per terminal-link entry, may sometimes affect preference and resistance to change in ways that are difficult to explain in terms of current models.     

Dual effects on choice of conditioned reinforcement frequency and conditioned reinforcement value

Pigeons were presented with a concurrent‐chains schedule in which the total time to primary reinforcement was equated for the two alternatives (VI 30 s VI 60 s vs. VI 60 s VI 30 s). In one set of conditions, the terminal links were signaled by the same stimulus, and in another set of conditions they were signaled by different stimuli. Choice was in favor of the shorter terminal link when the terminal links were differentially signaled but in favor of the shorter initial link (and longer terminal link) when the terminal links shared the same stimulus. Preference reversed regularly with reversals of the stimulus condition and was unrelated to the discrimination between the two terminal links during the nondifferential stimulus condition. The present results suggest that the relative value of the terminal‐link stimuli and the relative rate of conditioned reinforcer presentation are important influences on choice behavior, and that models of conditioned reinforcement need to include both factors.     

SOME EFFECTS OF PROCEDURAL VARIATIONS ON CHOICE RESPONDING IN CONCURRENT CHAINS

The present research used pigeons in a three‐key operant chamber and varied procedural features pertaining to both initial and terminal links of concurrent chains. The initial links randomly alternated on the side keys during a session, while the terminal links always appeared on the center key. Both equal and unequal initial‐link schedules were employed, with either differential or nondifferential terminal‐link stimuli across conditions. The research was designed to neutralize initial‐ and terminal‐link spatial cues in order to gain a clearer understanding of the roles of conditioned reinforcement and delayed primary reinforcement in choice. With both equal and unequal initial links and with differential terminal‐link stimuli, all pigeons reliably preferred the chain with the shorter terminal link. However, with equal initial links and nondifferential stimuli, all pigeons were indifferent. With unequal initial links and nondifferential stimuli, some pigeons were also indifferent, while others actually reversed and preferred the chain with the shorter initial link, even though it was followed by the longer terminal link. The decrease if not reversal of the previous preferences implies that preferences in concurrent chains are a function of the conditioned reinforcement afforded by terminal‐link stimuli, rather than delayed primary reinforcement.     

The Probability of Small Schedule Values and Preference for Random-Interval Schedules

Preference for working on variable schedules and temporal discrimination were simultaneously examined in two experiments using a discrete-trial, concurrent-chains arrangement with fixed interval (FI) and random interval (RI) terminal links. The random schedule was generated by first sampling a probability distribution after the programmed delay to reinforcement on the FI schedule had elapsed, and thus the RI never produced a component schedule value shorter than the FI and maintained a rate of reinforcement half that of the FI. Despite these features, the FI was not strongly preferred. The probability of obtaining the smallest programmed delay to reinforcement on the RI schedule was manipulated in Experiment 1, and the interaction of this probability and initial link length was examined in Experiment 2. As the probability of obtaining small values in the RI increased, preference for the schedule increased while the discriminated time of reinforcer availability in the terminal link decreased. Both of these effects were attenuated by lengthening the initial links. The results support the view that in addition to the delay to reinforcement, the probability of obtaining a short delay is an important choice-affecting variable that likely contributes to the robust preferences for variable, as opposed to fixed, schedules of reinforcement.     

Commitment using punishment.

Experimental parameters were adjusted so that pigeons' pairwise choices among three alternatives reflected the following order of preference: (a) a smaller-sooner reinforcer, (b) a larger-later reinforcer, and (c) the smaller-sooner reinforcer followed by a punishment (consisting of an extended blackout period). After this order of preference was established, the pigeons were exposed to a two-link, concurrent-chain-like choice procedure. One terminal link consisted of a choice between the smaller-sooner and the larger-later reinforcer; the other terminal link was identical to the first except that the smaller-sooner reinforcer was followed by blackout punishment. The pigeons' preference (in their initial-link choice) for the terminal link with the punished smaller-sooner alternative increased as the delay between the initial and terminal links increased. By choosing this terminal link, the pigeons are said to have "committed" themselves to obtaining the larger-later reinforcer. However, unlike prior studies of commitment (e.g., Rachlin & Green, 1972), it was still possible after making the commitment for the pigeons to choose the smaller-sooner reinforcer and undergo the punishment. The pigeons did in fact occasionally make this highly deleterious choice.     

Effects of reinforcement magnitude on pigeons' preference for different fixed-ratio schedules of reinforcement

In concurrent, two-member chains, the completion of one or the other of two initial percentage fixed-interval 90-sec links produced a terminal link in which the completion of a fixed ratio produced food reinforcement. The fixed ratios and the duration of reinforcement in the terminal links were varied. Relative response rate in initial links was proportional to the relative reinforcement duration per ratio response (reinforcement duration divided by fixed ratio) in terminal links. The rate of responding in the terminal fixed-ratio links was insensitive to both ratio size and reinforcement duration and therefore did not vary sufficiently to distinguish between responses per reinforcement and immediacy of reinforcement as controlling variables in terminal links.     

Motivational control of impulsive behavior interacts with choice opportunities

Impulsive behavior has been investigated through choice between a smaller/immediate reinforcer and a larger/delayed reinforcer, or through performance on a differential reinforcement of low rate (DRL) schedule. In the present study, we investigated a methodological divergence between these two procedures: in the former procedure, delay is a consequence of the subject's own choice, whereas in the later procedure, subjects are explicitly reinforced for delaying a response. In Experiment 1, 7 rats maintained at 80% of their free-feeding weights showed poorer efficiency of lever-pressing responses on a DRL 30-s schedule than when they were maintained at 90% of free-feeding weight. In Experiment 2, 16 rats were subjected to a concurrent chain schedule: the initial link was concurrent fixed ratio 1 fixed ratio 1, and each of these alternatives was followed by a short-DRL requirement with a one pellet reinforcer or a long-DRL requirement with a three pellet reinforcer. In one block of trials, rats were not allowed to choose between the two terminal links (forced-choice), whereas in the other block of trials rats were allowed to choose freely between the two terminal links (free-choice). Compared with rats maintained at 95% of their free-feeding weights, rats maintained at 80% of their free-feeding weights showed poorer efficiency in the terminal links’ DRL schedule performance (just as in Experiment 1), but this difference was shown only in the forced-choice blocks. These results indicate that motivational control of DRL schedule performance interacts with type of choice-making opportunity and highlight the direct comparison of motivational control of impulsive choice and DRL schedule performance.     

On the measurement of reinforcement frequency in the study of preference

In a two-link, concurrent-chain schedule, pigeons' pecks on each key during the initial link occasionally produced a terminal link, during which only that key was operative. Responses in the terminal link were reinforced with food on either fixed-interval or variable-interval schedules. In one experiment, relative amount of responding in the initial link equaled the relative harmonic rate of reinforcement in the terminal links. In a second experiment, the selection of interreinforcement intervals in variable-interval schedules in the terminal links was such that rates of reinforcement based on the harmonic or on the arithmetic means of the interreinforcement intervals predicted opposite preferences in the initial links. The observed preference was consistent with that predicted by the harmonic rather than by the arithmetic rates of reinforcement.     

Responding on concurrent-chains schedules in open and closed economies.

Pigeons' key pecks were reinforced according to concurrent-chains schedules of reinforcement. The programmed average time from the onset of the initial links to a terminal link entry was held constant across conditions while the value of variable-interval schedules in the terminal links was varied. Performance was assessed under two economic conditions: (a) an open economy in which session duration was limited to 1 hr and subjects were maintained at 80% of their free-feeding body weights with postsession food when necessary; and (b) a closed economy in which sessions were 23.5 hr long and no deprivation regimen was in effect. In all cases, the relative rate of responding in the initial links matched the reduction in overall delay to primary reinforcement correlated with entry into one terminal link relative to the reduction in delay correlated with entry into the other terminal link. Although the sum of responses made in the initial links and terminal links was found to increase, then decrease, as the rate of food presentation decreased in the closed economy, there was no consistent effect of overall rate of food presentation on total responding in the open economy. The choice data suggest that relative delay reduction predicts choice accurately, regardless of economic context.     

DOES SENSITIVITY TO MAGNITUDE DEPEND ON THE TEMPORAL DISTRIBUTION OF REINFORCEMENT?

Our research addressed the question of whether sensitivity to relative reinforcer magnitude in concurrent chains depends on the distribution of reinforcer delays when the terminal-link schedules are equal. In Experiment 1, 12 pigeons responded in a two-component procedure. In both components, the initial links were concurrent variable-interval 40-s variable-interval 40-s, and the terminal links were both 20-s interval schedules in which responses were reinforced by either 4-s of grain in one, or 2-s of grain in the other. The only difference between the components was whether the terminal-link schedules were fixed interval or variable intervals. For all subjects, the relative rate of responding in the initial links for the terminal link that produced the 4-s reinforcer was greater when the terminal links were fixed-interval schedules than when they were variable-interval schedules. This result is contrary to the prediction of Grace's (1994) contextual choice model, but is consistent with both Mazur's (2001) hyperbolic value-added model and Killeen's (1985) incentive theory. In Experiment 2, 4 pigeons responded in a concurrent-chains procedure in which 4-s or 2-s reinforcers were provided independently of responding according to equal fixed-time or mixed-time schedules. Preference for the 4-s reinforcer increased as the variability of the intervals comprising the mixed-time schedules was decreased. Generalized-matching sensitivity of initial-link response allocation to relative reinforcer magnitude was proportional to the geometric mean of the terminal-link delays.     

Choice between single and multiple reinforcers in concurrent-chains schedules

Pigeons responded on concurrent-chains schedules with equal variable-interval schedules as initial links. One terminal link delivered a single reinforcer after a fixed delay, and the other terminal link delivered either three or five reinforcers, each preceded by a fixed delay. Some conditions included a postreinforcer delay after the single reinforcer to equate the total durations of the two terminal links, but other conditions did not include such a postreinforcer delay. With short initial links, preference for the single-reinforcer alternative decreased when a postreinforcer delay was present, but with long initial links, the postreinforcer delays had no significant effect on preference. In conditions with a postreinforcer delay, preference for the single-reinforcer alternative frequently switched from above 50% to below 50% as the initial links were lengthened. This pattern of results was consistent with delay-reduction theory (Squires & Fantino, 1971), but not with the contextual-choice model (Grace, 1994) or the hyperbolic value-added model (Mazur, 2001) as they have usually been applied. However, the hyperbolic value-added model could account for the results if its calculations were expanded to include reinforcers delivered in later terminal links. The implications of these findings for models of concurrent-chains performance are discussed.     

Stimulus Equivalence: Effects Of A Default-response Option On Emergence Of Untrained Stimulus Relations

Human subjects were exposed to a concurrent-chains schedule in which reinforcer amounts, delays, or both were varied in the terminal links, and consummatory responses were required to receive points that were later exchangeable for money. Two independent variable-interval 30-s schedules were in effect during the initial links, and delay periods were defined by fixed-time schedules. In Experiment 1, subjects were exposed to three different pairs of reinforcer amounts and delays, and sensitivity to reinforcer amount and delay was determined based on the generalized matching law. The relative responding (choice) of most subjects was more sensitive to reinforcer amount than to reinforcer delay. In Experiment 2, subjects chose between immediate smaller reinforcers and delayed larger reinforcers in five conditions with and without timeout periods that followed a shorter delay, in which reinforcer amounts and delays were combined to make different predictions based on local reinforcement density (i.e., points per delay) or overall reinforcement density (i.e., points per total time). In most conditions, subjects' choices were qualitatively in accord with the predictions from the overall reinforcement density calculated by the ratio of reinforcer amount and total time. Therefore, the overall reinforcement density appears to influence the preference of humans in the present self-control choice situation.     

Conditioned reinforcement dynamics in three-link chained schedules.

In two experiments rats were trained on three-link concurrent-chains schedules of reinforcement. In Experiment 1, additional entries to one terminal link were added during one of the middle links to a baseline schedule that was otherwise equal for the two chains, and, depending on the condition, these additional terminal-link presentations ended either in food or in no food. When food occurred, preference was always in favor of the chain with the additional terminal-link presentations (which also entailed a higher rate of reinforcement). When no food occurred at the end of the additional terminal links, the outcome depended on the nature of the stimuli associated with these additional terminal links. When stimuli different from the reinforced baseline terminal links were used for the no-food terminal links, preference was against the choice alternative that led to the extra periods of extinction. When the same stimulus was used for the two kinds of terminal links, preference was near indifference, that is, significantly greater than when different stimuli were used. In Experiment 2, rats learned repeated reversals of a simultaneous discrimination under a three-link concurrent-chains schedule, in which the food or no-food choice outcomes were delayed until the end of the chain. Different conditions were defined by the point in the chain at which differential stimuli occurred. When the middle and terminal links provided no differential stimuli, discrimination was acquired more slowly than when differential stimuli occurred in both links. When differential stimuli occurred in the middle but not the terminal links, acquisition rates were intermediate. Both experiments together show that the effects of stimuli in a chain schedule are due partly to the time to food correlated with the stimuli and partly to the time to the next conditioned reinforcer in the sequence.     

Some effects of interreinforcement time upon choice

Pigeons' responses were reinforced on two identical and concurrently available chain variable-interval-schedules. Unlike the typical concurrent chains procedure, both links were operative throughout, thus producing three types of concurrency: (1) concurrent initial links; (2) concurrent initial and terminal links; (3) concurrent terminal links. Choice proportions in each of these three states suggested that the pigeons were sensitive to momentary likelihoods of reinforcement: choice proportions for a schedule were higher when the schedule had been operative for some time, resulting in a higher probability of reinforcement. The study also showed that the relative rates of responding did not match the relative rates of reinforcement in any of the three states of concurrency. Instead, the choice proportions in both the concurrent initial and in the concurrent terminal links were intermediate between the scheduled and the obtained relative rates of reinforcement, while the choice proportions for a terminal link concurrent with an initial link consistently overmatched the relative interreinforcement times (and were typically 1.00). These data indicate that an accurate characterization of choice may not be obtained by considering only the relative interreinforcement interval where one interreinforcement interval is segmented into a chain. Instead, the organism's choice for a schedule will be substantially lowered by the chaining operation.     

Choice and number of reinforcers

Pigeons were exposed to the concurrent-chains procedure in two experiments designed to investigate the effects of unequal numbers of reinforcers on choice. In Experiment 1, the pigeons were indifferent between long and short durations of access to variable-interval schedules of equal reinforcement density, but preferred a short high-density terminal link over a longer, lower density terminal link, even though in both sets of comparisons there were many more reinforcers per cycle in the longer terminal link. In Experiment 2, the pigeons preferred five reinforcers, the first of which was available after 30 sec, over a single reinforcer available at 30 sec, but only when the local interval between successive reinforcers was short. The pigeons were indifferent when this local interval was sufficiently long. The pigeons' behavior appeared to be under the control of local terminal-link variables, such as the intervals to the first reinforcer and between successive reinforcers, and was not well described in terms of transformed delays of reinforcement or reductions in average delay to reinforcement.     

Resistance to change produced by access to fixed-delay versus variable-delay terminal links

Pigeons' responding was reinforced on a multiple schedule consisting of two two-link chain schedules presented in regular alternation. Responding in initial links (always variable-interval 60-s) produced a key-color change and access to a terminal link. The terminal link for one chain provided food after a fixed delay (fixed-interval or fixed-time); the terminal link for the other provided food after a variable delay (variable-interval or variable-time). The average duration of the terminal-link schedules was varied across conditions, but in every condition the arithmetic mean of the variable-delay terminal-link schedule was equal to the duration of the fixed delay. Response rates were higher in the initial links of the chains with the variable-delay terminal links. Response-decreasing operations (satiation, extinction) were used after performances reached asymptote. Response rates maintained by access to variable-delay terminal links tended to be more resistant to change than were rates maintained by access to fixed-delay terminal links. These results are consistent with the preference for variable- over fixed-interval terminal links observed with concurrent-chains schedules, suggesting (1) that immediacy of reinforcement influences the conditioned reinforcing potency of access to a terminal link and (2) that choice in concurrent chains and resistance of responding to change may be manifestations of the same effect of reinforcement.     

Effects on preference of reinforcement delay, number of reinforcers, and terminal-link duration

Three experiments used concurrent-chains procedures to examine the effects of reinforcement delay, number of reinforcers, and terminal-link duration on preference. In Condition 30 of Experiment 1, food was delivered after 30 seconds in each 150-second terminal link, with four additional food deliveries occurring at 30-second intervals in one of the links. In Condition 5, food was delivered after 5 seconds in each 25-second terminal link, and the four additional reinforcers were delivered at 5-second intervals. Preferences for the multiple-food chain were greater in Condition 30. In Experiment 2, the terminal link(s) providing only one reinforcer terminated immediately after delivery of the reinforcer. Preferences for the multiple-food chain were smaller than in Experiment 1. In Condition 5 of Experiment 3, food was delivered after 5, 75, 100, 125, and 150 seconds in one 150-second link and after 5 seconds in the other. Condition 50 differed only in that the first (or only) reinforcer in each link was delivered after 50 seconds instead of after 5 seconds. Preferences for the multiple-food chain were greater in Condition 50. Results of Experiments 1 and 2 do not correspond to results obtained by Moore (1979).