Resistance to change produced by access to fixed-delay versus variable-delay terminal links

Pigeons' responding was reinforced on a multiple schedule consisting of two two-link chain schedules presented in regular alternation. Responding in initial links (always variable-interval 60-s) produced a key-color change and access to a terminal link. The terminal link for one chain provided food after a fixed delay (fixed-interval or fixed-time); the terminal link for the other provided food after a variable delay (variable-interval or variable-time). The average duration of the terminal-link schedules was varied across conditions, but in every condition the arithmetic mean of the variable-delay terminal-link schedule was equal to the duration of the fixed delay. Response rates were higher in the initial links of the chains with the variable-delay terminal links. Response-decreasing operations (satiation, extinction) were used after performances reached asymptote. Response rates maintained by access to variable-delay terminal links tended to be more resistant to change than were rates maintained by access to fixed-delay terminal links. These results are consistent with the preference for variable- over fixed-interval terminal links observed with concurrent-chains schedules, suggesting (1) that immediacy of reinforcement influences the conditioned reinforcing potency of access to a terminal link and (2) that choice in concurrent chains and resistance of responding to change may be manifestations of the same effect of reinforcement.     

Conditioned reinforcement by conditional discriminative stimuli.

A concurrent-chains schedule was used to examine how a delay to conditional discriminative stimuli affects conditioned reinforcement strength. Pigeons' key-peck responses in the initial link produced either of two terminal links according to independent variable-interval 30-s schedules. Each terminal link involved an identical successive conditional discrimination and was segmented into three links: a delay interval (green), a color conditional discriminative stimulus (blue or red), and a line conditional discriminative stimulus (vertical or horizontal lines). Food delivery occurred 45 s after entering the terminal link with a probability of .5, but its conditional probability (1.0 or 0) depended on the combination of the color and the line stimuli. One of the color stimuli occurred independently of further responding, 5 s after entry into the right terminal link, but it occurred 35 s after entry into the left terminal link. One of the line stimuli occurred independently of responding 40 s after entry into either terminal link, synchronized with the offset of the color stimulus. The initial-link relative response rate for the right was consistently higher in comparison with a control condition in which the color stimuli occurred 20 s after entry into either terminal link. The preference for the short delay to the color conditional discriminative stimuli suggests the possibility of conditioned reinforcement by information about the relation between the line conditional discriminative stimuli and the outcomes.     

The effect of conditioned reinforcement rate on choice: a review

We review the nature of conditioned reinforcement, including evidence that conditioned reinforcers maintain choice behavior in concurrent schedules and that they elevate responding in the terminal links of concurrent‐chains schedules. A question has resurfaced recently: Do theories of choice in concurrent‐chains schedules need to include a term reflecting greater preference for higher rates of conditioned reinforcement? The review of several studies addressing this point suggests that such a term is inappropriate. Elevated rates of conditioned reinforcement (and responding) in the terminal links of concurrent‐chains schedules do not lead to greater preference in the initial link leading to the higher rate of conditioned reinforcement. If anything, the opposite preference is likely to occur. This result is not surprising, since the additional putative conditioned reinforcers in the terminal link are not correlated with a reduction in time to primary reinforcement nor with an increase in value.     

Second-order schedules of token reinforcement: effects of varying the schedule of food presentation

In the initial link of a complex schedule, one discriminative stimulus was presented and lever pressing produced tokens on fixed-ratio schedules. In the terminal link, signalled by a second discriminative stimulus, deposits of the tokens produced food. With two rats, the terminal link was presented after each sixth component schedule of token reinforcement was completed. With the other two rats, the terminal link was presented following the first component schedule completed after a fixed interval. During the terminal link, each token deposit initially produced food. The schedule of food presentation was subsequently increased such that an increasing number of token deposits in the terminal link was required for each food presentation. Rates of lever pressing in the initial link were inversely related to the schedule of food presentation in the terminal link. These results are similar to those of experiments that have varied schedules of food presentation in chained schedules. Rates and patterns of responding controlled throughout the initial link were more similar to those ordinarily controlled by second-order brief-stimulus schedules than to those controlled by comparable extended chained schedules.     

Choice and conditioned reinforcement.

A potential weakness of one formulation of delay-reduction theory is its failure to include a term for rate of conditioned reinforcement, that is, the rate at which the terminal-link stimuli occur in concurrent-chains schedules. The present studies assessed whether or not rate of conditioned reinforcement has an independent effect upon choice. Pigeons responded on either modified concurrent-chains schedules or on comparable concurrent-tandem schedules. The initial link was shortened on only one of two concurrent-chains schedules and on only one of two corresponding concurrent-tandem schedules. This manipulation increased rate of conditioned reinforcement sharply in the chain but not in the tandem schedule. According to a formulation of delay-reduction theory, when the outcomes chosen (the terminal links) are equal, as in Experiment 1, choice should depend only on rate of primary reinforcement; thus, choice should be equivalent for the tandem and chain schedules despite a large difference in rate of conditioned reinforcement. When the outcomes chosen are unequal, however, as in Experiment 2, choice should depend upon both rate of primary reinforcement and relative signaled delay reduction; thus, larger preferences should occur in the chain than in the tandem schedules. These predictions were confirmed, suggesting that increasing the rate of conditioned reinforcement on concurrent-chains schedules may have no independent effect on choice.     

The effects of component duration on multiple-schedule performance in closed and open economies.

Pigeons responded on multiple variable-interval variable-interval schedules of reinforcement in an open and a closed economy. Equal duration components were increased in duration while the component rates of reinforcement were held constant, the component schedules were reversed, and component duration was decreased. In the open economy, daily sessions were limited to 1 hr, and subjects were maintained at 80% of their free-feeding weights through supplemental feeding when necessary in their home cages. In the closed economy, subjects were housed in their experimental chambers and no deprivation regimen was enforced. Relative response rate decreased as components were lengthened in the open economy, whereas in the closed economy relative rate increased as components were lengthened. Response proportions overmatched reinforcer proportions to a greater extent at long component durations in the closed economy, but there was no systematic effect of component duration on responding in the open economy.     

Choice and rate of reinforcement

Pigeons' responses in the presence of two concurrently available (initial-link) stimuli produced one of two different (terminal-link) stimuli. The rate of reinforcement in the presence of one terminal-link stimulus was three times that of the other. Three different pairs of identical but independent variable-interval schedules controlled entry into the terminal links. When the intermediate pair was in effect, the pigeons distributed their (choice) responses in the presence of the concurrently available stimuli of the initial links in the same proportion as reinforcements were distributed in the mutually exclusive terminal links. This finding was consistent with those of earlier studies. When either the pair of larger or smaller variable-interval schedules was in effect, however, proportions of choice responses did not match proportions of reinforcements. In addition, matching was not obtained when entry into the terminal links was controlled by unequal variable-interval schedules. A formulation consistent with extant data states that choice behavior is dependent upon the amount of reduction in the expected time to primary reinforcement, as signified by entry into one terminal link, relative to the amount of reduction in expected time to reinforcement signified by entry into the other terminal link.     

On the measurement of reinforcement frequency in the study of preference

In a two-link, concurrent-chain schedule, pigeons' pecks on each key during the initial link occasionally produced a terminal link, during which only that key was operative. Responses in the terminal link were reinforced with food on either fixed-interval or variable-interval schedules. In one experiment, relative amount of responding in the initial link equaled the relative harmonic rate of reinforcement in the terminal links. In a second experiment, the selection of interreinforcement intervals in variable-interval schedules in the terminal links was such that rates of reinforcement based on the harmonic or on the arithmetic means of the interreinforcement intervals predicted opposite preferences in the initial links. The observed preference was consistent with that predicted by the harmonic rather than by the arithmetic rates of reinforcement.     

Effects of reinforcement context on choice

Two experiments investigated the effects of successive reinforcement contexts on choice. In the first, concurrent variable-interval schedules of primary reinforcement operated during the initial links of concurrent chains. The rate of this reinforcement arranged by the concurrent schedules was decreased across conditions: When it was higher than the terminal-link rate, preference for the higher frequency initial-link schedule increased relative to baseline. (During baseline, a standard concurrent-schedule procedure was in effect). When the initial-link reinforcement rate was lower than the terminal-link rate, preference converged toward indifference. In the second experiment, a chain schedule was available on a third key while a concurrent schedule was in effect on the side keys. When the terminal link of the chain schedule was produced, the side keys became inoperative. Availability of the chain schedule did not affect choice between the concurrent schedules. These results show that only when successive reinforcement contexts are produced by choice responding do those successive contexts affect choice in concurrent schedules.     

Conditioned reinforcement versus time to reinforcement in chain schedules.

Pigeons were trained on three-component chain schedules in which the initial component was either a fixed-interval or variable-interval schedule. The middle and terminal components were varied among fixed-interval fixed-interval, variable-interval variable-interval, and an interdependent variable-interval variable-interval schedule in which the sum of the durations of the two variable-interval components was always equal to the sum of the fixed-interval fixed-interval components. At issue was whether the response rate in the initial component was controlled by its time to primary reinforcement or by the temporal parameters of the stimulus correlated with the middle terminal link. The fixed-interval initial-link schedule maintained much lower response rates than the variable-interval initial-link schedule regardless of the schedules in the middle and terminal links. Nevertheless, the intervening schedules played some role: With fixed-interval schedules in the initial links, response rates were consistently highest with independent variable-interval schedules in the middle and terminal links and intermediate with the interdependent variable-interval schedules; these initial-link differences were predicted by the response rates in the middle link of the chain. With variable-interval schedules in the initial links, response rates were lowest with the fixed-interval fixed-interval schedules following the initial link and were not systematically different for the two types of variable-interval variable-interval schedules. The results suggest that time to reinforcement itself accounts for little if any variance in initial-link responding.     

Choice for conditioned reinforcers in the signaled absence of primary reinforcement

Pigeons responded in a multiple schedule in which concurrent schedules of brief-stimulus presentation alternated with a component in which food was available (concurrent-chains component). In the initial links of the concurrent-chains component subjects chose either of two stimuli each correlated with the terminal link of one chain. The terminal links involved either variable-interval 30-second or variable-interval 60-second schedules. In the brief-stimulus component subjects chose between 0.5-second presentations of the terminal-link stimuli from the concurrent-chains component. Responding was generally maintained in the brief-stimulus component in two subjects for more than 300 sessions, suggesting that brief stimuli were conditioned reinforcers. During the brief-stimulus component, in 17 of 21 cases for which a minimal number of responses occurred, choice proportions above 0.55 were obtained for the brief-stimulus presentations correlated with the higher rate of primary reinforcement in the concurrent-chains component. These results support the suggestion that choice in conventional concurrent-chains procedures is partially controlled by production of the terminal-link stimuli.     

Comparing preference and resistance to change in constant- and variable-duration schedule components

Two experiments explored preference and resistance to change in concurrent chains in which the terminal links were variable-interval schedules that ended either after a single reinforcer had been delivered (variable duration) or after a fixed period of access to the schedule (constant duration). In Experiment 1, pigeons' preference between the same pair of terminal links overmatched relative reinforcement rate when the terminal links were of constant duration, but not when they were of variable duration. Responding during the richer terminal link decreased less, relative to baseline, when response-independent food was presented during the initial links according to a variable-time schedule. In Experiment 2, all subjects consistently preferred a terminal link that consisted of 20-s access to a variable-interval 20-s schedule over a terminal link that ended after one reinforcer had been delivered by the same schedule. Results of resistance-to-change tests corresponded to preference, as responding during the constant-duration terminal link decreased less, relative to baseline, when disrupted by both response-independent food during the initial links and prefeeding. Overall, these data extend the general covariation of preference and resistance to change seen in previous studies. However, they suggest that reinforcement numerosity, including variability in the number of reinforcers per terminal-link entry, may sometimes affect preference and resistance to change in ways that are difficult to explain in terms of current models.     

Preference for fixed-interval schedules: effects of unequal initial links

Six homing pigeons were trained on concurrent chain schedules in which the terminal links were fixed-interval schedules of 5 sec or 15 sec. One initial-link schedule was always VI 27-sec; the other was varied over conditions from VI 27-sec to VI 181-sec. Preference measured in the initial links varied as a joint function of the initial- and terminal-link schedules. When the initial links were varied with constant, but unequal, terminal links, the slope of the function relating the logarithm of the initial-link response ratio to the logarithm of the terminal-link entry ratio differed from that obtained with equal terminal links. This result indicates that biases attributable to the terminal-link schedules were not constant. The rate of change of preference, or degree of undermatching, in the initial links depended on whether the shorter initial link led to the shorter or the longer terminal link. These results raise the question of whether bias and undermatching in concurrent schedule performance are independent measures.     

Effects of primary reinforcement on pigeons' initial-link responding under a concurrent chains schedule with nondifferntial terminal links

The effect of primary reinforcement on initial-link responding under concurrent-chains schedules with nondifferential terminal links was assessed in 12 pigeons. The iniitial and terminal links were variable-interval schedules (always the same for both alternatives). The positions (left or right key) of the initial-link stimuli (red or green) were randomized while the correlation between color and food amount remained constant within each condition. The terminal-link stimuli were always presented on the center key. Except in two control groups and conditions, the terminal-link stimuli were the same color (nondifferential, blue or yellow). Over six conditions, the differences in food amont and the durations of the initial- and terminal-link schedules were manipulated. In 57 of 60 cases, birds generated choice proportions above .50 in favor of the initial-link stimlus that was correlated with the larger reinforcer. There was some indication that preference increased with shortened terminal-link durations. Because the terminal-link stimuli were nondifferential, differential responding in the initial links cannot be explained easily by conditioned reinforcement represented by the terminal-link stimuli. Thus, primiary reinforcement has a direct effect on initial-link responding in concurrent-chains schedules.     

Choice between two-component chained and tandem schedules

Pigeons were trained on a two-key choice procedure in which a pair of equal and concurrently available variable-interval schedules (initial links) arranged entry into one or the other of two mutually exclusive schedules (terminal links) that ended in primary reinforcement. The terminal links were two-component chained or tandem schedules. Responses during the initial links were distributed equally on the two keys whenever the terminal links were associated with the same sets of interreinforcement intervals. Whether or not the terminal-link interreinforcement intervals were the same on the two keys, initial-link responding was affected by neither the presence nor relative durations of differentially signalled components within a terminal-link schedule. The simplest interpretation of these results is that initial-link responding is maintained directly by delayed primary reinforcement, rather than conditioned reinforcement afforded by the stimuli correlated with the terminal-link schedule components. This finding suggests that aspects of chained schedule performance usually attributed to conditioned reinforcement might best be reinterpreted in terms of delayed primary reinforcement and various discriminative functions served by the component-correlated stimuli.     

Response strength in multiple periodic and aperiodic schedules

Responding in multiple periodic and aperiodic schedules of equal mean reinforcement rate was examined during extinction, satiation, and in the presence of various free-food schedules. In Experiments I and II, pigeons were trained on multiple variable-interval–fixed-interval schedules. Decreases in the rate of responding due to extinction, satiation, or food schedules were approximately equal regardless of the temporal pattern of reinforcer presentation. In Experiment III, pigeons responded on a two-component multiple schedule in which each component was a two-member homogeneous response chain terminating in a fixed-interval schedule during one component and in a variable-interval schedule during the other. The length of both terminal links was varied over a series of conditions. Initial-link responding in the fixed-interval component was reduced more by increasing terminal-link length than was initial-link responding in the variable-interval component. However, no differences in resistance to satiation and extinction were obtained across the fixed and variable components. If the relative decrease in responding produced by satiation and extinction is used as an index of the “value” of the conditions maintaining responding, then these data suggest that fixed and variable schedules of equal mean length are equally valued. This conclusion, however, is not consistent with findings of preference for variable over fixed schedules obtained in studies using concurrent-chain procedures.     

Preference and resistance to change with constant- and variable-duration terminal links: independence of reinforcement rate and magnitude

Pigeons responded in a three-component multiple concurrent-chains procedure in which the variable-interval reinforcement schedules were the same across components but magnitudes differed across components. The terminal links were arranged either as a variable delay followed by presentation of a reinforcer ("variable duration") or as a fixed period of access to the schedule during which a variable number of reinforcers could be earned ("constant duration"). Relative reinforcement rate was varied parametrically across both types of conditions. After baseline training in each condition, resistance to change of terminal-link responding was assessed by delivering food during the initial links according to a variable-time schedule. Both preference and resistance to change were more sensitive to reinforcement-rate differences in the constant-duration conditions. Sensitivities of preference and resistance to change to relative reinforcement rate did not change depending on relative reinforcement magnitude. Taken together, these results confirm and extend those of prior studies, and suggest that reinforcement rate and magnitude combine additively to determine preference and resistance to change. A single structural relation linking preference and resistance to change describes all the data from this and several related studies.     

Preference for less segmented fixed-time components in concurrent-chain schedules of reinforcement

A concurrent-chain procedure was used to examine choice between segmented and less segmented response-independent schedules of reinforcement. A pair of independent, concurrent variable-interval 60-s schedules were presented in the initial link, along with a 1.5-s changeover delay. A chained fixed-interval fixed-time and its corresponding tandem schedule constituted the terminal links. The length of the fixed-interval schedule in the terminal link was varied between 5 s and 30 s while that of the fixed-time schedule was kept at 5 s over conditions. The first components of both terminal-link schedules were accompanied by the same stimulus. Except in the baseline condition, the onset of the second component of the terminal-link chained schedule was accompanied by either a localized (key color) or a nonlocalized (dark houselight) stimulus change. Stimulus conditions were constant during the terminal-link tandem schedule. With three exceptions, pigeons demonstrated a slight preference for the tandem over the chained schedule in the terminal link. Furthermore, this preference varied inversely with the length of the first component. In general, these results are consistent with previous studies that reported an adverse effect on choice by segmenting an interval schedule into two or more components, but they are inconsistent with studies that reported preference for signaled over unsignaled delay of reinforcement.