Concurrent-chains schedules as a method to study choice between alcohol-associated conditioned reinforcers

An extensive body of research using concurrent-chains schedules of reinforcement has shown that choice for one of two differentially valued food-associated stimuli is dependent upon the overall temporal context in which those stimuli are embedded. The present experiments examined whether the concurrent chains procedure was useful for the study of behavior maintained by alcohol and alcohol-associated stimuli. In Experiment 1, rats responded on concurrent-chains schedules with equal variable-interval (VI) 10-s schedules in the initial links. Across conditions, fixed-interval schedules in the terminal links were varied to yield 1∶1, 9∶1, and 1∶9 ratios of alcohol delivery. Initial-link response rates reflected changes in terminal-link schedules, with greater relative responding in the rich terminal link. In Experiment 2, terminal-link schedules remained constant with a 9∶1 ratio of alcohol delivery rates while the length of two equal duration initial-link schedules was varied. Preference for the rich terminal link was less extreme when initial links were longer (i.e., the initial-link effect), as has been previously reported with food reinforcers. This result suggests that the conditioned reinforcing value of an alcohol-associated stimulus depends on the temporal context in which it is embedded. The concurrent-chains procedure and quantitative models of concurrent chains performance may provide a useful framework within which to study how contextual variables modulate preference for drug-associated conditioned reinforcers.     

Evidence against a constant-difference effect in concurrent-chains schedules

Savastano and Fantino (1996) reported that in concurrent-chains schedules, initial-link choice proportions remained constant as terminal-link durations increased as long as the subtractive difference between the two terminal-link schedules remained constant. Two experiments with pigeons were conducted to examine this constant-difference effect. Both experiments used equal variable-interval schedules as initial links. The terminal links were fixed delays to reinforcement in Experiment 1 and variable delays to reinforcement in Experiment 2. The durations of the terminal links were varied across conditions, but the difference between pairs of terminal links was always 10 s. In both experiments, preference for the shorter terminal link became less extreme as terminal-link durations increased, so a constant-difference effect was not found. It is argued, however, that this choice situation does not provide clear evidence for or against delay-reduction theory versus other theories of choice.     

Choice in the time-left procedure and in concurrent chains with a time-left terminal link.

In two experiments, rats chose between a standard fixed-duration food-associated stimulus and a stimulus whose duration was the time remaining to reinforcement in an elapsing comparison interval. In Experiment 1, 4 rats responded in a time-left procedure wherein a single initial-link variable-interval schedule set up two potential terminal links simultaneously. As time elapsed in the initial-link schedule, the choice was between a standard fixed-interval 30-s terminal link and a time-left terminal link whose programmed interval requirement equaled 90 s minus the elapsed time in the initial link. Rats generally responded more on the lever with the shortest programmed terminal-link duration, but the temporal parameters of the procedure were found to vary with response distributions. Contrary to previous reports, therefore, time-left data were well predicted by choice models that make no assumptions about animal timing. In Experiment 2, 8 rats responded on a concurrent-chains schedule with independent variable-interval initial links and a time-left terminal link in one of the choice schedules. On the time-left lever, the programmed terminal-link delay equaled 90 s minus the elapsed time in the time-left initial link. On the standard lever, terminal-link responses were reinforced according to a variable-interval schedule whose average value varied over four conditions. Relative time-left initial-link responses increased in the elapsing time-left initial-link schedule as the time-left terminal link became shorter relative to the standard terminal link. Scalar expectancy theory failed to predict the resultant data, but a modified version of the delay-reduction model made good predictions. An analysis of the elaboration of scalar expectancy theory for variable delays demonstrated that the model is poorly formulated for arithmetically distributed delays.     

Comparing preference and resistance to change in constant- and variable-duration schedule components

Two experiments explored preference and resistance to change in concurrent chains in which the terminal links were variable-interval schedules that ended either after a single reinforcer had been delivered (variable duration) or after a fixed period of access to the schedule (constant duration). In Experiment 1, pigeons' preference between the same pair of terminal links overmatched relative reinforcement rate when the terminal links were of constant duration, but not when they were of variable duration. Responding during the richer terminal link decreased less, relative to baseline, when response-independent food was presented during the initial links according to a variable-time schedule. In Experiment 2, all subjects consistently preferred a terminal link that consisted of 20-s access to a variable-interval 20-s schedule over a terminal link that ended after one reinforcer had been delivered by the same schedule. Results of resistance-to-change tests corresponded to preference, as responding during the constant-duration terminal link decreased less, relative to baseline, when disrupted by both response-independent food during the initial links and prefeeding. Overall, these data extend the general covariation of preference and resistance to change seen in previous studies. However, they suggest that reinforcement numerosity, including variability in the number of reinforcers per terminal-link entry, may sometimes affect preference and resistance to change in ways that are difficult to explain in terms of current models.     

Cued and uncued terminal links in concurrent-chains schedules

Pigeons were trained on a concurrent-chains schedule. The initial links were concurrent variable-interval schedules arranged on two side keys. Each terminal link was a fixed-interval schedule arranged on the center key. In cued conditions, different center-key colors signaled the two terminal-link schedules. In uncued conditions, the same center-key color appeared for both terminal links. Experiment 1 arranged unequal initial links and equal terminal links. Preference for the shorter initial-link schedule was greater when the terminal links were uncued. Experiment 2 arranged equal initial links and unequal terminal links. Preference for the shorter terminal-link schedule was greater when the terminal links were cued. Although the results of Experiment 2 successfully replicated previous research, the results of Experiment 1 are not easily reconciled with conditioned-reinforcement or discriminative-stimulus accounts of the role of terminal-link cues. Rather, terminal-link cues appear to decrease sensitivity to initial-link contingencies.     

Effects of primary reinforcement on pigeons' initial-link responding under a concurrent chains schedule with nondifferntial terminal links

The effect of primary reinforcement on initial-link responding under concurrent-chains schedules with nondifferential terminal links was assessed in 12 pigeons. The iniitial and terminal links were variable-interval schedules (always the same for both alternatives). The positions (left or right key) of the initial-link stimuli (red or green) were randomized while the correlation between color and food amount remained constant within each condition. The terminal-link stimuli were always presented on the center key. Except in two control groups and conditions, the terminal-link stimuli were the same color (nondifferential, blue or yellow). Over six conditions, the differences in food amont and the durations of the initial- and terminal-link schedules were manipulated. In 57 of 60 cases, birds generated choice proportions above .50 in favor of the initial-link stimlus that was correlated with the larger reinforcer. There was some indication that preference increased with shortened terminal-link durations. Because the terminal-link stimuli were nondifferential, differential responding in the initial links cannot be explained easily by conditioned reinforcement represented by the terminal-link stimuli. Thus, primiary reinforcement has a direct effect on initial-link responding in concurrent-chains schedules.     

Conditioned reinforcement value and choice.

The delay-reduction hypothesis of conditioned reinforcement states that the reinforcing value of a food-associated stimulus is determined by the delay to primary reinforcement signaled by the onset of the stimulus relative to the average delay to primary reinforcement in the conditioning situation. In contrast, most contemporary models of conditioned reinforcement strength posit that the reinforcing strength of a stimulus is some simple function only of the delay to primary reinforcement in the presence of stimulus. The delay-reduction hypothesis diverges from other conditioned reinforcement models in that it predicts that a fixed-duration food-paired stimulus will have different reinforcing values depending on the frequency of its presentation. In Experiment 1, pigeons' key pecks were reinforced according to concurrent-chains schedules with variable-interval 10-second and variable-interval 20-second terminal-link schedules. The initial-link schedule preceding the shorter terminal link was always variable-interval 60 seconds, and the initial-link schedule requirement preceding the longer terminal link was varied between 1 second and 60 seconds across conditions. In Experiment 2, the initial-link schedule preceding the longer of two terminal links was varied for each of three groups of pigeons. The terminal links of the concurrent chains for the three groups were variable-interval 10 seconds and 20 seconds, variable-interval 10 seconds and 30 seconds, and variable-interval 30 seconds and 50 seconds. In both experiments, preference for the shorter terminal link was either a bitonic function or an inverse function of the initial-link schedule preceding the longer terminal-link schedule. Consistent with the predictions of the delay-reduction hypothesis, the relative values of the terminal-link stimuli changed as a function of the overall frequency of primary reinforcement. Vaughan's (1985) melioration model, which was shown to be formally similar to Squires and Fantino's (1971) delay-reduction model, can be modified so as to predict these results without changing its underlying assumptions.     

Differences in delay, not ratios, control choice in concurrent chains

In two experiments, pigeons were trained with concurrent-chains schedules, wherein responding to equal initial links measured preference between variable-interval terminal links. Absolute terminal-link duration was varied by keeping constant the difference between the terminal-link delays and forcing their ratio to change. Delay-reduction theory scales value relative to a common temporal context and requires that delay differences control choice. Thus, preference should remain invariant. Most competing accounts, including the matching law and a strong form of Weber's law, require that preference vary with the delay ratio. Experiment 1 employed standard concurrent chains, in which terminal-link position and color were confounded. Although average preference remained constant, individual preferences were highly variable and inconsistent, possibly due to carryover of position biases across conditions. In an attempt to reduce variability, Experiment 2 used a modified concurrent-chains procedure. Preference at different terminal-link durations was assessed simultaneously to prevent order effects, and terminal-link position was alternated randomly across trials to minimize the impact of position biases. In Experiment 2, both individual and mean preferences showed the constant-difference invariance. Overall, choice was controlled by terminal-link differences, not ratios.     

Resistance to change produced by access to fixed-delay versus variable-delay terminal links

Pigeons' responding was reinforced on a multiple schedule consisting of two two-link chain schedules presented in regular alternation. Responding in initial links (always variable-interval 60-s) produced a key-color change and access to a terminal link. The terminal link for one chain provided food after a fixed delay (fixed-interval or fixed-time); the terminal link for the other provided food after a variable delay (variable-interval or variable-time). The average duration of the terminal-link schedules was varied across conditions, but in every condition the arithmetic mean of the variable-delay terminal-link schedule was equal to the duration of the fixed delay. Response rates were higher in the initial links of the chains with the variable-delay terminal links. Response-decreasing operations (satiation, extinction) were used after performances reached asymptote. Response rates maintained by access to variable-delay terminal links tended to be more resistant to change than were rates maintained by access to fixed-delay terminal links. These results are consistent with the preference for variable- over fixed-interval terminal links observed with concurrent-chains schedules, suggesting (1) that immediacy of reinforcement influences the conditioned reinforcing potency of access to a terminal link and (2) that choice in concurrent chains and resistance of responding to change may be manifestations of the same effect of reinforcement.     

Bias and sensitivity to reinforcement in a concurrent-chain schedule

Six pigeons were trained on concurrent-chain schedules in which the initial links were either dependent (Experiment 1) or independent (Experiment 2) concurrent variable-interval schedules and the terminal links were fixed-duration delays to reinforcement in blackout. A changeover delay of three seconds operated in the initial links. Both the initial- and terminal-link schedule values were varied over 61 experimental conditions. An analysis of the data using the generalized matching law showed that the sensitivity of initial-link response allocation to the frequency of terminal-link production was independent of both the duration of the terminal-link delays and, though less clearly, of the duration of the initial-link schedules. Sensitivity of initial-link response allocation to terminal-link reinforcer-rate ratios was a joint function of the terminal-link durations and the smaller average initial-link duration. The results showed that the generalized matching law is useful in analyzing concurrent-chain schedule performance and that a changeover delay in the initial links eliminates some terminal- to initial-link interactions that have made quantitative predictions for concurrent-chain performances difficult.     

Preference and resistance to change with constant- and variable-duration terminal links: independence of reinforcement rate and magnitude

Pigeons responded in a three-component multiple concurrent-chains procedure in which the variable-interval reinforcement schedules were the same across components but magnitudes differed across components. The terminal links were arranged either as a variable delay followed by presentation of a reinforcer ("variable duration") or as a fixed period of access to the schedule during which a variable number of reinforcers could be earned ("constant duration"). Relative reinforcement rate was varied parametrically across both types of conditions. After baseline training in each condition, resistance to change of terminal-link responding was assessed by delivering food during the initial links according to a variable-time schedule. Both preference and resistance to change were more sensitive to reinforcement-rate differences in the constant-duration conditions. Sensitivities of preference and resistance to change to relative reinforcement rate did not change depending on relative reinforcement magnitude. Taken together, these results confirm and extend those of prior studies, and suggest that reinforcement rate and magnitude combine additively to determine preference and resistance to change. A single structural relation linking preference and resistance to change describes all the data from this and several related studies.     

Choice between reliable and unreliable reinforcement alternatives revisited: Preference for unreliable reinforcement

Pigeons' choices between a reliable alternative that always provided food after a delay (i.e., 100% reinforcement) and an unreliable one that provided food or blackout equally often after a delay (i.e., 50% reinforcement) was studied using a discrete-trials concurrent-chains procedure modified to prevent choice between alternatives following a blackout outcome. Initial links were fixed-ratio 1 schedules, and terminal links were fixed-time schedules. Stimuli presented during the terminal-link delays were correlated with the food and blackout outcomes. In Experiment 1, terminal-link durations were varied. With short terminal links (i.e., 10 s), 6 of 8 subjects showed strong preference for the 50% side. As terminal-link duration increased to 30 s, preference, regardless of direction, became less extreme. In Experiment 2, the side-key location of the 50% and 100% alternatives was reversed for 3 subjects. Preference for the 50% alternative reoccurred following the key reversal. When a 5-s separation was subsequently interposed between the initial and terminal links for both alternatives, all birds reversed to a preference for the 100% side. In general, the strong preference for the 50% side was qualitatively consistent with the expectation that the procedure enhanced the conditioned-reinforcement effectiveness of the food-associated terminal-link stimulus on the 50% side. Implications of the results for various accounts of choice of the 50% alternative are discussed.     

Preference for fixed-interval schedules: effects of initial-link length

Six homing pigeons were trained on a variety of concurrent-chains schedules in which the initial links were equal variable-interval schedules and the terminal links were fixed-interval schedules. Both terminal-link and initial-link schedules were systematically varied. The results showed that preference for a particular terminal-link schedule combination was greater, the shorter the initial-link schedules. The data closely matched predictions from the model of choice suggested by Davison and Temple (1973), but did not match predictions from two other models. An alternative method for analyzing concurrent-chains performance by assuming that the schedule consists of both chained schedules and successive, discriminated components that comprise multiple schedules, was suggested.     

Rapid acquisition of choice and timing and the provenance of the terminal-link effect

Eight pigeons responded in a concurrent-chains procedure in which terminal-link schedules changed pseudorandomly across sessions. Pairs of terminal-link delays either summed to 15 s or to 45 s. Across sessions, the location of the shorter terminal link changed according to a pseudorandom binary sequence. On some terminal links, food was withheld to obtain start and stop times, measures of temporal control. Log initial-link response ratios stabilized within the first half of each session. Log response ratio was a monotonically-increasing but nonlinear function of programmed log terminal-link immediacy ratio. There was an effect of absolute terminal-link duration on log response ratio: For most subjects, preference for the relatively shorter terminal-link delay was stronger when absolute delays were long than when absolute delays were short. Polynomial regressions and model comparison showed that differences in degree of nonlinearity, not in sensitivity to log immediacy ratio, produced this effect. Temporal control of stop times was timescale invariant with scalar variability, but temporal control of start times was not consistent across subjects or terminal-link durations.     

A contextual model of concurrent-chains choice

An extension of the generalized matching law incorporating context effects on terminal-link sensitivity is proposed as a quantitative model of behavior under concurrent chains. The contextual choice model makes many of the same qualitative predictions as the delay-reduction hypothesis, and assumes that the crucial contextual variable in concurrent chains is the ratio of average times spent, per reinforcement, in the terminal and initial links; this ratio controls differential effectiveness of terminal-link stimuli as conditioned reinforcers. Ninety-two concurrent-chains data sets from 19 published studies were fitted to the model. Averaged across all studies, the model accounted for 90% of the variance in pigeons' relative initial-link responding. The model therefore demonstrates that a matching law analysis of concurrent chains—the assumption that relative initial-link responding equals relative terminal-link value—remains quantitatively viable. Because the model reduces to the generalized matching law when terminal-link duration is zero, it provides a quantitative integration of concurrent schedules and concurrent chains.     

Temporal context in concurrent chains: I. Terminal-link duration

Two experiments are reported in which the ratio of the average times spent in the terminal and initial links (Tt/Ti) in concurrent chains was varied. In Experiment 1, pigeons responded in a three-component procedure in which terminal-link variable-interval schedules were in constant ratio, but their average duration increased across components by a factor of two. The log initial-link response ratio was a negatively accelerated function of Tt/Ti. Overall, the data were well described by Grace's (1994) contextual choice model (CCM) with temporal context represented as (Tt/Ti)k or 2Tt/(Tt + Ti), and by Mazur's (2001) hyperbolic value-added model (HVA), with each model accounting for approximately 93% of the variance. In Experiment 2, fixed-parameter predictions for each model were generated, based on the data from Experiment 1, for conditions in which Tt/Ti was varied over a more extreme range. Data were consistent with the predictions of CCM with temporal context represented as 2Tt/(Tt + Ti) and to a lesser extent as (Tt/Ti)k, but not with HVA. Overall, these results suggest that preference increases as a hyperbolic function of Tt/Ti when terminal-link duration is increased relative to initial-link duration, with the terminal-link schedule ratio held constant.     

Independence of reinforcement delay and magnitude in concurrent chains

A three-component concurrent-chains procedure was used to investigate preference between terminal-link schedules that differed in delay and magnitude of reinforcement. Response and time allocation data were well described by a generalized matching model. Sensitivity to delay appeared to be lower when reinforcement magnitudes were unequal than when they were equal, but when obtained rather than programmed time spent responding in the initial links was used in the model, the difference vanished. The results support independence of delay and magnitude as separate dimensions of reinforcement value, as required by the matching law, and the assumption of the contextual choice model (Grace, 1994) that sensitivities to delay and magnitude are affected similarly by temporal context. Although there was statistical evidence for interaction between successive components, the effects were small and transient. The multiple-component concurrent-chains procedure should prove useful in future research on multidimensional preference, although it may be necessary to control obtained initial-link time more precisely.     

Preference for multiple versus mixed schedules of reinforcement

Five pigeons were trained in a concurrent-chain procedure. In the initial links, equal nonindependent variable-interval schedules were available concurrently on two keys. Completing the schedule on either key led to exclusive presentation of one of two further variable-interval schedules for a fixed period of time. During these terminal links, as many reinforcers as were scheduled could be obtained. If the response producing this terminal link occurred on one key, differential stimuli signaled which variable-interval schedule had been produced. If the response producing the terminal link occurred on the other key, no such differential stimuli were available. Once the fixed period of time elapsed, the initial links were reinstated. In Experiment 1, the period of time for which the terminal links were available was always 10 s and the absolute duration of the initial links was varied. Subjects preferred the alternative leading to the multiple schedule when the initial-link duration was short, but preferred the alternative leading to the mixed schedule when the initial-link durations were longer. In Experiment 2, both the initial-link duration and the duration of the terminal links were varied. The effect of initial-link duration was identical to that in Experiment 1 and there was no systematic effect of varying the terminal-link duration.