Effects of primary reinforcement on pigeons' initial-link responding under a concurrent chains schedule with nondifferntial terminal links

The effect of primary reinforcement on initial-link responding under concurrent-chains schedules with nondifferential terminal links was assessed in 12 pigeons. The iniitial and terminal links were variable-interval schedules (always the same for both alternatives). The positions (left or right key) of the initial-link stimuli (red or green) were randomized while the correlation between color and food amount remained constant within each condition. The terminal-link stimuli were always presented on the center key. Except in two control groups and conditions, the terminal-link stimuli were the same color (nondifferential, blue or yellow). Over six conditions, the differences in food amont and the durations of the initial- and terminal-link schedules were manipulated. In 57 of 60 cases, birds generated choice proportions above .50 in favor of the initial-link stimlus that was correlated with the larger reinforcer. There was some indication that preference increased with shortened terminal-link durations. Because the terminal-link stimuli were nondifferential, differential responding in the initial links cannot be explained easily by conditioned reinforcement represented by the terminal-link stimuli. Thus, primiary reinforcement has a direct effect on initial-link responding in concurrent-chains schedules.     

Bias and sensitivity to reinforcement in a concurrent-chain schedule

Six pigeons were trained on concurrent-chain schedules in which the initial links were either dependent (Experiment 1) or independent (Experiment 2) concurrent variable-interval schedules and the terminal links were fixed-duration delays to reinforcement in blackout. A changeover delay of three seconds operated in the initial links. Both the initial- and terminal-link schedule values were varied over 61 experimental conditions. An analysis of the data using the generalized matching law showed that the sensitivity of initial-link response allocation to the frequency of terminal-link production was independent of both the duration of the terminal-link delays and, though less clearly, of the duration of the initial-link schedules. Sensitivity of initial-link response allocation to terminal-link reinforcer-rate ratios was a joint function of the terminal-link durations and the smaller average initial-link duration. The results showed that the generalized matching law is useful in analyzing concurrent-chain schedule performance and that a changeover delay in the initial links eliminates some terminal- to initial-link interactions that have made quantitative predictions for concurrent-chain performances difficult.     

Time and rate measures in choice transitions

Three experiments with pigeons studied the relation between time and rate measures of behavior under conditions of changing preference. Experiment 1 studied a concurrent chain schedule with random-interval initial links and fixed-interval terminal links; Experiment 2 studied a multiple chained random-interval fixed-interval schedule; and Experiment 3 studied simple concurrent random-interval random-interval schedules. In Experiment 1, and to a lesser extent in the other two experiments, session-average initial-link wait-time differences were linearly related to session-average response-rate differences. In Experiment 1, and to a lesser extent in Experiment 3, ratios of session-average initial-link wait times and response rates were related by a power function. The weaker relations between wait and response measures in Experiment 2 appear to be due to the absence of competition between responses. In Experiments 1 and 2, initial-link changes lagged behind terminal-link changes. These findings may have implications for the relations between fixed- and variable-interval procedures and suggest that more attention should be paid to temporal measures in studies of free-operant choice.     

Preference for fixed-interval schedules: effects of unequal initial links

Six homing pigeons were trained on concurrent chain schedules in which the terminal links were fixed-interval schedules of 5 sec or 15 sec. One initial-link schedule was always VI 27-sec; the other was varied over conditions from VI 27-sec to VI 181-sec. Preference measured in the initial links varied as a joint function of the initial- and terminal-link schedules. When the initial links were varied with constant, but unequal, terminal links, the slope of the function relating the logarithm of the initial-link response ratio to the logarithm of the terminal-link entry ratio differed from that obtained with equal terminal links. This result indicates that biases attributable to the terminal-link schedules were not constant. The rate of change of preference, or degree of undermatching, in the initial links depended on whether the shorter initial link led to the shorter or the longer terminal link. These results raise the question of whether bias and undermatching in concurrent schedule performance are independent measures.     

Choice and delay of reinforcement: Effects of terminal-link stimulus and response conditions

In two experiments, pigeons were exposed to concurrent-chains schedules in which a single initial-link variable-interval schedule led to access to terminal links composed of fixed-interval or fixed-delay schedules. In Experiment 1, an 8-s (or 16-s) delay to reinforcement was associated with the standard key, while reinforcer delay values associated with the experimental key were varied from 4 to 32 s. The results of Experiment 1 showed undermatching of response ratios to delay ratios with terminal-link fixed-delay schedules, whereas in some pigeons matching or overmatching was evident with the fixed-interval schedules. In Experiment 2, one pair of reinforcer delay values, either 8 versus 16 s or 16 versus 32 s, was used. In the first condition of Experiment 2, different delays were associated with different keylight stimuli (cued condition). In the second condition, different terminal-link delays were associated with the same stimulus, either a blackout (uncued-blackout condition) or a white key (uncued-white condition). To examine the role of responses emitted during delays, the keys were retracted during a delay (key-absent condition) in the third condition and responses were required by a fixed-interval schedule in the fourth condition. Experiment 2 demonstrated that the choice proportions for the shorter delay were more extreme in the cued condition than in the uncued-blackout condition, and that the response requirement imposed by the fixed-interval schedules did not affect choice of the shorter delay, nor did the key-absent and key-present conditions. These results indicate that the keylight-stimulus conditions affected preference for the shorter of two delays and that the findings obtained in Experiment 1 depended mainly on the keylight-stimulus conditions of the terminal links (i.e., the conditioned reinforcing value of the terminal-link stimuli).     

Resistance to change and preference for variable versus fixed response sequences

In Experiment 1, 4 pigeons were trained on a multiple chain schedule in which the initial link was a variable-interval (VI) 20-s schedule signalled by a red or green center key, and terminal links required four responses made to the left (L) and/or right (R) keys. In the REPEAT component, signalled by red keylights, only LRLR terminal-link response sequences were reinforced, while in the VARY component, signalled by green keylights, terminal-link response sequences were reinforced if they satisfied a variability criterion. The reinforcer rate for both components was equated by adjusting the reinforcer probability for correct REPEAT sequences across sessions. Results showed that initial- and terminal-link responding in the VARY component was generally more resistant to prefeeding, extinction, and response-independent food than responding in the REPEAT component. In Experiment 2, the REPEAT and VARY contingencies were arranged as terminal links of a concurrent chain and the relative reinforcer rate was manipulated across conditions. For all pigeons, initial-link response allocation was biased toward the alternative associated with the VARY terminal link. These results replicate previous reports that operant variation is more resistant to change than operant repetition (Doughty & Lattal, 2001), and show that variation is preferred to repetition with reinforcer-related variables controlled. Behavioral momentum theory (Nevin & Grace, 2000) predicts the covariation of preference and resistance to change in Experiments 1 and 2, but does not explain why these aspects of behavior should depend on contingencies that require repetition or variation.     

Pigeons' preference for variable-interval water reinforcement under widely varied water budgets.

Water budget of pigeons was varied to assess the dependence of risk-sensitive preferences upon economic context such as has been reported for energy-budget manipulations with small animals in behavioral ecology research. Fixed- and variable-interval terminal-link water schedules reinforced choice between equal variable-interval initial-link schedules arranged on two pecking keys. While keeping a severely restrictive budget the same across three phases of the experiment, a contrasting distinct ample budget was arranged in each. To mimic typical methods in behavioral ecology studies, in each ample budget a more than three-fold increase in amount of water per reinforcer presentation was instituted simultaneously with significantly increased overall access to water. Total choice response rates plummeted in the ample budgets, and body weights either increased significantly or remained unchanged in different phases as expected by the nature of the different manipulations. Clear preferences for the variable-interval schedule were found throughout the experiment, except for rare instances of key bias. The results agree with similar operant food-reinforcement studies and extend conditions under which risk preference apparently does not depend upon economic context.     

Dissociation of value and response strength

Four pigeons were exposed to multiple schedules and later to concurrent-chains schedules, with terminal links that had previously been multiple-schedule components. For 2 birds, the terminal-link schedules arranged an inverse relationship between response rate and reinforcement rate; for the other 2 birds a direct corresponding relationship was arranged. Those response rates were further modified by differentially reinforcing either longer or shorter interresponse times, relative to the current means. Although the birds' initial-link responses indicated preferences for terminal links with higher rates of reinforcement, in half the cases the birds responded during the terminal links in such a way as to produce lower rates of reinforcement, rates their initial-link behavior indicated they did not prefer. That outcome is inconsistent with maximization theory, but consistent with a strengthening analysis of behavior on single-key schedules.     

Homogeneous chains, heterogeneous chains, and delay of reinforcement

Three pigeons responded on two-component chain schedules in which the required response topography in the initial and terminal links was similar (a homogeneous chain) or dissimilar (a heterogeneous chain). Key-peck responding in the initial link under a variable-interval 60-second (VI 60) schedule produced a terminal link in which, in different conditions, either key pecking or foot treadling was reinforced according to a VI 60 schedule. Multiple VI 60 VI 60 schedules, in which the responses required in the chain schedules were maintained by primary reinforcement in the two components, preceded and followed each type of chain. These multiple schedules were used to ensure that both responses occurred reliably prior to introducing the chain schedule. Key-peck response rates in the initial link of the chain consistently were higher during the homogeneous chain than during the heterogeneous chain. These results illustrate that intervening events during a period separating an operant response from primary reinforcement influence that operant, independently of the delay between the response and reinforcement.     

Rapid acquisition of choice and timing and the provenance of the terminal-link effect

Eight pigeons responded in a concurrent-chains procedure in which terminal-link schedules changed pseudorandomly across sessions. Pairs of terminal-link delays either summed to 15 s or to 45 s. Across sessions, the location of the shorter terminal link changed according to a pseudorandom binary sequence. On some terminal links, food was withheld to obtain start and stop times, measures of temporal control. Log initial-link response ratios stabilized within the first half of each session. Log response ratio was a monotonically-increasing but nonlinear function of programmed log terminal-link immediacy ratio. There was an effect of absolute terminal-link duration on log response ratio: For most subjects, preference for the relatively shorter terminal-link delay was stronger when absolute delays were long than when absolute delays were short. Polynomial regressions and model comparison showed that differences in degree of nonlinearity, not in sensitivity to log immediacy ratio, produced this effect. Temporal control of stop times was timescale invariant with scalar variability, but temporal control of start times was not consistent across subjects or terminal-link durations.     

Choice for periodic schedules of reinforcement

Pigeons' responses in the presence of two concurrently available (initial-link) stimuli produced one of two different (terminal-link) stimuli according to identical but independent variable-interval schedules. Responding in the mutually exclusive terminal links was reinforced with food according to fixed-ratio schedules for six pigeons and according to fixed-interval schedules for two pigeons. None of the pigeons matched the proportion of (choice) responses in the initial links to the proportion of the rates of reinforcement obtained during the terminal links. Instead, as the values of each of the terminal-link schedules were increased by a constant amount, the choice proportions for the stimulus associated with the smaller of the two values increased, even though the relative rates of reinforcement during the terminal links decreased. These results are incompatible with those from previous studies with aperiodic (variable-interval or variable-ratio) schedules. The present results do suggest, however, that in transforming aperiodic schedules into their periodic equivalents, it may be necessary to consider the size of the smallest interreinforcement interval comprising the terminal-link schedules.     

Transfer tests of stimulus value in concurrent chains

We report two experiments that use transfer tests to investigate whether in concurrent chains the value of a terminal-link stimulus is affected by the alternate terminal link. In Experiment 1, two groups of pigeons were trained on multiple concurrent-chains schedules in which switching between the schedules was via pecking a changeover key. For one group, the terminal links were fixed-interval 8 s versus fixed-interval 16 s in one component and fixed-interval 16 s versus fixed-interval 32 s in the other component. For a second group, the terminal links were variable-interval 10 s versus variable-interval 20 s in one component and variable-interval 20 s versus variable-interval 40 s in the other. After sufficient baseline training had been given so that performances had stabilized, transfer tests were conducted in which the two chains with equal terminal-link schedules were presented together as a new concurrent pair. For 6 of the 7 subjects, initial-link responding changed fairly rapidly during the test in the manner predicted if the values of the terminal links were equal. In Experiment 2, pigeons were trained on multiple concurrent chains using a two-key procedure, and the terminal links were the same variable-interval schedules as in Experiment 1. After baseline training, transfer tests were conducted that assessed (a) the relative reinforcing strength of the terminal-link stimuli in a novel initial-link situation and (b) the relative ability of those stimuli to evoke responding. The data from the reinforcing strength test were consistent with those from Experiment 1, but those from the evocation strength test were not. Although this discrepancy shows that responding in transfer tests is not solely a function of stimulus value, the results from both experiments suggest, overall, that value is determined by the stimulus—reinforcer relation independently of the alternative terminal link.     

Concurrent-chain performance: Effects of absolute and relative terminal-link entry frequency

Six pigeons were trained in a concurrent-chain procedure with constant variable-interval 6-s variable-interval 12-s terminal links. Five groups of conditions were arranged. Within a group of conditions, the duration of one initial-link schedule was held constant and the duration of the other initial link was varied. The duration of the varied initial link was always longer than, or equal to, the constant initial-link duration. The duration of the shorter initial link was varied across groups of conditions from 5 s to 70 s. The data from each group were well described by the generalized matching law. Sensitivity (a) to the terminal-link entry ratio increased as the shorter initial-link duration increased, but appeared to reach an asymptote at shorter initial-link durations greater than 32 s. Terminal-link bias did not change with changes in shorter initial-link duration for the response-allocation data, but showed a small increase with increasing shorter initial-link duration for the time-allocation data.     

Negatively reinforced key pecking

A reinforcement-switching procedure was used to produce negatively reinforced key pecking in pigeons. First, key pecking on a chain schedule (fixed-interval 10-sec variable-interval 60-sec) was conditioned using grain reinforcement. Second, intermittent shock in the initial link was introduced at a low intensity and gradually increased. Third, food reinforcement in the terminal link was eliminated. With shock at 90 V occurring on the average every 3 sec, initial-link pecking was maintained with no terminal-link food. Three of four pigeons responded consistently at shock intensities of 90, 70, and 50 V but not at 30 V. A fourth pigeon responded at but not below 90 V. Rate of response was directly related to shock frequency. Eliminating food deprivation did not affect the negatively reinforced performance.     

Response strength in multiple periodic and aperiodic schedules

Responding in multiple periodic and aperiodic schedules of equal mean reinforcement rate was examined during extinction, satiation, and in the presence of various free-food schedules. In Experiments I and II, pigeons were trained on multiple variable-interval–fixed-interval schedules. Decreases in the rate of responding due to extinction, satiation, or food schedules were approximately equal regardless of the temporal pattern of reinforcer presentation. In Experiment III, pigeons responded on a two-component multiple schedule in which each component was a two-member homogeneous response chain terminating in a fixed-interval schedule during one component and in a variable-interval schedule during the other. The length of both terminal links was varied over a series of conditions. Initial-link responding in the fixed-interval component was reduced more by increasing terminal-link length than was initial-link responding in the variable-interval component. However, no differences in resistance to satiation and extinction were obtained across the fixed and variable components. If the relative decrease in responding produced by satiation and extinction is used as an index of the “value” of the conditions maintaining responding, then these data suggest that fixed and variable schedules of equal mean length are equally valued. This conclusion, however, is not consistent with findings of preference for variable over fixed schedules obtained in studies using concurrent-chain procedures.     

Preference for fixed-interval schedules: effects of initial-link length

Six homing pigeons were trained on a variety of concurrent-chains schedules in which the initial links were equal variable-interval schedules and the terminal links were fixed-interval schedules. Both terminal-link and initial-link schedules were systematically varied. The results showed that preference for a particular terminal-link schedule combination was greater, the shorter the initial-link schedules. The data closely matched predictions from the model of choice suggested by Davison and Temple (1973), but did not match predictions from two other models. An alternative method for analyzing concurrent-chains performance by assuming that the schedule consists of both chained schedules and successive, discriminated components that comprise multiple schedules, was suggested.     

Stimulus functions in some chained fixed-ratio schedules of reinforcement

For two pigeons, behavior was compared in equivalent two-link chained and tandem schedules in which 100 responses were required for food reinforcement but the responses required in the two links of the chained schedule were varied over the values 2-98, 50-50, and 98-2. Initial pausing, response rate in each link, and the total time to complete each ratio were recorded. In chained components, initial-link pausing (for both birds) and total time to complete the ratio (for one bird) were generally shorter when the response requirement was 2-98 than when it was 50-50 or 98-2; for terminal-link pausing this relationship was reversed for both birds. There were also systematic changes in behavior in tandem components, and in the relationship between behavior in chained and tandem components. The results are discussed in terms of conditioned reinforcement.     

DOES SENSITIVITY TO MAGNITUDE DEPEND ON THE TEMPORAL DISTRIBUTION OF REINFORCEMENT?

Our research addressed the question of whether sensitivity to relative reinforcer magnitude in concurrent chains depends on the distribution of reinforcer delays when the terminal-link schedules are equal. In Experiment 1, 12 pigeons responded in a two-component procedure. In both components, the initial links were concurrent variable-interval 40-s variable-interval 40-s, and the terminal links were both 20-s interval schedules in which responses were reinforced by either 4-s of grain in one, or 2-s of grain in the other. The only difference between the components was whether the terminal-link schedules were fixed interval or variable intervals. For all subjects, the relative rate of responding in the initial links for the terminal link that produced the 4-s reinforcer was greater when the terminal links were fixed-interval schedules than when they were variable-interval schedules. This result is contrary to the prediction of Grace's (1994) contextual choice model, but is consistent with both Mazur's (2001) hyperbolic value-added model and Killeen's (1985) incentive theory. In Experiment 2, 4 pigeons responded in a concurrent-chains procedure in which 4-s or 2-s reinforcers were provided independently of responding according to equal fixed-time or mixed-time schedules. Preference for the 4-s reinforcer increased as the variability of the intervals comprising the mixed-time schedules was decreased. Generalized-matching sensitivity of initial-link response allocation to relative reinforcer magnitude was proportional to the geometric mean of the terminal-link delays.