Sensitivity to reinforcer duration in a self-control procedure

In a concurrent-chains procedure, pigeons' responses on left and right keys were followed by reinforcers of different durations at different delays following the choice responses. Three pairs of reinforcer delays were arranged in each session, and reinforcer durations were varied over conditions. In Experiment 1 reinforcer delays were unequal, and in Experiment 2 reinforcer delays were equal. In Experiment 1 preference reversal was demonstrated in that an immediate short reinforcer was chosen more frequently than a longer reinforcer delayed 6 s from the choice, whereas the longer reinforcer was chosen more frequently when delays to both reinforcers were lengthened. In both experiments, choice responding was more sensitive to variations in reinforcer duration at overall longer reinforcer delays than at overall shorter reinforcer delays, independently of whether fixed-interval or variable-interval schedules were arranged in the choice phase. We concluded that preference reversal results from a change in sensitivity of choice responding to ratios of reinforcer duration as the delays to both reinforcers are lengthened.     

Choice in a self-control paradigm: Quantification of experience-based differences

Previous quantitative models of choice in a self-control paradigm (choice between a larger, more-delayed reinforcer and a smaller, less-delayed reinforcer) have not described individual differences. Two experiments are reported that provide additional quantitative data on experience-based differences in choice between reinforcers of varying sizes and delays. In Experiment 1, seven pigeons in a self-control paradigm were exposed to a fading procedure that increased choices of the larger, more-delayed reinforcer through gradually decreasing the delay to the smaller of two equally delayed reinforcers. Three control subjects, exposed to each of the small-reinforcer delays to which the experimental subjects were exposed, but for fewer sessions, demonstrated that lengthy exposure to each of the conditions in the fading procedure may be necessary in order for the increase to occur. In Experiment 2, pigeons with and without fading-procedure exposure chose between reinforcers of varying sizes and delays scheduled according to a concurrent variable-interval variable-interval schedule. In both experiments, pigeons with fading-procedure exposure were more sensitive to variations in reinforcer size than reinforcer delay when compared with pigeons without this exposure. The data were described by the generalized matching law when the relative size of its exponents, representing subjects' relative sensitivity to reinforcer size and delay, were grouped according to subjects' experience.     

A comparison of delays and ratio requirements in self-control choice.

In a discrete-trial procedure, pigeons could choose between 2-s and 6-s access to grain by making a single key peck. In Phase 1, the pigeons obtained both reinforcers by responding on fixed-ratio schedules. In Phase 2, they received both reinforcers after simple delays, arranged by fixed-time schedules, during which no responses were required. In Phase 3, the 2-s reinforcer was available through a fixed-time schedule and the 6-s reinforcer was available through a fixed-ratio schedule. In all conditions, the size of the delay or ratio leading to the 6-s reinforcer was systematically increased or decreased several times each session, permitting estimation of an "indifference point," the schedule size at which a subject chose each alternative equally often. By varying the size of the schedule for the 2-s reinforcer across conditions, several such indifference points were obtained from both fixed-time conditions and fixed-ratio conditions. The resulting "indifference curves" from fixed-time conditions and from fixed-ratio conditions were similar in shape, and they suggested that a hyperbolic equation describes the relation between ratio size and reinforcement value as well as the relation between reinforcer delay and its reinforcement value. The results from Phase 3 showed that subjects chose fixed-time schedules over fixed-ratio schedules that generated the same average times between a choice response and reinforcement.     

Responding during reinforcement delay in a self-control paradigm.

Eight pigeons chose between a small, immediate reinforcer and a large, increasingly delayed reinforcer. Responding during the large-reinforcer delays was examined. During large-reinforcer delays, pecks on one key produced the small, immediate reinforcer; pecks on the other key had no effect. Thus, a pigeon could reverse its initial choice of the large, delayed reinforcer, or it could maintain its original choice. Pigeons that made a relatively high number of initial large-reinforcer choices tended to maintain these choices, and those pigeons that actually received a relatively high number of large reinforcers, tended to respond more frequently on the ineffective key during the delay periods. The findings suggest that some previous studies of self-control training in pigeons may have resulted in increased self-control partially due to a lack of opportunity for the pigeons to change their choices.     

Choice and delay of reinforcement: Effects of terminal-link stimulus and response conditions

In two experiments, pigeons were exposed to concurrent-chains schedules in which a single initial-link variable-interval schedule led to access to terminal links composed of fixed-interval or fixed-delay schedules. In Experiment 1, an 8-s (or 16-s) delay to reinforcement was associated with the standard key, while reinforcer delay values associated with the experimental key were varied from 4 to 32 s. The results of Experiment 1 showed undermatching of response ratios to delay ratios with terminal-link fixed-delay schedules, whereas in some pigeons matching or overmatching was evident with the fixed-interval schedules. In Experiment 2, one pair of reinforcer delay values, either 8 versus 16 s or 16 versus 32 s, was used. In the first condition of Experiment 2, different delays were associated with different keylight stimuli (cued condition). In the second condition, different terminal-link delays were associated with the same stimulus, either a blackout (uncued-blackout condition) or a white key (uncued-white condition). To examine the role of responses emitted during delays, the keys were retracted during a delay (key-absent condition) in the third condition and responses were required by a fixed-interval schedule in the fourth condition. Experiment 2 demonstrated that the choice proportions for the shorter delay were more extreme in the cued condition than in the uncued-blackout condition, and that the response requirement imposed by the fixed-interval schedules did not affect choice of the shorter delay, nor did the key-absent and key-present conditions. These results indicate that the keylight-stimulus conditions affected preference for the shorter of two delays and that the findings obtained in Experiment 1 depended mainly on the keylight-stimulus conditions of the terminal links (i.e., the conditioned reinforcing value of the terminal-link stimuli).     

Effects of response type on pigeons' sensitivity to variation in reinforcer amount and reinforcer delay

Twelve pigeons, divided into two groups, responded on concurrent nonindependent variable-interval schedules to obtain access to grain by either pecking keys or pressing treadles. Either the amount of grain or the delay to the receipt of grain was varied in separate conditions to determine the sensitivity of relative responding to variation in reinforcer amount (s_A), the sensitivity to variation in reinforcer delay (s_D), and s_A/s_D, a measure related to self-control. There were no significant differences between the two groups in the values of s_A, s_D, and s_A/s_D. These results suggest that the values of s_A, s_D, and s_A/s_D for pigeons may be similar across these two types of responses.     

Preference between variable-ratio and fixed-ratio schedules: local and extended relations.

Although it has repeatedly been demonstrated that pigeons, as well as other species, will often choose a variable schedule of reinforcement over an equivalent (or even richer) fixed schedule, the exact nature of that controlling relation has yet to be fully assessed. In this study pigeons were given repeated choices between concurrently available fixed-ratio and variable-ratio schedules. The fixed-ratio requirement (30 responses) was constant throughout the experiment, whereas the distribution of individual ratios making up the variable-ratio schedule changed across phases: The smallest and largest of these components were varied gradually, with the mean variable-ratio requirement constant at 60 responses. The birds' choices of the variable-ratio schedule tracked the size of the smallest variable-ratio component. A minimum variable-ratio component at or near 1 produced strong preference for the variable-ratio schedule, whereas increases in the minimum variable-ratio component resulted in reduced preference for the variable-ratio schedule. The birds' behavior was qualitatively consistent with Mazur's (1984) hyperbolic model of delayed reinforcement and could be described as approximate maximizing with respect to reinforcement value.     

Choice and segmented interreinforcement intervals

Pigeons were trained on a two-key concurrent schedule, where food reinforcers on one key were arranged by a simple variable-interval schedule and on the other key by a chain variable-interval variable-interval schedule. When the initial link of the chain was in effect, the pigeons tended to respond more on the simple variable-interval schedule, and hence less on the chain, than would be expected from a comparison of both the local and overall rates of reinforcement of the two schedules. When the terminal link of the chain was in effect, the pigeons responded more on the chain than would be expected from a comparison of the rates of reinforcement of the schedules then in effect. Overall responding on the chain was not proportional to overall reinforcement on the chain but rather was a by-product of responding during initial- and terminal-link phases.     

Effects of reinforcement context on choice

Two experiments investigated the effects of successive reinforcement contexts on choice. In the first, concurrent variable-interval schedules of primary reinforcement operated during the initial links of concurrent chains. The rate of this reinforcement arranged by the concurrent schedules was decreased across conditions: When it was higher than the terminal-link rate, preference for the higher frequency initial-link schedule increased relative to baseline. (During baseline, a standard concurrent-schedule procedure was in effect). When the initial-link reinforcement rate was lower than the terminal-link rate, preference converged toward indifference. In the second experiment, a chain schedule was available on a third key while a concurrent schedule was in effect on the side keys. When the terminal link of the chain schedule was produced, the side keys became inoperative. Availability of the chain schedule did not affect choice between the concurrent schedules. These results show that only when successive reinforcement contexts are produced by choice responding do those successive contexts affect choice in concurrent schedules.     

Stimulus effects on concurrent performance in transition

Six experimentally naive pigeons were exposed to concurrent variable-interval variable-interval schedules in a three-key procedure in which food reinforcement followed pecks on the side keys and pecks on the center key served as changeover responses. In Phase 1, 3 birds were exposed to 20 combinations of five variable-interval values, with each variable-interval value consistently associated with a different color on the side keys. Another 3 pigeons were exposed to the same 20 conditions, but with a more standard procedure that used a nondifferential discriminative stimulus on the two side keys throughout all conditions. In Phase 2, the differential and nondifferential stimulus conditions were reversed for each pigeon. Each condition lasted for one 5-hr session and one subsequent 1-hr session. In the last 14 conditions of each phase, the presence of differential discriminative stimuli decreased the time necessary for differential responding to develop and increased the sensitivity of behavior to reinforcement distribution in the 1st hr of training; during the last hours of training in each condition, however, the effects of the differential discriminative stimuli could not be distinguished from the effects of reinforcement distribution per se. These results show the importance of studying transitions in behavior as well as final performance. They may also be relevant to discrepancies in the results of previous experiments that have used nonhuman and human subjects.     

The effects of concurrent responding and reinforcement on behavioral output

Four birds key pecked on concurrent variable-interval one-minute variable-interval four-minute schedules with a two-second changeover delay. Response rates to the variable-interval one-minute key were then reduced by signaling its reinforcer availability and later by providing its reinforcers independently of responding. Each manipulation increased response rates to the variable-interval four-minute key even though relative reinforcement rates were unchanged. In a final phase, eliminating the variable-interval one-minute key and its schedule produced the highest rates of all to the variable-interval four-minute key. These results show that both reinforcement and response rates to one schedule influence response rates to another schedule. These results join those of Guilkey, Shull, & Brownstein (1975) in failing to replicate Catania (1963). Moreover, they violate the predictions of the equation for simple action (de Villiers & Herrnstein, 1976). In terms of a median-rate measure (reciprocal of the median interresponse time), rates to the variable-interval four-minute key were high when responding was not reduced to the variable-interval one-minute key and were low when it was reduced. This rate difference suggests a process difference between concurrent-schedule procedures that maintain high concurrent response rates versus those that do not. This process difference jeopardizes attempts to integrate single- and concurrent-operant performances within a single formulation.     

Choice, experience, and the generalized matching law

Five pigeons were exposed to different pairs of concurrent variable-interval, variable-interval schedules on nine experimental conditions of 30 sessions each. For every session, the parameters of the generalized matching equation were computed for the first five, six, seven, eight, and nine experimental conditions. The exponent a, both for response and time distribution, tended to decrease with increases in number of experimental conditions and to increase with number of sessions per condition, but values of k (bias) varied unsystematically. When the subjects were exposed to five new pairs of schedules, with 55 sessions per condition, the findings were confirmed. Data from the literature on the generalized matching law suggest that the variability of exponent values may be explained in part by the use of naive or experienced subjects in different investigations and by the variability in number of experimental conditions and in number of sessions per condition.