Choice for periodic schedules of reinforcement

Pigeons' responses in the presence of two concurrently available (initial-link) stimuli produced one of two different (terminal-link) stimuli according to identical but independent variable-interval schedules. Responding in the mutually exclusive terminal links was reinforced with food according to fixed-ratio schedules for six pigeons and according to fixed-interval schedules for two pigeons. None of the pigeons matched the proportion of (choice) responses in the initial links to the proportion of the rates of reinforcement obtained during the terminal links. Instead, as the values of each of the terminal-link schedules were increased by a constant amount, the choice proportions for the stimulus associated with the smaller of the two values increased, even though the relative rates of reinforcement during the terminal links decreased. These results are incompatible with those from previous studies with aperiodic (variable-interval or variable-ratio) schedules. The present results do suggest, however, that in transforming aperiodic schedules into their periodic equivalents, it may be necessary to consider the size of the smallest interreinforcement interval comprising the terminal-link schedules.     

An appraisal of preference for multiple versus mixed schedules

Pigeons' choice responses on either of two keys occasionally produced entry into a terminal link associated with that key. During the terminal links, responses produced access to grain according to mixed- or multiple-interval schedules. The multiple schedules provided stimuli correlated with the interval of time preceding reinforcement whereas the mixed schedules did not. The two subjects reliably preferred the multiple schedules to the mixed schedules throughout a series of replications. Preference for the multiple schedule was much smaller than suggested by earlier work comparing multiple and mixed schedules that had much higher rates of entry into the terminal links. Preference for the multiple schedule was greatly increased in this study when the rate of entry into the terminal schedules was increased. As in previous studies, these high preferences may have been the result of a sharp increase in the number of reinforcements on the multiple (as opposed to the mixed) schedule. The reliable but smaller preferences for the multiple schedule found with lower rates of entry into the terminal links were unconfounded by differences in the number of reinforcements obtained in the two terminal links.     

Choice and response contingencies

Two experiments investigated the extent to which response contingencies influence the choice between two schedules of reinforcement by exposing pigeons to a concurrent-chains procedure in which reinforcers in one terminal link were response-independent, and in the other terminal link, response-dependent. In Experiment 1, the pigeons were indifferent between an aperiodic, response-independent schedule and an aperiodic, response-dependent schedule that required a minimum rate of responding. This finding limits the generality of a required-rate contingency as a determinant of choice, which contingency had been previously demonstrated in a context of periodic reinforcement to evoke preference for an alternate schedule. In Experiment 2, the pigeons preferred a periodic, response-independent schedule to a periodic, response-dependent schedule that shared a feature with a required-rate schedule: there was a requirement to respond early in the interreinforcement interval, when responding produced reinforcement only later. The results of the two experiments suggest the following general interpretation: pigeons prefer a second schedule to the extent that the response contingencies of the first schedule must be satisfied during discriminable periods of nonreinforcement.     

Free-operant choice behavior: A molecular analysis

Pigeons' pecks to two concurrent initial-link stimuli occasionally produced one of two mutually exclusive terminal links. Further responding to the terminal-link stimulus produced food under fixed-interval or fixed-ratio schedules. In such concurrent chained schedules, investigators rarely use a changeover delay to control superstitious switching, although it is customary to use a changeover delay in simple concurrent schedules in which choice responses produce food directly. When terminal-link fixed-interval schedules were equal or similar in duration and no changeover delay was employed, conditional probabilities of choice for a schedule were found to be lower if the last choice was for that schedule than if the last choice was for the other schedule (“switching dependency”). Imposition of a changeover delay with equal or unequal terminal links produced the opposite pattern: conditional probabilities of choice for a schedule were higher if the last choice was for that schedule than if the last choice was for the other schedule. Turning off all chamber lights during the changeover delay interval attenuated these “repetition dependencies.” The results indicate that excessive switching can complicate the interpretation of data from concurrent chains much as from simple concurrent schedules, and that using blackouts to control switching may be preferable to using a changeover delay.     

The psychological distance to reward

Pigeons' responses in the presence of two concurrently available (initial-link) stimuli produced entry into one of two different and mutually exclusive terminal link stimuli according to identical but independent variable-interval schedules. In one experiment, a two-component chained fixed-interval schedule produced food in one terminal link while a simple fixed-interval schedule produced food in the other terminal link. When the interreinforcement intervals were equal in the two terminal links (i.e., the simple fixed-interval was twice the size of each of the components in the chained schedule) pigeons preferred the simple fixed-interval as measured by their relative rates of responding in the concurrently available initial links. This preference increased as the duration of the terminal links increased. The preference could be reversed by making the simple fixed-interval schedule sufficiently longer than the chained schedule. In the second experiment, the terminal links consisted of two- vs three-component chained fixed-intervals, again with equal interreinforcement intervals. Pigeons preferred the two-component chain to the three-component chain, although these results were less consistent and less dramatic than those in the first experiment. Again, preference increased as the duration of the terminal links increased. The results show that an organism's choice for a schedule will be substantially lowered by the chaining operation even when the interreinforcement interval remains constant.     

Choice between two-component chained and tandem schedules

Pigeons were trained on a two-key choice procedure in which a pair of equal and concurrently available variable-interval schedules (initial links) arranged entry into one or the other of two mutually exclusive schedules (terminal links) that ended in primary reinforcement. The terminal links were two-component chained or tandem schedules. Responses during the initial links were distributed equally on the two keys whenever the terminal links were associated with the same sets of interreinforcement intervals. Whether or not the terminal-link interreinforcement intervals were the same on the two keys, initial-link responding was affected by neither the presence nor relative durations of differentially signalled components within a terminal-link schedule. The simplest interpretation of these results is that initial-link responding is maintained directly by delayed primary reinforcement, rather than conditioned reinforcement afforded by the stimuli correlated with the terminal-link schedule components. This finding suggests that aspects of chained schedule performance usually attributed to conditioned reinforcement might best be reinterpreted in terms of delayed primary reinforcement and various discriminative functions served by the component-correlated stimuli.     

Tests of transitivity in choices between fixed and variable reinforcer delays.

This experiment tested for transitivity in pigeons' choices between variable-time (VT) and fixed-time (FT) schedules. In a discrete-trials procedure, a subject chose between two alternatives by making a single key peck. Each choice was between a "standard alternative," which was the same schedule throughout a condition, and an "adjusting alternative," in which the delay to reinforcement was systematically increased or decreased many times a session. These adjustments enabled an approximate indifference point to be identified--the value of the adjusting delay at which the subject chose each alternative about equally often. Each test of transitivity involved four conditions. In one, the standard alternative was a variable-time schedule with a 2-s reinforcer, and the adjusting alternative also delivered a 2-s reinforcer. A second condition was similar except that the adjusting alternative delivered a 5-s reinforcer. The indifference point from each of these conditions was then converted to a fixed-time schedule for subsequent comparisons in the third and fourth conditions, respectively. Each of these last two conditions compared one of the fixed-time schedules (based upon the previous conditions and including their different reinforcer durations) with an adjusting schedule that delivered the alternative reinforcer duration, to determine whether the obtained indifference points would be those predicted from the prior alternative-duration comparisons with the VT schedule. There was little evidence for intransitivity of choice: Averaged across subjects and replications, the obtained indifference points deviated from perfect transitivity by less than 8%, and these deviations were not statistically significant. These results contrast with those of Navarick and Fantino (1972), who found frequent violations of transitivity between periodic and aperiodic schedules using a concurrent-chains procedure with variable-interval schedules in the initial links.     

Psychological distance to reward: Segmentation of aperiodic schedules of reinforcement

College students responded for monetary rewards in two experiments on choice between differentially segmented aperiodic schedules of reinforcement. On a microcomputer, the concurrent chains were simulated as an air-defense video game in which subjects used two radars for detecting and destroying enemy aircraft. To earn more cash-exchangeable points, subjects had to shoot down as many planes as possible within a given period of time. For both experiments, access to one of two radar systems (terminal link) was controlled by a pair of independent concurrent variable-interval 60-s schedules (initial link) with a 4-s changeover delay always in effect. In Experiment 1, the appearance of an enemy aircraft in the terminal link was determined by a variable-interval (15 s or 60 s) schedule or a two-component chained variable-interval schedule of equal duration. Experiment 2 was similar to Experiment 1 except for the segmented schedule, which had three components. Subjects preferred the unsegmented schedule over its segmented counterpart in the conditions with variable-interval 60 s, and preference tended to be more pronounced with more components in the segmented schedule. These findings are compatible with those from previous studies of periodic and aperiodic schedules with pigeons or humans as subjects.     

Freedom and knowledge: an experimental analysis of preference in pigeons

Relative responding in initial links of concurrent-chain schedules showed that pigeons preferred free to forced choices and informative to uninformative stimuli. Variable-interval initial links on two lower keys (white) of a six-key chamber produced terminal links on either two upper-left keys (blue and/or amber) or two upper-right keys (green and/or red). Terminal.links in which pecks on either of two lit keys produced fixed-interval reinforcement (free choice) were preferred to links with only one lit fixed-interval key available (forced choice). Terminal links with different key colors correlated with concurrent fixed-interval reinforcement and extinction (informative stimuli) were preferred to links with these schedules operating on same-color keys (uninformative stimuli). Scheduling extinction for one of the two free-choice keys assessed preference for two lit keys over one lit key, but confounded number with whether stimuli were informative. Fixed-interval reinforcement for both keys in each terminal link, but with different-color keys in one link and same-color keys in the other, showed that preference for informative stimuli did not depend on stimulus variety. Preferences were independent of relative responses per reinforcement and other properties of terminal-link performance.     

Choice and rate of reinforcement

Pigeons' responses in the presence of two concurrently available (initial-link) stimuli produced one of two different (terminal-link) stimuli. The rate of reinforcement in the presence of one terminal-link stimulus was three times that of the other. Three different pairs of identical but independent variable-interval schedules controlled entry into the terminal links. When the intermediate pair was in effect, the pigeons distributed their (choice) responses in the presence of the concurrently available stimuli of the initial links in the same proportion as reinforcements were distributed in the mutually exclusive terminal links. This finding was consistent with those of earlier studies. When either the pair of larger or smaller variable-interval schedules was in effect, however, proportions of choice responses did not match proportions of reinforcements. In addition, matching was not obtained when entry into the terminal links was controlled by unequal variable-interval schedules. A formulation consistent with extant data states that choice behavior is dependent upon the amount of reduction in the expected time to primary reinforcement, as signified by entry into one terminal link, relative to the amount of reduction in expected time to reinforcement signified by entry into the other terminal link.     

Conditioned reinforcement dynamics in three-link chained schedules.

In two experiments rats were trained on three-link concurrent-chains schedules of reinforcement. In Experiment 1, additional entries to one terminal link were added during one of the middle links to a baseline schedule that was otherwise equal for the two chains, and, depending on the condition, these additional terminal-link presentations ended either in food or in no food. When food occurred, preference was always in favor of the chain with the additional terminal-link presentations (which also entailed a higher rate of reinforcement). When no food occurred at the end of the additional terminal links, the outcome depended on the nature of the stimuli associated with these additional terminal links. When stimuli different from the reinforced baseline terminal links were used for the no-food terminal links, preference was against the choice alternative that led to the extra periods of extinction. When the same stimulus was used for the two kinds of terminal links, preference was near indifference, that is, significantly greater than when different stimuli were used. In Experiment 2, rats learned repeated reversals of a simultaneous discrimination under a three-link concurrent-chains schedule, in which the food or no-food choice outcomes were delayed until the end of the chain. Different conditions were defined by the point in the chain at which differential stimuli occurred. When the middle and terminal links provided no differential stimuli, discrimination was acquired more slowly than when differential stimuli occurred in both links. When differential stimuli occurred in the middle but not the terminal links, acquisition rates were intermediate. Both experiments together show that the effects of stimuli in a chain schedule are due partly to the time to food correlated with the stimuli and partly to the time to the next conditioned reinforcer in the sequence.     

Preference for multiple versus mixed schedules of reinforcement

Five pigeons were trained in a concurrent-chain procedure. In the initial links, equal nonindependent variable-interval schedules were available concurrently on two keys. Completing the schedule on either key led to exclusive presentation of one of two further variable-interval schedules for a fixed period of time. During these terminal links, as many reinforcers as were scheduled could be obtained. If the response producing this terminal link occurred on one key, differential stimuli signaled which variable-interval schedule had been produced. If the response producing the terminal link occurred on the other key, no such differential stimuli were available. Once the fixed period of time elapsed, the initial links were reinstated. In Experiment 1, the period of time for which the terminal links were available was always 10 s and the absolute duration of the initial links was varied. Subjects preferred the alternative leading to the multiple schedule when the initial-link duration was short, but preferred the alternative leading to the mixed schedule when the initial-link durations were longer. In Experiment 2, both the initial-link duration and the duration of the terminal links were varied. The effect of initial-link duration was identical to that in Experiment 1 and there was no systematic effect of varying the terminal-link duration.     

Preference for fixed-interval schedules: an alternative model

Pigeons were trained under concurrent chain schedules in which the initial links were equal aperiodic schedules and the terminal links were fixed-interval schedules. Choice proportions in the initial links were measured in 26 experimental conditions. The data showed the inadequacy of previous models of concurrent chain performance. A new model was suggested in which choice is a joint function of terminal-link times, overall reinforcement rates, and terminal-link entries. This model accounted for 94% of the variance in the present data and for substantial percentages of the variance in previously reported data. The model simplifies to matching between response ratios and obtained reinforcement rate ratios for simple concurrent schedule performance.     

Effects of reinforcement context on choice

Two experiments investigated the effects of successive reinforcement contexts on choice. In the first, concurrent variable-interval schedules of primary reinforcement operated during the initial links of concurrent chains. The rate of this reinforcement arranged by the concurrent schedules was decreased across conditions: When it was higher than the terminal-link rate, preference for the higher frequency initial-link schedule increased relative to baseline. (During baseline, a standard concurrent-schedule procedure was in effect). When the initial-link reinforcement rate was lower than the terminal-link rate, preference converged toward indifference. In the second experiment, a chain schedule was available on a third key while a concurrent schedule was in effect on the side keys. When the terminal link of the chain schedule was produced, the side keys became inoperative. Availability of the chain schedule did not affect choice between the concurrent schedules. These results show that only when successive reinforcement contexts are produced by choice responding do those successive contexts affect choice in concurrent schedules.     

Choice and delay of reinforcement: Effects of terminal-link stimulus and response conditions

In two experiments, pigeons were exposed to concurrent-chains schedules in which a single initial-link variable-interval schedule led to access to terminal links composed of fixed-interval or fixed-delay schedules. In Experiment 1, an 8-s (or 16-s) delay to reinforcement was associated with the standard key, while reinforcer delay values associated with the experimental key were varied from 4 to 32 s. The results of Experiment 1 showed undermatching of response ratios to delay ratios with terminal-link fixed-delay schedules, whereas in some pigeons matching or overmatching was evident with the fixed-interval schedules. In Experiment 2, one pair of reinforcer delay values, either 8 versus 16 s or 16 versus 32 s, was used. In the first condition of Experiment 2, different delays were associated with different keylight stimuli (cued condition). In the second condition, different terminal-link delays were associated with the same stimulus, either a blackout (uncued-blackout condition) or a white key (uncued-white condition). To examine the role of responses emitted during delays, the keys were retracted during a delay (key-absent condition) in the third condition and responses were required by a fixed-interval schedule in the fourth condition. Experiment 2 demonstrated that the choice proportions for the shorter delay were more extreme in the cued condition than in the uncued-blackout condition, and that the response requirement imposed by the fixed-interval schedules did not affect choice of the shorter delay, nor did the key-absent and key-present conditions. These results indicate that the keylight-stimulus conditions affected preference for the shorter of two delays and that the findings obtained in Experiment 1 depended mainly on the keylight-stimulus conditions of the terminal links (i.e., the conditioned reinforcing value of the terminal-link stimuli).     

Resistance to change produced by access to fixed-delay versus variable-delay terminal links

Pigeons' responding was reinforced on a multiple schedule consisting of two two-link chain schedules presented in regular alternation. Responding in initial links (always variable-interval 60-s) produced a key-color change and access to a terminal link. The terminal link for one chain provided food after a fixed delay (fixed-interval or fixed-time); the terminal link for the other provided food after a variable delay (variable-interval or variable-time). The average duration of the terminal-link schedules was varied across conditions, but in every condition the arithmetic mean of the variable-delay terminal-link schedule was equal to the duration of the fixed delay. Response rates were higher in the initial links of the chains with the variable-delay terminal links. Response-decreasing operations (satiation, extinction) were used after performances reached asymptote. Response rates maintained by access to variable-delay terminal links tended to be more resistant to change than were rates maintained by access to fixed-delay terminal links. These results are consistent with the preference for variable- over fixed-interval terminal links observed with concurrent-chains schedules, suggesting (1) that immediacy of reinforcement influences the conditioned reinforcing potency of access to a terminal link and (2) that choice in concurrent chains and resistance of responding to change may be manifestations of the same effect of reinforcement.     

Choice for conditioned reinforcers in the signaled absence of primary reinforcement

Pigeons responded in a multiple schedule in which concurrent schedules of brief-stimulus presentation alternated with a component in which food was available (concurrent-chains component). In the initial links of the concurrent-chains component subjects chose either of two stimuli each correlated with the terminal link of one chain. The terminal links involved either variable-interval 30-second or variable-interval 60-second schedules. In the brief-stimulus component subjects chose between 0.5-second presentations of the terminal-link stimuli from the concurrent-chains component. Responding was generally maintained in the brief-stimulus component in two subjects for more than 300 sessions, suggesting that brief stimuli were conditioned reinforcers. During the brief-stimulus component, in 17 of 21 cases for which a minimal number of responses occurred, choice proportions above 0.55 were obtained for the brief-stimulus presentations correlated with the higher rate of primary reinforcement in the concurrent-chains component. These results support the suggestion that choice in conventional concurrent-chains procedures is partially controlled by production of the terminal-link stimuli.     

Choice and transformed interreinforcement intervals

Pigeons chose between two aperiodic, time-based schedules of reinforcement. The arithmetic mean interreinforcement interval of the first schedule was short, but the harmonic mean was long, whereas the arithmetic mean interreinforcement interval of the second schedule was long, but the harmonic mean was short. The pigeons preferred the schedule with the shorter harmonic mean in a concurrent-chains procedure when a terminal link ended after the first scheduled reinforcer had been gained on a terminal-link entry, but reversed their preferences, such that they preferred the schedule with the shorter arithmetic mean, when the terminal links ended after a fixed duration of exposure to the schedule. Moreover, the pigeons preferred the schedule with the shorter arithmetic mean in a two-key concurrent variable-interval variable-interval procedure, as well as in a concurrent variable-time variable-time, changeover-key procedure. The data suggest that an aggregate property of a schedule may not yield valid information about the responding that schedule will maintain as a choice alternative.     

Percentage reinforcement and choice

Pigeons responded on identical concurrent variable-interval schedules (choice phase), producing outcomes of either periodic reinforcement schedules always terminating in reinforcement (reliable schedule) or otherwise identical schedules providing reinforcement on only a percentage of instances (percentage reinforcement schedule). Comparisons of this type constituted two assessments of the generality of preference for percentage reinforcement reported by Kendall (1974). In a third set of conditions, a reliable schedule was pitted against a percentage reinforcement schedule in which the more negative outcome was a leaner schedule of reinforcement (rather than nonreinforcement, as in the other two conditions). In all three types of conditions, the schedule providing the higher rate of reinforcement was preferred. Results from a subsequent manipulation suggest that Kendall's contrasting results may have depended on the fact that the stimuli in his choice phase (unlit keys) were physically identical to the stimulus correlated with the nonchosen outcome in his outcome phase.