Choice, foraging, and reinforcer duration.

Pigeons were exposed to a foraging schedule characterized by three different states, beginning with a search state in which completion of a variable interval on a white key led to a choice state. In the choice state the subject could, by appropriate responding on a fixed ratio of three, either accept or reject the schedule offered. If the subject accepted the schedule, it entered a handling state in which the appropriate reinforcer amount was presented according to a variable-interval schedule. In Experiment 1 the shorter duration reinforcer was more likely to be accepted the longer the duration of the search state and the shorter the equal durations of the handling states. In Experiment 2 the shorter duration reinforcer was more likely to be accepted the longer the handling time preceding the longer duration reinforcer. All of the results were in qualitative--and some were in quantitative--agreement with those predicted by the delay-reduction hypothesis and the optimal-diet model.     

Choice for conditioned reinforcers in the signaled absence of primary reinforcement

Pigeons responded in a multiple schedule in which concurrent schedules of brief-stimulus presentation alternated with a component in which food was available (concurrent-chains component). In the initial links of the concurrent-chains component subjects chose either of two stimuli each correlated with the terminal link of one chain. The terminal links involved either variable-interval 30-second or variable-interval 60-second schedules. In the brief-stimulus component subjects chose between 0.5-second presentations of the terminal-link stimuli from the concurrent-chains component. Responding was generally maintained in the brief-stimulus component in two subjects for more than 300 sessions, suggesting that brief stimuli were conditioned reinforcers. During the brief-stimulus component, in 17 of 21 cases for which a minimal number of responses occurred, choice proportions above 0.55 were obtained for the brief-stimulus presentations correlated with the higher rate of primary reinforcement in the concurrent-chains component. These results support the suggestion that choice in conventional concurrent-chains procedures is partially controlled by production of the terminal-link stimuli.     

Choice and foraging: the effects of accessibility on acceptability.

Pigeons responded in a successive-encounters choice procedure in which accessibility of the less profitable of two outcomes varied either in terms of probability of encounter or search time to encounter (keeping search time to the more profitable outcome constant). When the less profitable outcome was made more probable its acceptance became more likely. However, when search time to encounter the less profitable outcome was shortened, its acceptance became less likely. Both results are consistent with the delay-reduction hypothesis and with an optimality model developed for application to the successive-encounters choice procedure.     

Choice and conditioned reinforcement.

A potential weakness of one formulation of delay-reduction theory is its failure to include a term for rate of conditioned reinforcement, that is, the rate at which the terminal-link stimuli occur in concurrent-chains schedules. The present studies assessed whether or not rate of conditioned reinforcement has an independent effect upon choice. Pigeons responded on either modified concurrent-chains schedules or on comparable concurrent-tandem schedules. The initial link was shortened on only one of two concurrent-chains schedules and on only one of two corresponding concurrent-tandem schedules. This manipulation increased rate of conditioned reinforcement sharply in the chain but not in the tandem schedule. According to a formulation of delay-reduction theory, when the outcomes chosen (the terminal links) are equal, as in Experiment 1, choice should depend only on rate of primary reinforcement; thus, choice should be equivalent for the tandem and chain schedules despite a large difference in rate of conditioned reinforcement. When the outcomes chosen are unequal, however, as in Experiment 2, choice should depend upon both rate of primary reinforcement and relative signaled delay reduction; thus, larger preferences should occur in the chain than in the tandem schedules. These predictions were confirmed, suggesting that increasing the rate of conditioned reinforcement on concurrent-chains schedules may have no independent effect on choice.     

A molecular analysis of choice on concurrent-chains schedules.

Six pigeons responded on concurrent-chains schedules with either independent or interdependent equal variable-interval schedules in the initial links and unequal variable-interval schedules, always in a 2:1 ratio, in the terminal links. Relative response rates in the initial links increased across conditions as initial-link duration was shortened and decreased across conditions as terminal-link duration was shortened, replicating previous findings. Responses in the initial links were recorded in 5-s bins, and local or molecular relative response rates were calculated in order to ascertain how relative response rate varied as a function of time since the onset of the initial links. Two distinct molecular patterns were found. With interdependent initial links, relative response rates for the preferred key were elevated for the first 10 or 20 s of the initial links and then declined to an asymptotic value. With independent initial links, a negative recency effect was found similar to that reported by Killeen (1970). These two molecular patterns were related to the different momentary reinforcement probabilities resulting from independent and interdependent scheduling.     

Choice: Some quantitative relations

Six pigeons responded in fifty-six conditions on a concurrent-chains procedure. Conditions included several with equal initial links and unequal terminal links, several with unequal initial links and equal terminal links, and several with both unequal initial and terminal links. Although the delay-reduction hypothesis accounted well for choice when the initial links were equal (mean deviation of .04), it fit the data poorly when the initial links were unequal (mean deviation of .18). A modification of the delay-reduction hypothesis, replacing the rates of reinforcement with the square roots of these rates, fit the data better than either the unmodified delay-reduction equation or Killeen's (1982) model. The modified delay-reduction equation was also consistent with data from prior studies using concurrent chains. The absolute rates of responding in each terminal link were well described by the same hyperbola (Herrnstein, 1970) that describes response rates on simple interval schedules.     

Choice in situations of time-based diminishing returns: immediate versus delayed consequences of action.

Pigeons chose between two schedules of food presentation, a fixed-interval schedule and a progressive-interval schedule that began at 0 s and increased by 20 s with each food delivery provided by that schedule. Choosing one schedule disabled the alternate schedule and stimuli until the requirements of the chosen schedule were satisfied, at which point both schedules were again made available. Fixed-interval duration remained constant within individual sessions but varied across conditions. Under reset conditions, completing the fixed-interval schedule not only produced food but also reset the progressive interval to its minimum. Blocks of sessions under the reset procedure were interspersed with sessions under a no-reset procedure, in which the progressive schedule value increased independent of fixed-interval choices. Median points of switching from the progressive to the fixed schedule varied systematically with fixed-interval value, and were consistently lower during reset than during no-reset conditions. Under the latter, each subject's choices of the progressive-interval schedule persisted beyond the point at which its requirements equaled those of the fixed-interval schedule at all but the highest fixed-interval value. Under the reset procedure, switching occurred at or prior to that equality point. These results qualitatively confirm molar analyses of schedule preference and some versions of optimality theory, but they are more adequately characterized by a model of schedule preference based on the cumulated values of multiple reinforcers, weighted in inverse proportion to the delay between the choice and each successive reinforcer.     

Choice between reliable and unreliable outcomes: mixed percentage-reinforcement in concurrent chains.

Pigeons' choices between alternatives that provided different percentages of reinforcement in mixed schedules were studied using the concurrent-chains procedure. In Experiment 1, the alternatives were terminal-link schedules that were equal in delay and magnitude of reinforcement, but that provided different percentages of reinforcement, with one schedule providing, reinforcement twice as reliably as the other. All pigeons preferred the more reliable schedule, and their level of preference was not systematically affected by variation in the absolute percentage values, or in the magnitude of reinforcement. In Experiment 2, preference for a schedule providing 100% reinforcement over one providing 33% reinforcement increased systematically with increases in the duration of the terminal links. In contrast, preference decreased systematically with increases in the duration of the initial links. Experiment 3 examined choice with equal percentages of reinforcement but unequal delays to reinforcement. Preference for the shorter delay to reinforcement was not systematically affected by variation in the absolute percentage of reinforcement. The overall pattern of results supported predictions based on an extension of the delay-reduction hypothesis to choice procedures involving mixed schedules of percentage reinforcement.     

Delay reduction and optimal foraging: variable-ratio search in a foraging analogue.

The present study investigated conditions under which the conditioned reinforcement principles of delay-reduction theory and views based on simple maximization of reinforcement rate make ordinally opposing predictions with respect to foraging-related choice behavior. The use of variable-ratio schedules in the choice phase also represents an extension of delay-reduction theory to schedules that may better mimic the effort involved in searching. Pigeons responded on modified concurrent-chains schedules in which equal variable-ratio schedules led to unequal variable-interval outcomes and unequal reinforcer amounts. All 4 subjects completed a minimum of two replications of conditions for which the predictions of delay-reduction theory and a simple rate-maximizing theory were opposed. Results were consistent with delay reduction's ordinal predictions in 11 of 11 replications of the divergent predictions favoring the smaller, more immediate alternative. The predictions of rate maximization were upheld only when they were consistent with those of delay reduction. Results are discussed in terms of conditioned reinforcement, sensitivity to reductions in delay to food, and possible rules of thumb that may be useful in characterizing foraging.     

Choice and delay of reinforcement: Effects of terminal-link stimulus and response conditions

In two experiments, pigeons were exposed to concurrent-chains schedules in which a single initial-link variable-interval schedule led to access to terminal links composed of fixed-interval or fixed-delay schedules. In Experiment 1, an 8-s (or 16-s) delay to reinforcement was associated with the standard key, while reinforcer delay values associated with the experimental key were varied from 4 to 32 s. The results of Experiment 1 showed undermatching of response ratios to delay ratios with terminal-link fixed-delay schedules, whereas in some pigeons matching or overmatching was evident with the fixed-interval schedules. In Experiment 2, one pair of reinforcer delay values, either 8 versus 16 s or 16 versus 32 s, was used. In the first condition of Experiment 2, different delays were associated with different keylight stimuli (cued condition). In the second condition, different terminal-link delays were associated with the same stimulus, either a blackout (uncued-blackout condition) or a white key (uncued-white condition). To examine the role of responses emitted during delays, the keys were retracted during a delay (key-absent condition) in the third condition and responses were required by a fixed-interval schedule in the fourth condition. Experiment 2 demonstrated that the choice proportions for the shorter delay were more extreme in the cued condition than in the uncued-blackout condition, and that the response requirement imposed by the fixed-interval schedules did not affect choice of the shorter delay, nor did the key-absent and key-present conditions. These results indicate that the keylight-stimulus conditions affected preference for the shorter of two delays and that the findings obtained in Experiment 1 depended mainly on the keylight-stimulus conditions of the terminal links (i.e., the conditioned reinforcing value of the terminal-link stimuli).     

Effects of reinforcer duration on responding in two-link chained interval schedules

Each of five pigeons was exposed to two or more durations of access to mixed grains on two-link, chained, interval schedules in which both links were identical fixed-interval or variable-interval schedules. Response rates were an increasing function of reinforcer duration for both initial and terminal links. For both types of interval schedules, initial-link response rates were more sensitive to reinforcer duration than were terminal-link response rates. The present results, together with prior ones, suggest that chaining and choice procedures are each sufficient for demonstrating substantial sensitivity of responding to changes in reinforcer duration.     

Responding on concurrent-chains schedules in open and closed economies.

Pigeons' key pecks were reinforced according to concurrent-chains schedules of reinforcement. The programmed average time from the onset of the initial links to a terminal link entry was held constant across conditions while the value of variable-interval schedules in the terminal links was varied. Performance was assessed under two economic conditions: (a) an open economy in which session duration was limited to 1 hr and subjects were maintained at 80% of their free-feeding body weights with postsession food when necessary; and (b) a closed economy in which sessions were 23.5 hr long and no deprivation regimen was in effect. In all cases, the relative rate of responding in the initial links matched the reduction in overall delay to primary reinforcement correlated with entry into one terminal link relative to the reduction in delay correlated with entry into the other terminal link. Although the sum of responses made in the initial links and terminal links was found to increase, then decrease, as the rate of food presentation decreased in the closed economy, there was no consistent effect of overall rate of food presentation on total responding in the open economy. The choice data suggest that relative delay reduction predicts choice accurately, regardless of economic context.     

Choice, contingency discrimination, and foraging theory

Four pigeons were trained on eight or nine pairs of independent concurrent variable-interval schedules. The range of reinforcement ratios included extreme ratios (up to 532 to 1). Large samples of stable performance were gathered. Contrary to the findings of Davison and Jones (1995), the generalized matching law described choice more accurately than a contingency-discriminability model. Taking small samples (5 to 10 sessions) and applying a more liberal stability criterion used by Davison and Jones only increased the unsystematic variance in the data and in estimates of generalized-matching-law sensitivity. Because changing to dependent scheduling and inserting a changeover delay had no systematic effect, the deviations from generalized matching reported by Davison and Jones probably arose from imperfectly discriminated stimuli. Analysis of visits revealed that visits to the nonpreferred alternative were brief and approximately constant. When choice between the preferred (rich) and nonpreferred (lean) alternatives, regardless of position, was analyzed according to the generalized matching law, sensitivities approximated 1.0, with bias in favor of the lean alternative. This bias, which arose from an excessive frequency of visits to the lean alternative, explains undermatching as the result of fitting one line to a choice relation that consists of two displaced lines, both with a slope of 1.0. The pattern of deviation from the generalized matching line confirmed this account. The findings suggest an alternative analysis of choice that focuses on probability of visiting the lean alternative as the dependent variable. This probability was directly proportional to ratio of reinforcement. Matching, undermatching, and overmatching may all be explained by a view of concurrent performance based on foraging theory, in which responding occurs primarily at the rich alternative and is occasionally interrupted by brief visits to the lean alternative.     

Choice between fixed-interval schedules: Graded versus step-like choice functions

Pigeons chose between two fixed-interval schedules of food reinforcement. A single peck on one of two lighted keys started the fixed-interval schedule correlated with that key. The schedule had to be completed before the next choice opportunity. The durations of the fixed intervals were varied over conditions from 15 s to 40 s. To maximize the rate of reinforcement, the pigeons had to choose exclusively the shorter of the two schedules. Nevertheless, choice was not all-or-none. Instead, relative choice, and the rates of producing the fixed intervals, varied in a graded fashion with the disparity between the two schedules. Choice ratios under this procedure (single response to choose) were highly sensitive to the ratios of the fixed-interval schedules.     

Response-independent Events In The Behavior Stream

The metaphor of the behavior stream provides a framework for studying the effects of response-independent food presentations intruded into an environment in which operant responding of pigeons was maintained by variable-interval schedules. In the first two experiments, response rates were reduced when response-independent food was intruded during the variable-interval schedule according to a concomitantly present fixed-time schedule. These reductions were not always an orderly function of the percentage of response-dependent food. Negatively accelerated patterns of key pecking across the fixed-time period occurred in Experiment 1 under the concomitant fixed-time variable-interval schedules. In Experiment 2, positively and negatively accelerated and linear response patterns occurred even though the schedules were similar to those used in Experiment 1. The variable findings in the first two experiments led to three subsequent experiments that were designed to further illuminate the controlling variables of the effects of intruded response-independent events. When the fixed and variable schedules were correlated with distinct operanda by employing a concurrent fixed-interval variable-interval schedule (Experiment 3) or with distinct discriminative stimuli (Experiments 4 and 5), negatively accelerated response patterns were obtained. Even in these latter cases, however, the response patterns were a joint function of the physical separation of the two schedules and the ratio of fixed-time or fixed-interval to variable-interval schedule food presentations. The results of the five experiments are discussed in terms of the contributions of both reinforcement variables and discriminative stimuli in determining the effects of intruding response-independent food into a stream of operant behavior.     

Delay reduction: current status.

Delay-reduction theory states that the effectiveness of a stimulus as a conditioned reinforcer may be predicted most accurately by the reduction in time to primary reinforcement correlated with its onset. We review support for the theory and then discuss two new types of experiments that assess it. One compares models of choice in situations wherein the less preferred outcome is made more accessible; the other investigates whether frequency of conditioned reinforcement affects choice beyond the effect exerted by frequency of primary reinforcement.     

Choice and segmented interreinforcement intervals

Pigeons were trained on a two-key concurrent schedule, where food reinforcers on one key were arranged by a simple variable-interval schedule and on the other key by a chain variable-interval variable-interval schedule. When the initial link of the chain was in effect, the pigeons tended to respond more on the simple variable-interval schedule, and hence less on the chain, than would be expected from a comparison of both the local and overall rates of reinforcement of the two schedules. When the terminal link of the chain was in effect, the pigeons responded more on the chain than would be expected from a comparison of the rates of reinforcement of the schedules then in effect. Overall responding on the chain was not proportional to overall reinforcement on the chain but rather was a by-product of responding during initial- and terminal-link phases.     

Choice with uncertain outcomes: conditioned reinforcement effects.

Pigeons responded on concurrent chains with equal initial- and terminal-link durations. In all conditions, the terminal links of one chain ended reliably in reinforcement; the terminal links on the alternative chain ended in either food or blackout. In Experiment 1, the terminal-link stimuli were correlated with (signaled) the outcome, and the durations of the initial and terminal links were varied across conditions. Preference did not vary systematically across conditions. In Experiment 2, terminal-link durations were varied under different stimulus conditions. The initial links were variable-interval 80-s schedules. Preference for the reliable alternative was generally higher in unsignaled than in signaled conditions. Preference increased with terminal-link durations only in the unsignaled conditions. There were no consistent differences between conditions with and without a common signal for reinforcement on the two chains. In the first series of conditions in Experiment 3, a single response was required in the initial links, and the stimulus conditions during 50-s terminal links were varied. Preference for the reliable outcome approached 1.0 in unsignaled conditions and was considerably lower (below .50 for 3 of 5 subjects) in signaled conditions. In a final series of signaled conditions with relatively long terminal links, preference varied with duration of the initial links. The results extend previous findings and are discussed in terms of the delay reduction signaled by terminal-link stimuli.     

Residence time and choice in concurrent foraging schedules

Five pigeons were trained on a concurrent-schedule analogue of the “some patches are empty” procedure. Two concurrently available alternatives were arranged on a single response key and were signaled by red and green keylights. A subject could travel between these alternatives by responding on a second yellow “switching” key. Following a changeover to a patch, there was a probability (p) that a single reinforcer would be available on that alternative for a response after a time determined by the value of λ, a probability of reinforcement per second. The overall scheduling of reinforcers on the two alternatives was arranged nonindependently, and the available alternative was switched after each reinforcer. In Part 1 of the experiment, the probabilities of reinforcement, ρ_red and ρ_green, were equal on the two alternatives, and the arranged arrival rates of reinforcers, λ_red and λ_green, were varied across conditions. In Part 2, the reinforcer arrival times were arranged to be equal, and the reinforcer probabilities were varied across conditions. In Part 3, both parameters were varied. The results replicated those seen in studies that have investigated time allocation in a single patch: Both response and time allocation to an alternative increased with decreasing values of λ and with increasing values of ρ, and residence times were consistently greater than those that would maximize obtained reinforcer rates. Furthermore, both response- and time-allocation ratios undermatched mean reinforcer-arrival time and reinforcer-frequency ratios.