Choice and conditioned reinforcement.

A potential weakness of one formulation of delay-reduction theory is its failure to include a term for rate of conditioned reinforcement, that is, the rate at which the terminal-link stimuli occur in concurrent-chains schedules. The present studies assessed whether or not rate of conditioned reinforcement has an independent effect upon choice. Pigeons responded on either modified concurrent-chains schedules or on comparable concurrent-tandem schedules. The initial link was shortened on only one of two concurrent-chains schedules and on only one of two corresponding concurrent-tandem schedules. This manipulation increased rate of conditioned reinforcement sharply in the chain but not in the tandem schedule. According to a formulation of delay-reduction theory, when the outcomes chosen (the terminal links) are equal, as in Experiment 1, choice should depend only on rate of primary reinforcement; thus, choice should be equivalent for the tandem and chain schedules despite a large difference in rate of conditioned reinforcement. When the outcomes chosen are unequal, however, as in Experiment 2, choice should depend upon both rate of primary reinforcement and relative signaled delay reduction; thus, larger preferences should occur in the chain than in the tandem schedules. These predictions were confirmed, suggesting that increasing the rate of conditioned reinforcement on concurrent-chains schedules may have no independent effect on choice.     

Income maximizing on concurrent ratio-interval schedules of reinforcement

Three experiments examined the effect of food availability on pigeons' choice behavior under concurrent schedules of reinforcement. In Experiment 1, 3 pigeons earned their daily food ration by choosing, in 30-min sessions, between concurrent variable-ratio 30 variable-interval 40-s schedules. Food presentations during both schedules lasted 2 or 12 s, depending upon the condition. Relative variable-ratio response rate was inversely related to hopper duration. In Experiment 2, 4 pigeons received their daily feeding by responding on the same schedule pair as in Experiment 1 (with 4-s food presentations) in sessions that varied in length from 10 to 30 min, depending on the condition. The length of a vertical slit projected on a response key increased with time so that “passage of time” might be more easily discriminable. As session duration decreased, relative variable-ratio response rate increased. In Experiment 3, 4 pigeons chose between two variable-interval 40-s schedules. One schedule operated without regard to the schedule selected, whereas the other operated only when the subject responded in its presence (dependent). Although these schedules had the same feedback function, preference for the dependent variable interval increased as session duration decreased from 30 to 10 min. The preference changes in these studies reveal the operation of an income-maximizing process in choice.     

Effects of delayed conditioned reinforcement in chain schedules.

The contingency between responding and stimulus change on a chain variable-interval 33-s, variable-interval 33-s, variable-interval 33-s schedule was weakened by interposing 3-s delays between either the first and second or the second and third links. No stimulus change signaled the delay interval and responses could occur during it, so the obtained delays were often shorter than the scheduled delay. When the delay occurred after the initial link, initial-link response rates decreased by an average of 77% with no systematic change in response rates in the second or third links. Response rates in the second link decreased an average of 59% when the delay followed that link, again with little effect on response rates in the first or third links. Because the effect of delaying stimulus change was comparable to the effect of delaying primary reinforcement in a simple variable-interval schedule, and the effect of the unsignaled delay was specific to the link in which the delay occurred, the results provide strong evidence for the concept of conditioned reinforcement.     

The effects of brief-stimulus presentations in fixed-ratio second-order schedules

Pigeons' responses were reinforced according to a three-component multiple schedule. In Component 1, key pecks produced food according to a fixed-ratio second-order schedule with fixed-ratio units. Here, a fixed number of fixed-ratio units produced food, and the brief stimulus terminating each unit also accompanied food. Responses in Component 2 produced food on an identical schedule except that the brief stimulus was not paired with food. Component 3 contained a simple fixed-ratio schedule whose response requirement equaled that of Components 1 and 2. Across conditions the size of the fixed-ratio unit (five, ten, twenty, forty, and eighty responses) and the total number of responses per reinforcement were parametrically manipulated. The highest response rates and shortest preratio pauses were observed in Component 3 (no brief stimulus). The lowest rates and longest pauses were found in the component with paired brief-stimulus presentations, indicating that the food-paired brief stimulus suppressed responding. The suppressive effects were greatest when the fixed-ratio units were small (e.g., fixed-ratio 5) and the total fixed-ratio requirement was large (e.g., fixed-ratio 160). Under no conditions did the paired brief stimulus facilitate responding. The nonpaired brief stimulus also suppressed responding but to a lesser extent. The suppressive effects of nonpaired brief stimuli were greatest when the fixed-ratio units were small and the total response requirement was large. These data suggest that the suppressive effects of the brief stimuli may have masked the conditioned-reinforcing effects reported in other studies, and that conditions that maximize suppression in second-order schedules involve the use of fixed-ratio schedule units and the presentation of many brief stimuli per reinforcer.     

Choice, experience, and the generalized matching law

Five pigeons were exposed to different pairs of concurrent variable-interval, variable-interval schedules on nine experimental conditions of 30 sessions each. For every session, the parameters of the generalized matching equation were computed for the first five, six, seven, eight, and nine experimental conditions. The exponent a, both for response and time distribution, tended to decrease with increases in number of experimental conditions and to increase with number of sessions per condition, but values of k (bias) varied unsystematically. When the subjects were exposed to five new pairs of schedules, with 55 sessions per condition, the findings were confirmed. Data from the literature on the generalized matching law suggest that the variability of exponent values may be explained in part by the use of naive or experienced subjects in different investigations and by the variability in number of experimental conditions and in number of sessions per condition.     

Conditioned reinforcement versus time to reinforcement in chain schedules.

Pigeons were trained on three-component chain schedules in which the initial component was either a fixed-interval or variable-interval schedule. The middle and terminal components were varied among fixed-interval fixed-interval, variable-interval variable-interval, and an interdependent variable-interval variable-interval schedule in which the sum of the durations of the two variable-interval components was always equal to the sum of the fixed-interval fixed-interval components. At issue was whether the response rate in the initial component was controlled by its time to primary reinforcement or by the temporal parameters of the stimulus correlated with the middle terminal link. The fixed-interval initial-link schedule maintained much lower response rates than the variable-interval initial-link schedule regardless of the schedules in the middle and terminal links. Nevertheless, the intervening schedules played some role: With fixed-interval schedules in the initial links, response rates were consistently highest with independent variable-interval schedules in the middle and terminal links and intermediate with the interdependent variable-interval schedules; these initial-link differences were predicted by the response rates in the middle link of the chain. With variable-interval schedules in the initial links, response rates were lowest with the fixed-interval fixed-interval schedules following the initial link and were not systematically different for the two types of variable-interval variable-interval schedules. The results suggest that time to reinforcement itself accounts for little if any variance in initial-link responding.     

Resistance to reinforcement change in multiple and concurrent schedules assessed in transition and at steady state

Behavioral momentum theory relates resistance to change of responding in a multiple-schedule component to the total reinforcement obtained in that component, regardless of how the reinforcers are produced. Four pigeons responded in a series of multiple-schedule conditions in which a variable-interval 40-s schedule arranged reinforcers for pecking in one component and a variable-interval 360-s schedule arranged them in the other. In addition, responses on a second key were reinforced according to variable-interval schedules that were equal in the two components. In different parts of the experiment, responding was disrupted by changing the rate of reinforcement on the second key or by delivering response-independent food during a blackout separating the two components. Consistent with momentum theory, responding on the first key in Part 1 changed more in the component with the lower reinforcement total when it was disrupted by changes in the rate of reinforcement on the second key. However, responding on the second key changed more in the component with the higher reinforcement total. In Parts 2 and 3, responding was disrupted with free food presented during intercomponent blackouts, with extinction (Part 2) or variable-interval 80-s reinforcement (Part 3) arranged on the second key. Here, resistance to change was greater for the component with greater overall reinforcement. Failures of momentum theory to predict short-term differences in resistance to change occurred with disruptors that caused greater change between steady states for the richer component. Consistency of effects across disruptors may yet be found if short-term effects of disruptors are assessed relative to the extent of change observed after prolonged exposure.     

Inelastic supply: An economic approach to simple interval schedules

Economic theory predicts an inverse relationship between the quantity of a commodity supplied to the marketplace and the equilibrium market price of that commodity. This prediction was tested in three experiments. Pigeons responded on simple variable-interval schedules, and quantity of reinforcement supplied was varied in a different way in each experiment. In Experiment 1, quantity supplied was varied by manipulating reinforcement rate while keeping session length constant. In Experiment 2, quantity supplied was varied by manipulating reinforcement rate while keeping reinforcers per session constant. In Experiment 3, quantity supplied was varied by manipulating reinforcer magnitude while keeping number of reinforcers constant. As predicted by economic theory, the obtained behavioral cost (responses per reinforcer) increased as supply decreased. The results could not be explained by simple artifacts such as satiation and time available to respond. In addition, the function relating response rate to reinforcement rate was bitonic in 7 of 9 animals in Experiments 1 and 2, which supports economic and regulatory theories over more traditional reinforcement theories.     

Effects of reinforcement context on choice

Two experiments investigated the effects of successive reinforcement contexts on choice. In the first, concurrent variable-interval schedules of primary reinforcement operated during the initial links of concurrent chains. The rate of this reinforcement arranged by the concurrent schedules was decreased across conditions: When it was higher than the terminal-link rate, preference for the higher frequency initial-link schedule increased relative to baseline. (During baseline, a standard concurrent-schedule procedure was in effect). When the initial-link reinforcement rate was lower than the terminal-link rate, preference converged toward indifference. In the second experiment, a chain schedule was available on a third key while a concurrent schedule was in effect on the side keys. When the terminal link of the chain schedule was produced, the side keys became inoperative. Availability of the chain schedule did not affect choice between the concurrent schedules. These results show that only when successive reinforcement contexts are produced by choice responding do those successive contexts affect choice in concurrent schedules.     

Signalled and unsignalled percentage reinforcement of performance under a chained schedule

Pigeons were trained to peck a key under a chained fixed-ratio 15 fixed-interval 25-sec schedule of food presentation. In Experiment 1, blocks of sessions in which 100%, 75%, 50%, and 25% of the sequences ended with food presentation were conducted. When food presentation was omitted, a timeout of equal duration replaced it. As the frequency of food presentation decreased so did the frequency of completing the chained schedule. In Experiment 2, 75%, 50%, or 25% of the sequences terminated with food presentation and outcomes were signalled, i.e., completion of the fixed ratio resulted in either a stimulus correlated with the fixed-interval 25-sec schedule or a stimulus correlated with extinction. As the frequency of food presentation decreased, the number of sequences completed per session increased for two pigeons and remained high for a third. In Experiments 3 and 4, assessments of the effects of signalling the outcome of the chained schedule were made with response-independent presentation of events at the end of the sequence. Again, signalling the outcome of the chained schedule led to more chains being completed per session than did not signalling the outcome. Stimuli differentially paired with food presentation have powerful behavioral effects that may be attributed to the potency of these stimuli as conditioned reinforcers.     

Contrast and induction in multiple schedules of discrete-trial concurrent reinforcement

Three pigeons were exposed to two-key discrete-trial concurrent schedules of reinforcement. Red and white key colors alternated irregularly and the assignment of reinforcers depended on key color. The red-key schedules were held constant, with the scheduled relative frequency of reinforcement for left-key pecks set at 0.75, while the white-key schedules varied. When the location of white-key reinforcement was changed from one side to the other, while its overall frequency was constant, red-key choices shifted in the same direction as white-key choices, an induction effect. When the overall frequency of white-key reinforcement was changed while its location remained constant, red key choices shifted in a direction opposite to white-key choices, a contrast effect. Both induction and contrast effects were clearer when the overall frequency of red-key reinforcement was reduced. These data demonstrate that the allocation of responding may exhibit schedule interaction effects similar to those commonly reported for response rate.     

Choice and delay of reinforcement: Effects of terminal-link stimulus and response conditions

In two experiments, pigeons were exposed to concurrent-chains schedules in which a single initial-link variable-interval schedule led to access to terminal links composed of fixed-interval or fixed-delay schedules. In Experiment 1, an 8-s (or 16-s) delay to reinforcement was associated with the standard key, while reinforcer delay values associated with the experimental key were varied from 4 to 32 s. The results of Experiment 1 showed undermatching of response ratios to delay ratios with terminal-link fixed-delay schedules, whereas in some pigeons matching or overmatching was evident with the fixed-interval schedules. In Experiment 2, one pair of reinforcer delay values, either 8 versus 16 s or 16 versus 32 s, was used. In the first condition of Experiment 2, different delays were associated with different keylight stimuli (cued condition). In the second condition, different terminal-link delays were associated with the same stimulus, either a blackout (uncued-blackout condition) or a white key (uncued-white condition). To examine the role of responses emitted during delays, the keys were retracted during a delay (key-absent condition) in the third condition and responses were required by a fixed-interval schedule in the fourth condition. Experiment 2 demonstrated that the choice proportions for the shorter delay were more extreme in the cued condition than in the uncued-blackout condition, and that the response requirement imposed by the fixed-interval schedules did not affect choice of the shorter delay, nor did the key-absent and key-present conditions. These results indicate that the keylight-stimulus conditions affected preference for the shorter of two delays and that the findings obtained in Experiment 1 depended mainly on the keylight-stimulus conditions of the terminal links (i.e., the conditioned reinforcing value of the terminal-link stimuli).     

Preference for conditioned reinforcement.

Pigeons were presented with a concurrent-chains schedule in which both choice alternatives led to the same terminal-link stimulus, which was followed by food. Superimposed on the food-reinforced presentations of the terminal-link stimulus was a second schedule of presentations of the same stimulus that were followed by no food. The absolute number of these no-food stimulus presentations was held constant while their relative frequency assigned to one or the other choice alternative was systematically varied. Preference for a given choice alternative tracked the relative frequency of these stimulus presentations, thus demonstrating that they served as reinforcers. These results resolve conflicts in the literature regarding the effect of conditioned reinforcement on choice.