Maximizing present value: A model to explain why moderate response rates obtain on variable-interval schedules

In Phases 1 and 3, two Japanese monkeys responded on a multiple variable-ratio 80 variable-interval X schedule, where the value of X was adjusted to ensure equal between-schedule reinforcement rates. Components strictly alternated following the delivery of a food pellet, and each session ended following 50 components. Phase 2 differed from the others only in that the 50 pellets previously earned during the session were delivered together at session's end. Variable-ratio response rates did not decrease across phases, but variable-interval response rates decreased substantially during the Phase 2 procedure. This rate decrease was attributed to the food-at-session's-end manipulation removing the greater immediacy of reinforcement provided by short interresponse times relative to long interresponse times. Without this time preference for short interresponse times, the variable-interval interresponse-time reinforcement feedback function largely controlled response emission, dictating a response-rate reduction. This result was explained in terms of the economic notion of “maximizing present value.”     

Responding of pigeons under variable-interval schedules of unsignaled, briefly signaled, and completely signaled delays to reinforcement

In Experiment 1, three pigeons' key pecking was maintained under a variable-interval 60-s schedule of food reinforcement. A 1-s unsignaled nonresetting delay to reinforcement was then added. Rates decreased and stabilized at values below those observed under immediate-reinforcement conditions. A brief stimulus change (key lit red for 0.5 s) was then arranged to follow immediately the peck that began the delay. Response rates quickly returned to baseline levels. Subsequently, rates near baseline levels were maintained with briefly signaled delays of 3 and 9 s. When a 27-s briefly signaled delay was instituted, response rates decreased to low levels. In Experiment 2, four pigeons' responding was first maintained under a multiple variable-interval 60-s (green key) variable-interval 60-s (red key) schedule. Response rates in both components fell to low levels when a 3-s unsignaled delay was added. In the first component delays were then briefly signaled in the same manner as Experiment 1, and in the second component they were signaled with a change in key color that remained until food was delivered. Response rates increased to near baseline levels in both components, and remained near baseline when the delays in both components were lengthened to 9 s. When delays were lengthened to 27 s, response rates fell to low levels in the briefly signaled delay component for three of four pigeons while remaining at or near baseline in the completely signaled delay component. In Experiment 3, low response rates under a 9-s unsignaled delay to reinforcement (tandem variable-interval 60 s fixed-time 9 s) increased when the delay was briefly signaled. The role of the brief stimulus as conditioned reinforcement may be a function of its temporal relation to food, and thus may be related to the eliciting function of the stimulus.     

Color preference in pigeons: stimulus intensity and reinforcement contingency effects in the avoidance of blue stimuli.

In a procedure intended to determine color preference in pigeons (which partially replicated Catania, Owens, & von Lossberg, 1983), two keys were illuminated by different colors drawn from a set of amber, red, green, or blue stimuli; this was followed by the presentation of grain when either of the two colors was pecked. The grain was illuminated alternately across trials with the colors presented on the keys. In Experiment 1 the intensity of the color stimuli used was not equalized, whereas in Experiment 2 the intensity of the colors was equalized. The low preference for blue found in Experiment 1, as measured by differential key pecking, was not found in Experiment 2. The discriminability of the intensity-equalized colors was confirmed in Experiment 2a, in which equal-intensity color discrimination problems were presented. In Experiment 3, as in Catania et al. (1983), a response-independent reinforcement schedule was used, but with intensity-equalized colors. In contrast to Experiment 2, very low preference for blue was found here and in Experiment 4, which used a within-subject procedure. These findings suggest that pigeon color preference may be a function of intensity, but all controlling variables have not as yet been identified.     

Factors affecting conditional discrimination learning by pigeons

In Experiment 1 (within subjects) and Experiment 2 (between subjects) it was shown that the sequential training of pigeons on a color discrimination and then on its reversal, each in a different floor-tilt/texture context, failed to produce conditional control of discriminative performance by those contexts. Daily alternation between the two problems (with correlated contexts) was successful, however. In each of these experiments conditional control was better reflected in generalization test performance in extinction than during sessions of training with reinforcement.     

Choice between sequences of fixed-ratio schedules: effects of ratio values and probability of food delivery.

Pigeons were exposed to schedules of food delivery that consisted of two sequential fixed ratios. When alternative sequences provided two food deliveries per 50 responses, the schedule with the shorter initial fixed-ratio value was consistently preferred. Progressively reducing from 1.0 to .25 the probability of food delivery following completion of the second fixed ratio of the sequence with the shorter initial fixed ratio did not reduce preference for this sequence. Moreover, the sequence with the shorter initial fixed ratio also was preferred when the probability of food delivery following completion of the initial ratio in that sequence was progressively reduced from 1.0 to .5, although preference shifted to the alternative when the probability was reduced to 0. These findings suggest that the length of the initial fixed ratio was a primary determinant of choice. Subsequent manipulations demonstrated, however, that when the initial fixed ratios of the two alternatives were equal, changes in the ratio value and probability of food delivery following completion of the second fixed ratio lawfully affected choice.     

Interaction of methadone, reinforcement history, and variable-interval performance.

In the present study, we examined how a reinforcement schedule history that generated high or low rates of responding influenced the effects of acute (Experiment 1) and chronic (Experiment 2) methadone administration. Initially, key-peck responses of pigeons were maintained under a variable-interval 90-s schedule of food presentation, and a methadone dose-response curve was determined with doses of 0.6, 1.2, and 2.4 mg/kg. The pigeons were then exposed, for at least 40 sessions, to either a fixed-ratio 50 schedule or a differential-reinforcement-of-low-rate 10-s schedule, or were given continued exposure to the variable-interval schedule. The methadone dose-response curve was redetermined after all pigeons again were responding under the variable-interval schedule. The effects of two different daily methadone doses (9.0 and 12.0 mg/kg/day) and withdrawal precipitated by naloxone also were assessed. Experience with a fixed-ratio or differential reinforcement of low rate schedule did not result in significantly different response rates under the variable-interval schedule and, in general, the acute effects of methadone did not have differential effects correlated with schedule history. However, for 2 of 4 subjects the rate-decreasing effects of methadone on rates of key pecking were greater following a history of low-rate responding, suggesting a possible interaction between schedule history and effects of methadone. Daily methadone administration under the variable-interval schedule revealed that pigeons with experience under the differential reinforcement of low rate schedule developed more rapid and complete tolerance to the rate-decreasing effects of methadone. Three of the 4 subjects in this group showed rate increases above drug-free baselines during chronic methadone dosing. Pigeons with a history of fixed-ratio responding also developed tolerance to the rate-decreasing effects of methadone but without the subsequent rate increases seen by subjects with low-rate histories. No subjects with variable-interval histories showed complete recovery of drug-free baselines, suggesting that interpolated training under other schedules may attenuate the rate-altering effects of chronically administered drugs. Naloxone (1.0 mg/kg), administered during the chronic methadone phase, resulted in greater disruption of responding by pigeons with a history of low-rate responding, as compared to subjects in the other two groups.(ABSTRACT TRUNCATED AT 400 WORDS)     

Consumption-leisure tradeoffs in pigeons: Effects of changing marginal wage rates by varying amount of reinforcement

Pigeons' rates of responding and food reinforcement under simple random-ratio schedules were compared with those obtained under comparable ratio schedules in which free food deliveries were added, but the duration of each food delivery was halved. These ratio-with-free-food schedules were constructed so that, were the pigeon to maintain the same rate of responding as it had under the simple ratio schedule, total food obtained (earned plus free) would remain unchanged. However, any reduction in responding would reduce total food consumption below that under the simple ratio schedule. These “compensated wage decreases” led to decreases in responding and decreases in food consumption, as predicted by an economic model of labor supply. Moreover, the reductions in responding increased as the ratio value increased (i.e., as wage rates decreased). Pigeons, therefore, substituted leisure for consumption. The relationship between these procedures and negative-income-tax programs is noted.     

Response-reinforcer relations and the maintenance of behavior

The effects on pigeons' key pecking of unsignaled delays of reinforcement and response-independent reinforcement were compared after either variable-interval or differential-reinforcement-of-low-rate baseline schedules. One 30-min session arranging delayed reinforcement and one 30-min session arranging response-independent reinforcement were conducted daily, 6 hr apart. A within-subject yoked-control procedure equated reinforcer frequency and distribution across the two sessions. Response rates usually were reduced more by response-independent than by delayed but response-contingent delivery of reinforcers. Under both schedules, response rates were lower when obtained delays were greater. These results bear upon methodological and conceptual issues regarding comparisons of contingencies that change the temporal response–reinforcer relations.     

A ROLE FOR NEGATIVE REINFORCEMENT OF RESPONSE OMISSION IN PUNISHMENT?

This experiment attempted to disentangle response-rate reductions controlled by the direct suppressive effects of a punisher from those due to negative reinforcement of response omission. Key-peck responding of pigeons was maintained by a conjoint variable-interval 3-min schedule of food presentation variable-interval 30-s schedule of response-dependent electric shock presentation. Omission of responses for 5, 10, or 30 s resulted in the possibility of canceling a scheduled shock. Response rates were a function of required pause duration, with lower rates occurring when longer periods of response omission were required for shock cancellation. These results show that, with several parameters of punishment held constant, response rates were controlled by the negative reinforcement contingency. Such a finding argues for renewed consideration of the role of negative reinforcement in punishment contingencies.