Key pecking of pigeons under variable-interval schedules of briefly signaled delayed reinforcement: effects of variable-interval value.

Key pecking of 4 pigeons was maintained under a multiple variable-interval 20-s variable-interval 120-s schedule of food reinforcement. When rates of key pecking were stable, a 5-s unsignaled, nonresetting delay to reinforcement separated the first peck after an interval elapsed from reinforcement in both components. Rates of pecking decreased substantially in both components. When rates were stable, the situation was changed such that the peck that began the 5-s delay also changed the color of the keylight for 0.5 s (i.e., the delay was briefly signaled). Rates increased to near-immediate reinforcement levels. In subsequent conditions, delays of 10 and 20 s, still briefly signaled, were tested. Although rates of key pecking during the component with the variable-interval 120-s schedule did not change appreciably across conditions, rates during the variable-interval 20-s component decreased greatly in 1 pigeon at the 10-s delay and decreased in all pigeons at the 20-s delay. In a control condition, the variable-interval 20-s schedule with 20-s delays was changed to a variable-interval 35-s schedule with 5-s delays, thus equating nominal rates of reinforcement. Rates of pecking increased to baseline levels. Rates of pecking, then, depended on the value of the briefly signaled delay relative to the programmed interfood times, rather than on the absolute delay value. These results are discussed in terms of similar findings in the literature on conditioned reinforcement, delayed matching to sample, and classical conditioning.     

Mechanisms underlying the effects of unsignaled delayed reinforcement on key pecking of pigeons under variable-interval schedules.

Three experiments were conducted to test an interpretation of the response-rate-reducing effects of unsignaled nonresetting delays to reinforcement in pigeons. According to this interpretation, rates of key pecking decrease under these conditions because key pecks alternate with hopper-observing behavior. In Experiment 1, 4 pigeons pecked a food key that raised the hopper provided that pecks on a different variable-interval-schedule key met the requirements of a variable-interval 60-s schedule. The stimuli associated with the availability of the hopper (i.e., houselight and keylight off, food key illuminated, feedback following food-key pecks) were gradually removed across phases while the dependent relation between hopper availability and variable-interval-schedule key pecks was maintained. Rates of pecking the variable-interval-schedule key decreased to low levels and rates of food-key pecks increased when variable-interval-schedule key pecks did not produce hopper-correlated stimuli. In Experiment 2, pigeons initially pecked a single key under a variable-interval 60-s schedule. Then the dependent relation between hopper presentation and key pecks was eliminated by arranging a variable-time 60-s schedule. When rates of pecking had decreased to low levels, conditions were changed so that pecks during the final 5 s of each interval changed the keylight color from green to amber. When pecking produced these hopper-correlated stimuli, pecking occurred at high rates, despite the absence of a peck-food dependency. When peck-produced changes in keylight color were uncorrelated with food, rates of pecking fell to low levels. In Experiment 3, details (obtained delays, interresponse-time distributions, eating times) of the transition from high to low response rates produced by the introduction of a 3-s unsignaled delay were tracked from session to session in 3 pigeons that had been initially trained to peck under a conventional variable-interval 60-s schedule. Decreases in response rates soon after the transition to delayed reinforcement were accompanied by decreases in eating times and alterations in interresponse-time distributions. As response rates decreased and became stable, eating times increased and their variability decreased. These findings support an interpretation of the effects of delayed reinforcement that emphasizes the importance of hopper-observing behavior.     

A comparison of signaled and unsignaled delay of reinforcement

Pigeons were trained on either a variable-interval 60-second schedule, or on a schedule that differentially reinforced responses that were spaced at least 20 seconds apart. The birds were then exposed to several durations of reinforcement delay, with comparisons between signaled and unsignaled delays. Although unsignaled delays of 5 and 10 seconds produced large decreases in response rate, signaled delays of up to 10 seconds produced only moderate decreases in response rates. In addition, some subjects responded more rapidly with a .5 or 1.0 second duration of unsignaled delay than with immediate reinforcement. These response rate changes occurred regardless of whether the rate of reinforcement concomitantly decreased or increased.     

Unsignalled delay of reinforcement in variable-interval schedules

Three pigeons responded on several tandem variable-interval fixed-time schedules in which the value of the fixed-time component was varied to assess the effects of different unsignalled delays of reinforcement. Actual (obtained) delays between the last key peck in an interval and reinforcement were consistently shorter than the nominal (programmed) delay. When nominal delays were relatively short, response rates were higher during the delay condition than during the corresponding nondelay condition. At longer nominal delay intervals, response rates decreased monotonically with increasing delays. The results were consistent with those obtained from delay-of-reinforcement procedures that impose either a stimulus change (signal) or a no-response requirement during the delay interval.     

Effects of variable-interval value and amount of training on stimulus generalization.

In Experiment 1 pigeons pecked a key that was illuminated with a 501-nm light and obtained food by doing so according to a variable-interval (VI) schedule of reinforcement, the mean value of which differed across groups: either 30 s, 120 s, or 240 s. The pigeons in all three groups were trained for 10 50-min sessions. Generalization testing was conducted in extinction with different wavelengths of light. Absolute and relative generalization gradients were similar in shape for the three groups. Experiment 2 was a systematic replication of Experiment 1 using line orientation as the stimulus dimension and a mean VI value of either 30 s or 240 s. Again, gradients of generalization were similar for the two groups. In Experiment 3 pigeons pecked a key that was illuminated with a 501-nm light and obtained food reinforcers according to either a VI 30-s or a 240-s schedule. Training continued until response rates stabilized (> 30 sessions). For subjects trained with the 30-s schedule, generalization gradients were virtually identical regardless of whether training was for 10 sessions (Experiment 1) or until response rates stabilized. For subjects trained with the VI 240-s schedule, absolute generalization gradients for subjects trained to stability were displaced upward relative to gradients for subjects trained for only 10 sessions (Experiment 1), and relative generalization gradients were slightly flatter. These results indicate that the shape of a generalization gradient does not necessarily depend on the rate of reinforcement during 10-session single-stimulus training but that the effects of prolonged training on stimulus generalization may be schedule dependent.     

Effects of methadone on alternative fixed-ratio fixed-interval performance: latent influences on schedule-controlled responding.

Effects of methadone on pigeons' key pecking were examined under four conditions selected to analyze the control of behavior under alternative fixed-ratio fixed-interval schedules. In Condition 1, pigeons pecked under one of three different alternative schedules (alternative fixed-ratio 50 fixed-interval 90 s, alternative fixed-ratio 75 fixed-interval 90 s and alternative fixed-ratio 200 fixed-interval 90 s) each week. In Condition 2, fixed-ratio 50 or fixed-ratio 75 schedules were in effect during baseline sessions, and alternative fixed-ratio 50 fixed-interval 90-s or alternative fixed-ratio 75 fixed-interval 90-s schedules were in effect during sessions in which methadone was administered. In Condition 3, effects of methadone on key pecking maintained under fixed-ratio 50 and fixed-ratio 75 schedules were examined, whereas in Condition 4 the effects of methadone on key pecking under a fixed-interval 90-s schedule as well as fixed-ratio 50 and fixed-ratio 75 schedules were investigated. Control by the fixed-interval contingency was assessed by computing the proportion of total session reinforcers delivered under the fixed-interval schedule. Methadone administration (0.5-4.0 mg/kg) shifted the predominant source of schedule control under the alternative schedule from the fixed-ratio schedule to the fixed-interval contingency. This shift was dependent on methadone dose and fixed-ratio size. Control by the fixed-interval contingency was greatest following extensive exposure to the interval component embedded within the alternative schedule (Condition 1), but was apparent to a lesser degree with even very limited exposure to the alternative fixed-ratio fixed-interval schedule (Condition 2). Interreinforcement intervals comparable to those under fixed-interval schedule were not observed under the fixed-ratio schedules presented alone (Condition 3). Repeated exposure to the fixed-interval contingency outside the context of the alternative fixed-ratio fixed-interval schedule did not engender performance changes under a fixed-ratio schedule which would mimic those of increased fixed-interval contingency control (Condition 4). These data suggest that drug administration can be used to unmask the influence of contingencies that are latent under baseline conditions and reveal influences of both past and present environmental variables.     

Responding of pigeons under variable-interval schedules of signaled-delayed reinforcement: effects of delay-signal duration.

Two experiments with pigeons examined the relation of the duration of a signal for delay ("delay signal") to rates of key pecking. The first employed a multiple schedule comprised of two components with equal variable-interval 60-s schedules of 27-s delayed food reinforcement. In one component, a short (0.5-s) delay signal, presented immediately following the key peck that began the delay, was increased in duration across phases; in the second component the delay signal initially was equal to the length of the programmed delay (27 s) and was decreased across phases. Response rates prior to delays were an increasing function of delay-signal duration. As the delay signal was decreased in duration, response rates were generally higher than those obtained under identical delay-signal durations as the signal was increased in duration. In Experiment 2 a single variable-interval 60-s schedule of 27-s delayed reinforcement was used. Delay-signal durations were again increased gradually across phases. As in Experiment 1, response rates increased as the delay-signal duration was increased. Following the phase during which the signal lasted the entire delay, shorter delay-signal-duration conditions were introduced abruptly, rather than gradually as in Experiment 1, to determine whether the gradual shortening of the delay signal accounted for the differences observed in response rates under identical delay-signal conditions in Experiment 1. Response rates obtained during the second exposures to the conditions with shorter signals were higher than those observed under identical conditions as the signal duration was increased, as in Experiment 1. In both experiments, rates and patterns of responding during delays varied greatly across subjects and were not systematically related to delay-signal durations. The effects of the delay signal may be related to the signal's role as a discriminative stimulus for adventitiously reinforced intradelay behavior, or the delay signal may have served as a conditioned reinforcer by virtue of the temporal relation between it and presentation of food.     

Representing within-session response rates proportionally and entirely.

In this technical article, methods for collecting and representing response rates maintained by schedules of reinforcement are presented. First, the time in a session that each important event (e.g., responses, reinforcers) occurs is collected and stored by a computer. Another computer program is used, then, to convert each response to a percentage of the total responses in a session and to plot these percentages cumulatively as a function of the time in the session that they occurred. In this manner, response rates may be expressed proportionally (i.e., using the same y-axis scale regardless of absolute response rate) without requiring the arbitrary selection of an interval over which responses are aggregated and expressed relative to the entire-session rate. A property of these records is that deviations in the slope of the obtained record from the diagonal, which connects (x, y) = (start of session, 0%) to (x, y) = (end of session, 100%), occurring at any point and for any duration, represent changes in the local response rate from the entire-session rate. This method of representing ongoing responding is illustrated by several records of key pecking of a pigeon on a variable-interval 60-s schedule of food reinforcement. Relative local response rates were also computed from these data at several levels of resolution (i.e., the time over which responses were aggregated), including the level typically employed by those interested in within-session changes in response rates.     

Effects of cocaine on briefly signaled versus completely signaled delays to reinforcement.

Key pecking by 4 pigeons was maintained by a multiple schedule consisting of two variable-interval 60-s schedules wherein each food presentation followed a nonresetting 27-s delay that was either briefly signaled at its outset or completely signaled. Brief-signal duration was adjusted so that response rates maintained by the briefly and completely signaled delays of reinforcement were similar. In general, acute administration of small to intermediate doses (0.3 to 3.0 mg/kg) of cocaine produced either small increases in response rates in both components or no change, and larger doses (5.6 to 13.0 mg/kg) decreased response rates. Chronic (i.e., daily) cocaine administration (10.0 mg/kg) resulted in tolerance to the rate-decreasing effects in both components. Cocaine's effects were generally similar whether delays were completely or briefly signaled. Discontinuation of cocaine administration and subsequent removal of the delay signals also had similar effects in both components of the multiple schedule. Taken together, these results are consistent with the view that the two types of delay signals were equally effective in maintaining responding during the variable-interval schedules.     

Homogeneous chains, heterogeneous chains, and delay of reinforcement

Three pigeons responded on two-component chain schedules in which the required response topography in the initial and terminal links was similar (a homogeneous chain) or dissimilar (a heterogeneous chain). Key-peck responding in the initial link under a variable-interval 60-second (VI 60) schedule produced a terminal link in which, in different conditions, either key pecking or foot treadling was reinforced according to a VI 60 schedule. Multiple VI 60 VI 60 schedules, in which the responses required in the chain schedules were maintained by primary reinforcement in the two components, preceded and followed each type of chain. These multiple schedules were used to ensure that both responses occurred reliably prior to introducing the chain schedule. Key-peck response rates in the initial link of the chain consistently were higher during the homogeneous chain than during the heterogeneous chain. These results illustrate that intervening events during a period separating an operant response from primary reinforcement influence that operant, independently of the delay between the response and reinforcement.     

Responding of pigeons under variable-interval schedules of unsignaled, briefly signaled, and completely signaled delays to reinforcement

In Experiment 1, three pigeons' key pecking was maintained under a variable-interval 60-s schedule of food reinforcement. A 1-s unsignaled nonresetting delay to reinforcement was then added. Rates decreased and stabilized at values below those observed under immediate-reinforcement conditions. A brief stimulus change (key lit red for 0.5 s) was then arranged to follow immediately the peck that began the delay. Response rates quickly returned to baseline levels. Subsequently, rates near baseline levels were maintained with briefly signaled delays of 3 and 9 s. When a 27-s briefly signaled delay was instituted, response rates decreased to low levels. In Experiment 2, four pigeons' responding was first maintained under a multiple variable-interval 60-s (green key) variable-interval 60-s (red key) schedule. Response rates in both components fell to low levels when a 3-s unsignaled delay was added. In the first component delays were then briefly signaled in the same manner as Experiment 1, and in the second component they were signaled with a change in key color that remained until food was delivered. Response rates increased to near baseline levels in both components, and remained near baseline when the delays in both components were lengthened to 9 s. When delays were lengthened to 27 s, response rates fell to low levels in the briefly signaled delay component for three of four pigeons while remaining at or near baseline in the completely signaled delay component. In Experiment 3, low response rates under a 9-s unsignaled delay to reinforcement (tandem variable-interval 60 s fixed-time 9 s) increased when the delay was briefly signaled. The role of the brief stimulus as conditioned reinforcement may be a function of its temporal relation to food, and thus may be related to the eliciting function of the stimulus.     

Effects of reinforcement history on responding under progressive-ratio schedules of reinforcement.

The effects of experimental history on responding under a progressive-ratio schedule of reinforcement were examined. Sixteen pigeons were divided into four equal groups. Groups 1 to 3 were trained to peck a key for food under a fixed-ratio, variable-ratio, or differential-reinforcement-of-low-rate schedule of reinforcement. After training, these pigeons were shifted to a progressive-ratio schedule, later were shifted back to their original schedule (with decreased rates of reinforcement), and finally were returned to the progressive-ratio schedule. Pigeons in Group 4 (control) were maintained on the progressive-ratio schedule for the entire experiment. To test for potential "latent history" effects, pigeons responding under the progressive-ratio schedule were injected with d-amphetamine and given behavioral-momentum tests of prefeeding and extinction. Experimental histories affected responding in the immediate transition to the progressive-ratio schedule; response rates of pigeons with variable-ratio and fixed-ratio histories were higher than rates of pigeons with differential-reinforcement-of-low-rate and progressive-ratio-only histories. Pigeons with differential-reinforcement-of-low-rate histories, and to a lesser degree pigeons with variable-ratio and fixed-ratio histories, also had shorter postreinforcement pauses than pigeons with only a progressive-ratio history. No consistent long-term effects of prior contingencies on responding under the progressive-ratio schedule were evident. d-Amphetamine and resistance-to-change tests failed to reveal consistent latent history effects. The data suggest that history effects are sometimes transitory and not susceptible to latent influences.     

Effects of alternative reinforcement sources: A reevaluation

The effects of two alternative sources of food delivery on the key-peck responding of pigeons were examined. Pecking was maintained by a variable-interval 3-min schedule. In the presence of this schedule in different conditions, either a variable-time 3-min schedule delivering food independently of responding or an equivalent schedule that required a minimum 2-s pause between a key peck and food delivery (a differential-reinforcement-of-other-behavior schedule) was added. The differential-reinforcement-of-other-behavior schedule reduced response rates more than did the variable-time schedule in most instances. The delay between a key peck and the next reinforcer consistently was longer under the differential-reinforcement-of-other-behavior schedule than under the variable-time schedule. Response rates and median delay between responses and reinforcers were negatively correlated. These results contradict earlier conclusions about the behavioral effects of alternative reinforcement. They suggest that an interpretation in terms of response–reinforcer contiguity is consistent with the data.     

Dual effects on choice of conditioned reinforcement frequency and conditioned reinforcement value

Pigeons were presented with a concurrent‐chains schedule in which the total time to primary reinforcement was equated for the two alternatives (VI 30 s VI 60 s vs. VI 60 s VI 30 s). In one set of conditions, the terminal links were signaled by the same stimulus, and in another set of conditions they were signaled by different stimuli. Choice was in favor of the shorter terminal link when the terminal links were differentially signaled but in favor of the shorter initial link (and longer terminal link) when the terminal links shared the same stimulus. Preference reversed regularly with reversals of the stimulus condition and was unrelated to the discrimination between the two terminal links during the nondifferential stimulus condition. The present results suggest that the relative value of the terminal‐link stimuli and the relative rate of conditioned reinforcer presentation are important influences on choice behavior, and that models of conditioned reinforcement need to include both factors.     

Conditioned reinforcement versus time to reinforcement in chain schedules.

Pigeons were trained on three-component chain schedules in which the initial component was either a fixed-interval or variable-interval schedule. The middle and terminal components were varied among fixed-interval fixed-interval, variable-interval variable-interval, and an interdependent variable-interval variable-interval schedule in which the sum of the durations of the two variable-interval components was always equal to the sum of the fixed-interval fixed-interval components. At issue was whether the response rate in the initial component was controlled by its time to primary reinforcement or by the temporal parameters of the stimulus correlated with the middle terminal link. The fixed-interval initial-link schedule maintained much lower response rates than the variable-interval initial-link schedule regardless of the schedules in the middle and terminal links. Nevertheless, the intervening schedules played some role: With fixed-interval schedules in the initial links, response rates were consistently highest with independent variable-interval schedules in the middle and terminal links and intermediate with the interdependent variable-interval schedules; these initial-link differences were predicted by the response rates in the middle link of the chain. With variable-interval schedules in the initial links, response rates were lowest with the fixed-interval fixed-interval schedules following the initial link and were not systematically different for the two types of variable-interval variable-interval schedules. The results suggest that time to reinforcement itself accounts for little if any variance in initial-link responding.     

Performance of rats under concurrent variable-interval schedules of negative reinforcement

The behavior of rats under concurrent variable-interval schedules of negative reinforcement was examined. A single one-minute variable-interval programmer determined the availability of 30-second timeouts from electric shock. These were assigned to one or the other of the two component schedules with a probability of 0 to 1.0. The response requirement for the component schedules was standing to the right or left of the center of the experimental chamber. With a six-second changeover delay, the relative time spent under one component schedule varied directly and linearly with the relative number of timeouts earned under that component schedule. The absolute number of changeovers was highest when a similar number of timeouts was earned under each component schedule, and lowest when all or nearly all timeouts were earned under one component schedule. In general, these relations are similar to those reported with concurrent variable-interval schedules of positive reinforcement.     

Preference for multiple versus mixed schedules of reinforcement

Five pigeons were trained in a concurrent-chain procedure. In the initial links, equal nonindependent variable-interval schedules were available concurrently on two keys. Completing the schedule on either key led to exclusive presentation of one of two further variable-interval schedules for a fixed period of time. During these terminal links, as many reinforcers as were scheduled could be obtained. If the response producing this terminal link occurred on one key, differential stimuli signaled which variable-interval schedule had been produced. If the response producing the terminal link occurred on the other key, no such differential stimuli were available. Once the fixed period of time elapsed, the initial links were reinstated. In Experiment 1, the period of time for which the terminal links were available was always 10 s and the absolute duration of the initial links was varied. Subjects preferred the alternative leading to the multiple schedule when the initial-link duration was short, but preferred the alternative leading to the mixed schedule when the initial-link durations were longer. In Experiment 2, both the initial-link duration and the duration of the terminal links were varied. The effect of initial-link duration was identical to that in Experiment 1 and there was no systematic effect of varying the terminal-link duration.     

Stimulus control of behavioral history.

Pigeons were exposed to two different reinforcement schedules under different stimulus conditions in each of two daily sessions separated by 6 hr (Experiments 1 and 2) or in a single session (Experiment 3). Following this, either a fixed-interval (Experiment 1) or a variable-interval schedule (Experiments 2 and 3) was effected in both stimulus conditions. In the first two experiments, exposure to fixed-ratio or differential-reinforcement-of-low-rate schedules led to response-rate, but not pattern, differences in subsequent performance on fixed- or variable-interval schedules that persisted for up to 60 sessions. The effects of reinforcement-schedule history on fixed-interval schedule performance generally were more persistent. In Experiment 3, a history of high and low response rates in different components of a multiple schedule resulted in subsequent response-rate differences under identical variable-interval schedules. Higher response rates initially occurred in the component previously correlated with high response rates. For 3 of 4 subjects, the differences persisted for 20 or more sessions. Previous demonstrations of behavioral history effects have been confined largely to between-subject comparisons. By contrast, the present results demonstrate strong behavioral effects of schedule histories under stimulus control within individual subjects.     

Probability and delay of reinforcement as factors in discrete-trial choice.

Pigeons chose between two alternatives that differed in the probability of reinforcement and the delay to reinforcement. A peck on the red key always produced a delay of 5 s and then a possible reinforcer. The probability of reinforcement for responding on this key varied from .05 to 1.0 in different conditions. A response on the green key produced a delay of adjustable duration and then a possible reinforcer, with the probability of reinforcement ranging from .25 to 1.0 in different conditions. The green-key delay was increased or decreased many times per session, depending on a subject's previous choices. The purpose of these adjustments was to estimate an indifference point, or a delay that resulted in a subject's choosing each alternative about equally often. In conditions where the probability of reinforcement was five times higher on the green key, the green-key delay averaged about 12 s at the indifference point. In conditions where the probability of reinforcement was twice as high on the green key, the green-key delay at the indifference point was about 8 s with high probabilities and about 6 s with low probabilities. An analysis based on these results and those from studies on delay of reinforcement suggests that pigeons' choices are relatively insensitive to variations in the probability of reinforcement between .2 and 1.0, but quite sensitive to variations in probability between .2 and 0.     

Yoked variable-ratio and variable-interval responding in pigeons

Pigeons' key pecks were maintained by variable-ratio or variable-interval schedules of food reinforcement. For pairs of pigeons in one group, variable-ratio reinforcement was arranged for one pigeon's pecks; for the second pigeon, reinforcement was arranged according to a variable-interval schedule yoked to the interreinforcement times produced by the first pigeon. For pairs of pigeons in another group, variable-interval reinforcement was arranged for one pigeon's pecks; for the second pigeon, reinforcement was arranged according to a variable-ratio schedule yoked to the interreinforcement responses produced by the first pigeon. For each pair, the yoking procedure was maintained for four or five consecutive sessions of 50 reinforcements each. In more than three-quarters of the pairs, variable-ratio response rates were higher than variable-interval rates within two sessions; in all cases, the rate difference developed within four sessions.