Negatively reinforced key pecking

A reinforcement-switching procedure was used to produce negatively reinforced key pecking in pigeons. First, key pecking on a chain schedule (fixed-interval 10-sec variable-interval 60-sec) was conditioned using grain reinforcement. Second, intermittent shock in the initial link was introduced at a low intensity and gradually increased. Third, food reinforcement in the terminal link was eliminated. With shock at 90 V occurring on the average every 3 sec, initial-link pecking was maintained with no terminal-link food. Three of four pigeons responded consistently at shock intensities of 90, 70, and 50 V but not at 30 V. A fourth pigeon responded at but not below 90 V. Rate of response was directly related to shock frequency. Eliminating food deprivation did not affect the negatively reinforced performance.     

Punishment of observing by the negative discriminative stimulus

To determine the effect of a negative discriminative stimulus on the response producing it, two pigeons were each studied in a three-key conditioning chamber. During alternating periods of unpredictable duration, pecking the center (food) key either was reinforced with grain on a variable-interval schedule or was never reinforced. On equal but independent variable-interval schedules, pecking either of the side (observing) keys changed the color of all keys for 30 sec from yellow to either green or red. When the schedule on the center key was variable-interval reinforcement, the color was green (positive discriminative stimulus); when no reinforcements were scheduled, the color was red (negative discriminative stimulus). Since pecking the side keys did not affect grain deliveries, changes in the rate of pecking could not be ascribed to changes in the frequency of primary reinforcement. In subsequent sessions, red was withheld as one of the possible consequences of pecking a given side key. When red was omitted, the rate on that key increased, and when red was restored, the rate decreased. It was concluded that red illumination of the keys, the negative discriminative stimulus, had a suppressive effect on the response that produced it.     

Key pecking of pigeons under variable-interval schedules of briefly signaled delayed reinforcement: effects of variable-interval value.

Key pecking of 4 pigeons was maintained under a multiple variable-interval 20-s variable-interval 120-s schedule of food reinforcement. When rates of key pecking were stable, a 5-s unsignaled, nonresetting delay to reinforcement separated the first peck after an interval elapsed from reinforcement in both components. Rates of pecking decreased substantially in both components. When rates were stable, the situation was changed such that the peck that began the 5-s delay also changed the color of the keylight for 0.5 s (i.e., the delay was briefly signaled). Rates increased to near-immediate reinforcement levels. In subsequent conditions, delays of 10 and 20 s, still briefly signaled, were tested. Although rates of key pecking during the component with the variable-interval 120-s schedule did not change appreciably across conditions, rates during the variable-interval 20-s component decreased greatly in 1 pigeon at the 10-s delay and decreased in all pigeons at the 20-s delay. In a control condition, the variable-interval 20-s schedule with 20-s delays was changed to a variable-interval 35-s schedule with 5-s delays, thus equating nominal rates of reinforcement. Rates of pecking increased to baseline levels. Rates of pecking, then, depended on the value of the briefly signaled delay relative to the programmed interfood times, rather than on the absolute delay value. These results are discussed in terms of similar findings in the literature on conditioned reinforcement, delayed matching to sample, and classical conditioning.     

Briefly delayed reinforcement: An interresponse time analysis

Key-peck responding of pigeons was compared under VI or DRL schedules arranging immediate reinforcement and briefly (.5 sec) delayed reinforcement. Delays were either signaled by a blackout in the chamber, unsignaled, or unsignaled with an additional requirement that responding not occur during the .5 sec interval immediately preceding reinforcement (response delay). Relative to the immediate reinforcement condition, response rates increased during the unsignaled delay, decreased during the signaled delay, and were inconsistent during the response delay condition. An analysis of interresponse times (IRTs) under the different conditions revealed a substantial increase in the frequency of short (0 to .5 sec) IRTs during the unsignaled condition and generally during the response delay conditions compared to that during the immediate reinforcement baseline. Signaled delays decreased the frequency of short (0 to .5 sec) IRTs relative to the immediate reinforcement condition. The results suggest that brief unsignaled delays and, in many instances, response delays increase the frequency of short IRTs by eliminating constraints on responding.     

Concurrent performances: inhibition of one response by reinforcement of another

In an analysis of interactions between concurrent performances, variable-interval reinforcement was scheduled, in various sequences, for both keys, for only one key, or for neither key of a two-key pigeon chamber. With changeover delays of 0.5 or 1.0 sec, and with each key's reinforcements discriminated on the basis of key-correlated feeder stimuli, reinforcement of pecks on one key reduced the pecking maintained by reinforcement on the other key. The decrease in pecking early after reinforcement was discontinued on one key was not substantially affected by whether pecks on the other key were reinforced, but after reinforcement was discontinued on both keys, reinstatement of reinforcement for one key sometimes produced transient increases in pecking on the other key. Correlating the availability of right-key reinforcements with a stimulus, which maintained right-key reinforcement while reducing right-key pecking to negligible levels, demonstrated that these interactions depended on concurrent reinforcement, not concurrent responding. Thus, reinforcement of a response, but not necessarily the occurrence of the response, inhibits other reinforced responses. Compared with accounts in terms of excitatory effects of extinction, often invoked in treatments of behavioral contrast, this inhibitory account has the advantage of dealing only with observed dimensions of behavior.     

Economic and biological influences on a pigeon's key peck

Pigeons were studied in a two-component multiple schedule. In the first phase of the experiment, key pecks were reinforced on a variable-interval 2-min schedule in both components and free food was delivered additionally during one component. When components alternated every 8 sec, all pigeons pecked at a much higher rate during the component with free food than during the other component. At a component duration of 16 min, the reverse was true: all pigeons pecked at a higher rate during the component without free food. In the second phase, the additional food during one component was made contingent on pecking. Responding during the component without the extra food remained essentially unchanged, as expected, since rate of reinforcement remained identical to that in the previous phase. However, rate of responding during the component with the extra food (now contingent on pecking) was elevated, compared to the rate in the first phase, and did not show the marked decline as component duration was increased.     

Inhibitory stimulus control and the magnitude of delayed reinforcement

The key pecking of pigeons was reinforced according to a variable-interval 1-min schedule during each of two successively presented stimuli. When the key was illuminated by a black line on a white background, reinforcement was delayed for 10 sec. When the key was illuminated by a plain white light, reinforcement was not delayed. For half of the subjects, the delayed reinforcer was 4.0-sec access to mixed grain, and for the remaining subjects it was 1.5-sec access. The immediate reinforcer was 1.5-sec access for all subjects. All subjects responded at a lower rate during the presentation of the black line; no between-group difference in terms of terminal response rate during the presentation of the line was found. However, subjects that received 4.0 sec of delayed reinforcement responded at a lower terminal rate during presentation of the plain white light than subjects that received 1.5 sec of delayed reinforcement. A subsequent generalization test along the line-orientation dimension produced flatter U-shaped gradients for subjects that received 4.0-sec of delayed reinforcement.     

Mechanisms underlying the effects of unsignaled delayed reinforcement on key pecking of pigeons under variable-interval schedules.

Three experiments were conducted to test an interpretation of the response-rate-reducing effects of unsignaled nonresetting delays to reinforcement in pigeons. According to this interpretation, rates of key pecking decrease under these conditions because key pecks alternate with hopper-observing behavior. In Experiment 1, 4 pigeons pecked a food key that raised the hopper provided that pecks on a different variable-interval-schedule key met the requirements of a variable-interval 60-s schedule. The stimuli associated with the availability of the hopper (i.e., houselight and keylight off, food key illuminated, feedback following food-key pecks) were gradually removed across phases while the dependent relation between hopper availability and variable-interval-schedule key pecks was maintained. Rates of pecking the variable-interval-schedule key decreased to low levels and rates of food-key pecks increased when variable-interval-schedule key pecks did not produce hopper-correlated stimuli. In Experiment 2, pigeons initially pecked a single key under a variable-interval 60-s schedule. Then the dependent relation between hopper presentation and key pecks was eliminated by arranging a variable-time 60-s schedule. When rates of pecking had decreased to low levels, conditions were changed so that pecks during the final 5 s of each interval changed the keylight color from green to amber. When pecking produced these hopper-correlated stimuli, pecking occurred at high rates, despite the absence of a peck-food dependency. When peck-produced changes in keylight color were uncorrelated with food, rates of pecking fell to low levels. In Experiment 3, details (obtained delays, interresponse-time distributions, eating times) of the transition from high to low response rates produced by the introduction of a 3-s unsignaled delay were tracked from session to session in 3 pigeons that had been initially trained to peck under a conventional variable-interval 60-s schedule. Decreases in response rates soon after the transition to delayed reinforcement were accompanied by decreases in eating times and alterations in interresponse-time distributions. As response rates decreased and became stable, eating times increased and their variability decreased. These findings support an interpretation of the effects of delayed reinforcement that emphasizes the importance of hopper-observing behavior.     

Concurrent performances: a baseline for the study of reinforcement magnitude

When a pigeon's pecking on a single key was reinforced by a variable-interval (VI) schedule of reinforcement, the rate of pecking was insensitive to changes in the duration of reinforcement from 3 to 6 sec. When, however, the pigeon's pecking on each of two keys was concurrently reinforced by two independent VI schedules, one for each key, the rate of pecking was directly proportional to the duration of reinforcement.     

Concurrent performances: synthesizing rate constancies by manipulating contingencies for a single response

An earlier experiment scheduled variable-interval reinforcement for pigeons' pecks on one key, and variable-interval reinforcement alternating with extinction, in a multiple schedule, for pecks on a second key. During the second key's extinction component, first-key pecking was relatively slow and continuous, rarely interrupted by second-key pecking; during the variable-interval component, first-key pecking was frequently interrupted by second-key pecking. When changeover delays operated, so that reinforced pecks on one key could not follow closely upon changeovers from the other key, rapid first-key pecking between interruptions compensated sufficiently for the time lost in second-key pecking that the overall rate of first-key pecking remained roughly constant across the alternating multiple-schedule components. The present experiments duplicated, on a single key, the temporal pattern of first-key pecking generated in the earlier experiments: components of continuous key availability were alternated with components of interrupted key availability. Approximately constant overall rates of responding were observed with a single-key equivalent of a changeover delay scheduled after interruptions and with manipulations of the on-off durations of the interruption cycle. Rate constancies in the original concurrent situation presumably depended on analogous contingencies that operated upon the concurrent responses, rather than on any constant “reserve” of responses.     

Punishment of observing by a stimulus associated with the lower of two reinforcement frequencies

Three pigeons were used to investigate the effects of a stimulus associated with the lower of two reinforcement frequencies on the response producing it. In a three-key chamber, pecking the center key produced grain on alternating variable-interval schedules with mean durations of 2 min or 30 sec. Initially, green illumination of the keys accompanied the more favorable (30-sec) schedule and red accompanied the less favorable (2-min) schedule. Then the keys remained yellow unless the bird pecked one of the side (observing) keys to produce the discriminative stimuli for a 30-sec period. Subsequently, when red was withheld as a possible consequence of pecking a particular side key, the rate on that key increased; when red was restored, the observing rate decreased. Thus the stimulus associated with less frequent reinforcement had a punishing effect on the behavior producing it. When green was withheld on one of the side keys and the other key produced both colors, observing behavior was not maintained on the red-only key, but was maintained on the key that produced both colors.     

Responding of pigeons under variable-interval schedules of signaled-delayed reinforcement: effects of delay-signal duration.

Two experiments with pigeons examined the relation of the duration of a signal for delay ("delay signal") to rates of key pecking. The first employed a multiple schedule comprised of two components with equal variable-interval 60-s schedules of 27-s delayed food reinforcement. In one component, a short (0.5-s) delay signal, presented immediately following the key peck that began the delay, was increased in duration across phases; in the second component the delay signal initially was equal to the length of the programmed delay (27 s) and was decreased across phases. Response rates prior to delays were an increasing function of delay-signal duration. As the delay signal was decreased in duration, response rates were generally higher than those obtained under identical delay-signal durations as the signal was increased in duration. In Experiment 2 a single variable-interval 60-s schedule of 27-s delayed reinforcement was used. Delay-signal durations were again increased gradually across phases. As in Experiment 1, response rates increased as the delay-signal duration was increased. Following the phase during which the signal lasted the entire delay, shorter delay-signal-duration conditions were introduced abruptly, rather than gradually as in Experiment 1, to determine whether the gradual shortening of the delay signal accounted for the differences observed in response rates under identical delay-signal conditions in Experiment 1. Response rates obtained during the second exposures to the conditions with shorter signals were higher than those observed under identical conditions as the signal duration was increased, as in Experiment 1. In both experiments, rates and patterns of responding during delays varied greatly across subjects and were not systematically related to delay-signal durations. The effects of the delay signal may be related to the signal's role as a discriminative stimulus for adventitiously reinforced intradelay behavior, or the delay signal may have served as a conditioned reinforcer by virtue of the temporal relation between it and presentation of food.     

Rate and temporal pattern of key pecking under autoshaping and omission schedules of reinforcement

The role of response-reinforcer contiguity on autoshaped key pecking in pigeons was studied by scheduling response-dependent nonreinforcement at the beginning or the end of brief (8-sec) discrete trials. Schedules that permitted chance conjunctions of key pecking and food sustained high rates of responding, whereas those that prevented the occurrence of key peck-food intervals shorter than 4 sec sustained low response rates. In addition, selective reinforcement schedules supported accelerating or decelerating rates of responding within individual trials. These effects were traceable to response-reinforcer (operant), but not stimulus-reinforcer (respondent) factors.     

Stimulus control of responding by response-reinforcer temporal contiguity

The stimulus properties of brief disruptions in response-reinforcer temporal contiguity were investigated using a discrete trial conditional discrimination procedure. Key pecking (nondelay) or key pecking followed by a brief interval of nonpecking (delay) in the sample component produced a stimulus change (choice component). Pecks in the choice component to one of two alternatives resulted in food or blackout, conditional upon which response requirement was met in producing the choice component. A baseline condition, in which key pecking always produced the choice component and correct choices were arranged randomly, alternated with experimental conditions that included nondelay and delay values of either 0.2, 0.5, or 1.0 sec between the last key peck and the initiation of the choice component. All subjects accurately discriminated brief temporal delays between a response and stimulus change, with choice accuracy increasing for three of four subjects as the temporal disruption in contiguity increase. Implications of the research for the study of delayed reinforcement, response-independent reinforcement, and the discrimination of causality are discussed.     

Alternative reinforcement effects on fixed-interval performance

Pigeons' key pecks were reinforced with food on a fixed-interval schedule. Food also was available at variable time periods either independently of responding or for not key pecking (a differential-reinforcement-of-other-behavior schedule). The latter condition arranged reinforcement following the first pause of t seconds after it became available according to a variable-time schedule. This schedule allowed separation of the effects of pause requirements ≤ five-seconds and reinforcement frequency. The time spent pausing increased as the duration of the pause required for reinforcement increased from 0 to 30 seconds and as the frequency of reinforcement for pausing increased from 0 to 2 reinforcers per minute. Key pecking was more evenly distributed within each fixed interval with shorter required pauses and with more frequent reinforcement for pausing. The results complement those obtained with other concurrent schedules in which the same operant response was reinforced in both components.     

Behavioral contrast as a function of the duration of an immediately preceding period of extinction

For four pigeons key-peck responding was reinforced on a variable-interval reinforcement schedule in the presence of a vertical white line. When response rates had stabilized a horizontal white line was introduced, in the presence of which reinforcement was not available (extinction). The horizontal line was presented once per session, immediately before the vertical line was presented. The duration of the horizontal line varied randomly from session to session, being either 0 sec (i.e., no presentation), 10 sec, 30 sec, 2 min, 10 min, 40 min, or 120 min. When the horizontal line was present for more than 0 sec, behavioral contrast was obtained in the presence of the following vertical line. Contrast increased with increasing durations of the horizontal line, asymptoting when the horizontal line was present for 40 min.     

Frequency of reinforcement as a determinant of extinction-induced aggression during errorless discrimination learning.

Seven pigeons were trained to discriminate without errors between a green keylight and a dark key. The key-pecking response was reinforced in the presence of green, and extinction was in effect in the presence of the dark key. The opportunity to attack a restrained target pigeon was present only during extinction. Both variable-interval 30-sec and fixed-ratio 1 schedules of reinforcement during the positive stimulus induced a higher rate of attack during extinction than a variable-interval 5-min schedule. The highest rate of attack during extinction occurred during the first 20 sec after the positive stimulus terminated. Hence, the withdrawal of the positive condition, rather than the consequences of the pecking response during extinction, appears to be one of the primary factors responsible for attack between pigeons during extinction. Behavioral contrast, defined as a decrease in the rate of responding when the positive stimulus was presented alone, was obtained from the four birds that displayed the lowest overall rates of attack while the three birds with the highest attack rates did not display behavioral contrast. For the birds without contrast, components of the attack response during the positive stimulus presumably competed with and reduced the rate of pecking the key, thereby recluding behavioral contrast.     

Absence of shock-elicited aggression in pigeons

Two pigeons that attacked a taxidermically prepared target pigeon during a schedule of positive reinforcement for key pecking, and two that did not, were shocked through implanted electrodes in the presence of the target. Shock intensities of 2 and 4 mA, durations of 0.1 and 1.3 sec, and frequencies of 2, 6, 20, and 35 per minute were delivered across 16 sessions with 180 shocks per session. No pigeon attacked the target; one pecked the shockplug on its back. The two pigeons that had not attacked during the positive reinforcement schedules were conditioned to peck the target for food reinforcement before another 16 sessions of shock. No attack was observed in these shock sessions. During subsequent positive reinforcement of key pecking, the target was attacked by the two pigeons that had originally attacked and by one that had not. Absence of shock-elicited attack in these pigeons may be related to the parameters of the experiment or may be yet another instance of the absence of shock-elicited attack in the class Aves. At least under the present conditions, it was not possible to predict the level of attack during electric shock from the level of attack during schedules of positive reinforcement for key pecking.     

Preference for mixed versus constant delays of reinforcement: Effect of probability of the short, mixed delay

Preference for mixed versus constant delays of reinforcement was studied with a concurrent-chain procedure. Lever pressing by rats in concurrently available variable-interval 60-second initial links occasionally produced mutually exclusive terminal-link reinforcement delays. A constant delay of reinforcement (either 15 seconds or 30 seconds) composed one terminal link and mixed delays (.2 second and twice the value of the constant delay) were arranged in the other terminal link. The proportion of .2-second delays in the mixed-delay terminal link took on values of 0, .1, .25, .5, .75, .9, and 1.0 over experimental conditions. Based on relative rates of responding in the initial links, preference for the mixed delays was a negatively accelerated function of the proportion of short, mixed delays. Three of five rats preferred the mixed delays to the constant delays when the proportion of short, mixed delays was .1 or higher, and all five rats preferred the mixed delays when the proportion of short, mixed delays was .25 or higher. Neither Squires and Fantino's (1971) delay-reduction model of choice nor a model based on the harmonic mean reinforcement delay provided a close estimate of choice proportions over the range of short-delay proportions studied. The delay-reduction model underestimated choice for the mixed delays at low and intermediate proportions of short delays, and the harmonic-mean-delay model overestimated choice for the mixed delays at intermediate and high proportions of short delays.     

Effects of response rate, reinforcement frequency, and the duration of a stimulus preceding response-independent food

Food-reinforced key pecking in the pigeon was maintained under a four-component multiple schedule. In two components, responding was maintained at high rates under a random-ratio schedule. In the other two components, responding was maintained at low rates under a schedule that specified a minimum interresponse time. For both high and low response rates, one of the schedule components was associated with a high reinforcement frequency and the other components with a lower reinforcement frequency. During performance under these schedules, a stimulus terminated by access to response-independent food was periodically presented. The duration of this pre-food stimulus was 5, 30, 60, or 120 sec. Changes in rate of key pecking during the pre-food stimulus were systematically related to baseline response rate and the duration of the stimulus. Both high and low response rates were increased during the 5-sec stimulus. At longer stimulus durations, low response rates were unaffected and high response rates were decreased during the stimulus. For two of three pigeons, high response rates maintained under a lower frequency of reinforcement tended to be decreased more than high response rates maintained under a higher reinforcement frequency. In general, the magnitude of decrease in high response rates was inversely related to the duration of the pre-food stimulus.