Parameters affecting the maintenance of negatively reinforced key pecking

Three negative reinforcement experiments employing a key-peck response are described. In Experiment I, pigeons shocked on the average of twice per minute (imposed condition) could produce, by pecking a key, an alternate condition with correlated stimuli. Delayed shocks were added, across sessions, to the alternate condition until pecking stopped. Two of three pigeons continued to peck despite a 100% increase in shock frequency. In Experiment II, pigeons were shocked in the imposed condition four times per minute. The postresponse delay to shock was held constant by delivering, in the alternate condition, the next shock, or the next two, three, or four shocks from the imposed-condition shock schedule. All three subjects continued to peck with no change in delay to the first two postresponse shocks but with a 75% reduction in shock frequency. In Experiment III, a response produced an immediate shock followed by a shock-free period. Three of four subjects continued to respond despite reduced delay to shock. Delay-to-shock or shock-frequency reduction was sufficient to maintain key pecking, but neither was necessary. The conditions that negatively reinforce the pigeon's key peck were similar to conditions that negatively reinforce the rat's bar press.     

Notes on fixed-ratio and fixed-interval escape responding in the pigeon

After learning to peck a key when each peck removed a slowly increasing series of electric shocks, pigeons were placed on fixed-ratio and fixed-interval escape schedules. The resulting behavior was comparable to that of other species on ratio and interval escape schedules. Thus, while the pigeon apparently requires special techniques for the initial shaping of a key-peck response with negative reinforcement, this response, once obtained, can be subjected to intermittent schedules of negative reinforcement with no great difficulty.     

The effects of reinforcement upon the prepecking behaviors of pigeons in the autoshaping experiment

The autoshaping procedure confounds the effects of pairing a keylight and food with the effect of adventitious food reinforcement of responses that typically occur before the pecking response. In Experiment I, acquisition of the orientation to the key, the approach toward the key, and the peck at the key were systematically monitored. Orientations to the key and approaches toward the key frequently occurred in contiguity with food presentation before peck acquisition. In Experiment II, a negative contingency procedure was used to assess the sensitivity of the approach toward the key to its consequences. When the approach toward the key resulted in nonreinforcement, the probability of occurrence of that response decreased to zero despite repeated light-food pairings. In Experiment III, peck probability was shown to be determined during the approach toward the key by the presence of stimuli that had previously been either paired or nonpaired with food. In Experiment IV, it was shown that the effects of the stimulus present during the approach toward the key were not due solely to the effects of pairing that stimulus with food. Autoshaped key pecking appears to be determined by the interacting effects of stimulus-reinforcer and response-reinforcer variables upon orientations to, approaches toward, and pecks at the lighted key.     

The development of stimulus control with and without a lighted key

In two experiments, the effect of an illuminated response key on the acquisition of stimulus control by an airflow stimulus was assessed. In the first experiment, pigeons were given nondifferential training with airflow emerging from behind the response key in one of three conditions of illumination: trained to peck a lighted key, trained to peck an unlighted key with a houselight present, trained to peck a key in total darkness. After 10 days of training on a variable-interval schedule of reinforcement, all subjects were given a generalization test on airflow velocity. The gradients for subjects trained in the dark were sharp, while those for subjects trained in lighted conditions were shallow. In the second experiment, the effect of an irrelevant keylight on the acquisition of an airflow velocity discrimination was assessed. Two groups of pigeons were trained to discriminate two airflow velocities. One group was trained with a lighted response key and the other was trained to peck the response key in total darkness. The dark-trained subjects acquired the discrimination more rapidly. The results demonstrate that the acquisition of stimulus control by airflow with either a differential or nondifferential training procedure can be overshadowed by keylight.     

Signal-controlled responding

Pigeons' key pecks were reinforced with grain, then extinguished. An 8-second tone preceded the availability of peck-dependent grain 1 second after tone offset. When a tone signalled grain and an 8-second clicking sound did not, three pigeons pecked during a high percentage of tone periods, but they pecked during a low percentage of click periods. When the roles of the tone and clicking sound were reversed, performance reversed. For other birds, when a key peck during the tone cancelled the availability of grain (omission procedure), the tendency to key peck during the tone decreased some, but still remained high. A third group of pigeons received the omission procedure with the addition that the tone could not end unless 2 seconds had elapsed without a key peck. The pigeons continued to respond in a high percentage of tone periods. The experiments favor an explanation based on the pairing of the tone with a reinforced response, such as Pavlovian conditioning.     

Stimulus aspects of aversive controls: the effects of response contingent shock

A tone ending with electrical shock was periodically presented to pigeons while they pecked a key for food. Pairs of birds were run simultaneously under a yoked program which insured that both birds received the same number and temporal distribution of shocks. For one of the birds, shock was always initiated by a peck; for the other, shock was unavoidable. Both procedures led to reduced rates of pecking in the presence of the tone, and gradients of stimulus generalization were obtained. But the effects of response contingent shock extinguished more rapidly than the effects of unavoidable shock. In general, birds exposed to unavoidable shock tended to respond at intermediate rates throughout tone, whereas those exposed to response contingent shock ceased to peck for part or all of the tone period.     

Auto-maintenance in the pigeon: sustained pecking despite contingent non-reinforcement

If a response key is regularly illuminated for several seconds before food is presented, pigeons will peck it after a moderate number of pairings; this “auto-shaping” procedure of Brown and Jenkins (1968) was explored further in the present series of four experiments. The first showed that pecking was maintained even when pecks turned off the key and prevented reinforcement (auto-maintenance); the second controlled for possible effects of generalization and stimulus change. Two other experiments explored procedures that manipulated the tendency to peck the negatively correlated key by introducing alternative response keys which had no scheduled consequences. The results indicate that pecking can be established and maintained by certain stimulus-reinforcer relationships, independent of explicit or adventitious contingencies between response and reinforcer.     

Some discriminative properties of fixed ratio performance in the pigeon

The discriminative control over a spatial choice response exercised by prior behavior was studied using a procedure involving discrete exposures to a two-member chained schedule. The initial member (red key) was either a smaller or larger fixed ratio (Mix FR:FR), the completion of which produced, after a 1-sec delay, two white response keys. If the larger FR had been completed as the initial chain member, a single peck on the right white key was reinforced; after the smaller FR, a peck on the left white key was reinforced. Frequencies of unreinforced responses (S(Delta) responses) were determined with several pairs of red-key FRs: 95-5, 75-25, 65-35, 60-40, 58-42 and 50-50. The S(Delta) response frequencies were low through the FR pair 65-35; sharp increases were obtained with pairs 60-40 and 58-42. Later, curves analogous to stimulus generalization functions were obtained using a probe procedure. Finally, the delay interval between completion of a red-key FR and the white-key choice response was manipulated: results were variable, but S(Delta) response frequencies tended to increase with increasing delays.     

Acquisition of the autoshaped key peck as a function of amount of preliminary magazine training

Three experiments evaluated the effect of magazine training on acquisition of the pigeon's key peck during autoshaping. In Experiment I, pigeons were exposed to two days of extended magazine training, followed on the third day by keylight-only presentations. All pigeons pecked the keylight early in the keylight-only session. Experiment II examined the relationship between the number of magazine-training trials and trials to the first peck. Pigeons were given either 0, 3, 10, or 25 magazine-training trials followed by the standard autoshaping procedure. The number of trials to the first peck was related to the number of magazine-training trials. In Experiment III, pigeons were exposed to the standard autoshaping procedure without prior magazine training. The data from Experiment III suggested that key pecking will occur only after the response of eating from the lighted hopper has occurred. Taken together, these results suggest that initial magazine training is an important variable in autoshaping. Key pecking is discussed as a generalized consummatory response.     

"Automaintenance": the role of reinforcement

A key was illuminated on the average of every 30 sec for a duration of 6 sec and this was followed by food presentations. When key pecks in the presence of the light produced immediate access to grain (autoshaping procedure) pigeons were likely to peck. When pecks terminated the keylight but prevented access to grain (automaintenance procedure) pigeons were much less likely to peck. Seven of 12 pigeons failed to develop responding during the automaintenance procedure. Four of the five pigeons that responded during the automaintenance procedure were exposed to a procedure in which responses could not immediately terminate the light. Three of the four ceased to respond during optimal automaintenance conditions, suggesting that the response-dependent offset of the keylight had been reinforcing their pecking. Responding during the automaintenance procedure was eliminated for a fifth pigeon by eliminating the contiguity of light-offset and food-onset on those trials in which the pigeon did not peck. These results suggest that: (1) automaintenance (unlike autoshaping) is not an effective procedure for reliably generating responding; (2) responding that does occur during the automaintenance procedure is reinforced by the response-dependent offset of the keylight.     

Behavioral control by an imprinted stimulus

Newly hatched ducklings were exposed to imprinting procedures and subsequently trained to peck a key by presenting the imprinting stimulus as the reinforcing (response contingent) event. It was found that the key peck was learned only when imprinting procedures were initiated during the first 6 to 8 hr after hatch. Additional studies revealed that: (1) the duckling's distress vocalizations were reduced in the presence of the imprinting stimulus and enhanced in its absence; (2) when the ducklings had constant access to the imprinted stimulus (via a key peck), pecking responses occurred in bursts and relatively few distress vocalizations occurred; (3) the initial effect of extinction procedures was an increase in key peck rate. When, however, repeated key pecks failed to produce the imprinted stimulus, distress vocalization ensued and peck rate declined; (4) both the presentation of an unfamiliar mechanical figure and delivery of electrical shock enhanced distress vocalization and key pecks; (5) for some ducklings, certain familiar objects in the environment influenced distress calls in a manner comparable to the imprinted stimulus in that distress calls increased when these objects were removed.     

Controls for and constraints on auto-shaping

Auto-shaping the pigeon's key-peck response was examined as a respondent conditioning procedure with the use of Rescorla's truly-random control procedure. In the first experiment, pigeons received presentations of brief light on the response key and brief presentations of food where the light and the food were independently presented. All birds failed to key peck after many light and food presentations, but explicit pairing of the light and food rapidly conditioned pecking to the light. Experiment 2 showed that even when an independent light/food presentation schedule was reduced to variable-time 30 sec, additional naive birds would not key peck and only one bird pecked when the schedules were variable-time 15 sec. A third experiment examined an explicit-unpairing control procedure, where the light and food were not only presented on independent schedules but were also separated by a minimum time, and found that auto-shaping did not occur. A fourth experiment investigated a number of control procedures and found them ineffective. A fifth experiment investigated the effects of a physical separation of the locus of the response key and the food dispenser, and a sixth experiment investigated using a tone in place of the light. It was concluded that pecking is generated by auto-shaping procedures only when an intermittently presented keylight is regularly paired with food.     

Stimulus control and response bias in an analogue prey-detection procedure

The present study compared the performance of 6 pigeons trained to detect luminance differences in two different signal-detection procedures. Exposed to a three-key array, the pigeons were trained to peck the left key when the brighter of two light intensities had been presented on the center key and to peck the right key when the dimmer of two light intensities had been presented on the center key. Procedure A was a standard signal-detection procedure in which left/bright and right/dim responses produced food reinforcement and left/dim and right/bright responses produced periods of timeout. Procedure B was designed to simulate some of the contingencies operating in a prey-detection situation. Left-key responses produced reinforcement following the brighter center-key stimulus and a period of timeout following the dimmer center-key stimulus. Right-key responses always produced a short period of timeout irrespective of the stimulus. Within each procedure, the duration of timeout arranged for false alarms (left/dim responses) was varied between 3 s and 120 s. Measures of accuracy and response bias were compared between the two procedures. The timeout manipulation produced systematic, but relatively small, changes in these measures when right/dim responses (i.e., correct rejections) produced reinforcement (Procedure A). Arranging timeout for right/dim responses in Procedure B produced greater variability in accuracy and response bias than did arranging reinforcement, but this variability was not related to timeout duration. Overall, discrimination accuracy was considerably higher when right/dim responses produced timeout than when they resulted in reinforcement, and accuracy was accompanied by a large bias toward the response associated with reinforcement. These results are consistent with a recently proposed model of signal detection.     

Signal probability, reinforcement and signal detection.

Five pigeons were trained to detect differences in light intensity. Two stimuli, S1 and S2, differing in intensity, were arranged on the center key of a three-key chamber according to set probabilities. A peck on the center key turned on the two side keys. When S1 was presented on the center key, a peck on the left key was "correct" and when S2 was presented, a peck on the right key was "correct." Correct responses produced reinforcement and incorrect responses produced 3-second blackout. Detection performance was measured under three procedures. The first was a standard signal-detection design in which the probability of S1 was varied and the number of reinforcements obtained for correct responses to S1 was allowed to covary. In the second procedure, the probability of S1 was again varied but the distribution of reinforcements between the two choices was kept equal. In the third procedure, probability of S1 was held constant while the distribution of reinforcements was varied between the two choices. Changes in response bias were a function of variations in the relative reinforcement ratio for the choice responses and not a function of variations in the probability of stimulus presentation. Discriminability remained constant across the three procedures.     

A method for the objective study of tool-using behavior

Key pecking for food was shaped in four crows within a conventional operant-conditioning test chamber. When pecking stabilized, a metal screen with openings 2.5 cm high by 1.0 cm wide, was placed over the response key, so that the crow could still see but could no longer peck the key. At the same time, several dozen wooden matchsticks, which could be used to operate the key, were placed in the test chamber. The crows made no use of these during 50 to 75 hr of exposure to this condition. Subsequently, the behavior of two crows was shaped so that they approached the matchsticks, picked one up in their beaks, approached the response key with the matchstick in their beak, and finally operated the response key by poking the matchstick through the screen. This shaping procedure was ineffective with the two other crows. However, these birds were successfully trained through positional fading of the tool. This involved suspending a metal rod from the ceiling so that it hung directly in front of the response key, and the crow had only to peck it to operate the key. Then, the rod was gradually lowered by lengthening its tether until it eventually rested on the floor of the test chamber. The principal advantage of this methodology is the automatic recording of the terminal (tool-using) behavior under study.     

The emergence of symmetry in a conditional discrimination task using different responses as propioceptive samples in pigeons

In Experiment 1, 10 pigeons were exposed to a successive symbolic matching-to-sample procedure in which the sample was generated by the pigeons' own behavior. Each trial began with both response keys illuminated white, one being the "correct" key and the other the "incorrect" key. The pigeons had no way of discriminating which key was correct and which incorrect, since these roles were assigned on a random basis with the same probability of 0.5 for each key. A fixed ratio of five responses was required on the correct key. However, each time the pigeon pecked the incorrect key, the correct key response counter reset. Five consecutive pecks on the correct key was the only way to end this component, and switch off both key lights. Two seconds later, these same keys were illuminated again, one green and the other red (comparison stimuli). Now, if the correct white key had been on the left, a peck at one color produced food, and if the correct white key had been on the right, a peck at the other color produced food. When the pigeons had learned this discrimination, they were exposed to several symmetry tests (simultaneous presentations of both keys illuminated the same color-i.e., both red or both green), in order to interchange the sample with the comparison stimuli. In Experiment 2, the importance of requiring discrimination between the samples and between the comparisons was analyzed. In Experiment 3, we compared the results of Experiment 1 with a slightly different experiment, which resulted in discrimination of key position, an exteroceptive stimulus. The results showed that symmetry emerged only when different responses were used as samples.     

Effects of barbiturates and other sedative hypnotics in pigeons trained to discriminate phencyclidine from saline.

Pigeons were trained to peck the center key (lighted white) of three response keys to turn off the center keylight and to light one of the side keys with a red keylight and the other side key with a green keylight. Five responses (fixed-ratio component) on either side key relighted the center key. Food was delivered following 10 fixed-ratio components on the red key if 1.5 mg/kg phencyclidine had been given before the session. The position of the red and green keylights on the side keys varied randomly each time they were lighted by a peck on the center key. Subsequently, increasing doses of phencyclidine, barbital, amobarbital, phenobarbital, methaqualone, methyprylon, diazepam, oxazepam, and d-amphetamine were substituted for the training dose of phencyclidine, using a cumulative dosing procedure. At low doses of the sedative hypnotics, birds pecked the keylight color associated with saline. At higher doses, birds pecked both key colors. At the highest doses of pentobarbital and amobarbital, some birds responded almost exclusively On he color associated with phencyclidine. When responding on keys of both colors occurred following administration of phencyclidine or other sedative hypnotics, this responding was controlled by key position rather than by key color.     

Responding under positive and negative response contingencies in pigeons and crows

Four crows were trained to key peck for food. Then, they were exposed to a positive response contingency that required them to peck the key when it was illuminated briefly (the trial) in order to receive food. This procedure resulted in consistent within-trial pecking. When the contingency changed so that food was presented at the end of a trial when no response occurred, but the trial terminated immediately and food was omitted when a response occurred (negative response contingency), responding decreased markedly. Eight pigeons were studied under the same change in contingencies. These birds varied in their response histories from naive to having several years' experience. The previously naive pigeons also showed rapid declines in responding under the negative contingency; the responding of the birds with extended training histories declined much more slowly. Eventually, however, six of the eight pigeons showed little or no responding under the negative contingency, while they responded consistently when re-exposed to the positive contingency. These findings question the power and the generality of the negative automaintenance phenomenon.     

Conjunctive schedules of reinforcement: I. Rate-dependent effects of pentobarbital and d-amphetamine

Key pecking of pigeons was maintained under conjunctive schedules of food presentation in which both a fixed-interval and a fixed-ratio schedule had to be completed before a peck produced food. For two pigeons, pecks on a single key completed both schedule requirements (fixed-interval 3-min, fixed-ratio 50 for one bird, fixed-interval 5-min, fixed-ratio 50 for the second). For two other pigeons, each requirement was scheduled on a separate key. On the two-key schedule, a peck after 5 min on the key scheduling the fixed-interval requirement produced food if at least 10 pecks had occurred on the ratio key (conjunctive fixed-interval 5-min, fixed-ratio 10). When each requirement was scheduled on a separate key, response rates on the fixed-ratio key were generally higher in the early portion of the interval and declined as the interval progressed; responding on the fixed-interval key, once initiated, typically remained at a constant rate throughout the interval. Responding under the single-key schedule was characterized by a high rate early in the interval; this then changed to a lower rate that continued until a peck produced food. For all pigeons, increases in response rates with pentobarbital and d-amphetamine were inversely related to the control rate of responding. When equivalent rates on each key of the two-key schedule were compared, both drugs increased rates on the fixed-ratio key less. Although the effects of both drugs were rate dependent, each drug differentially modified the pattern of responding under the single-key schedule.     

Conditioned reinforcement and choice with delayed and uncertain primary reinforcers.

In an adjusting-delay choice procedure, pigeons could peck on either a red key or a green key. A peck on the red key always led to a delay associated with red houselights and then food. The delay was adjusted over trials to estimate an indifference point--a delay at which the two keys were chosen about equally often. In some conditions, a peck on the green key led to food on all trials after delays of either 10 s or 30 s, and green houselights were lit during the delays. In other conditions, food was presented on only half of the green-key trials. If the green houselights continued to occur on both reinforcement and nonreinforcement trials, preference for the green key always decreased. Preference for the green key also decreased if half of the trials had 30-s houselights followed by food and the other half had no green houselights and no food. However, preference for the green key actually increased if half of the trials had 10-s green houselights followed by food and the other half had no green houselights followed by no food. The latter condition therefore demonstrated a case in which preference for an alternative increased when food was removed from half of the trials. The results suggest that the red and green houselights served as conditioned reinforcers. A hyperbolic decay model (Mazur, 1989) provided good predictions for all conditions by assuming that the strength of a conditioned reinforcer is inversely related to the total time spent in its presence before food is delivered.