Effect of varying the duration of grain presentation on automaintenance

In a series of three experiments the effects of variation in grain duration on automaintenance were evaluated. In the first experiment, key illumination was followed by grain only when pigeons did not peck the key. Each subject was exposed to 2-, 4-, and 8-second feeder durations in blocks of 10 sessions. Subjects pecked on a high percentage of trials at all feeder durations. The mean peck latency was shorter in the 8-second condition than in the two other conditions in five of six subjects. The conditional probability of pecking given successive keylight-grain pairings did not increase as the number of pairings increased. The second experiment was identical to the first, except that key pecking had no scheduled consequence. Under these conditions, all three subjects showed substantial responding. The recorded measures showed no systematic relationship to feeder duration in this study. In the third experiment, two different stimuli were followed by feeder presentations of either identical (2- or 8-second) or different (2- and 8-second) durations within each session. Subjects tended to respond sooner and with a higher overall rate in the presence of the stimulus associated with the longer feeder duration only when different feeder durations were presented within the same session. This result was confirmed by direct observation of the pigeons. The results of these experiments suggest that the effects of varying grain duration may be small, compared to the effects of varying other variables. The results also suggest that the location as well as the frequency of pecking may be an important measure in the analysis of factors controlling the pigeon's key peck.     

"Automaintenance": the role of reinforcement

A key was illuminated on the average of every 30 sec for a duration of 6 sec and this was followed by food presentations. When key pecks in the presence of the light produced immediate access to grain (autoshaping procedure) pigeons were likely to peck. When pecks terminated the keylight but prevented access to grain (automaintenance procedure) pigeons were much less likely to peck. Seven of 12 pigeons failed to develop responding during the automaintenance procedure. Four of the five pigeons that responded during the automaintenance procedure were exposed to a procedure in which responses could not immediately terminate the light. Three of the four ceased to respond during optimal automaintenance conditions, suggesting that the response-dependent offset of the keylight had been reinforcing their pecking. Responding during the automaintenance procedure was eliminated for a fifth pigeon by eliminating the contiguity of light-offset and food-onset on those trials in which the pigeon did not peck. These results suggest that: (1) automaintenance (unlike autoshaping) is not an effective procedure for reliably generating responding; (2) responding that does occur during the automaintenance procedure is reinforced by the response-dependent offset of the keylight.     

Acquisition of the autoshaped key peck as a function of amount of preliminary magazine training

Three experiments evaluated the effect of magazine training on acquisition of the pigeon's key peck during autoshaping. In Experiment I, pigeons were exposed to two days of extended magazine training, followed on the third day by keylight-only presentations. All pigeons pecked the keylight early in the keylight-only session. Experiment II examined the relationship between the number of magazine-training trials and trials to the first peck. Pigeons were given either 0, 3, 10, or 25 magazine-training trials followed by the standard autoshaping procedure. The number of trials to the first peck was related to the number of magazine-training trials. In Experiment III, pigeons were exposed to the standard autoshaping procedure without prior magazine training. The data from Experiment III suggested that key pecking will occur only after the response of eating from the lighted hopper has occurred. Taken together, these results suggest that initial magazine training is an important variable in autoshaping. Key pecking is discussed as a generalized consummatory response.     

Stimulus aspects of aversive controls: the effects of response contingent shock

A tone ending with electrical shock was periodically presented to pigeons while they pecked a key for food. Pairs of birds were run simultaneously under a yoked program which insured that both birds received the same number and temporal distribution of shocks. For one of the birds, shock was always initiated by a peck; for the other, shock was unavoidable. Both procedures led to reduced rates of pecking in the presence of the tone, and gradients of stimulus generalization were obtained. But the effects of response contingent shock extinguished more rapidly than the effects of unavoidable shock. In general, birds exposed to unavoidable shock tended to respond at intermediate rates throughout tone, whereas those exposed to response contingent shock ceased to peck for part or all of the tone period.     

Graded differential reinforcement: Response-dependent reinforcer amount

After key pecking had been autoshaped, six pigeons were exposed to a condition in which the duration of grain availability at the end of an 8-second trial depended on the number of responses emitted during the trial (0.25-second access to grain per response). This procedure, called correlated reinforcement, alternated across conditions with the automaintenance baseline in which the 8-second trial terminated with a constant 2.5-second access to grain. Two control procedures were run; in both, the reinforcer durations were yoked to those obtained in the last correlated session. In the yoked control no responses were required, but in the single-response yoked control at least one response was required to receive the yoked duration. The correlated condition maintained response rates above those produced by the two control conditions. These results may be accounted for by differential reinforcement.     

Key pecking under response-independent food presentation after long simple and compound stimuli

Sixteen pigeons were trained to peck a key using a response-independent (auto-shaping) procedure of food presentation. The 4-sec grain presentations were independent of responding but a keylight stimulus preceded each, with a 4-min interval between the grain presentation and the next stimulus. Subjects were divided into four groups, with two durations of the keylight (30 or 120 sec) and either one or four successive colors on the response key preceding food delivery. In Phase 2, the birds were continued with the same keylight duration but were presented the alternative number of key colors. All pigeons pecked the key during the stimulus. Birds in the two groups with the 30-sec stimulus duration began to respond significantly sooner than birds with the 120-sec duration. There were no significant differences in rate of pecking between groups by the last five days of Phase 1. In Phase 1, the pigeons exposed to the four stimulus components showed an increase in rate of pecking over the four components as grain presentation approached. The pigeons with one stimulus component did not exhibit this regularity. Analogous conditions in Phase 2 had similar results except for one group. The implications of the occurrence of key pecking due to response-independent food delivery for multiple and chained schedules were pointed out.     

Errorless discrimination established by differential autoshaping

In Experiment I, pigeons exposed to a differential autoshaping procedure pecked a key in the presence of the stimulus associated with reinforcement but did not peck, or pecked infrequently, in the presence of the stimulus associated with nonreinforcement. In Experiment II, pigeons were exposed to a differential autoshaping procedure in which one stimulus was associated with reinforcement and two stimuli were associated with nonreinforcement. The birds initially responded in the presence of one stimulus associated with nonreinforcement but never responded in the presence of the second stimulus associated with nonreinforcement. They were subsequently exposed to an autoshaping procedure in which reinforcement followed both these stimuli. The number of stimulus-reinforcement pairings required to establish pecking in the presence of the stimulus during which responses had not previously occurred suggested that such stimuli are inhibitory. These findings have implications for autoshaping, errorless discrimination, inhibition, and theories of discrimination byproducts.     

Failure to obtain positive contrast when pigeons press a bar

In Experiment 1, two groups of pigeons were exposed to multiple variable-interval 1-min variable-interval 1-min schedules of reinforcement. The required response was a key peck for one group, and a foot press of a bar for the other. When the procedure was shifted to multiple variable-interval 1-min extinction, positive contrast occurred with pigeons that key pecked, whereas negative induction occurred with those that bar pressed. Moreover, the absence of contrast could not be ascribed to the lack of inhibition, since negative generalization gradients after bar press training were U-shaped in Experiment 2. The results are discussed in terms of possible relationships between positive contrast and the phenomenon of autoshaping.     

Decision rules and signal detectability in a reinforcement-density discrimination

Two probabilistic schedules of reinforcement, one richer in reinforcement, the other leaner, were overlapping stimuli to be discriminated in a choice situation. One of two schedules was in effect for 12 seconds. Then, during a 6-second choice period, the first left-key peck was reinforced if the richer schedule had been in effect, and the first right-key peck was reinforced if the leaner schedule had been in effect. The two schedule stimuli may be viewed as two binomial distributions of the number of reinforcement opportunities. Each schedule yielded different frequencies of 16 substimuli. Each substimulus had a particular type of outcome pattern for the 12 seconds during which a schedule was in effect, and consisted of four consecutive light-cued 3-second T-cycles, each having 0 or 1 reinforced center-key pecks. Substimuli therefore contained 0 to 4 reinforcers. On any 3-second cycle, the first center-key peck darkened that key and was reinforced with probability .75 or .25 in the richer or leaner schedules, respectively. In terms of the theory of signal detection, detectability neared the maximum possible d′ for all four pigeons. Left-key peck probability increased when number of reinforcers in a substimulus increased, when these occurred closer to choice, or when pellets were larger for correct left-key pecks than for correct right-key pecks. Averaged over different temporal patterns of reinforcement in a substimulus, substimuli with the same number of reinforcers produced choice probabilities that matched relative expected payoff rather than maximized one alternative.     

Effects of search cost on foraging and feeding: a three-component chain analysis

An experiment determined whether pigeons minimize number of key pecks per food delivery and maintain their baseline intake of food while key pecking on a three-component chain schedule. Pigeons at either 80% or 100% body weight obtained all their food during baseline and contingency sessions. During baseline sessions, pecks on the left and center keys had no consequences; each peck on the right key activated the feeder. During contingency sessions, pigeons key pecked on a three-component chain schedule simulating components of a foraging chain. In the search component either 3, 9 or 15 key pecks (varied parametrically across blocks of sessions) on the left key produced a stimulus on the middle key, indicating an encounter with either the low-cost prey (3 key pecks) or an equally probable high-cost prey (21 key pecks). In the procurement component the pigeon pecked either: (a) the left key once, thus returning to the search component, or (b) the middle key either 3 or 21 times, which activated the right response key. In the handling component one peck on the right key operated the feeder. The pigeons always procured the low-cost prey and minimized the number of key pecks per hopper by procuring the high-cost prey when the search-cost ratio was high (15 key pecks) but not when it was low (3 key pecks). All pigeons maintained their baselines of eating during contingency sessions by key pecking more frequently and eating more efficiently. The 80% body-weight birds produced higher overall rates of key pecking and eating. These results have implications for ecological theories of optimal foraging and for psychological theories of learned performance.     

Mechanisms underlying the effects of unsignaled delayed reinforcement on key pecking of pigeons under variable-interval schedules.

Three experiments were conducted to test an interpretation of the response-rate-reducing effects of unsignaled nonresetting delays to reinforcement in pigeons. According to this interpretation, rates of key pecking decrease under these conditions because key pecks alternate with hopper-observing behavior. In Experiment 1, 4 pigeons pecked a food key that raised the hopper provided that pecks on a different variable-interval-schedule key met the requirements of a variable-interval 60-s schedule. The stimuli associated with the availability of the hopper (i.e., houselight and keylight off, food key illuminated, feedback following food-key pecks) were gradually removed across phases while the dependent relation between hopper availability and variable-interval-schedule key pecks was maintained. Rates of pecking the variable-interval-schedule key decreased to low levels and rates of food-key pecks increased when variable-interval-schedule key pecks did not produce hopper-correlated stimuli. In Experiment 2, pigeons initially pecked a single key under a variable-interval 60-s schedule. Then the dependent relation between hopper presentation and key pecks was eliminated by arranging a variable-time 60-s schedule. When rates of pecking had decreased to low levels, conditions were changed so that pecks during the final 5 s of each interval changed the keylight color from green to amber. When pecking produced these hopper-correlated stimuli, pecking occurred at high rates, despite the absence of a peck-food dependency. When peck-produced changes in keylight color were uncorrelated with food, rates of pecking fell to low levels. In Experiment 3, details (obtained delays, interresponse-time distributions, eating times) of the transition from high to low response rates produced by the introduction of a 3-s unsignaled delay were tracked from session to session in 3 pigeons that had been initially trained to peck under a conventional variable-interval 60-s schedule. Decreases in response rates soon after the transition to delayed reinforcement were accompanied by decreases in eating times and alterations in interresponse-time distributions. As response rates decreased and became stable, eating times increased and their variability decreased. These findings support an interpretation of the effects of delayed reinforcement that emphasizes the importance of hopper-observing behavior.     

Contributions of elicitation to measures of self-control

Pigeons' not pecking or pecking constituted choice between a delayed, large reinforcer and an immediate, small reinforcer (self-control) and at other times between a delayed reinforcer and no reinforcer (omission). Both a tone and a keylight were tested as choice signals, and the delayed reinforcer was either response independent or response dependent. Pigeons pecked during the choice signals on over 95% of the trials in the self-control procedure, and pecked during the choice signals on over 75% of the trials in the omission procedure. Consistent pecking was observed with either the tone or the keylight as a choice signal, with the exception that a tone paired with a response-independent delayed reinforcer did not maintain pecking in the omission procedure. Pigeons pecked during more choice signals when delayed reinforcers were response dependent than when the delayed reinforcers were response independent. These results indicate that Pavlovian conditioning influences self-control experiments, especially in single-key procedures.     

Topographical variations in behavior during autoshaping, automaintenance, and omission training

Three pigeons were exposed to an autoshaping and automaintenance procedure while a computer-controlled tracking system continuously recorded the position of the bird's head as it moved freely in the experimental chamber. Although only 2 birds pecked the key during the conditional stimulus (red keylight), all 3 birds exhibited stable patterns of approaching the conditional stimulus and withdrawing from the intertrial stimulus (white keylight). Subsequent exposure to an omission procedure, in which pecks on the red key cancelled the presentation of food upon the termination of the red keylight, greatly reduced key pecking, but approaching and pecking in the vicinity of the conditional stimulus were maintained at high levels. When the omission contingency was removed key pecking increased. During all phases the birds withdrew from the area of the white key and engaged in repetitive back-and-forth or circuiting movements during this intertrial stimulus. The data document (a) the strong control the conditional stimulus in autoshaping and automaintenance exerts over approach to the key and pecking motions whether or not the conditional stimulus elicits key pecking at a high level; and (b) withdrawal from the vicinity of the key and the occurrence of stereotypic behavior during the intertrial interval.     

Controls for and constraints on auto-shaping

Auto-shaping the pigeon's key-peck response was examined as a respondent conditioning procedure with the use of Rescorla's truly-random control procedure. In the first experiment, pigeons received presentations of brief light on the response key and brief presentations of food where the light and the food were independently presented. All birds failed to key peck after many light and food presentations, but explicit pairing of the light and food rapidly conditioned pecking to the light. Experiment 2 showed that even when an independent light/food presentation schedule was reduced to variable-time 30 sec, additional naive birds would not key peck and only one bird pecked when the schedules were variable-time 15 sec. A third experiment examined an explicit-unpairing control procedure, where the light and food were not only presented on independent schedules but were also separated by a minimum time, and found that auto-shaping did not occur. A fourth experiment investigated a number of control procedures and found them ineffective. A fifth experiment investigated the effects of a physical separation of the locus of the response key and the food dispenser, and a sixth experiment investigated using a tone in place of the light. It was concluded that pecking is generated by auto-shaping procedures only when an intermittently presented keylight is regularly paired with food.     

Marking, memory and superstition in the pigeon

Learning is generally poor if reinforcement is delayed, but it improves substantially if a brief stimulus is presented immediately after the response to be learned. The marking hypothesis suggests that the unexpected stimulus triggers a backward memory search, which effectively marks the preceding response in memory, making it more likely that it will be recalled when food is presented. In the present study, pigeons were occasionally reinforced after a 10-sec delay for pecking a split key. Reinforcement was presented regardless of which side was pecked, but for one group a marker followed a peck to the left half of the key during the delay preceding food, and for the other group a peck to the right. On non food trials these contingencies were reversed. Subjects developed a significant preference for the side marked on food trials, despite the absence of any contingency between responding to this side and food. In addition to providing further support for the marking hypothesis, these results favour theories of reinforcement emphasizing contiguity rather than contingency. Contiguity, however, needs to be interpreted within a memory framework. What is crucial is the contiguity of events within working memory, rather than in the real world.     

The influence of prior choices on current choice

Three pigeons chose between random-interval (RI) and tandem, continuous-reinforcement, fixed-interval (crf-FI) reinforcement schedules by pecking either of two keys. As long as a pigeon pecked on the RI key, both keys remained available. If a pigeon pecked on the crf-FI key, then the RI key became unavailable and the crf-FI timer began to time out. With this procedure, once the RI key was initially pecked, the prospective value of both alternatives remained constant regardless of time spent pecking on the RI key without reinforcement (RI waiting time). Despite this constancy, the rate at which pigeons switched from the RI to the crf-FI decreased sharply as RI waiting time increased. That is, prior choices influenced current choice-an exercise effect. It is argued that such influence (independent of reinforcement contingencies) may serve as a sunk-cost commitment device in self-control situations. In a second experiment, extinction was programmed if RI waiting time exceeded a certain value. Rate of switching to the crf-FI first decreased and then increased as the extinction point approached, showing sensitivity to both prior choices and reinforcement contingencies. In a third experiment, crf-FI availability was limited to a brief window during the RI waiting time. When constrained in this way, switching occurred at a high rate regardless of when, during the RI waiting time, the crf-FI became available.     

Transfer of oddity-from-sample performance in pigeons

Four pigeons were trained on a modified three-key oddity-from-sample task in which an observing response to the sample (center-key) stimulus lighted a single comparison (side-key) stimulus. If the comparison stimulus was different from the sample stimulus, a single peck to the lighted comparison was reinforced. If the comparison and sample stimuli were identical, the pigeons had to refrain from pecking the comparison for 4.6 seconds to terminate the matching comparison and to produce immediately a nonmatching comparison on the remaining side key. Each peck to the matching comparison reset the 4.6-second delay interval. Three hues were used during acquisition. During tests for transfer of the oddity performance, two novel hues were substituted either individually or together for one or two of the original training hues. For three birds, latencies to novel nonmatching hues were identical to baseline nonmatching latencies. Latencies to novel matching hues were shorter than baseline matching latencies but were consistently longer than novel nonmatching latencies. These transfer data demonstrate that the pigeons learned the oddity concept.     

Episodic-like memory in pigeons

It has been proposed that memory for personal experiences (episodic memory, rather than semantic memory) relies on the conscious review of past experience and thus is unique to humans. In an attempt to demonstrate episodic-like memory in animals, we first trained pigeons to respond to the (nonverbal) question "Did you just peck or did you just refrain from pecking?" by training them on a symbolic matching task with differential responding required to the two line-orientation samples and reinforcing the choice of a red comparison if they had pecked and the choice of a green comparison if they had not pecked. Then, in Experiment 1, after providing the conditions for (but not requiring) the pigeons to peck at one new stimulus (a yellow hue) but not at another (a blue hue), we tested them with the new hue stimuli and the red and green comparisons. In Experiment 2, we tested the pigeons with novel stimuli (a circle, which they spontaneously pecked, and a dark response key, which they did not peck) and the red and green comparisons. In both experiments, pigeons chose the comparison appropriate to the response made to the test stimulus. Thus, the pigeons demonstrated that they could remember specific details about their past experiences, a result consistent with the notion that they have the capacity for forming episodic-like memories.     

Economic and biological influences on a pigeon's key peck

Pigeons were studied in a two-component multiple schedule. In the first phase of the experiment, key pecks were reinforced on a variable-interval 2-min schedule in both components and free food was delivered additionally during one component. When components alternated every 8 sec, all pigeons pecked at a much higher rate during the component with free food than during the other component. At a component duration of 16 min, the reverse was true: all pigeons pecked at a higher rate during the component without free food. In the second phase, the additional food during one component was made contingent on pecking. Responding during the component without the extra food remained essentially unchanged, as expected, since rate of reinforcement remained identical to that in the previous phase. However, rate of responding during the component with the extra food (now contingent on pecking) was elevated, compared to the rate in the first phase, and did not show the marked decline as component duration was increased.