Second-order autoshaped key pecking based on an auditory stimulus.

In Experiment 1, pigeons were exposed either to paired or to unpaired presentations of a tone and grain, and then to paired presentations of a keylight with the tone. Substantial second-order conditioned pecking to the keylight was produced in the birds that had received paired presentations of tone and grain. In Experiment 2, second-order pecking to the keylight increased in probability across four groups that had received, respectively, 20, 80, 140, or 200 paired presentations of tone and grain. In Experiment 3, the amount of pecking directed towards a keylight which predicted the first-order, tone CS was as substantial in birds without a prior history of key pecking as in birds with such a history. A further experiment failed to discover any significant differences in the levels of second-order pecking to a keylight paired with a first-order tone CS or with a first-order keylight CS. Thus, an auditory signal that does not itself support pecking may enable a localized visual stimulus to evoke key pecking.     

Topographical variations in behavior during autoshaping, automaintenance, and omission training

Three pigeons were exposed to an autoshaping and automaintenance procedure while a computer-controlled tracking system continuously recorded the position of the bird's head as it moved freely in the experimental chamber. Although only 2 birds pecked the key during the conditional stimulus (red keylight), all 3 birds exhibited stable patterns of approaching the conditional stimulus and withdrawing from the intertrial stimulus (white keylight). Subsequent exposure to an omission procedure, in which pecks on the red key cancelled the presentation of food upon the termination of the red keylight, greatly reduced key pecking, but approaching and pecking in the vicinity of the conditional stimulus were maintained at high levels. When the omission contingency was removed key pecking increased. During all phases the birds withdrew from the area of the white key and engaged in repetitive back-and-forth or circuiting movements during this intertrial stimulus. The data document (a) the strong control the conditional stimulus in autoshaping and automaintenance exerts over approach to the key and pecking motions whether or not the conditional stimulus elicits key pecking at a high level; and (b) withdrawal from the vicinity of the key and the occurrence of stereotypic behavior during the intertrial interval.     

Key pecking under response-independent food presentation after long simple and compound stimuli

Sixteen pigeons were trained to peck a key using a response-independent (auto-shaping) procedure of food presentation. The 4-sec grain presentations were independent of responding but a keylight stimulus preceded each, with a 4-min interval between the grain presentation and the next stimulus. Subjects were divided into four groups, with two durations of the keylight (30 or 120 sec) and either one or four successive colors on the response key preceding food delivery. In Phase 2, the birds were continued with the same keylight duration but were presented the alternative number of key colors. All pigeons pecked the key during the stimulus. Birds in the two groups with the 30-sec stimulus duration began to respond significantly sooner than birds with the 120-sec duration. There were no significant differences in rate of pecking between groups by the last five days of Phase 1. In Phase 1, the pigeons exposed to the four stimulus components showed an increase in rate of pecking over the four components as grain presentation approached. The pigeons with one stimulus component did not exhibit this regularity. Analogous conditions in Phase 2 had similar results except for one group. The implications of the occurrence of key pecking due to response-independent food delivery for multiple and chained schedules were pointed out.     

Key pecking in pigeons produced by pairing keylight with inaccessible grain

In Experiment I, keylight was paired with inaccessible grain delivery (under two conditions of keylight intensity) to determine if autoshaping would occur in the absence of primary reinforcement. In Experiment II, the procedure was repeated with accessible grain, for comparison. In Experiment III, the procedures were repeated with explicitly unpaired presentations of keylight and either inaccessible or accessible grain. The results indicated that key pecking occurred as quickly in the presence of keylight pairings with inaccessible grain as with accessible grain, though (except for one bird) key pecking was not maintained with inaccessible grain. Furthermore, compared to the dim keylight, the bright keylight greatly suppressed key pecking when paired with inaccessible grain, and reduced the rate of key pecking when paired with accessible grain. Little key pecking occurred in groups exposed to explicitly unpaired presentations of keylight (whether bright or dim) and grain (whether accessible or inaccessible). When the birds in Experiment III were retested with explicitly paired presentations of keylight and grain, little key pecking was observed, suggesting suppressive effects of prior explicitly unpaired presentations. It is suggested that the effects of key-brightness manipulation were produced by the association of grain with cues other than the response key, or by distraction produced by partial illumination of the grain hopper.     

Selective sensitivity of schedule-induced activity to an operant suppression contingency.

The sensitivity of pigeons' schedule-induced activity to operant consequences was studied in two experiments. During a 30-s interval between food presentations, a keylight stimulus brightened incrementally. Stable terminal key pecking and interim locomotor activity developed. An operant "setback" contingency was applied to activity. The contingency arranged for locomotor movements (detected by a nine-panel floorboard) to be followed by a resetting of the keylight brightness to a dimmer value and a 1-s delay of reinforcement (for individual responses). Experiment 1 showed that activity patterns were highly sensitive to their operant consequences. Accompanying key-peck rates were only transiently affected. In Experiment 2, the setback contingency was imposed during restricted portions of the trial, and differential operant control of activity was demonstrated. However, birds in this study produced higher rates of key pecking as activity rates were reduced. These results suggest that although schedule-induced activity arises in response to the temporal arrangement of stimulus events, this behavior may retain considerable sensitivity to response-consequence relations.     

A comparison of pecking generated by serial, delay, and trace autoshaping procedures

Pigeons were exposed to serial, delay, and trace autoshaping procedures. In Experiment I, all conditioned stimuli (CSs) were changes in illumination of the response key. The number of trials to acquisition of the keypeck increased from serial, to 4-sec delay, 8-sec delay, and 8-sec trace procedures, in that order. In Experiment II, which used a longer intertrial interval, trials to criterion increased from 8-sec delay, to 28-sec delay, 8-sec trace, and 28-sec trace procedures, in that order. In Experiment III, two groups received serial procedures in which the first CS was either a tone or a houselight, and the second was a keylight. The tone group acquired the key peck more rapidly than the houselight group. Early in conditioning in these experiments, and when the conditioned stimulus was a change in the keylight, there was a short latency to the onset of pecking and pecking was directed at the CS. After extensive conditioning, or when the CS was relatively diffuse, pecking still occurred, but had a longer latency and was not reliably directed toward the conditioned stimulus.     

Discriminated interresponse times: role of autoshaped responses.

When discriminated interresponse-time (IRT) procedures have been used to assess preference relations among temporally extended operants, deviations from matching have been obtained. Using a yoked-control procedure, the present study found that key pecking in a discriminated IRT procedure has two sources of strength--that arising from the response-reinforcer contingency that is explicitly arranged, and that arising from a stimulus-reinforcer contingency that is a by-product of the explicitly arranged contingency. The key pecking of all lead birds, and that of 3 of the 4 birds exposed to a yoked autoshaping procedure, was controlled by the keylight that signaled the lead birds' criterion IRTs. Because stimulus control of key pecking by the keylight, whether autoshaped or discriminative, fosters deviations from matching, the discriminated IRT procedure does not provide an appropriate basis for conclusions about preference relations among IRTs.     

Autoshaping the pigeon's gape response: acquisition and topography as a function of reinforcer type and magnitude.

The pigeon's key-pecking response is experimentally dissociable into transport (head movement) and gape (jaw movement) components. During conditioning of the key-pecking response, both components come under the control of the conditioned stimulus. To study the acquisition of gape conditioned responses and to clarify the contribution of unconditioned stimulus (reinforcer) variables to the form of the response, gape and key-contact responses were recorded during an autoshaping procedure and reinforcer properties were systematically varied. One group of 8 pigeons was food deprived and subgroups of 2 birds each were exposed to four different pellet sizes as reinforcers, each reinforcer signaled by a keylight conditioned stimulus. A second group was water deprived and received water reinforcers paired with the conditioned stimulus. Water- or food-deprived control groups received appropriate water or food reinforcers that were randomly delivered with respect to the keylight stimulus. Acquisition of the conditioned gape response frequently preceded key-contact responses, and gape conditioned responses were generally elicited at higher rates than were key contacts. The form of the conditioned gape was similar to, but not identical with, the form of the unconditioned gape. The gape component is a critical topographical feature of the conditioned key peck, a sensitive measure of conditioning during autoshaping, and an important source of the observed similarities in the form of conditioned and consummatory responses.     

Effects of d-amphetamine, diazepam, and pentobarbital on the schedule-controlled pecking and locomotor activity of pigeons

Pigeons were trained on a variant of the autoshaping procedure devised by Matthews and Lerer (1987) in which a keylight stimulus ramp of increasing brightness signaled the passing of a 30-s interfood interval. This procedure generates two distinct behavioral components: key pecking and locomotor activity. The effects of three psychoactive drugs on these behavior classes were measured. d-Amphetamine had negligible effects on both types of behavior, whereas diazepam and pentobartital increased key pecking and decreased activity in a dose-dependent fashion. In Experiment 2, the possibility that drug effects were suppressed by excessively strong stimulus control exerted in Experiment 1 was tested by decreasing the discriminability of the stimulus ramp. The direction of the effects of diazepam and pentobarbital was the same as in Experiment 1 but the magnitude of the effects tended to be larger. The effects of d-amphetamine, however, remained quite small, suggesting that, under these conditions, locomotor activity and key pecking are less sensitive to d-amphetamine. In Experiments 3 and 4, key pecking was eliminated by removing the keylight. Reinforcers were presented at fixed intervals in Experiment 3 and at variable intervals in Experiment 4. The drug effects on activity observed in Experiments 1 and 2 disappeared in both Experiments 3 and 4. The results suggest that diazepam and pentobarbital affect activity indirectly by increasing key-pecking behavior, which, in turn, competitively decreases activity.     

Incentive processes and the peak shift

Intradimensional operant discrimination schedules were employed, which eliminated the covariation of response and reinforcement rates that are found on most operant baselines. In Phase 1, one keylight (S(1)) controlled an increase in pigeons' treadle pressing, relative to another keylight (S(2)), while being correlated with a decrease in frequency of reinforcement. In Phase 2 both treadle pressing and reinforcement increased in the presence of one keylight, relative to the second. In Phase 1 the relatively flat treadle-press generalization gradients peaked at S(1), whereas the peaks of those in Phase 2 were shifted from S(1) in a direction away from S(2). It was postulated that these positive and negative stimulus-reinforcement contingencies influence the likelihood of obtaining peak shift through the operation of a classically conditioned "central motive state." How response-reinforcement and stimulus-reinforcement contingencies might contribute to the development of inhibitory effects of S(2) is discussed. Autoshaped key pecking also was produced by these procedures. During manipulations of stimuli, the gradients obtained for autoshaped key pecking were narrow and sharply peaked at the food-correlated stimulus (S(2)) in Phase 1. This failure to obtain peak shift for an elicited response suggests a difference in discriminative processes operating in classical and instrumental learning.     

Frequency of reinforcement as a determinant of extinction-induced aggression during errorless discrimination learning.

Seven pigeons were trained to discriminate without errors between a green keylight and a dark key. The key-pecking response was reinforced in the presence of green, and extinction was in effect in the presence of the dark key. The opportunity to attack a restrained target pigeon was present only during extinction. Both variable-interval 30-sec and fixed-ratio 1 schedules of reinforcement during the positive stimulus induced a higher rate of attack during extinction than a variable-interval 5-min schedule. The highest rate of attack during extinction occurred during the first 20 sec after the positive stimulus terminated. Hence, the withdrawal of the positive condition, rather than the consequences of the pecking response during extinction, appears to be one of the primary factors responsible for attack between pigeons during extinction. Behavioral contrast, defined as a decrease in the rate of responding when the positive stimulus was presented alone, was obtained from the four birds that displayed the lowest overall rates of attack while the three birds with the highest attack rates did not display behavioral contrast. For the birds without contrast, components of the attack response during the positive stimulus presumably competed with and reduced the rate of pecking the key, thereby recluding behavioral contrast.     

The maintenance of key pecking by stimulus-contingent and response-independent food presentation

Three naive pigeons were exposed to a series of two-component multiple schedules of response-independent food presentation. The component schedules were sometimes identical (non-differential procedures) and sometimes different (differential procedures). High rates of key pecking were maintained in all the differential procedures, and pecking decreased substantially in non-differential procedures, even when the frequency of food presentation in non-differential procedures was higher than in differential procedures. It is suggested that the high rates of key pecking were maintained not by adventitious response-reinforcer contingencies, but by differential contingencies between the stimulus (keylight) and food. The role of such contingencies in the phenomenon of behavioral contrast is discussed.     

Higher order autoshaping

A series of experiments is reported on appetitive higher order conditioning in the pigeon. Experiment I showed that second order autoshaping can be produced by pairing a neutral keylight with a keylight of another colour, previously paired with food. Experiment II employed an omission procedure to show that second order autoshaping is a consequence of the contingency between first and second order stimuli. In Experiment III, extinction of responding to the first order stimulus was shown to reduce responding to the second order stimulus. Experiments IV and V showed firstly that this reduction is not due to generalization of extinction, and secondly that second order key pecks may be produced in the absence of any pecking to the first order stimulus. The results suggest that second order autoshaping is based largely on a direct association between the first and second order stimuli.     

Mechanisms underlying the effects of unsignaled delayed reinforcement on key pecking of pigeons under variable-interval schedules.

Three experiments were conducted to test an interpretation of the response-rate-reducing effects of unsignaled nonresetting delays to reinforcement in pigeons. According to this interpretation, rates of key pecking decrease under these conditions because key pecks alternate with hopper-observing behavior. In Experiment 1, 4 pigeons pecked a food key that raised the hopper provided that pecks on a different variable-interval-schedule key met the requirements of a variable-interval 60-s schedule. The stimuli associated with the availability of the hopper (i.e., houselight and keylight off, food key illuminated, feedback following food-key pecks) were gradually removed across phases while the dependent relation between hopper availability and variable-interval-schedule key pecks was maintained. Rates of pecking the variable-interval-schedule key decreased to low levels and rates of food-key pecks increased when variable-interval-schedule key pecks did not produce hopper-correlated stimuli. In Experiment 2, pigeons initially pecked a single key under a variable-interval 60-s schedule. Then the dependent relation between hopper presentation and key pecks was eliminated by arranging a variable-time 60-s schedule. When rates of pecking had decreased to low levels, conditions were changed so that pecks during the final 5 s of each interval changed the keylight color from green to amber. When pecking produced these hopper-correlated stimuli, pecking occurred at high rates, despite the absence of a peck-food dependency. When peck-produced changes in keylight color were uncorrelated with food, rates of pecking fell to low levels. In Experiment 3, details (obtained delays, interresponse-time distributions, eating times) of the transition from high to low response rates produced by the introduction of a 3-s unsignaled delay were tracked from session to session in 3 pigeons that had been initially trained to peck under a conventional variable-interval 60-s schedule. Decreases in response rates soon after the transition to delayed reinforcement were accompanied by decreases in eating times and alterations in interresponse-time distributions. As response rates decreased and became stable, eating times increased and their variability decreased. These findings support an interpretation of the effects of delayed reinforcement that emphasizes the importance of hopper-observing behavior.     

The effects of reinforcement upon the prepecking behaviors of pigeons in the autoshaping experiment

The autoshaping procedure confounds the effects of pairing a keylight and food with the effect of adventitious food reinforcement of responses that typically occur before the pecking response. In Experiment I, acquisition of the orientation to the key, the approach toward the key, and the peck at the key were systematically monitored. Orientations to the key and approaches toward the key frequently occurred in contiguity with food presentation before peck acquisition. In Experiment II, a negative contingency procedure was used to assess the sensitivity of the approach toward the key to its consequences. When the approach toward the key resulted in nonreinforcement, the probability of occurrence of that response decreased to zero despite repeated light-food pairings. In Experiment III, peck probability was shown to be determined during the approach toward the key by the presence of stimuli that had previously been either paired or nonpaired with food. In Experiment IV, it was shown that the effects of the stimulus present during the approach toward the key were not due solely to the effects of pairing that stimulus with food. Autoshaped key pecking appears to be determined by the interacting effects of stimulus-reinforcer and response-reinforcer variables upon orientations to, approaches toward, and pecks at the lighted key.     

Automaintenance without stimulus-change reinforcement: Temporal control of key pecks

Yoked pairs of experimentally naive pigeons were exposed to a modified autoshaping procedure in which key pecking by the leader birds postponed both keylight termination and access to grain for the leader and the follower bird. Key pecking developed and was maintained in all birds and continued through two reversals of roles in the yoked procedure. Although temporal control developed more slowly in follower birds, asymptotic temporal distributions of key pecking were similar for all birds in both leader and follower roles; maximum responding occurred soon after keylight onset and decreased to a minimum prior to reinforcement. Response distributions for both leader and follower birds were described by Killeen's (1975) mathematical model of temporal control. Follower birds received response-independent reinforcement, and the development by these birds of temporal distributions which are minimal immediately prior to reinforcement is without precedent in Pavlovian appetitive conditioning. However, maintenance of key pecking by the leader birds, whose responses postponed both stimulus-change and food reinforcement, supports an interpretation of autoshaped and automaintained key pecking as responding elicited by signaled grain presentation.     

Control of behavior by an establishing stimulus

Seventeen pigeons were exposed to a three-key discrete-trial procedure in which a peck on the lit center key produced food if, and only if, the left keylight was lit. The center key was illuminated by a peck on the lit right key. Of interest was whether subjects pecked the right key before or after the response-independent onset of the left keylight. Pecks on the right key after left-keylight onset suggest control of behavior by the left keylight—an establishing stimulus. In three experiments, the strength of center-keylight onset as conditioned reinforcer for a response on the right key was manipulated by altering the size of the reduction in time to food delivery correlated with its onset. Control of pigeons' key pecks by onset of the left keylight occurred on more trials per session when the center keylight was a relatively weak conditioned reinforcer and on fewer trials per session when the center keylight was a relatively strong condtioned reinforcer. Differences across conditions in the degree of control by onset of the establishing stimulus were greatest when changes in conditioned reinforcer strength occurred relatively frequently and were signaled. The results provide evidence of the function of an establishing stimulus.     

Contributions of elicitation to measures of self-control

Pigeons' not pecking or pecking constituted choice between a delayed, large reinforcer and an immediate, small reinforcer (self-control) and at other times between a delayed reinforcer and no reinforcer (omission). Both a tone and a keylight were tested as choice signals, and the delayed reinforcer was either response independent or response dependent. Pigeons pecked during the choice signals on over 95% of the trials in the self-control procedure, and pecked during the choice signals on over 75% of the trials in the omission procedure. Consistent pecking was observed with either the tone or the keylight as a choice signal, with the exception that a tone paired with a response-independent delayed reinforcer did not maintain pecking in the omission procedure. Pigeons pecked during more choice signals when delayed reinforcers were response dependent than when the delayed reinforcers were response independent. These results indicate that Pavlovian conditioning influences self-control experiments, especially in single-key procedures.     

On conditioned reinforcing effects of negative discriminative stimuli

Observing responses by pigeons were studied during sessions in which a food key and an observing key were available continuously. A variable-interval schedule and extinction alternated randomly on the food key. In one condition, food-key pecking during extinction decreased reinforcement frequency during the next variable-interval component, and in the other condition such pecking did not affect reinforcement frequency. Observing responses either changed both keylight colors from white to green (S+) or to red (S−) depending on the condition on the food key, or the observing responses never produced the S+ but produced the S− when extinction was in effect on the food key. Observing responses that produced only S− were maintained only when food-key pecking during extinction decreased reinforcement frequency in the subsequent variable-interval component. The red light conformed to conventional definitions of a negative discriminative stimulus, rendering results counter to previous findings that production of S− alone does not maintain observing. Rather than offering support for an informational account of conditioned reinforcement, the results are discussed in terms of a molar analysis to account for how stimuli acquire response-maintaining properties.