Responding of pigeons under variable-interval schedules of unsignaled, briefly signaled, and completely signaled delays to reinforcement

In Experiment 1, three pigeons' key pecking was maintained under a variable-interval 60-s schedule of food reinforcement. A 1-s unsignaled nonresetting delay to reinforcement was then added. Rates decreased and stabilized at values below those observed under immediate-reinforcement conditions. A brief stimulus change (key lit red for 0.5 s) was then arranged to follow immediately the peck that began the delay. Response rates quickly returned to baseline levels. Subsequently, rates near baseline levels were maintained with briefly signaled delays of 3 and 9 s. When a 27-s briefly signaled delay was instituted, response rates decreased to low levels. In Experiment 2, four pigeons' responding was first maintained under a multiple variable-interval 60-s (green key) variable-interval 60-s (red key) schedule. Response rates in both components fell to low levels when a 3-s unsignaled delay was added. In the first component delays were then briefly signaled in the same manner as Experiment 1, and in the second component they were signaled with a change in key color that remained until food was delivered. Response rates increased to near baseline levels in both components, and remained near baseline when the delays in both components were lengthened to 9 s. When delays were lengthened to 27 s, response rates fell to low levels in the briefly signaled delay component for three of four pigeons while remaining at or near baseline in the completely signaled delay component. In Experiment 3, low response rates under a 9-s unsignaled delay to reinforcement (tandem variable-interval 60 s fixed-time 9 s) increased when the delay was briefly signaled. The role of the brief stimulus as conditioned reinforcement may be a function of its temporal relation to food, and thus may be related to the eliciting function of the stimulus.     

Effects of differences between stimuli, responses, and reinforcer rates on conditional discrimination performance

In a discrete-trial conditional discrimination procedure, 4 pigeons obtained food reinforcers by pecking a key with a short latency on trials signaled by one stimulus and by pecking the same key with a long latency on trials signaled by a second stimulus. The physical difference between the two stimuli and the temporal separation between the latency values required for reinforcement were varied factorially over four sets of conditions, and the ratio of reinforcer rates for short and long latencies was varied within each set of conditions. Stimulus discrimination varied directly with both stimulus and response differences and was unaffected by the reinforcer ratio. Sensitivity to reinforcement, estimated by generalized-matching-law fits to the data within each set of conditions, varied directly with the response difference but inversely with the stimulus difference arranged between sets of conditions. Because variations in stimulus differences, response differences, and reinforcer differences did not have equivalent effects, these findings question the functional equivalence of the three terms of the discriminated operant: antecedent stimuli, behavior, and consequences.     

Unsignalled delay of reinforcement in variable-interval schedules

Three pigeons responded on several tandem variable-interval fixed-time schedules in which the value of the fixed-time component was varied to assess the effects of different unsignalled delays of reinforcement. Actual (obtained) delays between the last key peck in an interval and reinforcement were consistently shorter than the nominal (programmed) delay. When nominal delays were relatively short, response rates were higher during the delay condition than during the corresponding nondelay condition. At longer nominal delay intervals, response rates decreased monotonically with increasing delays. The results were consistent with those obtained from delay-of-reinforcement procedures that impose either a stimulus change (signal) or a no-response requirement during the delay interval.     

Contiguity of briefly presented stimuli with food reinforcement

Pigeons performed on second-order schedules of reinforcement consisting of four fixed-interval components. Only the terminal component ended with food. Performance was studied both when a brief stimulus followed the completion of each of the first three fixed intervals (brief-stimulus schedule) and when the stimulus was omitted (tandem schedule). Variations in the temporal contiguity of the last presentation of the stimulus and the presentation of food indicated that the shorter the delay, the greater was the enhancement of rate of responding in comparison with tandem performance. A positively accelerated pattern of responding within fixed-interval components was a function of the contiguity of the brief stimulus and reinforcement; this pattern was absent for all tandem-schedule performance.     

Marking, memory and superstition in the pigeon

Learning is generally poor if reinforcement is delayed, but it improves substantially if a brief stimulus is presented immediately after the response to be learned. The marking hypothesis suggests that the unexpected stimulus triggers a backward memory search, which effectively marks the preceding response in memory, making it more likely that it will be recalled when food is presented. In the present study, pigeons were occasionally reinforced after a 10-sec delay for pecking a split key. Reinforcement was presented regardless of which side was pecked, but for one group a marker followed a peck to the left half of the key during the delay preceding food, and for the other group a peck to the right. On non food trials these contingencies were reversed. Subjects developed a significant preference for the side marked on food trials, despite the absence of any contingency between responding to this side and food. In addition to providing further support for the marking hypothesis, these results favour theories of reinforcement emphasizing contiguity rather than contingency. Contiguity, however, needs to be interpreted within a memory framework. What is crucial is the contiguity of events within working memory, rather than in the real world.     

Serial conditioning as a function of stimulus, response, and temporal dependencies

Six experiments were used to examine the effects of explicit response, stimulus, and temporal dependencies on responding in an interfood interval. The first two experiments demonstrated that 10-segment 60-s interfood clocks controlled similar distributions of key pecking in pigeons regardless of whether response–reinforcement contiguity was required, allowed, or precluded. The third and fourth experiments found that in the absence of an explicit response–reinforcement dependency, systematic explicit stimuli in an interfood interval were sufficient to establish and maintain the characteristic distribution of key pecking and that an interval without an explicit clock failed to establish or maintain key pecking. The last two experiments demonstrated that the interfood interval need not be of fixed length, and that a simple correlation of stimuli with increments from either a minimum to a maximum imminency or probability of food presentation controlled behavior in a similar manner. Successively higher rates generally occurred to successively later stimuli in the upper half of the range.     

Preference for signalled reinforcement

Key pecking was reinforced on a two-component multiple schedule. A variable-interval schedule controlled reinforcement in both components. During one component, access to reinforcement was preceded by a tone; in the other component, a standard unsignalled schedule was in effect. After performance stabilized, subjects were given a choice between the signalled and unsignalled schedules. They were placed in the chamber with the unsignalled schedule in effect on the right key. A single response on the left, or changeover, key produced the signalled schedule for 1 min. Both pigeons in Experiment I pecked the changeover key at a rate sufficient to remain under the signalled schedule for over 90% of the session. Removing and reintroducing the tone demonstrated that the changeover-key responses were due to the occurrence of the tone. In Experiment II, when pecking the changeover key produced the unsignalled schedule, pecking the changeover key declined. The results may be explained either in terms of Hendry's information hypothesis or as escape from an intermittent positive reinforcement schedule.     

Behavioral interactions and stimulus control during conditional discriminations

Two pigeons were exposed to factorial combinations of two values of line tilt and two frequencies of houselight flashes. During each of four baseline stages, key pecking in the presence of all four combinations was reinforced according to a variable-interval schedule. The baseline phases were followed by four different conditional discrimination training procedures in which reinforcement availability for pecking in the presence of the line tilts depended upon the houselight frequency. The subjects acquired each conditional discrimination. Behavioral contrast occurred during the acquisition and abolition of the discriminations. Generalization tests, given after each conditional discrimination, revealed that both the line tilt and houselight frequency dimensions controlled pecking only after conditional discriminations in which reinforcement availability depended upon the value of both dimensions.     

Food-avoidance in hungry pigeons, and other perplexities

Twenty-three pigeons were subjected to a series of procedures in which the key-peck's effects ranged from immediate, differential food reinforcement, through delayed reinforcement, the production of stimulus changes with and without probable secondary reinforcement, the prevention of food presentation ("food-avoidance"), to extinction. Neither primary nor secondary food reinforcement appeared to be essential for the maintenance or acquisition of key pecking. The food-avoidance contingency failed to suppress responding in any subject. Only complete extinction, when pecking produced neither food nor stimulus changes, eliminated all pecking for most subjects. A combination of stimulus-change reinforcement and food reinforcement appeared to account for the results, but only if it could be assumed that the presence of food in a procedure enhanced the reinforcing power of stimulus change, whether or not the food was also dependent upon responding. Such an interaction between reinforcers may be involved in the phenomenon of autoshaping.     

Pavlovian contingencies and resistance to change in a multiple schedule

According to theoretical accounts of behavioral momentum, the Pavlovian stimulus—reinforcer contingency determines resistance to change. To assess this prediction, 8 pigeons were exposed to an unsignaled delay-of-reinforcement schedule (a tandem variable-interval fixed-time schedule), a signaled delay-of-reinforcement schedule (a chain variable-interval fixed-time schedule), and an immediate, zero-delay schedule of reinforcement in a three-component multiple schedule. The unsignaled delay and signaled delay schedules employed equal fixed-time delays, with the only difference being a stimulus change in the signaled delay schedule. Overall rates of reinforcement were equated for the three schedules. The Pavlovian contingency was identical for the unsignaled and immediate schedules, and response—reinforcer contiguity was degraded for the unsignaled schedule. Results from two disruption procedures (prefeeding subjects prior to experimental sessions and adding a variable-time schedule to timeout periods separating baseline components) demonstrated that response—reinforcer contiguity does play a role in determining resistance to change. The results from the extinction manipulation were not as clear. Responding in the unsignaled delay component was consistently less resistant to change than was responding in both the immediate and presignaled segments of the signaled delay components, contrary to the view that Pavlovian contingencies determine resistance to change. Probe tests further supported the resistance-to-change results, indicating consistency between resistance to change and preference, both of which are putative measures of response strength.     

Rate and temporal pattern of key pecking under autoshaping and omission schedules of reinforcement

The role of response-reinforcer contiguity on autoshaped key pecking in pigeons was studied by scheduling response-dependent nonreinforcement at the beginning or the end of brief (8-sec) discrete trials. Schedules that permitted chance conjunctions of key pecking and food sustained high rates of responding, whereas those that prevented the occurrence of key peck-food intervals shorter than 4 sec sustained low response rates. In addition, selective reinforcement schedules supported accelerating or decelerating rates of responding within individual trials. These effects were traceable to response-reinforcer (operant), but not stimulus-reinforcer (respondent) factors.     

Time-place learning by pigeons, Columba livia

In each of two experiments, 2 pigeons received discrimination training in which food reinforcement for key pecking was conditional upon both spatial and temporal cues. In Experiment 1, food was available for periods of 30 s at each of three locations (pecking keys) during trials that lasted 90 s. In Experiment 2, food was available for periods of 15 min at each of four locations (pecking keys) during a 60-min trial. In both experiments, pigeons' key pecking was jointly controlled by the spatial and temporal cues. These data, and other recent experiments, suggest that animals learn relationships between temporal and spatial cues that predict stable patterns of food availability.     

Attention and generalization during a conditional discrimination

A conditional discrimination was established and analyzed, using four pigeons. The discrimination was among four compound stimuli projected on the response key—a white circle or triangle on a red or green background—during two conditions of illumination in the chamber, no illumination or flashing illumination. The two lighting conditions indicated whether the stimuli on the key containing triangles or those containing red would be the occasion for reinforcement. After the discrimination formed, generalization to intermediate and extreme values of the conditional stimulus and the attention of the birds to separate aspects of the stimulus on the key under each of the conditional stimuli were studied. All subjects generalized across values of the conditional stimulus, the lighting of the chamber. But subjects differed in the manner in which they treated the compound stimuli: two tended to attend to one or the other aspect of the stimulus on the key depending on the conditional stimulus, and two offered no evidence of such selective attention. Thus, the differential control of responding by the conditional stimuli cannot be attributed to a shift in attention between the figure and ground aspects of the compound stimuli.     

Inhibitory control and errorless discrimination learning

Pigeons learned to discriminate between a positive stimulus (white key) and a negative stimulus (red or green key, depending on the subject) via Terrace's fading procedure. Generalization tests, conducted with intermittent reinforcement for key pecking at various wavelengths, yielded minima at the value of the negative stimulus in most “errorless” birds. Terrace's contrary finding of flat gradients in errorless subjects probably resulted from a floor-effect (i.e., virtually zero responding) produced by his extinction-test procedure. The present and other findings do not support Terrace's conclusions that the negative stimulus of an errorless discrimination is behaviorally neutral; inhibition apparently develops to the nonreinforced stimulus even during errorless discrimination learning. A negative correlation between stimulus and reinforcer seems the crucial factor in producing an inhibitory stimulus.     

Acquisition and maintenance of autoshaped key pecking as a function of food stimulus and key stimulus similarity.

The results of a number of recent studies suggest that acquisitions of autoshaped key pecking in pigeons is affected by the similarity of the grain-hopper stimulus and response-key stimulus. In Experiment 1 this hypothesis was tested by training independent groups of pigeons to key peck under six different hopper-stimulus and key-stimulus similarity conditions, and three procedures containing either immediate reinforcement, variable delay of reinforcement, or omission of reinforcement for key pecking. Number of trials to acquisition was found to be related to the similarity variable. Maintained responding was affected by the response-reinforcer contingency. This effect was found both within and between subjects. Under two of the contingencies (automaintenance and omission), maintained responding continued to be affected by the similarity of the hopper stimulus and key stimulus. In Experiment 2 pigeons were given omission training with a hopper light on or off. Both acquisition and maintenance of key pecking were facilitated by the presence of the hopper light. The present findings suggest that much of the responding reported in various automatic shaping and training procedures may reflect the effects of key stimulus/food stimulus similarity.     

Periodic Reinforcement and Second-Order Schedules

In two experiments, rats were trained initially on a recycling conjunctive schedule in which a food pellet was delivered after 30 s provided at least one response had occurred; otherwise the next cycle began immediately. This produced low rate responding characterized by either a pause-respond-pause pattern or else a pause-respond pattern. The schedule then was changed so that half of the intervals ended with the presentation of a brief stimulus instead of food. Patterning after food was little affected, whereas patterning after the brief stimulus varied across rats. Generally, a short pause after the brief stimulus was followed by an initial increase in responding that led to either a fairly constant rate, or else to a decrease in rate throughout the interval. In later conditions, when the incidence of response-food and response- stimulus contiguity were manipulated separately, only the former increased response rate; this was so even when the brief stimulus was paired with food in some conditions. Rate increases were accompanied by changes in patterning across all intervals. These results do not support a simple conditioned reinforcement interpretation of the control acquired by a brief stimulus on a second-order schedule with fixed-interval components. Rather, they suggest that a number of interrelated variables combine to maintain responding. These variables include response-reinforcer contiguity, the temporal location of the response dependency, and the contingency between the brief stimulus and the reinforcer.

     

Stimulus control of respondent and operant key pecking: A single key procedure

Pigeons' responses to a uniformly illuminated response key were either reinforced on a variable-interval one-minute schedule of reinforcement or extinguished for one-minute periods. When 1.5 second signals were presented at the beginning of each component, so as to differentially predict reinforcement, the pigeons pecked at the signals, at rates higher than rates during the remainder of the component. When the brief signals were not differentially predictive of reinforcement, pecking in their presence decreased to near zero levels. Similar results were obtained with signals based upon colors and upon line orientations. Changes in rates of (unreinforced) pecking occurred during the signal whether pigeons responded differentially during the remainder of the component or not. Experiment II demonstrated that the presence of the signal correlated with extinction was not necessary for pecking to develop at the signal which preceded the component in which responding was intermittently reinforced. The experiments demonstrated a clear dissociation of respondent control from operant control of a response. In addition, operant behavior was shown to be relatively insensitive to differing rates of reinforcement, as compared to the sensitivity of respondent behavior to differing rates of reinforcement produced by the very same operant behavior.     

Parameters affecting the maintenance of negatively reinforced key pecking

Three negative reinforcement experiments employing a key-peck response are described. In Experiment I, pigeons shocked on the average of twice per minute (imposed condition) could produce, by pecking a key, an alternate condition with correlated stimuli. Delayed shocks were added, across sessions, to the alternate condition until pecking stopped. Two of three pigeons continued to peck despite a 100% increase in shock frequency. In Experiment II, pigeons were shocked in the imposed condition four times per minute. The postresponse delay to shock was held constant by delivering, in the alternate condition, the next shock, or the next two, three, or four shocks from the imposed-condition shock schedule. All three subjects continued to peck with no change in delay to the first two postresponse shocks but with a 75% reduction in shock frequency. In Experiment III, a response produced an immediate shock followed by a shock-free period. Three of four subjects continued to respond despite reduced delay to shock. Delay-to-shock or shock-frequency reduction was sufficient to maintain key pecking, but neither was necessary. The conditions that negatively reinforce the pigeon's key peck were similar to conditions that negatively reinforce the rat's bar press.     

Response stereotypy without automaticity in pigeons

Previous research has shown that when pigeons are required to peck each of two keys four times in any order for reinforcement, stereotyped response sequences develop though they are not demanded by the task. The present experiment explored whether the functional value of sequence stereotypy was that sequences became automatized, freeing cognitive resources for other activities. Pigeons were exposed to variants of the task requiring four pecks on each key that differed in the degree of task stringency. Concurrent with the sequence task, pigeons were required to process the color of the keys in order to complete successfully either a matching or a conditional discrimination task. The delay between sequence execution and matching or discrimination choice was varied between 0 and 10 sec. Matching and discrimination accuracy were decreasing functions of delay, but the stringency of the concurrent sequence task had no effect on choice accuracy, suggesting that stereotyped sequences were automatized. However, the matching and discrimination tasks produced substantial disruptions of sequence performance, suggesting that stereotyped sequences were not automatized. The data were taken to suggest that response stereotypy need not imply automaticity.     

Behavioral contrast produced by a signaled decrease in local rate of reinforcement

Pigeons' key pecking in the presence of one stimulus (S1) was reinforced according to a response-dependent variable-interval schedule. Pecking rate during S1 increased (behavioral contrast) when a second stimulus (S2) [associated with either a response-dependent fixed-interval schedule (Experiment I) or a response-independent reinforcement schedule in which reinforcement availability was signaled by visual (Experiment II) or temporal (Experiment III) stimuli] alternated with S1. These experiments suggest that a discriminable, signaled decrease in local reinforcement rate during S2 is an antecedent of the behavioral contrast response rate increases during S1.