Acquisition and maintenance of autoshaped key pecking as a function of food stimulus and key stimulus similarity.

The results of a number of recent studies suggest that acquisitions of autoshaped key pecking in pigeons is affected by the similarity of the grain-hopper stimulus and response-key stimulus. In Experiment 1 this hypothesis was tested by training independent groups of pigeons to key peck under six different hopper-stimulus and key-stimulus similarity conditions, and three procedures containing either immediate reinforcement, variable delay of reinforcement, or omission of reinforcement for key pecking. Number of trials to acquisition was found to be related to the similarity variable. Maintained responding was affected by the response-reinforcer contingency. This effect was found both within and between subjects. Under two of the contingencies (automaintenance and omission), maintained responding continued to be affected by the similarity of the hopper stimulus and key stimulus. In Experiment 2 pigeons were given omission training with a hopper light on or off. Both acquisition and maintenance of key pecking were facilitated by the presence of the hopper light. The present findings suggest that much of the responding reported in various automatic shaping and training procedures may reflect the effects of key stimulus/food stimulus similarity.     

A test of the reinforcing properties of stimuli correlated with nonreinforcement

The information hypothesis of conditioned reinforcement predicts that a stimulus that “reduces uncertainty” about the outcome of a trial will acquire reinforcing properties, even when the stimulus reliably predicts nonreinforcement. Four pigeons' key pecks produced one of two 5-sec stimuli with 0.50 probability according to a discriminated variable-interval schedule. One stimulus was followed by reinforcement; a second stimulus was followed by blackout. To the same extent, therefore, both stimuli reduced uncertainty about the possibility that food would arrive at the termination of the schedule interval. When a second key in the chamber was lighted, each peck on it could produce the stimulus preceding reinforcement, the stimulus preceding nonreinforcement, a novel stimulus, or no stimulus, across separate conditions. The stimulus preceding food maintained responding at substantial levels on the second, stimulus-producing, key. Such responding was not maintained by other stimuli. These data, replicated when the stimuli were reversed on the variable-interval schedule, do not support the prediction that uncertainty-reducing stimuli are necessarily conditioned reinforcers.     

Stimulus control of responding by response-reinforcer temporal contiguity

The stimulus properties of brief disruptions in response-reinforcer temporal contiguity were investigated using a discrete trial conditional discrimination procedure. Key pecking (nondelay) or key pecking followed by a brief interval of nonpecking (delay) in the sample component produced a stimulus change (choice component). Pecks in the choice component to one of two alternatives resulted in food or blackout, conditional upon which response requirement was met in producing the choice component. A baseline condition, in which key pecking always produced the choice component and correct choices were arranged randomly, alternated with experimental conditions that included nondelay and delay values of either 0.2, 0.5, or 1.0 sec between the last key peck and the initiation of the choice component. All subjects accurately discriminated brief temporal delays between a response and stimulus change, with choice accuracy increasing for three of four subjects as the temporal disruption in contiguity increase. Implications of the research for the study of delayed reinforcement, response-independent reinforcement, and the discrimination of causality are discussed.     

Choice with certain and uncertain reinforcers in an adjusting-delay procedure.

A discrete-trials adjusting-delay procedure was used to investigate the conditions under which pigeons might show a preference for partial reinforcement over 100% reinforcement, an effect reported in a number of previous experiments. A peck on a red key always led to a delay with red houselights and then food. In each condition, the duration of the red-houselight delay was adjusted to estimate an indifference point. In 100% reinforcement conditions, a peck on a green key always led to a delay with green houselights and then food. In partial-reinforcement conditions, a peck on the green key led either to the green houselights and food or to white houselights and no food. In some phases of the experiment, statistically significant preference for partial reinforcement over 100% reinforcement was found, but this effect was observed in only about half of the pigeons. The effect was largely eliminated when variability in the delay stimulus colors was equated for 50% reinforcement conditions and 100% reinforcement conditions. Idiosyncratic preferences for certain colors or for stimulus variability may be at least partially responsible for the effect.     

Conditioned reinforcement and choice with delayed and uncertain primary reinforcers.

In an adjusting-delay choice procedure, pigeons could peck on either a red key or a green key. A peck on the red key always led to a delay associated with red houselights and then food. The delay was adjusted over trials to estimate an indifference point--a delay at which the two keys were chosen about equally often. In some conditions, a peck on the green key led to food on all trials after delays of either 10 s or 30 s, and green houselights were lit during the delays. In other conditions, food was presented on only half of the green-key trials. If the green houselights continued to occur on both reinforcement and nonreinforcement trials, preference for the green key always decreased. Preference for the green key also decreased if half of the trials had 30-s houselights followed by food and the other half had no green houselights and no food. However, preference for the green key actually increased if half of the trials had 10-s green houselights followed by food and the other half had no green houselights followed by no food. The latter condition therefore demonstrated a case in which preference for an alternative increased when food was removed from half of the trials. The results suggest that the red and green houselights served as conditioned reinforcers. A hyperbolic decay model (Mazur, 1989) provided good predictions for all conditions by assuming that the strength of a conditioned reinforcer is inversely related to the total time spent in its presence before food is delivered.     

Aversive aspects of a schedule of positive reinforcement

Six male White Carneaux pigeons were trained to peck at one of two keys to obtain food on several fixed-ratio schedules of reinforcement. Concurrently, the first response on a second key could, I—change the conditions of visual stimulation and remove the food reinforcement contingency, II—change the conditions of stimulation and have no effect upon the reinforcement contingency, or III—do nothing. The second response on the stimulus change key always restored baseline conditions. When second-key responses produced a stimulus change, the number of such responses was a function of the ratio value on the first key. Typically, second-key responses occurred before the start of fixed-ratio runs. The duration of stimulus change periods was an exponential function of the number of responses required for reinforcement when the possibility for reinforcement was not disturbed by periods of stimulus change (Condition II).     

"Turning back the clock" on serial-stimulus sign tracking.

Two experiments examined the effects of a negative (setback) response contingency on key pecking engendered by a changing light-intensity stimulus clock (ramp stimulus) signaling fixed-time 30-s food deliveries. The response contingency specified that responses would immediately decrease the light-intensity value, and, because food was delivered only after the highest intensity value was presented, would delay food delivery by 1 s for each response. The first experiment examined the acquisition and maintenance of responding for a group trained with the contingency in effect and for a group trained on a response-independent schedule with the ramp stimulus prior to introduction of the contingency. The first group acquired low rates of key pecking, and, after considerable exposure to the contingency, those rates were reduced to low levels. The rates of responding for the second group were reduced very rapidly (within four to five trials) after introduction of the setback contingency. For both groups, rates of responding increased for all but 1 bird when the contingency was removed. A second experiment compared the separate effects of each part of the response contingency. One group was exposed only to the stimulus setback (stimulus only), and a second group was exposed only to the delay of the reinforcer (delay only). The stimulus-only group's rates of responding were immediately reduced to moderate levels, but for most of the birds, these rates recovered quickly when the contingency was removed. The delay-only groups's rates decreased after several trials, to very low levels, and recovery of responding took several sessions once the contingency was removed. The results suggest that (a) sign-tracking behavior elicited by an added clock stimulus may be reduced rapidly and persistently when a setback contingency is imposed, and (b) the success of the contingency is due both to response-dependent stimulus change and response-dependent alterations in the frequency of food delivery. The operation of the contingency is compared with the effects of secondary reinforcement and punishment procedures.     

Interresponse-time shaping by variable-interval-like interresponse-time reinforcement contingencies

The interresponse-time reinforcement contingencies and distributions of interreinforcement intervals characteristic of certain variable-interval schedules were mimicked by reinforcing each key peck with a probability equal to the duration of the interresponse time it terminated, divided by the scheduled mean interreinforcement interval. The interresponse-time reinforcement contingency was then eliminated by basing the probability of reinforcement on the fifth interresponse time preceding the key peck. Even though distributions of interreinforcement intervals were unaffected by this manipulation, response rates consistently increased. A second experiment replicated this effect and showed it to combine additively with that of mean reinforcement rate. These results provide strong support for the contention that current analyses of variable-interval response rates that ignore the inherent interresponse-time reinforcement contingency may be seriously in error.     

Redundant information in an observing-response procedure

In three observing-response experiments relevant to the information hypothesis of conditioned reinforcement, the basic procedure was one in which an observing response produced one stimulus on trials that terminated in non-contingent reinforcement and another stimulus on trials that terminated in a brief timeout. In Experiment I, the observing response consisted of a single peck or a short fixed-ratio schedule (FR 3 or FR 6), depending on the type of trial. If the single peck produced the negative stimulus and the fixed ratio produced the positive stimulus, observing responses were maintained. If the single peck produced the positive stimulus and the fixed-ratio produced the negative stimulus, observing responses were not maintained on negative trials. In the second experiment, the response key was either white or dark at the beginning of a trial, indicating whether it was a positive or negative trial. Observing responses continued to be maintained on positive trials but not on negative trials. In Experiment III, only positive or negative trials were scheduled for several sessions. Observing responses extinguished regardless of whether positive or negative trials were scheduled. The results do not support the hypothesis that making the stimuli produced by observing responses redundant will reduce observing responses.     

Stimulus control and the response-reinforcement contingency

Pigeons were trained under a schedule in which reinforcement was made available at varying periods of time after a prior reinforcement. The first key peck after a reinforcer was available began a timer and a second key peck, which exceeded a specified minimal time interval, produced the reinforcer. It was shown that a contingency which contains a minimal interresponse time does not necessarily weaken stimulus control by an exteroceptive stimulus.     

The effects of reinforcement upon the prepecking behaviors of pigeons in the autoshaping experiment

The autoshaping procedure confounds the effects of pairing a keylight and food with the effect of adventitious food reinforcement of responses that typically occur before the pecking response. In Experiment I, acquisition of the orientation to the key, the approach toward the key, and the peck at the key were systematically monitored. Orientations to the key and approaches toward the key frequently occurred in contiguity with food presentation before peck acquisition. In Experiment II, a negative contingency procedure was used to assess the sensitivity of the approach toward the key to its consequences. When the approach toward the key resulted in nonreinforcement, the probability of occurrence of that response decreased to zero despite repeated light-food pairings. In Experiment III, peck probability was shown to be determined during the approach toward the key by the presence of stimuli that had previously been either paired or nonpaired with food. In Experiment IV, it was shown that the effects of the stimulus present during the approach toward the key were not due solely to the effects of pairing that stimulus with food. Autoshaped key pecking appears to be determined by the interacting effects of stimulus-reinforcer and response-reinforcer variables upon orientations to, approaches toward, and pecks at the lighted key.     

TEMPORAL CONTROL ON INTERVAL SCHEDULES: WHAT DETERMINES THE POSTREINFORCEMENT PAUSE?

On fixed-interval or response-initiated delay schedules of reinforcement, the average pause following food presentation is proportional to the interfood interval. Moreover, when a number of intervals of different durations occur in a programmed cyclic series, postreinforcement pauses track the changes in interval value. What controls the duration of postreinforcement pauses under these conditions? Staddon, Wynne, and Higa (1991), in their linear waiting model, propose control by the preceding interfood interval. Another possibility is that delay to reinforcement, signaled by a key peck and/or stimulus change, determines the subsequent pause. The experiments reported here examined the role of these two possible time markers by studying the performance of pigeons under a chained cyclic fixed-interval procedure. The data support the linear waiting model, but suggest that more than the immediately preceding interfood interval plays a role in temporal control.     

Effects of alternative reinforcement sources: A reevaluation

The effects of two alternative sources of food delivery on the key-peck responding of pigeons were examined. Pecking was maintained by a variable-interval 3-min schedule. In the presence of this schedule in different conditions, either a variable-time 3-min schedule delivering food independently of responding or an equivalent schedule that required a minimum 2-s pause between a key peck and food delivery (a differential-reinforcement-of-other-behavior schedule) was added. The differential-reinforcement-of-other-behavior schedule reduced response rates more than did the variable-time schedule in most instances. The delay between a key peck and the next reinforcer consistently was longer under the differential-reinforcement-of-other-behavior schedule than under the variable-time schedule. Response rates and median delay between responses and reinforcers were negatively correlated. These results contradict earlier conclusions about the behavioral effects of alternative reinforcement. They suggest that an interpretation in terms of response–reinforcer contiguity is consistent with the data.     

Short-term remembering of discriminative stimuli in pigeons.

Pigeons learned to peck the left or right of two white keys depending on whether a red or a green stimulus was displayed on a third key. The opportunity to peck the white keys was then dealyed for zero to six seconds after the red or green (to-be-remembered) stimulus. On half the trials, the feeder operated during the delay to interrupt behavior that might mediate discriminated responding. No events were scheduled on the remaining trials. In a later condition, the pigeons had the opportunity to peck the white keys during the delay. In general, accuracy decreased as delay increased in all conditions, but performance was least accurate following feeder operations and most accurate when pecking was allowed during the delay. The procedures may be analogous to varying the opportunity for rehearsal in studies of human short-term memory.     

Pigeons' spatial memory: factors affecting delayed matching of key location.

The delayed-matching-to-sample procedure was modified to study pigeons' spatial memory. Nine pecking keys, arranged as a three-by-three matrix, served as the spatial cues. Trials began with a brief "ready" stimulus (dimming of the houselight). Then a randomly chosen key was lit briefly as a sample. After a short delay the sample key was lit again along with one of the other eight keys. A peck at the key that had served as the sample produced grain reinforcement, where as a peck to the other key produced only the intertrial interval. After delayed matching of key location was learned, the effects of sample and delay duration, number of keys illuminated as sample and comparisons, and organization of three-key samples were studied. Matching accuracy decreased as sample duration decreased, delay increased, the number of locations serving as samples increased, the number and proximity of comparisons increased, and when the three-key samples were "discontinuous" rather than "lines".     

Conditioning of within-trial patterns of key pecking in pigeons

The possibility of conditioning systematic patterns of responding during brief discrete trials was studied by requiring hungry pigeons to key peck and then pause or to pause and then key peck in order to gain access to food. These schedules were highly effective in promoting decelerated and accelerated rates of responding, respectively, within individual trials; indeed, performance was quite similar to that observed when explicit external stimuli were correlated with “peck” and “pause” portions of the daily trials. Finally, schedules of reinforcement that did not selectively reinforce peck-pause or pause-peck patterns neither generated these patterns nor maintained them at the previous high levels. The results, therefore, confirm Shimp's (1976) proposal that organized groupings of discrete responses may function as operants—even in the absence of strict response-reinforcer contiguity.     

Conditioned reinforcement and discrimination performance

Pigeons were trained in a three-key chamber to peck one side key in the presence of a vertical line on the center key and to peck the other side key in the presence of a horizontal line. Correct choice responses were reinforced with food according to fixed- and variable-ratio, fixed-interval, and differential-reinforcement-of-long-latency schedules of reinforcement. For each schedule, the birds performed under each of two conditions: (1) each correct choice response produced a brief presentation of stimuli intermittently paired with food, then the next trial; (2) each correct choice response produced an intertrial interval only. For all schedules except one long latency schedule, response rates were higher under the condition of brief stimulus presentation than under the comparable control condition. Presentation of brief magazine stimuli increased choice accuracy. The amount of change in accuracy was correlated with the rate of food presentation. Performance under the schedules with highest food reinforcement rates showed no enhancement; performance under the schedules with the lowest reinforcement rates showed the greatest enhancement.     

Response acquisition with delayed reinforcement

Discrete responses of experimentally naive, food-deprived White Carneaux pigeons (key pecks) or Sprague-Dawley rats (bar or omnidirectional lever presses) initiated unsignaled delay periods that terminated with food delivery. Each subject first was trained to eat from the food source, but no attempt was made to shape or to otherwise train the response. In both species, the response developed and was maintained. Control procedures excluded the simple passage of time, response elicitation or induction by food presentation, type of operandum, food delivery device location, and adventitious immediate reinforcement of responding as the basis for the effects. Results revealed that neither training nor immediate reinforcement is necessary to establish new behavior. The conditions that give rise to both the first and second response are discussed, and the results are related to other studies of the delay of reinforcement and to explanations of behavior based on contingency or correlation and contiguity.     

Preference for intermittent reinforcement

Two experiments were conducted demonstrating that under certain conditions pigeons may peck at a higher rate on a key that produces intermittent reinforcement following a delay than on one that always produces reinforcement following the same delay duration. In both experiments, concurrent chain schedules were employed. In Experiment I, a single peck on one key led to a white light and a delay of 15 sec, which always terminated with food. A peck on the other key led to its illumination by one of two colored lights and a delay period of 15 sec. The delay was followed by either food presentation or timeout, either one lasting 3 sec. In a control group, the lights on this key were not correlated with food or timeout. Under the correlated stimuli, birds more often pecked the key leading to intermittent reinforcement, whereas with uncorrelated stimuli they pecked the key leading to the white light and 100% reinforcement. In Experiment II, concurrent variable-interval schedules were employed in the first link. The results showed generally that the relative rate was higher on the key leading to intermittent reinforcement when the stimuli were correlated with reinforcement and timeout than on the key leading to 100% reinforcement. There was some indication that this performance was affected by (1) the duration of the delay, (2) the percentage of reinforcement on the key yielding the higher percentage of reinforcement (the key with the white light), and (3) prior experimental conditions.     

Delayed temporal discrimination in pigeons: A comparison of two procedures

A within-subjects comparison was made of pigeons' performance on two temporal discrimination procedures that were signaled by differently colored keylight samples. During stimulus trials, a peck on the key displaying a slanted line was reinforced following short keylight samples, and a peck on the key displaying a horizontal line was reinforced following long keylight samples, regardless of the location of the stimuli on those two choice keys. During position trials, a peck on the left key was reinforced following short keylight samples and a peck on the right key was reinforced following long keylight samples, regardless of which line stimulus appeared on the correct key. Thus, on stimulus trials, the correct choice key could not be discriminated prior to the presentation of the test stimuli, whereas on position trials, the correct choice key could be discriminated during the presentation of the sample stimulus. During Phase 1, with a 0-s delay between sample and choice stimuli, discrimination learning was faster on position trials than on stimulus trials for all 4 birds. During Phase 2, 0-, 0.5-, and 1.0-s delays produced differential loss of stimulus control under the two tasks for 2 birds. Response patterns during the delay intervals provided some evidence for differential mediation of the two delayed discriminations. These between-task differences suggest that the same processes may not mediate performance in each.