Inhibitory stimulus control and the magnitude of delayed reinforcement

The key pecking of pigeons was reinforced according to a variable-interval 1-min schedule during each of two successively presented stimuli. When the key was illuminated by a black line on a white background, reinforcement was delayed for 10 sec. When the key was illuminated by a plain white light, reinforcement was not delayed. For half of the subjects, the delayed reinforcer was 4.0-sec access to mixed grain, and for the remaining subjects it was 1.5-sec access. The immediate reinforcer was 1.5-sec access for all subjects. All subjects responded at a lower rate during the presentation of the black line; no between-group difference in terms of terminal response rate during the presentation of the line was found. However, subjects that received 4.0 sec of delayed reinforcement responded at a lower terminal rate during presentation of the plain white light than subjects that received 1.5 sec of delayed reinforcement. A subsequent generalization test along the line-orientation dimension produced flatter U-shaped gradients for subjects that received 4.0-sec of delayed reinforcement.     

Free-operant forgetting: Delayed stimulus control of multiple-schedule performance

Pigeons' responses were reinforced in two components of several multiple variable-interval variable-interval schedules of food reinforcement. In one component, the key was illuminated green for 15 seconds and white for 45 seconds. In the other component, the key was illuminated red for 15 seconds and white for 45 seconds. Values for the exponent of the power functions relating response ratios to reinforcement ratios were higher in the presence of the discriminative stimuli (green or red) than in the presence of white. Sensitivity of response ratios to changes in reinforcement ratios provided an index of the extent to which responding was under delayed stimulus control by prior discriminative stimuli.     

Dimensional stimulus control following brief wavelength training

Pigeons with extensive training pecking a key illuminated by a white line then had brief training with the key illuminated by 555 nanometers. This was immediately followed by a wavelength generalization test in extinction. Dimensional stimulus control about the training wavelength increased with the duration and number of reinforcements given on variable-interval 30-sec and variable-interval 10-sec schedules in Experiment I. In Experiment II, dimensional stimulus control was obtained after only 4 min of wavelength training from birds with prior and independent discrimination training. Experiment III provided groups equated in number of reinforcers with groups in Experiment I and two 8-min duration groups. Analyses, which included results from both Experiments I and III, showed that dimensional stimulus control increased: (a) more rapidly as a function of the duration of variable-interval 10-sec than variable-interval 30-sec reinforcement; (b) at the same rate across variable-interval reinforcement schedules, as a function of the number of reinforcers available during brief wavelength training.     

Stimulus control and delayed reinforcement

Pigeons were tested for generalization along the line-orientation dimension, after being trained on various two-component multiple schedules. The first component contained either a variable-interval 1-min schedule of immediate reinforcement or an extinction schedule and was associated with a plain white key (S1). The second component contained a variable-interval 1-min schedule of delayed reinforcement and was associated with a black line on a white background (S2). The major results showed that (a) decremental gradients were obtained around the stimulus associated with the delayed reinforcement component when S1 was associated with extinction, but that incremental gradients were obtained when S1 was associated with immediate reinforcement, (b) the subjects' pretraining did not affect the generalization gradients if sufficient training on the terminal multiple schedule was provided, and (c) changing the S1 schedule from immediate reinforcement to extinction produced behavioral contrast if reinforcement was delayed for 10 sec during S2, but not if it was delayed for 20 sec.     

Application of Bower's one-element model to paired-associate learning by pigeons

Bower's (1961) all-or-none model of human paired-associate learning was applied to individual data supplied by three pigeons. When the center one of three keys was illuminated with red light or with three white dots in a vertical array on a black ground, pecking on the left key was reinforced. When the center key was lighted green or with a horizontal array of three white dots on a black ground, pecking on the right key was reinforced. The left and right keys were illuminated with white light. The task was considered to be analogous to learning a paired-associate list of four pairs involving four stimulus items and two response items. The model was evaluated by comparing the following model predictions with values obtained from each animal: trials-to-criterion, standard deviation of trials-to-criterion, standard deviation of errors-to-criterion, mean error runs, mean error runs of lengths one to four, and autocorrelations of errors of lags one to three. Most of the predictions based upon the model were in close agreement with the obtained data.     

Some effects of relative reinforcement rate and changeover delay in response-independent concurrent schedules of reinforcement

Reinforcements were arranged independently of the pigeon's behavior by concurrent variable-interval schedules. The reinforcements arranged by one of the schedules occurred when the chamber was illuminated with amber light, and the reinforcements arranged by the other schedule occurred when the chamber was illuminated with blue light. Both schedules functioned concurrently, but reinforcers were delivered by each only in the presence of the appropriate stimulus condition. A response on a white key, the only key in the chamber, alternated the stimulus condition and the effective schedule. The results of this procedure were similar to those obtained with concurrent response-dependent variable-interval schedules of reinforcement. The proportion of the total session time spent in the presence of a schedule component approximated the proportion of the total number of reinforcements in the component. Changeover rate was a decreasing function of the changeover delay and of the difference between the relative rates of reinforcement for each pair of concurrent schedules.     

Stimulus control of responding during a fixed-interval reinforcement schedule

During training sessions, pecks by pigeons on a response key illuminated by a vertical line of white light resulted in reinforcement and an ensuing blackout according to a fixed-interval schedule. Training sessions were followed by dimensional stimulus control test sessions during which the orientation of the line present throughout the fixed interval was varied. Inverted U-shaped (excitatory) gradients of responding, with maximum responding occurring in the presence of the vertical line, were observed during the terminal part of the fixed interval. U-shaped (inhibitory) gradients of responding, with minimum responding occurring in the presence of the vertical line, were observed during the early part of the fixed interval when the preceding interval had terminated with reinforcement and blackout but not when the preceding interval had terminated with blackout only. These results suggest that the dimensional control by the stimulus present throughout the fixed interval is of a conditional variety. Whether the fixed-interval stimulus exerts inhibitory or excitatory dimensional control depends upon the presence and absence, respectively, of stimuli associated with reinforcement.     

Concurrent performances: stimulus-control gradients during schedules of signalled and unsignalled concurrent reinforcement

On one key, pigeons' pecks were reinforced according to a variable-interval schedule in the presence of vertical lines, and were not reinforced in the presence of oblique lines. On a second key, pecks were reinforced according to a variable-interval schedule in the presence of blue, according to a signalled variable-interval schedule in the presence of red, and were not reinforced in the presence of white. Subsequently, during extinction, stimulus-control gradients were obtained by presenting eight different line orientations on the first key concurrent with each of the three colors on the second key. On the first key, line-orientation gradients tended to be lower, narrower, and less shifted in peak or area when the second-key stimulus was blue or red, the stimuli respectively correlated with unsignalled and signalled reinforcement, than when it was white, the stimulus correlated with extinction. Thus, the effect on first-key line-orientation gradients depended on second-key stimuli correlated with concurrent reinforcement, whether or not these stimuli were also correlated with concurrent responding. As a function of first-key line orientation, an inverted gradient was obtained on the second key during blue; during both red and white, rates of pecking on the second key were near zero.     

Discriminative stimulus control and the effects of concurrent operants.

Discriminative-stimulus-control functions were investigated in a concurrent operant context. Variable-interval reinforcement schedules were arranged for pigeons on two response keys. One key, illuminated with a white vertical line on green background (position irrelevant), was programmed with a given schedule value across groups. For different groups of pigeons, the alternative key, illuminated with green alone, was programmed with twice, the same, or half the reinforcement frequency of the other key. Stimulus-control gradients were collected from both keys as line orientation was varied. On the green-plus-line alternative, flattest gradients were observed when twice the reinforcement frequency was concurrently programmed and the steepest were observed when equal values were concurrently programmed. Also examined were the effects of a programmed changeover delay, important in maintaining the independence of concurrent operants. The changeover delay was found to have relatively minor effects upon stimulus control, despite its typical and marked effects upon steady-state responding.     

Generalization gradients of inhibition after different amounts of training

Five groups of pigeons received seven sessions of variable-interval reinforcement for pecking a blank white key, followed by either 1, 2, 4, 8, or 16 sessions of training on a successive discrimination in which the positive stimulus was the blank white key and the negative stimulus was a black vertical line on the white key. After training, a generalization test was administered along the line-tilt continuum. Relative gradients of inhibition became steeper with increased amounts of training, and reliably nonhorizontal absolute gradients were obtained only from groups of subjects with at least four days of training. Therefore, inhibitory stimulus control improves with added training. Several problems with the concept of “inhibition” are examined and some implications of the results for theoretical analyses of operant discrimination learning are discussed.     

Stimulus generalization from feeder to response key in the acquisition of autoshaped pecking

During autoshaping, a 6-second presentation of one stimulus and a variable time 30-second presentation of a second stimulus alternated in appearance on a pigeon key. Grain always was delivered for 3 seconds at the end of the first stimulus interval. In the first experiment, autoshaped pecking of the stimulus preceding grain delivery began much sooner when that stimulus was a black vertical line on a white background and the other stimulus was green than when the opposite stimulus arrangement was used. Because these two stimuli differed in form, hue, brightness, and similarity in hue and brightness to the illumination of the raised feeder, three subsequent experiments examined whether the differential speed of autoshaping in the two groups was due to a feature-positive, feature-negative effect, a preference for brighter over darker stimuli, a simple preference for white over green, or stimulus generalization from the brightness or hue of the illuminated, raised feeder to the stimulus on the key preceding grain delivery. The data from these experiments showed that the first autoshaped key peck was most likely to be made to the stimulus of the same hue as that illuminating the feeder, regardless of whether that stimulus was positively or negatively associated with grain delivery. At least under some conditions, therefore, stimulus-generalization mediated response transfer of pecking grain in the presence of the hue illuminating the feeder to pecking the key illuminated by a similar hue appears to account for the occurrence of autoshaped key pecking.     

Some factors that influence the acquisition of complex, stereotyped, response sequences in pigeons

Two pigeons were required to peck six to nine illuminated response keys. A response on any one of the keys darkened that key. When each key had been darkened, a reinforcer was delivered. No specific sequence of key pecking was ever required. The keys were presented in various matrices: three by two, three by three, horizontal rows, and vertical columns. The keys either presented the same stimulus, white light; or each key presented a different stimulus, a color or form. The results indicated that although there were 720 to 362,880 different sequences that would produce reinforcement, each bird developed a particular, stereotyped sequence that dominated its behavior. Variability among the birds across phases yielded less than 60 sequences, .0001 to 6 percent of the possible sequences. The data suggest that a reinforcement contingency that includes “free choice” of response sequence will produce stereotypical response sequences that function as complex “units” of behavior.     

Facilitation of food-reinforced responding by a signal for response-independent food

Five pigeons whose key pecking was maintained by 4-sec access to grain on a variable-interval 2-min schedule received Pavlovian differential conditioning trials superimposed upon the instrumental baseline. The conditioned stimuli were changes in the stimulus on the key from white to red, or to a white horizontal line against a dark background. The positive conditioned stimulus was 20 sec long, and was followed immediately by 8-sec access to grain. The negative conditioned stimulus, also 20 sec long, was never paired with response-independent food. All pigeons responded more rapidly in the presence of the positive conditioned stimulus than in the presence of the negative one. The positive conditioned stimulus produced an increase in response rate over the pre-conditioned stimulus period. The negative conditioned stimulus had no marked effect upon response rate. When the roles of the positive and negative stimuli were reversed, and the duration of the response-independent reinforcement was reduced to 4 sec, the new positive conditioned stimulus came to facilitate responding, and the new negative conditioned stimulus no longer produced facilitation. A second discrimination reversal produced similar outcomes. When a third reversal was initiated, and the duration of response-independent reinforcement was reduced to 2 sec, the difference between the effects of the positive and negative stimuli diminished.     

Delayed and current stimulus control in successive discriminations

In a successive discrimination in which successively alternating red and green hues signaled component variable-interval schedules, sensitivity of the ratio of responses in the two components to variation in the component reinforcer ratio decreased systematically during the course of the component. This decrease in stimulus control or discrimination over the course of the component was shown to be the result of delayed control of responding during the component by the stimulus transition between components. When the red–green stimulus transition was altered by interpolating a white stimulus at the end of each 60-s component, discrimination at the beginning of the component (measured by the power-function exponent for sensitivity to reinforcement) was reduced. Conditions with the white stimulus inserted in other quarters of the component indicated that the current discriminative stimulus exerts control over responding throughout the component, whereas during about the first half of the component, response differentials are influenced by the transition between discriminative stimuli.     

Differential reinforcement and stimulus control of not responding

Pigeons were trained to respond with equal variable-interval reinforcement in the presence of a white key and also a white key with a vertical line. They were then trained not to respond to the vertical line by extinguishing the response or by reinforcing its non-occurrence at various frequencies. During training, the rate of key-pecking in the presence of the white key, maintained by a constant variable-interval schedule of reinforcement, depended on the frequency of reinforcement in the presence of the line. When lines of different orientations were presented in a generalization test, birds trained with extinction responded more to other orientations than to the vertical line, whereas those trained with high frequencies of reinforcement for not responding tended to respond equally at all line orientations. Intermediate frequencies of reinforcement gave mixed results.     

Enhancement of conditioned reinforcement by uncertainty

Pigeons were trained in three conditions. In the baseline condition, the birds responded on a fixed-interval schedule with the response key white. When the interval was completed, the key turned either red or green for a delay interval that was terminated by a grain presentation dependent on no key pecks during the final 2 sec. In the uncertainty condition, no grain was presented at the end of the delay periods when the key was red. In the certainty condition, the white light appeared only on occasions when pecking would turn the key green and produce food. Otherwise, the key was illuminated red throughout the total time period. The highest response rate in white occurred in the uncertainty condition, the next highest in the certainty condition, and the lowest in baseline. The results suggest that uncertainty facilitated responding, although uncertainty is not a necessary condition for conditioned reinforcement.     

Preferences among stimulus matches in the pigeon

Three pigeons were trained to peck two to five illuminated response keys. A peck to any of the keys changed the stimulus on the key. When all keys showed the same stimulus (i.e., a stimulus match), an additional key was illuminated with white light. A peck on this key produced three-second access to grain, a three-second intertrial interval, and the next trial. For most sessions, no particular stimulus match was required. Although there were often several stimuli available, each bird preferred a particular stimulus match. With up to 12 stimuli available, birds matched a particular stimulus 60% to 100% of the time.     

A facilitative effect of punishment on unpunished behavior

The key pecking of two pigeons was reinforced on a variable-interval schedule of reinforcement during the presentation of each of two stimuli. In various phases of the experiment, punishment followed every response emitted in the presence of one of the stimuli. In general, when the rate of punished responding changed during the presentation of one stimulus, the rate of unpunished responding during the other stimulus changed in the opposite direction. This sort of change in rate is an example of behavioral contrast. When punishment was introduced, the rate of punished responding decreased and the rate of unpunished responding increased as functions of shock intensity. When the rate of previously punished responding increased after the termination of the shock, the rate of the always unpunished responding decreased. When the procedure correlated with a red key was changed from variable-interval reinforcement and punishment for each response to extinction and no punishment, the rate of reinforced responding during presentations of a green key decreased and then increased while the rate of the previously punished responding during red first increased and then decreased during extinction.     

Stimulus control of respondent and operant key pecking: A single key procedure

Pigeons' responses to a uniformly illuminated response key were either reinforced on a variable-interval one-minute schedule of reinforcement or extinguished for one-minute periods. When 1.5 second signals were presented at the beginning of each component, so as to differentially predict reinforcement, the pigeons pecked at the signals, at rates higher than rates during the remainder of the component. When the brief signals were not differentially predictive of reinforcement, pecking in their presence decreased to near zero levels. Similar results were obtained with signals based upon colors and upon line orientations. Changes in rates of (unreinforced) pecking occurred during the signal whether pigeons responded differentially during the remainder of the component or not. Experiment II demonstrated that the presence of the signal correlated with extinction was not necessary for pecking to develop at the signal which preceded the component in which responding was intermittently reinforced. The experiments demonstrated a clear dissociation of respondent control from operant control of a response. In addition, operant behavior was shown to be relatively insensitive to differing rates of reinforcement, as compared to the sensitivity of respondent behavior to differing rates of reinforcement produced by the very same operant behavior.     

Some effects of response-dependent clock stimuli in a fixed-interval schedule

Two experiments studied the effects of brief response-dependent clock stimuli in fixed-interval schedules of reinforcement. In the first experiment, two pigeons were exposed to a fixed-interval schedule. Two conditions were compared. In both conditions each peck on the key produced a brief stimulus. In one condition, pecks produced a different stimulus in successive sixths of the interval. This was the clock condition. In the other condition, the same stimulus was produced throughout the interval. Response rates were lower and the pause after reinforcement was longer in the clock condition. In the second experiment, a two-key optional clock procedure was used. Responding on the clock key produced one of three stimuli correlated with the three successive minutes of a fixed-interval schedule. A response on the other key produced grain at the end of the 3 min. When the final stimulus was removed from the situation and pecking produced nothing during the third minute, responding to the clock key declined to a very low rate. When the first two stimuli were removed and the third one replaced, responding to the clock key was resumed.