Inhibitory control and errorless discrimination learning

Pigeons learned to discriminate between a positive stimulus (white key) and a negative stimulus (red or green key, depending on the subject) via Terrace's fading procedure. Generalization tests, conducted with intermittent reinforcement for key pecking at various wavelengths, yielded minima at the value of the negative stimulus in most “errorless” birds. Terrace's contrary finding of flat gradients in errorless subjects probably resulted from a floor-effect (i.e., virtually zero responding) produced by his extinction-test procedure. The present and other findings do not support Terrace's conclusions that the negative stimulus of an errorless discrimination is behaviorally neutral; inhibition apparently develops to the nonreinforced stimulus even during errorless discrimination learning. A negative correlation between stimulus and reinforcer seems the crucial factor in producing an inhibitory stimulus.     

Contrast and reallocation of extraneous reinforcers between multiple-schedule components

Four pigeons responded in components of multiple schedules in which two responses were available and reinforced with food. Pecks on the left key (“main” key) were reinforced at a constant rate in one component and at a rate that varied over conditions in the other component. When reinforcer rate was varied, behavioral contrast occurred in the constant component. On the right key (“extra” key), five variable-interval schedules and one variable-ratio schedule, presented conjointly, arranged reinforcers for responses in all conditions. These conjoint schedules were common to both multiple-schedule components—rather than unique to particular components—and reinforcers from these schedules could therefore be arranged in one component and obtained during the other component. In this way, the additional reinforcers were analogous to the “extraneous” reinforcers thought to maintain behavior other than pecking in conventional multiple schedules. Response rate on the extra key did not change systematically over conditions in the constant component, and in the varied component extra responding was inversely related to main-key reinforcement. All subjects obtained more extra-key reinforcers in whichever component arranged fewer main-key reinforcers. Consistent with the theory that reallocation of extraneous reinforcers may cause behavioral contrast, absolute reinforcer rate for the extra key in the constant component was low in conditions that produced positive contrast on the main key and high in those that produced negative contrast. Also consistent with this theory, behavioral contrast was reduced in two conditions that canceled extra-key reinforcers that had been arranged but not obtained at the end of components. Thus, a constraint on reallocation markedly reduced the extent of contrast.     

Stimulus properties of conspecific behavior

Two experiments identified the conditions in which the behavior of one bird acquired discriminative control of the behavior of a second bird. The schedule-controlled behaviors of the “stimulus” bird were differentially correlated with the components of a multiple schedule according to which the pecking of an “experimental” bird produced food. In Experiment 1, three pairs of pigeons acquired a successive discrimination and two reversals with the conspecific stimuli. Experiment 2 included a control condition in which no systematic relationship existed between the conspecific stimuli and the component schedules. While differential responding during the components of the multiple schedule was again found when the conspecific stimuli were available, differential responding did not occur in the control condition. Test conditions included in the experiments indicated that (a) the differential responding was not dependent on the discriminative properties of reinforcement, (b) the pecking of the stimulus and experimental birds was temporally interrelated, (c) the visual conspecific stimuli were critical to the maintenance of the discrimination, and (d) the observed stimulus control immediately generalized to an unfamiliar conspecific.     

Punishment of observing by the negative discriminative stimulus

To determine the effect of a negative discriminative stimulus on the response producing it, two pigeons were each studied in a three-key conditioning chamber. During alternating periods of unpredictable duration, pecking the center (food) key either was reinforced with grain on a variable-interval schedule or was never reinforced. On equal but independent variable-interval schedules, pecking either of the side (observing) keys changed the color of all keys for 30 sec from yellow to either green or red. When the schedule on the center key was variable-interval reinforcement, the color was green (positive discriminative stimulus); when no reinforcements were scheduled, the color was red (negative discriminative stimulus). Since pecking the side keys did not affect grain deliveries, changes in the rate of pecking could not be ascribed to changes in the frequency of primary reinforcement. In subsequent sessions, red was withheld as one of the possible consequences of pecking a given side key. When red was omitted, the rate on that key increased, and when red was restored, the rate decreased. It was concluded that red illumination of the keys, the negative discriminative stimulus, had a suppressive effect on the response that produced it.     

Generalization gradients of inhibition after different amounts of training

Five groups of pigeons received seven sessions of variable-interval reinforcement for pecking a blank white key, followed by either 1, 2, 4, 8, or 16 sessions of training on a successive discrimination in which the positive stimulus was the blank white key and the negative stimulus was a black vertical line on the white key. After training, a generalization test was administered along the line-tilt continuum. Relative gradients of inhibition became steeper with increased amounts of training, and reliably nonhorizontal absolute gradients were obtained only from groups of subjects with at least four days of training. Therefore, inhibitory stimulus control improves with added training. Several problems with the concept of “inhibition” are examined and some implications of the results for theoretical analyses of operant discrimination learning are discussed.     

Sign-tracking with an interfood clock

Food was presented to pigeons, irrespective of their behavior. The fixed 60-s interfood interval was segmented into ten 6-s periods, each signaled by a distinctive stimulus color, ordered by wavelength. This “interfood clock” reliably generated and maintained successively higher rates of key pecking at stimuli successively closer to food. Under extinction, key pecking ceased. When the standard stimulus sequence was changed to a different sequence for each bird, accelerated responding again emerged and was sustained under each of the new color sequences. However, responding was neither maintained nor acquired when each successive interfood interval provided a different random sequence of the ten stimuli. Thus, the interfood clock generated and maintained sign-tracking under stimulus control, and the resulting behavior was attributable neither to stimulus generalization nor to a simple temporal gradient.     

Determinants of contrast in the signal-key procedure: Evidence against additivity theory

Two experiments are reported that challenge the interpretation of previous results with the signal-key procedure, in which the discriminative stimuli are located on a response key different from the key associated with the operant response requirement. Experiment 1 replicated the procedure of Keller (1974), and found that contrast effects on the operant key occurred reliably for only one of four subjects. High rates to the signal key initially occurred for only one subject, but modifications of the procedure produced substantial rates to the signal key for all subjects. In all cases, however, signal-key behavior was greatly reduced by the addition of a changeover delay which prevented reinforcement within 2 seconds of the last peck to the signal key, suggesting that signal-key pecking was maintained primarily by adventitious reinforcement. Experiment 2 modified the signal-key procedure by using three response keys, so that the discriminative stimuli on the signal key controlled different responses during all phases of training. With this modification, reliable contrast effects on the operant key occurred for all subjects, suggesting that the failure to find contrast in previous studies has been due to the confounding of changes in the discrimination requirements with changes in relative rate of reinforcement. The results challenge the additivity theory of contrast, and suggest that “elicited” behavior plays a minor role, if any, in the determination of contrast effects in multiple schedules.     

Auto-maintenance in the pigeon: sustained pecking despite contingent non-reinforcement

If a response key is regularly illuminated for several seconds before food is presented, pigeons will peck it after a moderate number of pairings; this “auto-shaping” procedure of Brown and Jenkins (1968) was explored further in the present series of four experiments. The first showed that pecking was maintained even when pecks turned off the key and prevented reinforcement (auto-maintenance); the second controlled for possible effects of generalization and stimulus change. Two other experiments explored procedures that manipulated the tendency to peck the negatively correlated key by introducing alternative response keys which had no scheduled consequences. The results indicate that pecking can be established and maintained by certain stimulus-reinforcer relationships, independent of explicit or adventitious contingencies between response and reinforcer.     

On conditioned reinforcing effects of negative discriminative stimuli

Observing responses by pigeons were studied during sessions in which a food key and an observing key were available continuously. A variable-interval schedule and extinction alternated randomly on the food key. In one condition, food-key pecking during extinction decreased reinforcement frequency during the next variable-interval component, and in the other condition such pecking did not affect reinforcement frequency. Observing responses either changed both keylight colors from white to green (S+) or to red (S−) depending on the condition on the food key, or the observing responses never produced the S+ but produced the S− when extinction was in effect on the food key. Observing responses that produced only S− were maintained only when food-key pecking during extinction decreased reinforcement frequency in the subsequent variable-interval component. The red light conformed to conventional definitions of a negative discriminative stimulus, rendering results counter to previous findings that production of S− alone does not maintain observing. Rather than offering support for an informational account of conditioned reinforcement, the results are discussed in terms of a molar analysis to account for how stimuli acquire response-maintaining properties.     

General attentiveness effects of discriminative training

Using a design that permitted the simultaneous assessment of intra-, inter-, and extradimensional effects of discriminative training, the generality of discriminative effects that have been said to reflect increases in “general attentiveness” was assessed. Pigeons received either discriminative training with two stimuli correlated with reinforcement and one stimulus correlated with nonreinforcement, or nondifferential reinforcement (control) training. One positive stimulus was part of an intradimensional task and the other was not. After training, generalization tests were conducted to assess stimulus control along several dimensions. Discriminative training resulted in increased control along dimensions of the positive stimulus involved in the intradimensional task, but not along any dimensions of the other positive stimulus. The results suggested that discriminative training leads to increases in attention that are neither so general as suggested by the “general attentiveness” view nor so specific as to be revealed solely by intradimensional effects.     

Some factors that influence the acquisition of complex, stereotyped, response sequences in pigeons

Two pigeons were required to peck six to nine illuminated response keys. A response on any one of the keys darkened that key. When each key had been darkened, a reinforcer was delivered. No specific sequence of key pecking was ever required. The keys were presented in various matrices: three by two, three by three, horizontal rows, and vertical columns. The keys either presented the same stimulus, white light; or each key presented a different stimulus, a color or form. The results indicated that although there were 720 to 362,880 different sequences that would produce reinforcement, each bird developed a particular, stereotyped sequence that dominated its behavior. Variability among the birds across phases yielded less than 60 sequences, .0001 to 6 percent of the possible sequences. The data suggest that a reinforcement contingency that includes “free choice” of response sequence will produce stereotypical response sequences that function as complex “units” of behavior.     

Separating the effects of salience and disparity on the rate of observing

Pigeons producing deliveries of grain on a mixed variable-interval, extinction schedule by pecking a center key could also produce discriminative stimuli on concurrent variable-interval schedules by pecking the left or right observing key. The stimuli produced by each observing key were varied independently. In the first experiment, the negative discriminative stimulus was at the far end of the spectrum from the key illumination accompanying the mixed schedule and from the positive discriminative stimulus. When the magnitude of the difference between the latter two stimuli (salience) was varied, more pecks occurred on the observing key producing the larger of the two differences than on the key producing the smaller difference. In the second experiment, the stimulus accompanying the mixed schedule was at the far end of the spectrum, and the magnitude of the difference between the two discriminative stimuli (disparity) was varied. The proportion of pecks occurring on each observing key shifted systematically in the direction of the key producing the larger difference. The salience of the discriminative stimuli and their disparity each has an independent influence on the frequency of observing when the other is controlled, but the effect of the salience appears to be the more substantial.     

The role of the response-reinforcer contingency in negative automaintenance

When a response key is briefly illuminated before a grain reinforcer is presented, key pecking is reliably developed and maintained in pigeons, even if pecking prevents reinforcement (negative automaintenance). This experiment demonstrated that pigeons are sensitive to a negative response-reinforcer contingency, even though it does not eliminate responding. Within individual pigeons, two kinds of trials were compared: red key trials, in which reinforcement was negatively contingent on responding, and white key trials, in which reinforcement was unrelated to responding. Reinforcement frequency in non-contingent trials was yoked to the obtained reinforcement frequency in negatively contingent trials. All eight pigeons pecked substantially more on the non-contingent key than on the negative key, and preferred the non-contingent key to the negative key on occasional “choice” trials where both were presented together. When the stimuli correlated with the two conditions were reversed, the pigeons' behavior also shifted. These response differences are taken as evidence that pigeons are sensitive to the negative response-reinforcer contingency.     

Behavioral contrast produced by a signaled decrease in local rate of reinforcement

Pigeons' key pecking in the presence of one stimulus (S1) was reinforced according to a response-dependent variable-interval schedule. Pecking rate during S1 increased (behavioral contrast) when a second stimulus (S2) [associated with either a response-dependent fixed-interval schedule (Experiment I) or a response-independent reinforcement schedule in which reinforcement availability was signaled by visual (Experiment II) or temporal (Experiment III) stimuli] alternated with S1. These experiments suggest that a discriminable, signaled decrease in local reinforcement rate during S2 is an antecedent of the behavioral contrast response rate increases during S1.     

Stimulus-reinforcer contingencies and local behavioral contrast

Four pigeons were exposed to a series of multiple schedules of variable-interval reinforcement in which pecks were required on one key (operant key) and components were signalled on a second key (signal key). Four additional pigeons experienced identical conditions, except that a yoking procedure delivered food on variable-time schedules, with no key pecks required. One of the components of the multiple schedule was constant throughout the experiment as a variable-interval (or variable-time) 30-second schedule. Operant-key responding during the constant component was uniform throughout the component, uninfluenced by changes in the duration of the variable component, and only slightly influenced by changes in reinforcement frequency correlated with the variable component. By comparison, signal-key response rate during the constant component was highest at the onset of the component, was higher when the variable component was 60-sec long than when it was 1-sec long, and was higher when no reinforcement occurred in the variable component than when reinforcement was scheduled in the variable component. These characteristics of signal-key pecking matched characteristics of local positive behavioral contrast. These data are taken to support the “additivity theory” of behavioral contrast and to suggest that Pavlovian stimulus-reinforcer relations contribute primarily to the phenomenon of local positive contrast.     

Analysis of discriminative control by social behavioral stimuli

Visual discriminative control of the behavior of one rat by the behavior of another was studied in a two-compartment chamber. Each rat's compartment had a food cup and two response keys arranged vertically next to the clear partition that separated the two rats. Illumination of the leader's key lights signaled a “search” period when a response by the leader on the unsignaled and randomly selected correct key for that trial illuminated the follower's keys. Then, a response by the follower on the corresponding key was reinforced, or a response on the incorrect key terminated the trial without reinforcement. Accuracy of following the leader increased to 85% within 15 sessions. Blocking the view of the leader reduced accuracy but not to chance levels. Apparent control by visual behavioral stimuli was also affected by auditory stimuli and a correction procedure. When white noise eliminated auditory cues, social learning was not acquired as fast nor as completely. A reductionistic position holds that behavioral stimuli are the same as nonsocial stimuli; however, that does not mean that they do not require any separate treatment. Behavioral stimuli are usually more variable than nonsocial stimuli, and further study is required to disentangle behavioral and nonsocial contributions to the stimulus control of social interactions.     

Schedule-induced mirror responding in the pigeon

Two pigeons that were previously exposed to a multiple schedule of reinforcement in the presence of a stuffed and a live pigeon, and two of three naive pigeons, responded on a mirror during exposure to multiple fixed-ratio, fixed-ratio schedules of reinforcement for key pecking. Both the topography and temporal pattern of mirror responding were comparable to schedule-induced “attack” on live and stuffed targets. Rate of target responding was reduced when either the mirror was covered with paper or when the multiple schedule was removed. A reversal in the relationship between reinforcement schedules and discriminative stimuli demonstrated that mirror responding was controlled by the stimulus correlated with the higher fixed-ratio schedule. With one component of the multiple schedule held constant at fixed ratio 25 and the ratio requirement of the other component varying from 25 to 150, there was an inverted U-shaped relationship between rate of mirror responding and fixed-ratio schedule in the varied component. As in Flory's study (1969b) there was an inverted U-shaped relationship between target responding and inter-food intervals. The combined results of these studies suggest that the relationship between rate of target responding and reinforcement schedules is controlled primarily by the inter-food intervals resulting from the schedules.     

Concurrent performances: synthesizing rate constancies by manipulating contingencies for a single response

An earlier experiment scheduled variable-interval reinforcement for pigeons' pecks on one key, and variable-interval reinforcement alternating with extinction, in a multiple schedule, for pecks on a second key. During the second key's extinction component, first-key pecking was relatively slow and continuous, rarely interrupted by second-key pecking; during the variable-interval component, first-key pecking was frequently interrupted by second-key pecking. When changeover delays operated, so that reinforced pecks on one key could not follow closely upon changeovers from the other key, rapid first-key pecking between interruptions compensated sufficiently for the time lost in second-key pecking that the overall rate of first-key pecking remained roughly constant across the alternating multiple-schedule components. The present experiments duplicated, on a single key, the temporal pattern of first-key pecking generated in the earlier experiments: components of continuous key availability were alternated with components of interrupted key availability. Approximately constant overall rates of responding were observed with a single-key equivalent of a changeover delay scheduled after interruptions and with manipulations of the on-off durations of the interruption cycle. Rate constancies in the original concurrent situation presumably depended on analogous contingencies that operated upon the concurrent responses, rather than on any constant “reserve” of responses.     

The role of autoshaping in cooperative two-player games between starlings

We report a study of the behavior of starlings in laboratory situations inspired by the “prisoner's dilemma.” Our purpose is to investigate some possible mechanisms for the maintenance of cooperation by reciprocity and to investigate the process of autoshaping at a trial-by-trial level. In Experiment 1, pairs of starlings housed in adjacent cages played a discrete-trial “game” in which food could be obtained only by “cooperation.” In this game, pecking at a response key eliminated the opportunity to obtain food but produced food for the partner. If neither bird pecked, neither had the opportunity to obtain food in that trial. Some level of cooperation persisted for several sessions whether the birds had been pretrained for a high or low probability of pecking at the key. The probability of a cooperative response was higher after trials in which the partner responded (and a reward was obtained) than after trials in which neither bird responded (and no reward was obtained), but the probability of a response was even higher after trials in which the same bird had responded, even though no reward was obtained by the actor in these trials. This behavior did not require visual presence of another player, because similar results were obtained in Experiment 2 (a replicate of Experiment 1 in which the members of the pair could not see each other) and in Experiment 3, a game in which each starling played with a computer responding with “tit for tat.” Using an omission schedule, in which food was given in all trials in which the bird did not peck, Experiment 4 showed that pecking could be maintained by autoshaping. In this experiment, overall probability of pecking decreased with experience, due to a drop in the tendency to peck in consecutive trials. The probability of pecking in trials following a reinforced trial did not decrease with experience. An implementation of the Rescorla–Wagner model for this situation was capable of reproducing molar, but not molecular, aspects of our results. The results violate the predictions of several game-theoretical models for the evolution of cooperation, including tit for tat, generous tit for tat, and the superior win-stay-lose-shift.     

Stress-induced breakdown of an appetitive discrimination

Rats trained to discriminate between S^D and S^Δ for food reinforcement showed marked impairments in this discrimination when strong, unavoidable shocks occurred at the termination of a third stimulus. The predominant feature of this impairment was a supernormal rate of unreinforced (S^Δ) behavior. Shocks delivered without exteroceptive warning also led to a discriminative breakdown. The effect was a direct function of shock intensity. When behavior was strongly suppressed in the third stimulus by response-correlated shock (“punishment”), instead of unavoidable shock, breakdowns were only temporary; as soon as responding recovered from its overall suppression, discriminative performance returned to normal. The discriminative deterioration may be interpreted as an emotional by-product of frequent aversive stimulation, but accidental contingencies could also have played a role.