Secondary reinforcement and number of primary reinforcements

Pigeons' pecks on either of two concurrently available response keys produced secondary reinforcers according to independent one-minute variable-interval schedules. Different secondary reinforcers, in the presence of which the rates of primary reinforcement were equal, were associated with each key. The rate of pecking maintained by each secondary reinforcer varied directly, but nonproportionally, with the number of primary reinforcements given in the presence of the secondary reinforcer.     

Concurrent performances: synthesizing rate constancies by manipulating contingencies for a single response

An earlier experiment scheduled variable-interval reinforcement for pigeons' pecks on one key, and variable-interval reinforcement alternating with extinction, in a multiple schedule, for pecks on a second key. During the second key's extinction component, first-key pecking was relatively slow and continuous, rarely interrupted by second-key pecking; during the variable-interval component, first-key pecking was frequently interrupted by second-key pecking. When changeover delays operated, so that reinforced pecks on one key could not follow closely upon changeovers from the other key, rapid first-key pecking between interruptions compensated sufficiently for the time lost in second-key pecking that the overall rate of first-key pecking remained roughly constant across the alternating multiple-schedule components. The present experiments duplicated, on a single key, the temporal pattern of first-key pecking generated in the earlier experiments: components of continuous key availability were alternated with components of interrupted key availability. Approximately constant overall rates of responding were observed with a single-key equivalent of a changeover delay scheduled after interruptions and with manipulations of the on-off durations of the interruption cycle. Rate constancies in the original concurrent situation presumably depended on analogous contingencies that operated upon the concurrent responses, rather than on any constant “reserve” of responses.     

Concurrent performances: rate constancies without changeover delays

Pigeons' pecks on two keys were maintained, without changeover delays, by independent variable-interval schedules of food reinforcement. Four regularly cycling 2-min components scheduled reinforcement respectively for both keys, left key only, both keys, and right key only. Initially, reinforcement scheduled for one key alone produced more responding on that key than reinforcement scheduled concurrently for both keys. Continued sessions reduced this difference; response rate on a given key approached constancy, or invariance with respect to the performance on and schedule for the other key. When extinction replaced the reinforcement schedule on either key, responding on that key decreased more during components that scheduled reinforcement for the other key than during those that did not. This demonstration that responses on one key were not supported by reinforcers on the other key suggested that the alternation of concurrent responding and either-key-alone responding prevented concurrent superstitions from developing.     

EFFECTS OF ALTERNATIVE REINFORCEMENT: DOES THE SOURCE MATTER?1

In a chamber with a single response key, pigeon's key pecks were reinforced with food according to a variable-interval schedule. In addition, extra reinforcements occurred concurrently according to an independent schedule. In one condition, availability of the extra reinforcements was signalled by a change in key color from white to red. The extra reinforcements occurred after a peck on the red key. In a second condition, the extra reinforcements were unsignalled and occurred only after a 2-sec pause in pecking for one group of subjects and were unsignalled and occurred freely as scheduled for another group of subjects. In the first two conditions, duration of reinforcement was varied. A third condition duplicated the second but varied rate rather than duration of reinforcement. The rate of pecking varied inversely with the amount of extra reinforcement per unit time according to the same function, regardless of the condition regulating occurrence of the extra reinforcements, and regardless of whether or not a 2-sec pause was required for their occurrence. The shape of this function was predicted by Herrnstein's (1970) matching law.     

On the directionality of key pecking during signals for appetitive and aversive events

Pigeons were exposed to a signal paired with either blackout or blackout plus shock and to another signal paired with food superimposed on a baseline of concurrent variable-interval reinforcement of pecks on two keys. The signals were changes of color of one of the two keys. The rate of pecking both keys during the signal paired with blackout or blackout plus shock was lower than the baseline rate of pecking (a conditioned emotional response), but the decrease in pecking was greater on the signal key. When the intensity of shock was increased, the rate of pecking did not decrease further on the signal key but did decrease on the other key. Rate of pecking during the signal paired with food increased sharply on the signal key (an autoshaping effect) and decreased sharply on the other key. These results support a view that there are two effects of the interaction between classical and instrumental conditioning, a stimulus-directed effect and a generalemotional effect.     

On some determinants of choice in pigeons

A pigeon's pecking at each of two or three simultaneously available red keys was reinforced at different frequencies with a conditioned reinforcer, an orange key, on which 25 pecks resulted in a presentation of grain. Pecking was occasionally punished with a period of no reinforcement during which each key was dark. Both with two and with three keys, the relative frequency of pecking on a key was equal to the relative frequency of reinforcement obtained by pecks on that key. Also, the absolute frequency of pecking on each key was a linear function with zero intercept of the absolute frequency of reinforcement associated with that key. The slope of this function varied with the number of available keys; it was steeper with two than with three. The relative frequency of switching from any key (two successive pecks on different keys) approximated a linear function with zero intercept and slope slightly greater than 1.0 of the total relative frequency of reinforcement associated with the keys to which the bird could switch. However, the relative frequency of switching to a particular key often showed systematic irregularities. The invariance in these data is the equality between the relative frequency of pecks on one of two or three keys and the relative frequency of reinforcement associated with that key.     

Interval reinforcement of choice behavior in discrete trials

Pigeons were trained to peck at red or green keys presented simultaneously in discrete trials. In one experiment, reinforcements were arranged by concurrent variable-interval schedules. The proportion of responses to green approximately matched the proportion of reinforcements produced by pecking green. Detailed analysis of responding revealed a systematic decrease in the probability of switching from green to red within sequences of trials after reinforcement. This trend corresponded to sequential changes in the relative frequency of reinforcement, and not to sequential changes in probability of reinforcement. In a second experiment, reinforcements were scheduled probabilistically every seventh trial. Even though there were no contingencies on pecking during the first six post-reinforcement trials, choices of green on the first response after reinforcement matched the proportion of reinforcements for pecking green. These results extend the generality of overall matching under concurrent reinforcement.     

Time allocation on concurrent schedules with asymmetrical response requirements

Pigeons were trained on concurrent schedules in which key pecking was required by both schedules (concurrent variable-interval variable-interval schedules) and on concurrent schedules in which key pecking was required by only one of the schedules (concurrent variable-interval variable-time schedules). The distribution of reinforcements was systematically varied with both types of concurrent schedules. The distribution of time between the schedules depended on the reinforcement distribution and was independent of the symmetry of the response requirement. The relation between time and reinforcement distributions appears to be invariant over a wide range of manipulations of responding maintained by concurrent schedules.     

Disruption of temporally organized behavior by morphine

Four pigeons pecked keys in two different procedures commonly used in the study of timing, or temporal discrimination. Sessions consisted of 40 trials. During half of the trials, two keys were presented for 50 s. Left-key pecks were reinforced according to a variable-interval 67.86-s schedule during the first 25 s of the trial, and right-key pecks were not reinforced. During the second 25 s of the trial, right-key pecks were reinforced according to the same schedule, and left-key pecks were not reinforced. In the other half of the 40-trial session, the center key was presented. The majority of these trials arranged fixed-interval 2.5-s schedules. Occasionally a probe, or peak-interval, trial was presented. These trials were 100 s in duration and terminated without reinforcement. These two procedures were used to examine the effects of morphine on indexes of timing and on patterns of responding. Morphine altered behavior in a race-dependent manner in both procedures. Low baseline (saline) response rates were increased following morphine administration, and high baseline rates were either unaffected or decreased slightly. Rate-dependent effects appeared as leftward shifts in the timing index for two-key trials and decreases in the index of curvature for fixed-interval trials. Despite large changes in response rates, no consistent shift of the peak time was observed during peak-interval trials. These results are discussed primarily in terms of rate dependency; that is, rates of responding following drug administration tend to be determined in large part by rates of responding under baseline conditions.     

Punishment of observing by the negative discriminative stimulus

To determine the effect of a negative discriminative stimulus on the response producing it, two pigeons were each studied in a three-key conditioning chamber. During alternating periods of unpredictable duration, pecking the center (food) key either was reinforced with grain on a variable-interval schedule or was never reinforced. On equal but independent variable-interval schedules, pecking either of the side (observing) keys changed the color of all keys for 30 sec from yellow to either green or red. When the schedule on the center key was variable-interval reinforcement, the color was green (positive discriminative stimulus); when no reinforcements were scheduled, the color was red (negative discriminative stimulus). Since pecking the side keys did not affect grain deliveries, changes in the rate of pecking could not be ascribed to changes in the frequency of primary reinforcement. In subsequent sessions, red was withheld as one of the possible consequences of pecking a given side key. When red was omitted, the rate on that key increased, and when red was restored, the rate decreased. It was concluded that red illumination of the keys, the negative discriminative stimulus, had a suppressive effect on the response that produced it.     

Concurrent performances: rate and accuracy of free-operant oddity responding

In pigeon's oddity performances, maintained by variable-interval reinforcement of pecks on the odd key of three keys in a triangular array, accuracy and response rate varied inversely with the rate of variable-interval reinforcement scheduled concurrently for pecks on a fourth, spatially isolated key. But when variable-interval and extinction components alternated in a multiple schedule for pecks on the spatially isolated key, oddity accuracy was greater during variable-interval components than during extinction components. Oddity response rate was not affected systematically by the alternating components. Changeovers between the oddity keys and the spatially isolated key were frequent during variable-interval components; responding occurred almost exclusively on the oddity keys during extinction components. This difference in performance during the two components was eliminated by arranging stimulus-correlated variable-interval reinforcement in the multiple schedule on the spatially isolated key: a stimulus was presented in the variable-interval components only when reinforcement became available, thereby reducing responding on this key to near-zero levels in both components while maintaining the variable-interval reinforcement. The effect of the multiple-schedule components on oddity accuracy was not altered, however, and thus apparently depended directly on concurrent reinforcement and not on differential sequential properties of concurrent responding during the two components.     

Alternative reinforcement effects on fixed-interval performance

Pigeons' key pecks were reinforced with food on a fixed-interval schedule. Food also was available at variable time periods either independently of responding or for not key pecking (a differential-reinforcement-of-other-behavior schedule). The latter condition arranged reinforcement following the first pause of t seconds after it became available according to a variable-time schedule. This schedule allowed separation of the effects of pause requirements ≤ five-seconds and reinforcement frequency. The time spent pausing increased as the duration of the pause required for reinforcement increased from 0 to 30 seconds and as the frequency of reinforcement for pausing increased from 0 to 2 reinforcers per minute. Key pecking was more evenly distributed within each fixed interval with shorter required pauses and with more frequent reinforcement for pausing. The results complement those obtained with other concurrent schedules in which the same operant response was reinforced in both components.     

Reinforcer effectiveness as a function of reinforcer rate and magnitude: a comparison of concurrent performances

Pigeons' pecks on each of two concurrently available response keys were reinforced under a variable-interval schedule that sometimes allotted food-pellet deliveries to one key and sometimes to the other. The keys differed in the number of reinforcements assigned to each and in the number of pellets delivered during each reinforcement. When the total quantity of food associated with each key during a session was constant, the proportion of responses to a key depended on the particular combinations of reinforcer rate and reinforcer magnitude scheduled on each key. A given quantity of food generated more responding on a key when it was delivered frequently in small amounts than when it was delivered infrequently in large amounts.     

Auto-maintenance in the pigeon: sustained pecking despite contingent non-reinforcement

If a response key is regularly illuminated for several seconds before food is presented, pigeons will peck it after a moderate number of pairings; this “auto-shaping” procedure of Brown and Jenkins (1968) was explored further in the present series of four experiments. The first showed that pecking was maintained even when pecks turned off the key and prevented reinforcement (auto-maintenance); the second controlled for possible effects of generalization and stimulus change. Two other experiments explored procedures that manipulated the tendency to peck the negatively correlated key by introducing alternative response keys which had no scheduled consequences. The results indicate that pecking can be established and maintained by certain stimulus-reinforcer relationships, independent of explicit or adventitious contingencies between response and reinforcer.     

Effects of delayed reinforcement in a concurrent situation

Pigeons were trained to peck either of two response keys for food reinforcement on equated aperiodic schedules. Delays of reinforcement for pecks at one key reduced the relative frequency of pecking exponentially as a function of the delay interval. Similar functions were obtained when other dependent variables were plotted against the delay interval.     

Concurrent performances: stimulus-control gradients during schedules of signalled and unsignalled concurrent reinforcement

On one key, pigeons' pecks were reinforced according to a variable-interval schedule in the presence of vertical lines, and were not reinforced in the presence of oblique lines. On a second key, pecks were reinforced according to a variable-interval schedule in the presence of blue, according to a signalled variable-interval schedule in the presence of red, and were not reinforced in the presence of white. Subsequently, during extinction, stimulus-control gradients were obtained by presenting eight different line orientations on the first key concurrent with each of the three colors on the second key. On the first key, line-orientation gradients tended to be lower, narrower, and less shifted in peak or area when the second-key stimulus was blue or red, the stimuli respectively correlated with unsignalled and signalled reinforcement, than when it was white, the stimulus correlated with extinction. Thus, the effect on first-key line-orientation gradients depended on second-key stimuli correlated with concurrent reinforcement, whether or not these stimuli were also correlated with concurrent responding. As a function of first-key line orientation, an inverted gradient was obtained on the second key during blue; during both red and white, rates of pecking on the second key were near zero.     

Reinforcement schedules: the role of responses preceding the one that produces the reinforcer

In a two-key pigeon chamber, variable-interval reinforcement was scheduled for a specified number of pecks, emitted either on a single key or in a particular sequence on the two keys. Although the distribution of pecks between the two keys was affected by whether pecks were required on one or on both keys, the total pecks emitted was not; the change from a one-key to a two-key requirement simply moved some pecks from one key to the other. Thus, each peck preceding the one that produced the reinforcer contributed independently to the subsequent rate of responding; the contribution of a particular peck in the sequence was determined by the time between its emission and the delivery of the reinforcer (delay of reinforcement), and was identified by the proportion of pecks moved from one key to the other when the response requirement at that point in the sequence was moved from one key to the other.     

Soft commitment: self-control achieved by response persistence.

With reinforcement contingent on a single peck on either of two available keys (concurrent continuous reinforcement schedules) 4 pigeons, at 80% of free-feeding weights, preferred a smaller-sooner reinforcer (2.5 s of mixed grain preceded by a 0.5-s delay) to a larger-later reinforcer (4.5 s of mixed grain preceded by a 3.5-s delay). However, when the smaller-sooner and larger-later reinforcers were contingent on a concurrent fixed-ratio 31 schedule (the first 30 pecks distributed in any way on the two keys), all pigeons obtained the larger-later reinforcer much more often than they did when only a single peck was required. This "self-control" was achieved by beginning to peck the key leading to the larger-later reinforcer and persisting on that key until reinforcement occurred. We call this persistence "soft commitment" to distinguish it from strict commitment, in which self-control is achieved by preventing changeovers. Soft commitment also effectively achieved self-control when a brief (1-s) signal was inserted between the 30th and 31st response of the ratio and with concurrent fixed-interval 30-s schedules (rather than ratio schedules) of reinforcement. In a second experiment with the same subjects, the fixed ratio was interrupted by darkening both keys and lighting a third (center) key on which pecking was required for various fractions of the fixed-ratio count. The interruption significantly reduced self-control. When interruption was complete (30 responses on the center key followed by a single choice response), pigeons chose the smaller-sooner reinforcer as frequently as they did when only a single choice response was required.     

Mechanisms underlying the effects of unsignaled delayed reinforcement on key pecking of pigeons under variable-interval schedules.

Three experiments were conducted to test an interpretation of the response-rate-reducing effects of unsignaled nonresetting delays to reinforcement in pigeons. According to this interpretation, rates of key pecking decrease under these conditions because key pecks alternate with hopper-observing behavior. In Experiment 1, 4 pigeons pecked a food key that raised the hopper provided that pecks on a different variable-interval-schedule key met the requirements of a variable-interval 60-s schedule. The stimuli associated with the availability of the hopper (i.e., houselight and keylight off, food key illuminated, feedback following food-key pecks) were gradually removed across phases while the dependent relation between hopper availability and variable-interval-schedule key pecks was maintained. Rates of pecking the variable-interval-schedule key decreased to low levels and rates of food-key pecks increased when variable-interval-schedule key pecks did not produce hopper-correlated stimuli. In Experiment 2, pigeons initially pecked a single key under a variable-interval 60-s schedule. Then the dependent relation between hopper presentation and key pecks was eliminated by arranging a variable-time 60-s schedule. When rates of pecking had decreased to low levels, conditions were changed so that pecks during the final 5 s of each interval changed the keylight color from green to amber. When pecking produced these hopper-correlated stimuli, pecking occurred at high rates, despite the absence of a peck-food dependency. When peck-produced changes in keylight color were uncorrelated with food, rates of pecking fell to low levels. In Experiment 3, details (obtained delays, interresponse-time distributions, eating times) of the transition from high to low response rates produced by the introduction of a 3-s unsignaled delay were tracked from session to session in 3 pigeons that had been initially trained to peck under a conventional variable-interval 60-s schedule. Decreases in response rates soon after the transition to delayed reinforcement were accompanied by decreases in eating times and alterations in interresponse-time distributions. As response rates decreased and became stable, eating times increased and their variability decreased. These findings support an interpretation of the effects of delayed reinforcement that emphasizes the importance of hopper-observing behavior.