Choice between response units: The rate constancy model

In a conjoint schedule, reinforcement is available simultaneously on two or more schedules for the same response. The present experiments provided food for key pecking on both a random-interval and a differential-reinforcement-of-low-rate (DRL) schedule. Experiment 1 involved ordinary DRL schedules; Experiment 2 added an external stimulus to indicate when the required interresponse time had elapsed. In both experiments, the potential reinforcer frequency from each component was varied by means of a second-order fixed-ratio schedule, and the DRL time parameter was changed as well. Response rates were described by a model stating that time allocation to each component matches the relative frequency of reinforcement for that component. When spending time in a given component, the subject is assumed to respond at the rate characteristic of baseline performance. This model appeared preferable to the absolute-rate version of the matching law. The model was shown to be applicable to multiple-response concurrent schedules as well as to conjoint schedules, and it described some of the necessary conditions for response matching, undermatching, and bias. In addition, the pigeons did not optimize reinforcer frequency.     

Elimination of reinforced behavior: intermittent schedules of not-responding

Pigeons' key pecking resulted in food according to either a variable-ratio or a variable-interval schedule. At the same time, food was available for not pecking for a specified time. The required time of not-pecking was segmented into not-responding units, and these units were followed by food according to a fixed-ratio schedule. Both unit duration and the number required were varied. In general, the shorter the time unit or the smaller the ratio, the lower was response rate. When total required not-responding time was constant, but changes in unit duration and the number required altered how the total was achieved, shorter units produced lower rates. Other conditions involved substitution of food delivered independent of responding for the not-responding schedule. With low and moderate total times to food presentation, the not-responding schedule produced lower rates; with the longest times, the response-independent schedule generated less responding. When considered in terms of relative frequency of food presentation available from a source other than pecking, the not-responding schedule reduced rate more effectively than did the response-independent schedule. Comparisons with other research suggested that food presented dependent on not responding compared favorably with punishment as a procedure for reducing response rate. Transient effects differed. Although punishment temporarily depresses rate when first imposed and temporarily enhances it when first removed, food given for not responding quickly generated steady-state rates.     

Changing the response unit from a single peck to a fixed number of pecks in fixed-interval schedules

Each of three pigeons was studied first under a standard fixed-interval schedule. With the fixed interval held constant, the schedule was changed to a second-order schedule in which the response unit was the behavior on a small fixed-ratio schedule (first a fixed-ratio 10 and then a fixed-ratio 20 schedule). That is, every completion of the fixed-ratio schedule produced a 0.7-sec darkening of the key and reset the response count to zero for the next ratio. The first fixed-ratio completed after the fixed-interval schedule elapsed produced the 0.7-sec blackout followed immediately by food. These manipulations were carried out under two different fixed-interval durations for each bird ranging from 3 min to 12 min. The standard fixed-interval schedules produced the typical pause after reinforcement followed by responding at a moderate rate until the next reinforcement. The second-order schedules also engendered a pause after reinforcement, but responding occurred in bursts separated by brief pauses after each blackout. For a particular fixed-interval duration, post-reinforcement pauses increased slightly as the number of pecks in the response unit increased despite large differences in the rate and pattern of key pecking. Post-reinforcement pause increased with the fixed-interval duration under all response units. These data confirm that the allocation of time between pausing and responding is relatively independent of the rate and topography of responding after the pause.     

Aversive aspects of a fixed-interval schedule of food reinforcement

The key pecking of pigeons was reinforced according to a fixed-interval schedule of reinforcement. The pigeons were also given the opportunity to attack a restrained target pigeon. The attack rates during the sessions of fixed-interval reinforcement were higher than during the operant level sessions in four of the five pigeons. Most attack occurred during the post-reinforcement pause in key pecking. It was suggested that a fixed-interval schedule of positive reinforcement possesses aversive properties, the most aversive of which are located during the post-reinforcement pause.     

The effects of different component response requirements in multiple and concurrent schedules

Six pigeons were trained on multiple and concurrent schedules. The reinforcement rates were varied systematically (a) when lever pressing was required in one component and key pecking in the successive component; (b) when lever pressing was required in both multiple components; (c) when key pecking was required in both multiple components; and (d) when key pecking was required on one schedule and lever pressing was required on the concurrently-available schedule. Only the absolute level of responding was changed by different response requirements. Analyzed by the generalized matching law, performance under different response requirements resulted in a bias toward key pecking, and the measured response bias was the same in multiple and concurrent schedule arrangements. The bias in time measures obtained from concurrent schedule performance was reliably smaller than the obtained response biases. The sensitivity to reinforcement-rate changes was ordered: concurrent key-lever; multiple key-key; multiple lever-key; and, the least sensitive, multiple lever-lever. The results confirm that requirements of different topographical responses can be handled by the generalized matching law mainly in the bias parameter, but problems for this type of analysis may be caused by the changing sensitivity to reinforcement in multiple schedule performance as response requirements are changed.     

Reinforcing the absence of fixed-ratio performance

Pigeons received food for key pecking according to a fixed-ratio schedule, while, at the same time, food also was available for not pecking for a specified time. With a fixed ratio of 60, responding was not affected by not-pecking times of 80 or 40 seconds, and was eliminated completely at 10 seconds. With ratios of 180, pecking stopped with not-pecking times of 80 seconds or less; with ratios of 300, it stopped at 120 seconds or less. Not-responding schedules produced steady-state performance immediately following contact with the schedule. With return to the fixed-ratio schedule alone, response rate sometimes was elevated temporarily. When response-independent food presentations replaced the not-pecking schedule, response rate often was enhanced, and the ratio pattern was lost. Only the highest densities of food delivery eliminated responding, even with a fixed ratio of 300. In general, the effects corresponded to those of punishment, except that contrast had appeared both during and after punishment, and now appeared only after the response elimination procedure was suspended.     

Reinforcement magnitude and the inhibiting effect of reinforcement

In a two-key concurrent variable-interval schedule (using pigeons), if the reinforcement frequency for one response is held constant while that for the other is increased, the rate of response on the constant key decreases. The immediate reinforcement for key pecking can usually be conceptualized as the change from a condition in which the key light is on and the food hopper light is off to one in which the key light is off and the hopper light is on. The prechange condition is associated with a delay to food of one-half the average interreinforcement interval in effect during this condition. The postchange condition is associated with a delay to food of about .5 seconds. The programming of additional reinforcement results in a decrease in the delay to food associated with the prechange stimulus condition, and thus a decrease in the value of the improvement that results from the change. This would appear to be analogous to a decrease in the amount of reinforcement, and thus sufficient explanation for the decrease in the rate of the response.     

Mirror pecking and timeout under a multiple fixed-ratio schedule of food delivery

Pigeons were trained to peck a key under a multiple fixed-ratio 25 fixed-ratio 175 schedule of food presentation. In the first condition, either a mirror or the opportunity to produce a 30-second timeout were available. In a second condition, mirror and timeout availability were reversed for the two groups. Following a return to the initial condition, mirror and timeout keys were presented together for all birds. Mirror and timeout responses occurred predominantly in the pause in the larger fixed-ratio component, regardless of whether the opportunities for the two responses were available singly or together. Mirror responding occurred in a greater proportion of the pauses than did timeouts. When the opportunities for both mirror pecking and timeout were available concurrently, they occurred with probabilities similar to those under the single conditions. Within the pause itself, mirror responses most frequently occurred immediately after reinforcement. Timeouts occurred most frequently toward the end of the pause, and some timeouts occurred in the early part of the run. Longer preratio pausing occurred in the larger fixed-ratio component in the conditions in which the mirror was present, whether or not any mirror pecks were recorded.     

EFFECTS OF ALTERNATIVE REINFORCEMENT: DOES THE SOURCE MATTER?1

In a chamber with a single response key, pigeon's key pecks were reinforced with food according to a variable-interval schedule. In addition, extra reinforcements occurred concurrently according to an independent schedule. In one condition, availability of the extra reinforcements was signalled by a change in key color from white to red. The extra reinforcements occurred after a peck on the red key. In a second condition, the extra reinforcements were unsignalled and occurred only after a 2-sec pause in pecking for one group of subjects and were unsignalled and occurred freely as scheduled for another group of subjects. In the first two conditions, duration of reinforcement was varied. A third condition duplicated the second but varied rate rather than duration of reinforcement. The rate of pecking varied inversely with the amount of extra reinforcement per unit time according to the same function, regardless of the condition regulating occurrence of the extra reinforcements, and regardless of whether or not a 2-sec pause was required for their occurrence. The shape of this function was predicted by Herrnstein's (1970) matching law.     

Comparisons between one-key and two-key versions of the sinewave schedule for pigeons

When the rate of reinforcement for pigeons' key pecking varied over time following a sine waveform, performances were more consistent and reliable if a constant-rate reinforcement schedule was concurrently available on a second key than if only the sinewave-varying reinforcement schedule was available. In the two-key version, response rates clearly followed varying reinforcement rates with the same frequency, with no phase lag, and without breaks. In both versions, pecking rate was a power function of reinforcement rate. Sinewave-schedule performance waveforms qualified for engineering methods of frequency analysis and met criteria for a standard measurement system.     

Preference for mixed versus constant delay of reinforcement

Preference for constant and mixed delay of reinforcement was studied using concurrent equal variable-interval schedules. For four pigeons, pecking one key was reinforced following constant delays of 8 sec and mixed delays of 6 or 10 and 2 or 14 sec. Pecking a second key was reinforced following constant delays of 0, 8, 16, and 32 sec. For two additional pigeons, pecking one key was reinforced following delays of 30, 15 or 45, 5 or 55, and 0 or 60 sec. Reinforcements on the other key were delayed 30 sec. It was found that (a) pigeons preferred mixed relative to constant delay of reinforcement, and (b) preference for mixed delay of reinforcement increased as the range of delay interval variability increased.     

Choice performance in several concurrent key-peck treadle-press reinforcement schedules

Five pigeons were exposed to several concurrent variable-interval food reinforcement schedules. For three subjects, one component of the schedule required a key-pecking response, the other a treadle-pressing response. For the other two subjects, both schedule components required treadle-pressing responses. The relative probability of reinforcement associated with the manipulanda was varied from 0 to 1.0 in 13 experimental conditions for the Key-Treadle subjects and nine conditions for the Treadle-Treadle subjects. The results indicated that the logarithms of relative time spent responding, and the logarithms of relative number of responses emitted on a manipulandum, approximated direct linear functions of logarithms of the relative frequencies of reinforcement associated with that manipulandum. No systematic bias in favor of time spent key pecking over time spent treadle pressing was apparent for the Key-Treadle subjects. All subjects exhibited undermatching, in that the ratios of time and response allocation at the alternatives systematically differed from the ratios of reinforcers obtained from the alternatives in the direction of indifference. Key pecking appeared to have no special link to food beyond treadle pressing or what would be expected on the basis of the reinforcement dependencies alone.     

Conjunctive schedules of reinforcement: III. A fixed-interval adjusting fixed-ratio schedule

Key pecking of three pigeons was studied under a conjunctive schedule that specified both a fixed-interval and an adjusting fixed-ratio requirement. The fixed-interval schedule was 6 min for one pigeon and 3 min for the other two. The size of the ratio requirement was determined within each cycle of the fixed interval by the duration of the pause before responding began. The fixed-ratio value was at maximum at the start of each fixed interval and decreased linearly until the first response occurred (adjusting fixed-ratio schedule). A peck produced food when the number of responses remaining on the fixed-ratio schedule was completed and when the fixed interval had elapsed. If no response occurred during the interval, the fixed-ratio requirement decreased to one and a single response after the interval elapsed produced food. The initial value of the adjusting fixed-ratio schedule was studied over a range of 0 to 900. Increases in the adjusting fixed-ratio schedule to about 300 responses increased both pause duration and running response rate and also modified the pattern of responding from that obtained under the fixed-interval schedule. Higher values of the adjusting fixed ratio generally decreased pause duration and running response rate and also disrupted responding. Interreinforcement time under the conjunctive schedule was increased substantially when the adjusting fixed-ratio size exceeded 300 responses.     

The role of intermittent food in the induction of attack in pigeons

The present experiments evaluated whether transitions in reinforcer probability are necessary to induce attack in pigeons. In Experiment I, three of six pigeons exposed to response-contingent constant-probability food schedules and a photograph of a conspecific as a target exhibited sustained postreinforcement attack on the target. The postreinforcement pattern of attack developed over the course of the experiment and was accompanied by a reduction in the rate of postreinforcement key pecking and an increase in the postreinforcement pause in key pecking. These effects on key pecking resulted in unprogrammed variations in the probability of reinforcement which may have been responsible for the induction of attack. In Experiment II, the attack-inducing properties of a constant-probability response-independent food schedule were compared to a periodic food schedule matched for overall rate of food delivery and to a no-food condition. In addition to attack, the spatial location of the subjects was monitored during each interfood interval. The periodic and aperiodic food schedules generated very different patterns of spatial location. Postfood attack was induced by both food schedules, although the constant-probability schedule induced attack in fewer birds. The no-food condition was not effective in inducing attack in any birds. These experiments indicate that intermittent food schedules without reductions in reinforcer probability are sufficient to induce attack in some pigeons, although not as effective as schedules with transitions in reinforcer probability.     

The role of discriminative stimuli in concurrent performances

Key pecking in pigeons was examined under concurrent and parallel arrangements of two independent and simultaneously available variable-interval schedules. Pecks on the changeover key alternated the schedule of reinforcement for responses on the main key. Under concurrent schedules, discriminative stimuli were paired with the reinforcement schedule arranged in each component and changeover responses also alternated these stimuli. Under parallel schedules, changeover responses alternated the effective reinforcement schedule, but did not change the discriminative stimulus. On concurrent procedures, changeover response rate was inversely related to the difference in reinforcement rate between the two components, whereas on parallel schedules no consistent relationship was found. With both schedules, absolute response and reinforcement rates were positively related, although for a given set of reinforcement frequencies, rates were often higher on the concurrent schedules. On concurrent schedules, relative response rates and relative times were equal to relative reinforcement rates. On parallel schedules these ratios were positively related, but response and time ratios were much smaller than were obtained with comparable concurrent schedules. This inequality was most pronounced when absolute reinforcement frequencies were lowest.     

Temporal control of behavior: schedule interactions

In Experiment I the response that terminated the postreinforcement pauses occurring under a fixed-interval 60-second schedule was reinforced, if the pause duration exceeded 30 seconds. The percentage of such pauses, rather than increasing, decreased. There were complex effects on the discriminative control of the pause by the reinforcer terminating the previous fixed interval, depending on whether the fixed interval and the added reinforcer were the same or different. In Experiments II(a) and II(b), each reinforcement initiated an alternative fixed-interval interresponse-time-greater-than-t-sec schedule, the schedule values being systematically varied. When the response following a pause exceeding a given duration was reinforced, fewer such pauses occurred than when they were not reinforced, i.e., on the comparable simple fixed-interval schedule. There was no systematic relationship between mean interrinforcement interval and duration of the postreinforcement pause. The pause duration initiated by reinforcement was directly related to the dependency controlling the shortest pause at that time, regardless of changes in mean interreinforcement interval.     

Positive reinforcement and the elimination of reinforced responses

Key pecking was maintained on a fixed-interval schedule while either a differential-reinforcement-of-not-responding or a fixed-time schedule was imposed simultaneously. The lower the time parameter of the not-responding schedule, the lower was the response rate. Similar effects occurred with the fixed-time schedule, if the pigeons had experience with reinforcement for not responding. Otherwise the effects were less orderly, to the extent that rate could reach maximum with the lowest-valued fixed-time schedule. The not-responding and the response-independent schedules had similar effects on rate in experienced pigeons only when the time parameter or nominal frequency of food presentation was considered. When considered in terms of obtained frequency of food presentation, reinforcement of not responding produced larger decrements in rate than did the fixed-time schedule.     

Preference for signalled reinforcement

Key pecking was reinforced on a two-component multiple schedule. A variable-interval schedule controlled reinforcement in both components. During one component, access to reinforcement was preceded by a tone; in the other component, a standard unsignalled schedule was in effect. After performance stabilized, subjects were given a choice between the signalled and unsignalled schedules. They were placed in the chamber with the unsignalled schedule in effect on the right key. A single response on the left, or changeover, key produced the signalled schedule for 1 min. Both pigeons in Experiment I pecked the changeover key at a rate sufficient to remain under the signalled schedule for over 90% of the session. Removing and reintroducing the tone demonstrated that the changeover-key responses were due to the occurrence of the tone. In Experiment II, when pecking the changeover key produced the unsignalled schedule, pecking the changeover key declined. The results may be explained either in terms of Hendry's information hypothesis or as escape from an intermittent positive reinforcement schedule.     

Escape from rich‐to‐lean transitions: Stimulus change and timeout

Extended pausing during discriminable transitions from rich‐to‐lean conditions can be viewed as escape (i.e., rich‐to‐lean transitions function aversively). In the current experiments, pigeons’ key pecking was maintained by a multiple fixed‐ratio fixed‐ratio schedule of rich or lean reinforcers. Pigeons then were provided with another, explicit, mechanism of escape by changing the stimulus from the transition‐specific stimulus used in the multiple schedule to a mixed‐schedule stimulus (Experiment 1) or by producing a period of timeout in which the stimulus was turned off and the schedule was suspended (Experiment 2). Overall, escape was under joint control of past and upcoming reinforcer magnitudes, such that responses on the escape key were most likely during rich‐to‐lean transitions, and second‐most likely during lean‐to‐lean transitions. Even though pigeons pecked the escape key, they paused before doing so, and the latency to begin the fixed ratio (i.e., the pause) remained extended during rich‐to‐lean transitions. These findings suggest that although the stimulus associated with rich‐to‐lean transitions functioned aversively, pausing is more than simply escape responding from the stimulus.     

Within-session changes in responding during several simple schedules

Pigeons' key pecking was reinforced by food delivered by several fixed-interval, variable-ratio, and differential-reinforcement-of-low-rate schedules. Rate of responding, number of responses per reinforcer, length of postreinforcement pause, running response rate, and the time required to collect an available reinforcer changed systematically within sessions when the schedules provided high rates of reinforcement, but usually not when they provided low rates. These results suggest that the factors that produce within-session changes in responding are generally similar for different schedules of reinforcement. However, a separate factor may also contribute during variable-ratio schedules. The results question explanations for within-session changes that are related solely to the passage of time, to responding, and to one interpretation of attention. They support the idea that one or more factors related to reinforcement play a role.