Effects of response rate, reinforcement frequency, and the duration of a stimulus preceding response-independent food

Food-reinforced key pecking in the pigeon was maintained under a four-component multiple schedule. In two components, responding was maintained at high rates under a random-ratio schedule. In the other two components, responding was maintained at low rates under a schedule that specified a minimum interresponse time. For both high and low response rates, one of the schedule components was associated with a high reinforcement frequency and the other components with a lower reinforcement frequency. During performance under these schedules, a stimulus terminated by access to response-independent food was periodically presented. The duration of this pre-food stimulus was 5, 30, 60, or 120 sec. Changes in rate of key pecking during the pre-food stimulus were systematically related to baseline response rate and the duration of the stimulus. Both high and low response rates were increased during the 5-sec stimulus. At longer stimulus durations, low response rates were unaffected and high response rates were decreased during the stimulus. For two of three pigeons, high response rates maintained under a lower frequency of reinforcement tended to be decreased more than high response rates maintained under a higher reinforcement frequency. In general, the magnitude of decrease in high response rates was inversely related to the duration of the pre-food stimulus.     

Some limitations on behavioral contrast and induction during successive discrimination

A pigeon's rate of pecking on a red key, reinforced at a constant frequency, may be changed by increasing or decreasing the frequency of reinforcement of pecking on a successively presented green key. The changes in the rate of pecking on red, called interactions, are of two types: contrast, in which the changes in the rates of pecking on the two colors are in opposite directions; and, induction, in which the changes in the rates are in the same direction. In previous data, a change in the frequency of reinforcement associated with the green key produced a corresponding change in the rate of pecking the green key and an opposite change (contrast) in the rate of pecking on the red key. The present data suggest that the magnitude of contrast is very small if pecking on the red key is reinforced at a high enough frequency (about 40 reinforcements per hr in the present experiment). Also, given that interactions occur, induction rather than contrast may result from small changes in a low frequency of reinforcement associated with green.     

Matching, contrast, and equalizing in the concurrent lever-press responding of rats

Five rats pressed levers for food reinforces delivered by several concurrent variable-interval variable-interval schedules. The rate of reinforcement available for responding on one component schedule was held constant at 60 reinforcers per hour. The rate of reinforcement available for responding on the other schedule varied from 30 to 240 reinforcers per hour. The behavior of the rats resembled the behavior of pigeons pecking keys for food reinforcers. The ratio of the overall rates of responding emitted under, and the ratio of the time spent responding under, the two components of each concurrent schedule were approximately equal to the ratio of the overall rates of reinforcement obtained from the components. The overall rate of responding emitted under, and the time spent responding under, the variable component schedule varied directly with the overall rate of reinforcement from that schedule. The overall rate of responding emitted under, and the time spent responding under, the constant component schedule varied inversely with the overall rate of reinforcement obtained from the variable component. The local rates of responding emitted under, and the local rates of reinforcement obtained from, the two components did not differ consistently across subjects. But they were not exactly equal either.     

Alternative reinforcement effects on fixed-interval performance

Pigeons' key pecks were reinforced with food on a fixed-interval schedule. Food also was available at variable time periods either independently of responding or for not key pecking (a differential-reinforcement-of-other-behavior schedule). The latter condition arranged reinforcement following the first pause of t seconds after it became available according to a variable-time schedule. This schedule allowed separation of the effects of pause requirements ≤ five-seconds and reinforcement frequency. The time spent pausing increased as the duration of the pause required for reinforcement increased from 0 to 30 seconds and as the frequency of reinforcement for pausing increased from 0 to 2 reinforcers per minute. Key pecking was more evenly distributed within each fixed interval with shorter required pauses and with more frequent reinforcement for pausing. The results complement those obtained with other concurrent schedules in which the same operant response was reinforced in both components.     

Associative interaction: joint control of key pecking by stimulus-reinforcer and response-reinforcer relationships

The joint control of rate of key pecking in pigeons by stimulus-reinforcer and response-reinforcer relationships was studied in the context of a two-component multiple schedule of reinforcement. Food presentation was always associated with one component and extinction with the other. The stimulus-reinforcer relationship was manipulated by varying the relative durations of the two components. In the food-presentation component, a fixed rate of reinforcement, independent of rate of responding, was generated by a schedule referred to as “T*”. One aspect of the response-reinforcer relationship, contiguity, was manipulated by varying the percentage of delayed reinforcers. With the multiple T* extinction schedule, stimulus-reinforcer and response-reinforcer relationships could be varied independently of one another. Rate of key pecking was sensitive to manipulations of both relationships. However, significant differential effects due to either the stimulus-reinforcer or response-reinforcer relationship were obtained only when the other relationship was weak: stimulus-reinforcer and response-reinforcer relationships interacted in the joint control of responding.     

Response strength in multiple schedules

In several different experiments, pigeons were trained with one schedule or condition of food reinforcement for pecking in the presence of one key color, and a different schedule or condition in the presence of a second key color. After responding in both of these multiple schedule components stabilized, response-independent food was presented during dark-key periods between components, and the rates of pecking in both schedule components decreased. The decrease in responding relative to baseline depended on the frequency, magnitude, delay, or response-rate contingencies of reinforcement prevailing in that component. When reinforcement was terminated, decreases in responding relative to baseline rates were ordered in the same way as with response-independent food. The relations between component response rates were power functions. Internal consistencies in the data, in conjunction with parallel findings in the literature, suggest that the concept of response strength summarizes the effects of diverse procedures, where response strength is identified with relative resistance to change. The exponent of the power function relating response rates may provide the basis for scaling response strength.     

Economic and biological influences on key pecking and treadle pressing in pigeons

Pigeons were studied on a two-component multiple schedule in which the required operant was, in different conditions, biologically relevant (i.e., key pecking) or nonbiologically relevant (i.e., treadle pressing). Responding was reinforced on a variable-interval (VI) 2-min schedule in both components. In separate phases, additional food was delivered on a variable-time (VT) 15-s schedule (response independent) or a VI 15-s schedule (response dependent) in one of the components. The addition of response-independent food had different effects on responding depending on the operant response and on the frequency with which the components alternated. When components alternated frequently (every 10 s), all pigeons keypecked at a much higher rate during the component with the additional food deliveries, whether response dependent or independent. In comparison, treadle pressing was elevated only when the additional food was response dependent; rate of treadling was lower when the additional food was response independent. When components alternated infrequently (every 20 min), pigeons key pecked at high rates at points of transition into the component with the additional food deliveries. Rate of key pecking decreased with time spent in the 20-min component when the additional food was response independent, whereas rate of pecking remained elevated in that component when the additional food was response dependent. Under otherwise identical test conditions, rate of treadle pressing varied only as a function of its relative rate of response-dependent reinforcement. Delivery of response-independent food thus had different, but predictable, effects on responding depending on which operant was being studied, suggesting that animal-learning procedures can be integrated with biological considerations without the need to propose constraints that limit general laws of learning.     

The role of discriminative stimuli in concurrent performances

Key pecking in pigeons was examined under concurrent and parallel arrangements of two independent and simultaneously available variable-interval schedules. Pecks on the changeover key alternated the schedule of reinforcement for responses on the main key. Under concurrent schedules, discriminative stimuli were paired with the reinforcement schedule arranged in each component and changeover responses also alternated these stimuli. Under parallel schedules, changeover responses alternated the effective reinforcement schedule, but did not change the discriminative stimulus. On concurrent procedures, changeover response rate was inversely related to the difference in reinforcement rate between the two components, whereas on parallel schedules no consistent relationship was found. With both schedules, absolute response and reinforcement rates were positively related, although for a given set of reinforcement frequencies, rates were often higher on the concurrent schedules. On concurrent schedules, relative response rates and relative times were equal to relative reinforcement rates. On parallel schedules these ratios were positively related, but response and time ratios were much smaller than were obtained with comparable concurrent schedules. This inequality was most pronounced when absolute reinforcement frequencies were lowest.     

Choice between response units: The rate constancy model

In a conjoint schedule, reinforcement is available simultaneously on two or more schedules for the same response. The present experiments provided food for key pecking on both a random-interval and a differential-reinforcement-of-low-rate (DRL) schedule. Experiment 1 involved ordinary DRL schedules; Experiment 2 added an external stimulus to indicate when the required interresponse time had elapsed. In both experiments, the potential reinforcer frequency from each component was varied by means of a second-order fixed-ratio schedule, and the DRL time parameter was changed as well. Response rates were described by a model stating that time allocation to each component matches the relative frequency of reinforcement for that component. When spending time in a given component, the subject is assumed to respond at the rate characteristic of baseline performance. This model appeared preferable to the absolute-rate version of the matching law. The model was shown to be applicable to multiple-response concurrent schedules as well as to conjoint schedules, and it described some of the necessary conditions for response matching, undermatching, and bias. In addition, the pigeons did not optimize reinforcer frequency.     

Key pecking of pigeons under variable-interval schedules of briefly signaled delayed reinforcement: effects of variable-interval value.

Key pecking of 4 pigeons was maintained under a multiple variable-interval 20-s variable-interval 120-s schedule of food reinforcement. When rates of key pecking were stable, a 5-s unsignaled, nonresetting delay to reinforcement separated the first peck after an interval elapsed from reinforcement in both components. Rates of pecking decreased substantially in both components. When rates were stable, the situation was changed such that the peck that began the 5-s delay also changed the color of the keylight for 0.5 s (i.e., the delay was briefly signaled). Rates increased to near-immediate reinforcement levels. In subsequent conditions, delays of 10 and 20 s, still briefly signaled, were tested. Although rates of key pecking during the component with the variable-interval 120-s schedule did not change appreciably across conditions, rates during the variable-interval 20-s component decreased greatly in 1 pigeon at the 10-s delay and decreased in all pigeons at the 20-s delay. In a control condition, the variable-interval 20-s schedule with 20-s delays was changed to a variable-interval 35-s schedule with 5-s delays, thus equating nominal rates of reinforcement. Rates of pecking increased to baseline levels. Rates of pecking, then, depended on the value of the briefly signaled delay relative to the programmed interfood times, rather than on the absolute delay value. These results are discussed in terms of similar findings in the literature on conditioned reinforcement, delayed matching to sample, and classical conditioning.     

A test of the negative discriminative stimulus as a reinforcer of observing

Five pigeons were used to test the hypothesis that the source of reinforcement for observing behavior is the information that it provides concerning the schedule of primary reinforcement. On a variable-interval schedule, pecking the left-hand key produced a 30-sec display of such information. During this 30-sec period, when pecking the right-hand key was reinforced on a random-interval schedule, both keys were green; when no reinforcement was scheduled (extinction) both keys were red. Later, this baseline procedure, in which both red and green were available, was replaced for blocks of sessions by procedures in which either (a) the red was eliminated and only the green could be produced; or (b) the green was eliminated and only the red could be produced. The results were that green maintained rates of pecking on the left key that were as high or higher than when both colors were available and that red maintained no responding. It was concluded that the reinforcing value of a stimulus depends on the positive or negative direction of its correlation with primary reinforcement, rather than upon the amount of information that it conveys.     

Changeover behavior and preference in concurrent schedules

Pigeons were trained on a multiple schedule of reinforcement in which separate concurrent schedules occurred in each of two components. Key pecking was reinforced with milo. During one component, a variable-interval 40-s schedule was concurrent with a variable-interval 20-s schedule; during the other component, a variable-interval 40-s schedule was concurrent with a variable-interval 80-s schedule. During probe tests, the stimuli correlated with the two variable-interval 40-s schedules were presented simultaneously to assess preference, measured by the relative response rates to the two stimuli. In Experiment 1, the concurrently available variable-interval 20-s schedule operated normally; that is, reinforcer availability was not signaled. Following this baseline training, relative response rate during the probes favored the variable-interval 40-s alternative that had been paired with the lower valued schedule (i.e., with the variable-interval 80-s schedule). In Experiment 2, a signal for reinforcer availability was added to the high-value alternative (i.e., to the variable-interval 20-s schedule), thus reducing the rate of key pecking maintained by that schedule but leaving the reinforcement rate unchanged. Following that baseline training, relative response rates during probes favored the variable-interval 40-s alternative that had been paired with the higher valued schedule. The reversal in the pattern of preference implies that the pattern of changeover behavior established during training, and not reinforcement rate, determined the preference patterns obtained on the probe tests.     

Elimination of reinforced behavior: intermittent schedules of not-responding

Pigeons' key pecking resulted in food according to either a variable-ratio or a variable-interval schedule. At the same time, food was available for not pecking for a specified time. The required time of not-pecking was segmented into not-responding units, and these units were followed by food according to a fixed-ratio schedule. Both unit duration and the number required were varied. In general, the shorter the time unit or the smaller the ratio, the lower was response rate. When total required not-responding time was constant, but changes in unit duration and the number required altered how the total was achieved, shorter units produced lower rates. Other conditions involved substitution of food delivered independent of responding for the not-responding schedule. With low and moderate total times to food presentation, the not-responding schedule produced lower rates; with the longest times, the response-independent schedule generated less responding. When considered in terms of relative frequency of food presentation available from a source other than pecking, the not-responding schedule reduced rate more effectively than did the response-independent schedule. Comparisons with other research suggested that food presented dependent on not responding compared favorably with punishment as a procedure for reducing response rate. Transient effects differed. Although punishment temporarily depresses rate when first imposed and temporarily enhances it when first removed, food given for not responding quickly generated steady-state rates.     

Punishment of observing by the negative discriminative stimulus

To determine the effect of a negative discriminative stimulus on the response producing it, two pigeons were each studied in a three-key conditioning chamber. During alternating periods of unpredictable duration, pecking the center (food) key either was reinforced with grain on a variable-interval schedule or was never reinforced. On equal but independent variable-interval schedules, pecking either of the side (observing) keys changed the color of all keys for 30 sec from yellow to either green or red. When the schedule on the center key was variable-interval reinforcement, the color was green (positive discriminative stimulus); when no reinforcements were scheduled, the color was red (negative discriminative stimulus). Since pecking the side keys did not affect grain deliveries, changes in the rate of pecking could not be ascribed to changes in the frequency of primary reinforcement. In subsequent sessions, red was withheld as one of the possible consequences of pecking a given side key. When red was omitted, the rate on that key increased, and when red was restored, the rate decreased. It was concluded that red illumination of the keys, the negative discriminative stimulus, had a suppressive effect on the response that produced it.     

Concurrent performances: a baseline for the study of reinforcement magnitude

When a pigeon's pecking on a single key was reinforced by a variable-interval (VI) schedule of reinforcement, the rate of pecking was insensitive to changes in the duration of reinforcement from 3 to 6 sec. When, however, the pigeon's pecking on each of two keys was concurrently reinforced by two independent VI schedules, one for each key, the rate of pecking was directly proportional to the duration of reinforcement.     

Preference for signalled reinforcement

Key pecking was reinforced on a two-component multiple schedule. A variable-interval schedule controlled reinforcement in both components. During one component, access to reinforcement was preceded by a tone; in the other component, a standard unsignalled schedule was in effect. After performance stabilized, subjects were given a choice between the signalled and unsignalled schedules. They were placed in the chamber with the unsignalled schedule in effect on the right key. A single response on the left, or changeover, key produced the signalled schedule for 1 min. Both pigeons in Experiment I pecked the changeover key at a rate sufficient to remain under the signalled schedule for over 90% of the session. Removing and reintroducing the tone demonstrated that the changeover-key responses were due to the occurrence of the tone. In Experiment II, when pecking the changeover key produced the unsignalled schedule, pecking the changeover key declined. The results may be explained either in terms of Hendry's information hypothesis or as escape from an intermittent positive reinforcement schedule.     

On conditioned reinforcing effects of negative discriminative stimuli

Observing responses by pigeons were studied during sessions in which a food key and an observing key were available continuously. A variable-interval schedule and extinction alternated randomly on the food key. In one condition, food-key pecking during extinction decreased reinforcement frequency during the next variable-interval component, and in the other condition such pecking did not affect reinforcement frequency. Observing responses either changed both keylight colors from white to green (S+) or to red (S−) depending on the condition on the food key, or the observing responses never produced the S+ but produced the S− when extinction was in effect on the food key. Observing responses that produced only S− were maintained only when food-key pecking during extinction decreased reinforcement frequency in the subsequent variable-interval component. The red light conformed to conventional definitions of a negative discriminative stimulus, rendering results counter to previous findings that production of S− alone does not maintain observing. Rather than offering support for an informational account of conditioned reinforcement, the results are discussed in terms of a molar analysis to account for how stimuli acquire response-maintaining properties.     

Control rate of response or reinforcement and amphetamine's effect on behavior.

The roles of control response rate and reinforcement frequency in producing amphetamine's effect on operant behavior were evaluated independently in rats. Two multiple schedules were arranged in which one variable, either response rate or reinforcement frequency, was held constant and the other variable manipulated. A multiple differential-reinforcement-of-low-rate seven-second yoked variable-interval schedule was used to equate reinforcement frequencies at different control response rates between multiple-schedule components. Amphetamine increased responding under the variable-interval component. In contrast, amphetamine decreased responding equivalently between components of a multiple random-ratio schedule that produced similar control response rates at different reinforcement frequencies. The results provide experimental support to the rate-dependency principle that control rate of responding is an important determinant of amphetamine's effect on operant behavior.     

Preference for mixed- versus fixed-ratio schedules

Pigeons' pecks on one key produced a stimulus correlated with a mixed-ratio schedule of food reinforcement. Pecks on a second key produced a stimulus correlated with a fixed-ratio schedule. When the arithmetic mean of the mixed ratios equaled the fixed ratio, the former stimulus maintained a higher rate of pecking. When the fixed ratio was sufficiently smaller, preference shifted to it. The pigeons' relative preference for the schedules could be described by comparing the geometric mean of the reinforcement rates in the several mixed-ratio components with the reinforcement rates in the fixed-ratio components.     

Interaction of methadone, reinforcement history, and variable-interval performance.

In the present study, we examined how a reinforcement schedule history that generated high or low rates of responding influenced the effects of acute (Experiment 1) and chronic (Experiment 2) methadone administration. Initially, key-peck responses of pigeons were maintained under a variable-interval 90-s schedule of food presentation, and a methadone dose-response curve was determined with doses of 0.6, 1.2, and 2.4 mg/kg. The pigeons were then exposed, for at least 40 sessions, to either a fixed-ratio 50 schedule or a differential-reinforcement-of-low-rate 10-s schedule, or were given continued exposure to the variable-interval schedule. The methadone dose-response curve was redetermined after all pigeons again were responding under the variable-interval schedule. The effects of two different daily methadone doses (9.0 and 12.0 mg/kg/day) and withdrawal precipitated by naloxone also were assessed. Experience with a fixed-ratio or differential reinforcement of low rate schedule did not result in significantly different response rates under the variable-interval schedule and, in general, the acute effects of methadone did not have differential effects correlated with schedule history. However, for 2 of 4 subjects the rate-decreasing effects of methadone on rates of key pecking were greater following a history of low-rate responding, suggesting a possible interaction between schedule history and effects of methadone. Daily methadone administration under the variable-interval schedule revealed that pigeons with experience under the differential reinforcement of low rate schedule developed more rapid and complete tolerance to the rate-decreasing effects of methadone. Three of the 4 subjects in this group showed rate increases above drug-free baselines during chronic methadone dosing. Pigeons with a history of fixed-ratio responding also developed tolerance to the rate-decreasing effects of methadone but without the subsequent rate increases seen by subjects with low-rate histories. No subjects with variable-interval histories showed complete recovery of drug-free baselines, suggesting that interpolated training under other schedules may attenuate the rate-altering effects of chronically administered drugs. Naloxone (1.0 mg/kg), administered during the chronic methadone phase, resulted in greater disruption of responding by pigeons with a history of low-rate responding, as compared to subjects in the other two groups.(ABSTRACT TRUNCATED AT 400 WORDS)