Effects of response rate, reinforcement frequency, and the duration of a stimulus preceding response-independent food

Food-reinforced key pecking in the pigeon was maintained under a four-component multiple schedule. In two components, responding was maintained at high rates under a random-ratio schedule. In the other two components, responding was maintained at low rates under a schedule that specified a minimum interresponse time. For both high and low response rates, one of the schedule components was associated with a high reinforcement frequency and the other components with a lower reinforcement frequency. During performance under these schedules, a stimulus terminated by access to response-independent food was periodically presented. The duration of this pre-food stimulus was 5, 30, 60, or 120 sec. Changes in rate of key pecking during the pre-food stimulus were systematically related to baseline response rate and the duration of the stimulus. Both high and low response rates were increased during the 5-sec stimulus. At longer stimulus durations, low response rates were unaffected and high response rates were decreased during the stimulus. For two of three pigeons, high response rates maintained under a lower frequency of reinforcement tended to be decreased more than high response rates maintained under a higher reinforcement frequency. In general, the magnitude of decrease in high response rates was inversely related to the duration of the pre-food stimulus.     

Conditioned suppression or facilitation as a function of the behavioral baseline

Rats were exposed to a multiple schedule of reinforcement. During one component, a bar-press was followed by reinforcement only if it occurred between 15 and 20 sec after the previous response. This differential-reinforcement-of-low-rate (DRL) schedule produced a typical slow rate of responding. During the other component, reinforcement followed the first response to be emitted during limited periods of time which occurred at fixed intervals. These fixed-interval schedules with a limited hold produced higher response rates, described as `interval' or `ratio-like' behavior. Responding during the DRL component increased in frequency during a tone which ended with an unavoidable shock of low intensity, but decreased during the tone when the shock intensity was raised. The `interval' and `ratio-like' responding decreased in frequency during the tone at all shock intensities. Initial acceleration of the DRL responding appeared to be due to adventitious punishment of collateral behavior which was observed between the bar-presses. The more severe conditioned suppression during the fixed-interval components might be the result of the lower probability of reinforcement after any single response.     

Multiple determinants of the effects of reinforcement magnitude on free-operant response rates

Four experiments examined the effects of increasing the number of food pellets given to hungry rats for a lever-press response. On a simple variable-interval 60-s schedule, increased number of pellets depressed response rates (Experiment 1). In Experiment 2, the decrease in response rate as a function of increased reinforcement magnitude was demonstrated on a variable-interval 30-s schedule, but enhanced rates of response were obtained with the same increase in reinforcement magnitude on a variable-ratio 30 schedule. In Experiment 3, higher rates of responding were maintained by the component of a concurrent variable-interval 60-s variable-interval 60-s schedule associated with a higher reinforcement magnitude. In Experiment 4, higher rates of response were produced in the component of a multiple variable-interval 60-s variable-interval 60-s schedule associated with the higher reinforcement magnitude. It is suggested that on simple schedules greater reinforcer magnitudes shape the reinforced pattern of responding more effectively than do smaller reinforcement magnitudes. This effect is, however, overridden by another process, such a contrast, when two magnitudes are presented within a single session on two-component schedules.     

Rate of conditioned reinforcement affects observing rate but not resistance to change

The effects of rate of conditioned reinforcement on the resistance to change of operant behavior have not been examined. In addition, the effects of rate of conditioned reinforcement on the rate of observing have not been adequately examined. In two experiments, a multiple schedule of observing-response procedures was used to examine the effects of rate of conditioned reinforcement on observing rates and resistance to change. In a rich component, observing responses produced a higher frequency of stimuli correlated with alternating periods of random-interval schedule primary reinforcement or extinction. In a lean component, observing responses produced similar schedule-correlated stimuli but at a lower frequency. The rate of primary reinforcement in both components was the same. In Experiment 1, a 4:1 ratio of stimulus production was arranged by the rich and lean components. In Experiment 2, the ratio of stimulus production rates was increased to 6:1. In both experiments, observing rates were higher in the rich component than in the lean component. Disruptions in observing produced by presession feeding, extinction of observing responses, and response-independent food deliveries during intercomponent intervals usually were similar in the rich and lean components. When differences in resistance to change did occur, observing tended to be more resistant to change in the lean component. If resistance to change is accepted as a more appropriate measure of response strength than absolute response rates, then the present results provide no evidence that higher rates of stimuli generally considered to function as conditioned reinforcers engender greater response strength.     

Arousal, changeover responses, and preference in concurrent schedules

Pigeons were trained on multiple schedules that provided concurrent reinforcement in each of two components. In Experiment 1, one component consisted of a variable-interval (VI) 40-s schedule presented with a VI 20-s schedule, and the other a VI 40-s schedule presented with a VI 80-s schedule. After extended training, probe tests measured preference between the stimuli associated with the two 40-s schedules. Probe tests replicated the results of Belke (1992) that showed preference for the 40-s schedule that had been paired with the 80-s schedule. In a second condition, the overall reinforcer rate provided by the two components was equated by adding a signaled VI schedule to the component with the lower reinforcer rate. Probe results were unchanged. In Experiment 2, pigeons were trained on alternating concurrent VI 30-s VI 60-s schedules. One schedule provided 2-s access to food and the other provided 6-s access. The larger reinforcer magnitude produced higher response rates and was preferred on probe trials. Rate of changeover responding, however, did not differ as a function of reinforcer magnitude. The present results demonstrate that preference on probe trials is not a simple reflection of the pattern of changeover behavior established during training.     

Response persistence under variable-time schedules following immediate and unsignalled delayed reinforcement

Key pecking of three pigeons was maintained in separate components of a multiple schedule by either immediate reinforcement (i.e., tandem variable-time fixed-interval schedule) or unsignalled delayed reinforcement (i.e., tandem variable-interval fixed-time schedule). The relative rate of food delivery was equal across components, and this absolute rate differed across conditions. Immediate reinforcement always generated higher response rates than did unsignalled delayed reinforcement. Then, variable-time schedules of food delivery replaced the contingencies just described such that food was delivered at the same rate but independently of responding. In most cases, response rates decreased to near-zero levels. In addition, response persistence was not systematically different between multiple-schedule components across pigeons. The implications of the results for the concepts of response strength and the response-reinforcer relation are noted.     

Preference for signalled reinforcement

Key pecking was reinforced on a two-component multiple schedule. A variable-interval schedule controlled reinforcement in both components. During one component, access to reinforcement was preceded by a tone; in the other component, a standard unsignalled schedule was in effect. After performance stabilized, subjects were given a choice between the signalled and unsignalled schedules. They were placed in the chamber with the unsignalled schedule in effect on the right key. A single response on the left, or changeover, key produced the signalled schedule for 1 min. Both pigeons in Experiment I pecked the changeover key at a rate sufficient to remain under the signalled schedule for over 90% of the session. Removing and reintroducing the tone demonstrated that the changeover-key responses were due to the occurrence of the tone. In Experiment II, when pecking the changeover key produced the unsignalled schedule, pecking the changeover key declined. The results may be explained either in terms of Hendry's information hypothesis or as escape from an intermittent positive reinforcement schedule.     

Response rate, reinforcement frequency, and behavioral contrast

Pigeons responded for food on a multiple schedule in which periods of green-key illumination alternated with periods of red-key illumination. When behavior had stabilized with a variable-interval 2-min schedule of reinforcement operating during both stimuli, low rates of responding (interresponse times greater than 2 sec) were differentially reinforced during the green component. Conditions during the red stimulus were unchanged. Response rates during the green component fell without changing the frequency of reinforcement but there were no unequivocal contrast effects during the red stimulus. The frequency of reinforcement during the green component was then reduced by changing to a variable-interval 8-min schedule without reducing the response rates in that component, which were held at a low level by the spacing requirement. Again, the conditions during the red stimulus were unchanged but response rates during that stimulus increased. These results show that reductions in reinforcement frequency, independently of response rate, can produce interactions in multiple schedules.     

Control rate of response or reinforcement and amphetamine's effect on behavior.

The roles of control response rate and reinforcement frequency in producing amphetamine's effect on operant behavior were evaluated independently in rats. Two multiple schedules were arranged in which one variable, either response rate or reinforcement frequency, was held constant and the other variable manipulated. A multiple differential-reinforcement-of-low-rate seven-second yoked variable-interval schedule was used to equate reinforcement frequencies at different control response rates between multiple-schedule components. Amphetamine increased responding under the variable-interval component. In contrast, amphetamine decreased responding equivalently between components of a multiple random-ratio schedule that produced similar control response rates at different reinforcement frequencies. The results provide experimental support to the rate-dependency principle that control rate of responding is an important determinant of amphetamine's effect on operant behavior.     

The roles of stimulus control and reinforcement frequency in modulating the behavioral effects of d-amphetamine in the rat.

The behavioral effects of d-amphetamine have been shown to be modulated by stimulus control, with less impairment of performance occurring when control is great. When the fixed-consecutive-number schedule is used (on which at least a specified consecutive number of responses must be made on one operandum before a single response on another will produce a reinforcer), response rate tends to be invariant but reinforcement frequency is not. This study asks whether the differences in reinforcement frequency that usually accompany changes in stimulus control could themselves be responsible for the performance differences. Two versions of the fixed-consecutive-number schedule of reinforcement were combined into a multiple schedule within which stimulus control was varied but differences in reinforcement frequency were minimized by omitting some reinforcer deliveries during the component that usually had the higher reinforcement frequency. In one component, a compound discriminative stimulus was added with the eighth consecutive response on the first lever; a single response on the second lever was then reinforced. In the other component, no such stimulus was presented. With no added stimulus, large decreases occurred in the number of runs satisfying the minimum requirement for reinforcement at doses of drug that produced only minimal changes when an added stimulus controlled behavior. Thus, increased stimulus control diminishes the behavioral changes produced by d-amphetamine even when the possible contribution by baseline reinforcement rate is minimized.     

Conditioned reinforcement value and resistance to change

Three experiments examined the effects of conditioned reinforcement value and primary reinforcement rate on resistance to change using a multiple schedule of observing-response procedures with pigeons. In the absence of observing responses in both components, unsignaled periods of variable-interval (VI) schedule food reinforcement alternated with extinction. Observing responses in both components intermittently produced 15 s of a stimulus associated with the VI schedule (i.e., S+). In the first experiment, a lower-valued conditioned reinforcer and a higher rate of primary reinforcement were arranged in one component by adding response-independent food deliveries uncorrelated with S+. In the second experiment, one component arranged a lower valued conditioned reinforcer but a higher rate of primary reinforcement by increasing the probability of VI schedule periods relative to extinction periods. In the third experiment, the two observing-response components provided similar rates of primary reinforcement but arranged different valued conditioned reinforcers. Across the three experiments, observing-response rates were typically higher in the component associated with the higher valued conditioned reinforcer. Resistance to change was not affected by conditioned reinforcement value, but was an orderly function of the rate of primary reinforcement obtained in the two components. One interpretation of these results is that S+ value does not affect response strength and that S+ deliveries increase response rates through a mechanism other than reinforcement. Alternatively, because resistance to change depends on the discriminative stimulus-reinforcer relation, the failure of S+ value to impact resistance to change could have resulted from a lack of transfer of S+ value to the broader discriminative context.     

Response discriminative and reinforcement factors in stimulus control of performance on multiple and chained schedules of reinforcement

Discriminative control of the response rates of two groups of rats was equated by training them to cease bar pressing in light-out no-tone ( + ) and to respond during tone and light. Multiple-schedule subjects received food at the same rate for responding during tone or light as for nonresponding in + . For the chained-schedule subjects responding in tone or light only produced + where food was received for nonresponding. In extinction tests multiple-schedule subjects emitted approximately twice the responses to tone-plus-light as to tone or light presented individually (additive summation). The rats trained on the chained schedule, in which the tone and light each controlled substantial response rates but were never paired with food, showed no summation when the tone and light were presented together. The results indicate that discriminative control of response rates and reinforcement differences between schedule components determine stimulus control.     

Homogeneous chains, heterogeneous chains, and delay of reinforcement

Three pigeons responded on two-component chain schedules in which the required response topography in the initial and terminal links was similar (a homogeneous chain) or dissimilar (a heterogeneous chain). Key-peck responding in the initial link under a variable-interval 60-second (VI 60) schedule produced a terminal link in which, in different conditions, either key pecking or foot treadling was reinforced according to a VI 60 schedule. Multiple VI 60 VI 60 schedules, in which the responses required in the chain schedules were maintained by primary reinforcement in the two components, preceded and followed each type of chain. These multiple schedules were used to ensure that both responses occurred reliably prior to introducing the chain schedule. Key-peck response rates in the initial link of the chain consistently were higher during the homogeneous chain than during the heterogeneous chain. These results illustrate that intervening events during a period separating an operant response from primary reinforcement influence that operant, independently of the delay between the response and reinforcement.     

Some effects of discriminative training with equated frequency of reinforcement

Pigeons were exposed to a multiple schedule which provided equally frequent reinforcement in the presence of two stimuli but which produced markedly different rates of key-pecking. Generalization gradients were displaced away from the stimulus associated with the lower rate of key-pecking. Another group of pigeons had similar training, except that a low rate of key-pecking was established in a stimulus with a much higher frequency of food reinforcement. In this case, the generalization gradients were not affected by the training on the schedule producing a low response rate.     

The effect of reinforcement differences on choice and response distribution during stimulus compounding

In Experiments I and II, rats were trained to respond on one lever during light and another during tone. The absence of tone and light controlled response cessation. In the multiple schedule of Experiment I, all reinforcements were received for responding in tone or light; in the chain schedule of Experiment II, all reinforcements were received in no tone + no light for not responding. Experiment I subjects, for which tone and light were associated with response and reinforcement increase, responded significantly more to tone-plus-light than to tone or light alone (additive summation). Experiment II subjects, for which tone and light were associated with response increase and reinforcement decrease, responded comparably to tone, light, and tone + light. Thus, additive summation was observed when stimulus-response and stimulus-reinforcer associations in tone and light were both positive, but not when they were conflicting. All subjects in both experiments responded predominantly on the light-correlated lever during tone + light, even when light intensity was reduced in testing. Furthermore, when a light was presented to a subject engaged in tone-associated responding, all subjects immediately switched the locus of responding to the light-correlated lever. No change in locus occurred when a tone was presented to a subject engaged in light-associated responding, irrespective of the stimulus-reinforcer association conditioned to tone. The light-lever preference in tone + light indicates that the heightened responding observed in Experiment I was not the summation of tone-associated behavior with light-associated behavior. Rather, it appears to be the result of a facilitation of one operant (light-associated responding) by the reinforcement-associated cue for the other.     

Unsignaled delay of reinforcement, relative time, and resistance to change

Two experiments with pigeons examined the effects of unsignaled, nonresetting delays of reinforcement on responding maintained by different reinforcement rates. In Experiment 1, 3-s unsignaled delays were introduced into each component of a multiple variable-interval (VI) 15-s VI 90-s VI 540-s schedule. When considered as a proportion of the preceding immediate reinforcement baseline, responding was decreased similarly for the three multiple-schedule components in both the first six and last six sessions of exposure to the delay. In addition, the relation between response rates and reinforcement rates was altered such that both parameters of the single-response version of the matching law (i.e., k and Re) were decreased. Experiment 2 examined the effects of unsignaled delays ranging from 0.5 s to 8.0 s on responding maintained by a multiple VI 20-s VI 120-s schedule of reinforcement. Response rates in both components increased with brief unsignaled delays and decreased with longer delays. As in Experiment 1, response rates as a proportion of baseline were affected similarly for the two components in both the first six and last six sessions of exposure to the delay. Unlike delays imposed between two stimulus events, the effects of delays between responses and reinforcers do not appear to be attenuated when the average time between reinforcers is longer. In addition, the disruptions produced by unsignaled delays appear to be inconsistent with the general finding that responding maintained by higher rates of reinforcement is less resistant to change.     

Intermittent punishment of human responding maintained by intermittent reinforcement

To determine the effects of variable-interval shock punishment on behavior maintained by variable-interval and variable-ratio reinforcement, human subjects' key-pressing behavior was reinforced with money on a four-component multiple schedule. Components 1 and 2 were variable-interval 30-sec, and Components 3 and 4 were variable-ratio 210. After responding was stabilized, response-contingent electric shock was scheduled on a variable-interval 10-sec schedule during the second and fourth components of each cycle. Subjects instructed as to the reinforcement contingencies showed gradually increasing suppression of variable-interval responding at increasing shock intensities and either very high or very low rates of variable-ratio responding at higher intensities. Minimally instructed subjects showed suppression at higher shock intensities, but no clear differential suppression as a function of reinforcement schedule. Recovery from initial suppression was observed within sessions.     

The role of discriminative stimuli in concurrent performances

Key pecking in pigeons was examined under concurrent and parallel arrangements of two independent and simultaneously available variable-interval schedules. Pecks on the changeover key alternated the schedule of reinforcement for responses on the main key. Under concurrent schedules, discriminative stimuli were paired with the reinforcement schedule arranged in each component and changeover responses also alternated these stimuli. Under parallel schedules, changeover responses alternated the effective reinforcement schedule, but did not change the discriminative stimulus. On concurrent procedures, changeover response rate was inversely related to the difference in reinforcement rate between the two components, whereas on parallel schedules no consistent relationship was found. With both schedules, absolute response and reinforcement rates were positively related, although for a given set of reinforcement frequencies, rates were often higher on the concurrent schedules. On concurrent schedules, relative response rates and relative times were equal to relative reinforcement rates. On parallel schedules these ratios were positively related, but response and time ratios were much smaller than were obtained with comparable concurrent schedules. This inequality was most pronounced when absolute reinforcement frequencies were lowest.     

Effects of reinforcement magnitude and ratio values on behaviour maintained by a cyclic ratio schedule of reinforcement

In Experiments 1 and 2, lever pressing by rats was reinforced on a cyclic ratio schedule of food reinforcement, comprising a repeated sequence of fixed-ratio component schedules. Reinforcement magnitude was varied, on occasional sessions in Experiment 1 and across blocks of sessions in Experiment 2, from one to two or three 45-mg food pellets. In the one-pellet condition, post-reinforcement pauses increased with component schedule value. At higher magnitudes, post-reinforcement pauses increased, and overall response rates declined. Response rate on component schedules was a decreasing linear function of the obtained rate of reinforcement in all conditions. Plotted against component schedule value, response rate increased exponentially to an asymptote that decreased when reinforcement magnitude increased. These findings are consistent with regulatory accounts of food reinforced behaviour. In Experiment 3, rats were trained under a cyclic ratio schedule comprising fixed-ratio components including higher values, and some inverted U-shaped response functions were obtained. Those rats that did not showthis relationship were trained on cyclic ratios with even higher values, and all showed inverted U-shaped response functions. This suggests that behaviour on cyclic ratio schedules can reflect activating of reinforcement as well as the satiating effects seen in Experiments 1 and 2.     

Behavioral aftereffects of reinforcement and its omission as a function of reinforcement magnitude

Rats responded on a multiple fixed-interval fixed-interval schedule of reinforcement. Each complete cycle of the multiple schedule was separated from the next by a relatively long period of timeout from all schedule contingencies. A response at the end of the second component of each cycle was always reinforced with an invariant reinforcement magnitude, while reinforcement magnitude and reinforcement omission were systematically varied in the first component. Response rate in the first component was a monotonic function of reinforcement magnitude in that component. These changes in response rate in the first component did not affect response rate in the second component. When reinforcement was omitted on 50% of occasions in the first component, following reinforcement there was a reduction in response rate in the second component that was monotonically related to reinforcement magnitude. Following reinforcement omission there was an increase in response rate in the second component that was unrelated to reinforcement magnitude. When reinforcement was omitted on 100% of occasions in the first component, behavioral contrast was observed.