Response persistence under ratio and interval reinforcement schedules

In Experiment 1, rats were exposed to progressive-ratio schedules of food reinforcement while other rats were exposed simultaneously to yoked-interval schedules that arranged equivalent interreinforcer intervals but required only a single response at the end of the interval for food delivery. In Experiment 2, a within-subject yoked-control procedure was employed in which pigeons were exposed to alternating sessions (one per day) of progressive-ratio schedules and yoked-interval schedules as described above. In both experiments, responding under the yoked-interval schedule persisted beyond the point at which responding under the progressive-ratio schedule had ceased. The progressive-ratio schedules controlled break-and-run distributions, and the yoked-interval schedules controlled more even distributions of responses in time. Response rates decreased and postreinforcement pauses increased over time within individual sessions under both schedules. The results suggest that responding maintained by interval schedules is more persistent than that maintained by ratio schedules. The limitations and implications of this conclusion are discussed in the context of other investigations of response strength and behavioral momentum.     

The effects of unsignalled delayed reinforcement

Pigeons' pecks were reinforced according to a variable-interval schedule. A delay-of-reinforcement procedure was then added to the schedule, or a yoked-control procedure was arranged where the reinforcers occurred independently of responding according to the same variable-interval schedule. During the delay-of-reinforcement procedure, the first peck after a reinforcer was scheduled began a delay timer and the reinforcer was delivered at the end of the interval. No stimulus change signalled the delay interval and responses could occur during it, so that the obtained delays were often shorter than those scheduled. Responding under this procedure was highly variable but, in general, behavior was substantially reduced even with the shortest delay used, 3 sec. In addition, the rates maintained by delayed reinforcement were only slightly greater than those maintained by the yoked-control procedure, suggesting that adventitious pairings of response and reinforcer were responsible for some of the maintenance of behavior that did occur. The results challenge recent conceptions of reinforcement as involving response-reinforcer correlations and re-emphasize the role of temporal proximity between response and reinforcer.     

Response duration is sensitive to both immediate and delayed reinforcement

We investigated the duration of lever pressing by rats when the delivery of appetitive reinforcers was contingent upon response duration. In the first experiment, response durations increased when duration requirements were imposed, and they decreased when duration requirements were removed. This effect occurred whether reinforcers were immediate or delayed by 8 s. In order to maintain the integrity of the delay intervals, reinforcer delivery was dependent upon both lever depression and release. In a second experiment, lever depression only and a response duration of at least 4 s were required for reinforcer delivery. Compared to immediate reinforcement conditions, delayed reinforcers increased both variability and the length of the maximum response durations. In a third experiment, immediate reinforcers were delivered contingent upon lever depression and release under a variety of duration requirements. Median lever‐press durations tracked the contingencies rapidly. Across all three experiments, rats emitted numerous response durations that were too short to satisfy the reinforcer requirements, and bimodal distributions similar to those produced by differential reinforcement of low rate schedules were evident for most rats. In many aspects, response duration responds to reinforcement parameters in a fashion similar to rate of discrete responding, but an examination of this continuous dimension of behavior may provide additional information about environment–behavior relationships.     

The reductive effects of noncontingent reinforcement fixed-time versus variable-time schedules.

The effects of fixed‐time (FT) and variable‐time (VT) schedules on responding were evaluated with 2 adults with mental retardation. Multielement and reversal designs were used to compare the effects of FT and VT schedules in reducing responses previously maintained on variable‐ratio reinforcement schedules. The schedules were equally effective in reducing the target behavior.     

Concurrent performances: stimulus-control gradients during schedules of signalled and unsignalled concurrent reinforcement

On one key, pigeons' pecks were reinforced according to a variable-interval schedule in the presence of vertical lines, and were not reinforced in the presence of oblique lines. On a second key, pecks were reinforced according to a variable-interval schedule in the presence of blue, according to a signalled variable-interval schedule in the presence of red, and were not reinforced in the presence of white. Subsequently, during extinction, stimulus-control gradients were obtained by presenting eight different line orientations on the first key concurrent with each of the three colors on the second key. On the first key, line-orientation gradients tended to be lower, narrower, and less shifted in peak or area when the second-key stimulus was blue or red, the stimuli respectively correlated with unsignalled and signalled reinforcement, than when it was white, the stimulus correlated with extinction. Thus, the effect on first-key line-orientation gradients depended on second-key stimuli correlated with concurrent reinforcement, whether or not these stimuli were also correlated with concurrent responding. As a function of first-key line orientation, an inverted gradient was obtained on the second key during blue; during both red and white, rates of pecking on the second key were near zero.     

Matching and maximizing with variable-time schedules.

Pigeons were offered choices between a variable-time schedule that arranged reinforcers throughout the session and a variable-time schedule that arranged reinforcers only when the pigeon was spending time on it. The subjects could maximize the overall rate of reinforcement in this situation by biasing their time allocation towards the latter schedule. This arrangement provides an alternative to concurrent variable-interval variable-ratio schedules for testing whether animals maximize overall rates or match relative rates, and has the advantage of being free of the asymmetrical response requirements present with those schedules. The results were contrary to those predicted by maximizing: The bias it predicts did not appear.     

Signalled and unsignalled percentage reinforcement of performance under a chained schedule

Pigeons were trained to peck a key under a chained fixed-ratio 15 fixed-interval 25-sec schedule of food presentation. In Experiment 1, blocks of sessions in which 100%, 75%, 50%, and 25% of the sequences ended with food presentation were conducted. When food presentation was omitted, a timeout of equal duration replaced it. As the frequency of food presentation decreased so did the frequency of completing the chained schedule. In Experiment 2, 75%, 50%, or 25% of the sequences terminated with food presentation and outcomes were signalled, i.e., completion of the fixed ratio resulted in either a stimulus correlated with the fixed-interval 25-sec schedule or a stimulus correlated with extinction. As the frequency of food presentation decreased, the number of sequences completed per session increased for two pigeons and remained high for a third. In Experiments 3 and 4, assessments of the effects of signalling the outcome of the chained schedule were made with response-independent presentation of events at the end of the sequence. Again, signalling the outcome of the chained schedule led to more chains being completed per session than did not signalling the outcome. Stimuli differentially paired with food presentation have powerful behavioral effects that may be attributed to the potency of these stimuli as conditioned reinforcers.     

Response acquisition with delayed reinforcement

Discrete responses of experimentally naive, food-deprived White Carneaux pigeons (key pecks) or Sprague-Dawley rats (bar or omnidirectional lever presses) initiated unsignaled delay periods that terminated with food delivery. Each subject first was trained to eat from the food source, but no attempt was made to shape or to otherwise train the response. In both species, the response developed and was maintained. Control procedures excluded the simple passage of time, response elicitation or induction by food presentation, type of operandum, food delivery device location, and adventitious immediate reinforcement of responding as the basis for the effects. Results revealed that neither training nor immediate reinforcement is necessary to establish new behavior. The conditions that give rise to both the first and second response are discussed, and the results are related to other studies of the delay of reinforcement and to explanations of behavior based on contingency or correlation and contiguity.     

Effects of delayed conditioned reinforcement in chain schedules.

The contingency between responding and stimulus change on a chain variable-interval 33-s, variable-interval 33-s, variable-interval 33-s schedule was weakened by interposing 3-s delays between either the first and second or the second and third links. No stimulus change signaled the delay interval and responses could occur during it, so the obtained delays were often shorter than the scheduled delay. When the delay occurred after the initial link, initial-link response rates decreased by an average of 77% with no systematic change in response rates in the second or third links. Response rates in the second link decreased an average of 59% when the delay followed that link, again with little effect on response rates in the first or third links. Because the effect of delaying stimulus change was comparable to the effect of delaying primary reinforcement in a simple variable-interval schedule, and the effect of the unsignaled delay was specific to the link in which the delay occurred, the results provide strong evidence for the concept of conditioned reinforcement.     

Responding under discrete-trial fixed-interval schedules of reinforcement

A fixed-interval schedule of reinforcement was modified by dividing each interval into 4-sec trial periods. No more than one response could occur during each trial because the operandum was inactivated for the remainder of any trial in which a response occurred. For example, under a 28-sec schedule, no more than seven responses could be emitted between reinforcements. Probabilities of responding by pigeons under six values of this discrete-trial fixed-interval schedule were best described by a two-state model: responding was either absent or infrequent immediately after reinforcement; then, at some variable time after reinforcement, there was an abrupt transition to a high and constant probability of responding on each trial. Performances under the discrete-trial procedure were less affected by uncontrolled sources of variance than performances under equivalent free-operant fixed-interval schedules.     

Group behavior of rats under schedules of reinforcement

Groups of three rats were placed in a chamber containing one response lever and one water dispenser. A variety of schedule conditions were explored including fixed ratio, extinction, satiation, fixed interval, fixed time, differential reinforcement of low rates, and discrimination learning. Each group was treated as a single unit, with the collective lever responses emitted by the three rats being the main dependent variable. Group responding was found to be controlled by the reinforcement schedules in an orderly and consistent manner. However, the groups often paused less and responded faster than individual rats working under identical conditions.     

Properties of behavior under random interval reinforcement schedules

In a temporally defined system of reinforcement schedules, the fixed interval case is defined when reinforcement probability, P, is equal to unity for the first response in any cycle length, T; when P is less than 1.0, random interval schedules emerge wherein T/P specifies the expected interval between reinforcements. Key-pecking rates were found to be: (a) inversely related to T/P; (b) higher at T=1.0 second than at other T parameter values; (c) low and linear at several T and T/P values. The mean post-reinforcement pause, if initially small, increased, and if initially large, decreased, as T/P increased.     

Mechanisms underlying the effects of unsignaled delayed reinforcement on key pecking of pigeons under variable-interval schedules.

Three experiments were conducted to test an interpretation of the response-rate-reducing effects of unsignaled nonresetting delays to reinforcement in pigeons. According to this interpretation, rates of key pecking decrease under these conditions because key pecks alternate with hopper-observing behavior. In Experiment 1, 4 pigeons pecked a food key that raised the hopper provided that pecks on a different variable-interval-schedule key met the requirements of a variable-interval 60-s schedule. The stimuli associated with the availability of the hopper (i.e., houselight and keylight off, food key illuminated, feedback following food-key pecks) were gradually removed across phases while the dependent relation between hopper availability and variable-interval-schedule key pecks was maintained. Rates of pecking the variable-interval-schedule key decreased to low levels and rates of food-key pecks increased when variable-interval-schedule key pecks did not produce hopper-correlated stimuli. In Experiment 2, pigeons initially pecked a single key under a variable-interval 60-s schedule. Then the dependent relation between hopper presentation and key pecks was eliminated by arranging a variable-time 60-s schedule. When rates of pecking had decreased to low levels, conditions were changed so that pecks during the final 5 s of each interval changed the keylight color from green to amber. When pecking produced these hopper-correlated stimuli, pecking occurred at high rates, despite the absence of a peck-food dependency. When peck-produced changes in keylight color were uncorrelated with food, rates of pecking fell to low levels. In Experiment 3, details (obtained delays, interresponse-time distributions, eating times) of the transition from high to low response rates produced by the introduction of a 3-s unsignaled delay were tracked from session to session in 3 pigeons that had been initially trained to peck under a conventional variable-interval 60-s schedule. Decreases in response rates soon after the transition to delayed reinforcement were accompanied by decreases in eating times and alterations in interresponse-time distributions. As response rates decreased and became stable, eating times increased and their variability decreased. These findings support an interpretation of the effects of delayed reinforcement that emphasizes the importance of hopper-observing behavior.     

A temporal parameter influencing choice between signalled and unsignalled shock schedules

The present study investigated whether choice of a signalled variable-time shock schedule over an unsignalled one was influenced by the average intershock interval. Eight rats were given a choice between signalled and unsignalled shock schedules in a series of conditions with average intershock intervals of 510, 270, 150, 90, 60, and 45 sec. Each test condition was preceded by a training-baseline condition, and schedule values were arranged in an ascending (four subjects) or descending (four subjects) order. Choice of the signalled conditions was directly related to the average intershock interval of the variable-time schedule for six of the eight subjects. The per cent of time in the signalled condition was highest when the average intershock interval was 150 sec or longer and lowest when the average intershock interval was 45 sec. The findings were interpreted as being due to changes in the safety features of the signalled schedule, rather than to changes in the average intershock interval per se.     

Effects of D-amphetamine on response acquisition with immediate and delayed reinforcement.

The present study examined in 8-hour sessions the effects of d-amphetamine (1.0, 5.6, and 10 mg/kg) on the acquisition of lever-press responding in rats that were exposed to procedures in which water delivery was delayed by 0, 8, or 16 seconds relative to the response that produced it. Both nonresetting- and resetting-delay conditions were studied. Although neither shaping nor autoshaping occurred, substantial levels of operative-lever responding developed under all conditions in which responses produced water. The lowest dose (1.0 mg/kg) of d-amphetamine either had no effect on or increased operative-lever pressing, whereas higher doses typically produced an initial reduction in lever pressing. Nonetheless, overall rates of operative-lever pressing at these doses were as high as, or higher than, those observed with vehicle. Thus, response acquisition was observed under all reinforcement procedures at all drug doses. In the absence of the drug, most responding occurred on the operative lever when reinforcement was immediate. Such differential responding also developed under both nonresetting- and resetting-delay procedures when the delay was 8 seconds, but not when it was 16 seconds. d-Amphetamine did not affect the development of differential responding under any procedure. Thus, consistent with d-amphetamine's effects under repeated acquisition procedures, the drug had no detrimental effect on learning until doses that produced general behavioral disruption were administered.     

Effects of reinforcement history on responding under progressive-ratio schedules of reinforcement.

The effects of experimental history on responding under a progressive-ratio schedule of reinforcement were examined. Sixteen pigeons were divided into four equal groups. Groups 1 to 3 were trained to peck a key for food under a fixed-ratio, variable-ratio, or differential-reinforcement-of-low-rate schedule of reinforcement. After training, these pigeons were shifted to a progressive-ratio schedule, later were shifted back to their original schedule (with decreased rates of reinforcement), and finally were returned to the progressive-ratio schedule. Pigeons in Group 4 (control) were maintained on the progressive-ratio schedule for the entire experiment. To test for potential "latent history" effects, pigeons responding under the progressive-ratio schedule were injected with d-amphetamine and given behavioral-momentum tests of prefeeding and extinction. Experimental histories affected responding in the immediate transition to the progressive-ratio schedule; response rates of pigeons with variable-ratio and fixed-ratio histories were higher than rates of pigeons with differential-reinforcement-of-low-rate and progressive-ratio-only histories. Pigeons with differential-reinforcement-of-low-rate histories, and to a lesser degree pigeons with variable-ratio and fixed-ratio histories, also had shorter postreinforcement pauses than pigeons with only a progressive-ratio history. No consistent long-term effects of prior contingencies on responding under the progressive-ratio schedule were evident. d-Amphetamine and resistance-to-change tests failed to reveal consistent latent history effects. The data suggest that history effects are sometimes transitory and not susceptible to latent influences.