Varying wheel-running reinforcer duration within a session: effect on the revolution-postreinforcement pause relation

Previous investigations of wheel-running reinforcement that manipulated reinforcer duration across conditions showed a strong relation between wheel-running rate and average postreinforcement pause (PRP) duration. To determine if the basis of this relation across conditions was a local effect of fatigue or satiation, the correlation between revolutions run and the duration of the immediately following PRP was investigated under conditions in which reinforcer duration was either constant or variable within a session. Seven male Wistar rats pressed a lever on a fixed-interval 60-s reinforcement schedule with the opportunity to run for 60 s as the reinforcing consequence. In the constant-duration condition, the duration of the reinforcer was always 60 s. In the variable-duration condition, the duration of the reinforcer varied between 2 and 240 s with a mean of 60 s. Mean correlations between revolutions run and the next PRP duration for constant, variable, and constant conditions were -.07, .20, and -.07, respectively. Although the positive correlation in the variable-duration condition is consistent with an effect of momentary fatigue or satiation, little of the variance in PRP duration appears to be attributable to these factors.     

Effects of time between trials on rats' and pigeons' choices with probabilistic delayed reinforcers

Parallel experiments with rats and pigeons examined reasons for previous findings that in choices with probabilistic delayed reinforcers, rats' choices were affected by the time between trials whereas pigeons' choices were not. In both experiments, the animals chose between a standard alternative and an adjusting alternative. A choice of the standard alternative led to a short delay (1 s or 3 s), and then food might or might not be delivered. If food was not delivered, there was an "interlink interval," and then the animal was forced to continue to select the standard alternative until food was delivered. A choice of the adjusting alternative always led to food after a delay that was systematically increased and decreased over trials to estimate an indifference point--a delay at which the two alternatives were chosen about equally often. Under these conditions, the indifference points for both rats and pigeons increased as the interlink interval increased from 0 s to 20 s, indicating decreased preference for the probabilistic reinforcer with longer time between trials. The indifference points from both rats and pigeons were well described by the hyperbolic-decay model. In the last phase of each experiment, the animals were not forced to continue selecting the standard alternative if food was not delivered. Under these conditions, rats' choices were affected by the time between trials whereas pigeons' choices were not, replicating results of previous studies. The differences between the behavior of rats and pigeons appears to be the result of procedural details, not a fundamental difference in how these two species make choices with probabilistic delayed reinforcers.     

Effects of reinforcer delays on choice as a function of income level

Three rats earned their daily food ration by responding during individual trials either on a lever that delivered one food pellet immediately or on a second lever that delivered three pellets after a delay that was continuously adjusted to ensure substantial responding to both alternatives. Choice of the delayed reinforcer increased when the number of trials per session was reduced. This result suggests that models seeking closure on choice effects must include a parameter reflecting how preference changes with sessionwide income. Moreover, models positing that reinforcer probability and immediacy (1/delay) function equivalently in choice are called into question by the finding that probability and immediacy produce opposing effects when income level is changed.     

Response–reinforcer dependency and resistance to change

The effects of the response–reinforcer dependency on resistance to change were studied in three experiments with rats. In Experiment 1, lever pressing produced reinforcers at similar rates after variable interreinforcer intervals in each component of a two‐component multiple schedule. Across conditions, in the fixed component, all reinforcers were response‐dependent; in the alternative component, the percentage of response‐dependent reinforcers was 100, 50 (i.e., 50% response‐dependent and 50% response‐independent) or 10% (i.e., 10% response‐dependent and 90% response‐independent). Resistance to extinction was greater in the alternative than in the fixed component when the dependency in the former was 10%, but was similar between components when this dependency was 100 or 50%. In Experiment 2, a three‐component multiple schedule was used. The dependency was 100% in one component and 10% in the other two. The 10% components differed on how reinforcers were programmed. In one component, as in Experiment 1, a reinforcer had to be collected before the scheduling of other response‐dependent or independent reinforcers. In the other component, response‐dependent and ‐independent reinforcers were programmed by superimposing a variable‐time schedule on an independent variable‐interval schedule. Regardless of the procedure used to program the dependency, resistance to extinction was greater in the 10% components than in the 100% component. These results were replicated in Experiment 3 in which, instead of extinction, VT schedules replaced the baseline schedules in each multiple‐schedule component during the test. We argue that the relative change in dependency from Baseline to Test, which is greater when baseline dependencies are high rather than low, could account for the differential resistance to change in the present experiments. The inconsistencies in results across the present and previous experiments suggest that the effects of dependency on resistance to change are not well understood. Additional systematic analyses are important to further understand the effects of the response–reinforcer relation on resistance to change and to the development of a more comprehensive theory of behavioral persistence.     

Running and responding reinforced by the opportunity to run: effect of reinforcer duration.

The present study investigated the effect of reinforcer duration on running and on responding reinforced by the opportunity to run. Eleven male Wistar rats responded on levers for the opportunity to run in a running wheel. Opportunities to run were programmed to occur on a tandem fixed-ratio 1 variable-interval 30-s reinforcement schedule. Reinforcer duration varied across conditions from 30 to 120 s. As reinforcer duration increased, the rates of running and lever pressing declined, and latency to lever press increased. The increase in latency to respond was consistent with findings that unconditioned inhibitory aftereffects of reinforcement increase with reinforcer magnitude. The decrease in local lever-pressing rates, however, was inconsistent with the view that response strength increases with the duration of the reinforcer. Response rate varied inversely, not directly, with reinforcer duration. Furthermore, within-session data challenge satiation, fatigue, and response deprivation as determinants of the observed changes in running and responding. In sum, the results point to the need for further research with nonappetitive forms of reinforcement.     

Reinforcement value and fixed‐interval performance

The concept of reinforcement value summarizes the effect of different variables, such as reinforcement delay, reinforcement magnitude, and deprivation level, on behavior. In the present set of experiments, we evaluated the effect of reinforcement devaluation on performance under FI schedules. The literature on timing and reinforcement value suggests that devaluation generates longer expected times to reinforcement than the same intervals trained under control conditions. We devalued reinforcement with delay in Experiments 1A, 1B, and 2, and diminished deprivation in Experiments 3A and 3B. Devaluation reduced response rates, increased the number of one‐response intervals, and lengthened postreinforcement pauses, but had inconsistent effects on other timing measures such as quarter life and breakpoint. The results of delayed reinforcement and diminished deprivation manipulations are well summarized as reinforcement devaluation effects. These results suggest that devaluation may reduce stimulus control. In addition, we argue that the process by which delayed reinforcement affects behavior might also explain some effects observed in other devaluation procedures through the concept of reinforcement value.     

Relative sensitivity to reinforcer amount and delay in a self-control choice situation.

Rats were exposed to concurrent-chains schedules in which a single variable-interval schedule arranged entry into one of two terminal-link delay periods (fixed-interval schedules). The shorter delay ended with the delivery of a single food pellet; the longer day ended with a larger number of food pellets (two under some conditions and six under others). In Experiment 1, the terminal-link delays were selected so that under all conditions the ratio of delays would exactly equal the ratio of the number of pellets. But the absolute duration of the delays differed across conditions. In one condition, for example, rats chose between one pellet delayed 5 s and six pellets delayed 30 s; in another condition rats chose between one pellet delayed 10 s and six pellets delayed 60 s. The generalized matching law predicts indifference between the two alternatives, assuming that the sensitivity parameters for amount and delay of reinforcement are equal. The rats' choices were, in fact, close to indifference except when the choice was between one pellet delayed 5 s and six pellets delayed 30 s. That deviation from indifference suggests that the sensitivities to amount and delay differ from each other depending on the durations of the delays. In Experiment 2, rats chose between one pellet following a 5-s delay and six pellets following a delay that was systematically increased over sessions to find a point of indifference. Indifference was achieved when the delay to the six pellets was approximately 55 s. These results are consistent with the possibility that the relative sensitivities to amount and delay differ as a function of the delays.     

The general matching law describes choice on concurrent variable-interval schedules of wheel-running reinforcement.

Six male Wistar rats were exposed to concurrent variable-interval schedules of wheel-running reinforcement. The reinforcer associated with each alternative was the opportunity to run for 15 s, and the duration of the changeover delay was 1 s. Results suggested that time allocation was more sensitive to relative reinforcement rate than was response allocation. For time allocation, the mean slopes and intercepts were 0.82 and 0.008, respectively. In contrast, for response allocation, mean slopes and intercepts were 0.60 and 0.03, respectively. Correction for low response rates and high rates of changing over, however, increased slopes for response allocation to about equal those for time allocation. The results of the present study suggest that the two-operant form of the matching law can be extended to wheel-running reinforcement. 'I'he effects of a low overall response rate, a short Changeover delay, and long postreinforcement pausing on the assessment of matching in the present study are discussed.     

Determinants of reinforcer accumulation during an operant task.

Responses by rats on an earn lever made available food pellets that were delivered to a food cup by responses on a second, collect, lever. The rats could either collect and immediately consume or accumulate (defined as the percentage of multiple earn responses and as the number of pellets earned before a collect response) earned pellets. In Experiment 1, accumulation varied as a function of variations in the earn or collect response requirements and whether the earn and collect levers were proximal (31 cm) or distal (248 cm) to one another. Some accumulation occurred under all but one of the conditions, but generally was higher when the earn and collect levers were distal to one another, particularly when the earn response requirement was fixed-ratio (FR) 1. In Experiment 2, the contributions of responses and time to accumulation were assessed by comparing an FR 20 earn response requirement to a condition in which only a single earn response was required at the end of a time interval nominally yoked to the FR interval. When 248 cm separated the earn and collect levers, accumulation was always greater in the FR condition, and it was not systematically related to reinforcement rate. In Experiment 3, increasing the earn response requirement with a progressive-ratio schedule that reset only with a collect response increased the likelihood of accumulation when the collect and earn levers were 248 cm apart, even though such accumulation increased the next earn response requirement. Reinforcer accumulation is an understudied dimension of operant behavior that relates to the analysis of such phenomena as hoarding and self-control, in that they too involve accumulating versus immediately collecting or consuming reinforcers.     

Effects of pre-trial response requirements on self-control choices by rats and pigeons

Parallel experiments with rats and pigeons examined whether the size of a pre-trial ratio requirement would affect choices in a self-control situation. In different conditions, either 1 response or 40 responses were required before each trial. In the first half of each experiment, an adjusting-ratio schedule was used, in which subjects could choose a fixed-ratio schedule leading to a small reinforcer, or an adjusting-ratio schedule leading to a larger reinforcer. The size of the adjusting ratio requirement was increased and decreased over trials based on the subject's responses, in order to estimate an indifference point-a ratio at which the two alternatives were chosen about equally often. The second half of each experiment used an adjusting-delay procedure-fixed and adjusting delays to the small and large reinforcers were used instead of ratio requirements. In some conditions, particularly with the reinforcer delays, the rats had consistently longer adjusting delays with the larger pre-trial ratios, reflecting a greater tendency to choose the larger, delayed reinforcer when more responding was required to reach the choice point. No consistent effects of the pre-trial ratio were found for the pigeons in any of the conditions. These results may indicate that rats are more sensitive to the long-term reinforcement rates of the two alternatives, or they may result from a shallower temporal discounting rate for rats than for pigeons, a difference that has been observed in previous studies.     

Effects of reinforcement rate and delay on the acquisition of lever pressing by rats.

The acquisition of lever pressing by naive rats, in the absence of shaping, was studied as a function of different rates and unsignaled delays of reinforcement. Groups of 3 rats were each exposed to tandem schedules that differed in either the first or the second component. First-component schedules were either continuous reinforcement or random-interval 15, 30, 60 or 120 s; second-component schedules were fixed-time 0, 1, 3, 6, 12, or 24 s. Rate of responding was low under continuous immediate reinforcement and higher under random-interval 15 s. Random interval 30-s and 60-s schedules produced lower rates that were similar to each other. Random-interval 120 s controlled the lowest rate in the immediate-reinforcement condition. Adding a constant 12-s delay to each of the first-component schedule parameters controlled lower response rates that did not vary systematically with reinforcement rate. The continuous and random-interval 60-s schedules of immediate reinforcement controlled higher global and first-component response rates than did the same schedules combined with longer delays, and first-component rates showed some graded effects of delay duration. In addition, the same schedules controlled higher second-component response rates in combination with a 1-s delay than in combination with longer delays. These results were related to those from previous studies on acquisition with delayed reinforcement as well as to those from similar reinforcement procedures used during steady-state responding.     

Theories of probabilistic reinforcement.

In three experiments, pigeons chose between two alternatives that differed in the probability of reinforcement and the delay to reinforcement. A peck at a red key led to a delay of 5 s and then a possible reinforcer. A peck at a green key led to an adjusting delay and then a certain reinforcer. This delay was adjusted over trials so as to estimate an indifference point, or a duration at which the two alternatives were chosen about equally often. In Experiments 1 and 2, the intertrial interval was varied across conditions, and these variations had no systematic effects on choice. In Experiment 3, the stimuli that followed a choice of the red key differed across conditions. In some conditions, a red houselight was presented for 5 s after each choice of the red key. In other conditions, the red houselight was present on reinforced trials but not on nonreinforced trials. Subjects exhibited greater preference for the red key in the latter case. The results were used to evaluate four different theories of probabilistic reinforcement. The results were most consistent with the view that the value or effectiveness of a probabilistic reinforcer is determined by the total time per reinforcer spent in the presence of stimuli associated with the probabilistic alternative. According to this view, probabilistic reinforcers are analogous to reinforcers that are delivered after variable delays.     

Choice between constant and variable alternatives by rats: effects of different reinforcer amounts and energy budgets

Two experiments, using rats as subjects, investigated the effect of different reinforcer amounts and energy budgets on choice between constant and variable alternatives under a closed economy. Rats were housed in the chamber and were exposed to a modified concurrent-chains schedule in which the choice phase was separated from a rest phase during which the rats could engage in other activities. In the choice phase, a single variable-interval schedule arranged entry into one of two equal terminal links (fixed-interval schedules). The constant terminal link ended with the delivery of a fixed number of food pellets (two or three, depending on the condition), whereas the variable terminal link ended with a variable number of food pellets (means of two or three, depending on the condition). Energy budget was defined as positive when body weights were over 90% of free-feeding weights, and as negative when they were under 80% of free-feeding weights. The different body weights were produced by varying the duration of the equal terminal-link schedules within daily 3-hr sessions. In Experiment 1, rats chose between a constant and a variable three pellets under both energy budgets. Rats preferred the constant three pellets more under the positive energy budget, whereas they were indifferent under the negative energy budget. In Experiment 2, rats chose between a constant three pellets and a variable two pellets, and chose between a constant two pellets and a variable three pellets under both energy budgets. The rats strongly preferred the constant three pellets over the variable two pellets under both energy budgets. In contrast, rats preferred the variable three pellets over the constant two pellets only under the negative energy budget, whereas they were indifferent under the positive energy budget. These results indicate that rats choices are sensitive to the difference in reinforcer amounts and to the energy budgets defined by the level of body weight. The present results are consistent with those obtained with small granivorous birds as well as with the predictions of a recent risk-sensitive foraging theory.     

Tests of behavioral-economic assessments of relative reinforcer efficacy: economic substitutes

This experiment was conducted to test predictions of two behavioral-economic approaches to quantifying relative reinforcer efficacy. According to the first of these approaches, characteristics of averaged normalized demand curves may be used to predict progressive-ratio breakpoints and peak responding. The second approach, the demand analysis, rejects the concept of reinforcer efficacy, arguing instead that traditional measures of relative reinforcer efficacy (breakpoint, peak response rate, and choice) correspond to specific characteristics of non-normalized demand curves. The accuracy of these predictions was evaluated in rats' responding for food or fat: two reinforcers known to function as partial substitutes. Consistent with the first approach, predicted peak normalized response output values (Omax) obtained under single-schedule conditions ordinally predicted progressive-ratio breakpoints and peak responding. Predictions of the demand analysis had mixed success. Pmax and Omax were significantly correlated with PR breakpoints and peak responding (respectively) when fat, but not when food, was the reinforcer. Relative consumption of food and fat under single schedules of reinforcement did not predict preference better than chance. The normalized demand analysis is supplemented with the economic concept of diminishing marginal utility, to predict preference shifts across the range of food and fat prices examined.     

Effects of reinforcer probability, delay, and response requirements on the choices of rats and pigeons: possible species differences

In Experiment 1 with rats, a left lever press led to a 5-s delay and then a possible reinforcer. A right lever press led to an adjusting delay and then a certain reinforcer. This delay was adjusted over trials to estimate an indifference point, or a delay at which the two alternatives were chosen about equally often. Indifference points increased as the probability of reinforcement for the left lever decreased. In some conditions with a 20% chance of food, a light above the left lever was lit during the 5-s delay on all trials, but in other conditions, the light was only lit on those trials that ended with food. Unlike previous results with pigeons, the presence or absence of the delay light on no-food trials had no effect on the rats' indifference points. In other conditions, the rats showed less preference for the 20% alternative when the time between trials was longer. In Experiment 2 with rats, fixed-interval schedules were used instead of simple delays, and the presence or absence of the fixed-interval requirement on no-food trials had no effect on the indifference points. In Experiment 3 with rats and Experiment 4 with pigeons, the animals chose between a fixed-ratio 8 schedule that led to food on 33% of the trials and an adjusting-ratio schedule with food on 100% of the trials. Surprisingly, the rats showed less preference for the 33% alternative in conditions in which the ratio requirement was omitted on no-food trials. For the pigeons, the presence or absence of the ratio requirement on no-food trials had little effect. The results suggest that there may be differences between rats and pigeons in how they respond in choice situations involving delayed and probabilistic reinforcers.     

Intertrial-interval effects on sensitivity (A') and response bias (B") in a temporal discrimination by rats.

Killeen and Fetterman's (1988) behavioral theory of animal timing predicts that decreases in the rate of reinforcement should produce decreases in the sensitivity (A') of temporal discriminations and a decrease in miss and correct rejection rates (decrease in bias toward "long" responses). Eight rats were trained on a 10- versus 0.1-s temporal discrimination with an intertrial interval of 5 s and were subsequently tested on probe days on the same discrimination with intertrial intervals of 1, 2.5, 5, 10, or 20 s. The rate of reinforcement declined for all animals as intertrial interval increased. Although sensitivity (A') decreased with increasing intertrial interval, all rats showed an increase in bias to make long responses.     

Responding for sucrose and wheel-running reinforcement: effects of sucrose concentration and wheel-running reinforcer duration

Six male albino rats were placed in running wheels and exposed to a fixed-interval 30-s schedule of lever pressing that produced either a drop of sucrose solution or the opportunity to run for a fixed duration as reinforcers. Each reinforcer type was signaled by a different stimulus. In Experiment 1, the duration of running was held constant at 15 s while the concentration of sucrose solution was varied across values of 0, 2.5. 5, 10, and 15%. As concentration decreased, postreinforcement pause duration increased and local rates decreased in the presence of the stimulus signaling sucrose. Consequently, the difference between responding in the presence of stimuli signaling wheel-running and sucrose reinforcers diminished, and at 2.5%, response functions for the two reinforcers were similar. In Experiment 2, the concentration of sucrose solution was held constant at 15% while the duration of the opportunity to run was first varied across values of 15, 45, and 90 s then subsequently across values of 5, 10, and 15 s. As run duration increased, postreinforcement pause duration in the presence of the wheel-running stimulus increased and local rates increased then decreased. In summary, inhibitory aftereffects of previous reinforcers occurred when both sucrose concentration and run duration varied; changes in responding were attributable to changes in the excitatory value of the stimuli signaling the two reinforcers.     

Relationship between response rate and reinforcement frequency in variable-interval schedules: II. Effect of the volume of sucrose reinforcement

Three rats were exposed to variable-interval schedules specifying a range of different reinforcement frequencies, using three different volumes of .32 molar sucrose (.10, .05, and .02 milliliters) as the reinforcer. With each of the three volumes, the rates of responding of all three rats were increasing, negatively accelerated functions of reinforcement frequency, the data conforming closely to Herrnstein's equation. In each rat the value of the constant K_H, which expresses the reinforcement frequency needed to obtain the half-maximal response rate, increased with decreasing reinforcer volume, the values obtained with .02 milliliters being significantly greater than the values obtained with .10 milliliters. The values of the constant R_max, which expresses the theoretical maximum response rate, were not systematically related to reinforcer volume. The effect of reinforcer volume upon the relationship between response rate and reinforcement frequency is thus different from the effect of the concentration of sucrose reinforcement: In a previous experiment (Bradshaw, Szabadi, & Bevan, 1978) it was found that sucrose concentration influenced the values of both constants, R_max increasing and K_H decreasing with increasing sucrose concentration.     

Effects of cocaine on fixed-interval responding reinforced by the opportunity to run

Rate-dependent drug effects have been observed for operant responding maintained by food, water, heat, light onset, electrical brain stimulation, shock-stimulus termination, and shock presentation. The present study sought to determine if the effects of cocaine on lever pressing maintained by the opportunity to run could also be described as rate dependent. Seven male Wistar rats were trained to respond on levers for the opportunity to run in a wheel. The schedule of reinforcement was fixed-interval 60 s, and the reinforcing consequence was the opportunity to run for 60 s. On this schedule, overall rates of responding were low, usually below six presses per minute, and pauses frequently exceeded the 60-s interval. Despite these differences, an overall scalloped pattern of lever pressing was evident for each rat. Doses of 1, 2, 4, 8, and 16 mg/kg cocaine were administered 10 min prior to a session. Only at the 16 mg/kg dose did the responding of the majority of rats change in a manner suggestive of a rate-dependent drug effect. Specifically, lower response rates at the beginning of the intervals increased and higher rates at the end of the intervals decreased, as indicated by the fact that slopes from the regression of drug rates on control rates decreased. These data provide tentative support for the generalization of rate-dependent effects to operant responding maintained by wheel running. Differences in the baseline performance maintained by wheel running compared to those for food and water point to the need for further experimentation before this effect can be firmly established.