Behavioral economics and within-session changes in responding

Pigeons and rats responded on fixed-ratio schedules with requirements ranging from 5 to 120 responses. Consistent with past results from several schedules and procedures, responding usually changed systematically within experimental sessions. The within-session changes were usually larger and were less symmetrical around the middle of the session for schedules that provided higher, rather than lower, rates of reinforcement. These results suggest that similar variables contribute to within-session changes in responding under different schedules. When an economic demand function was fit to the data, the intensity and elasticity of demand for food and the percentage of the variance accounted for decreased within sessions, although the trend for elasticity did not reach statistical significance for pigeons. These results suggest that relatively short sessions should be used to study economic demand in open economies and that demand may differ at different times in a session and in sessions of different lengths. Within-session changes in intensity, but not necessarily elasticity, of demand are consistent with behavioral economic theories.     

Tests of behavior momentum in simple and multiple schedules with rats and pigeons.

Four experiments examined the relationship between rate of reinforcement and resistance to change in rats' and pigeons' responses under simple and multiple schedules of reinforcement. In Experiment 1, 28 rats responded under either simple fixed-ratio, variable-ratio, fixed-interval, or variable-interval schedules; in Experiment 2, 3 pigeons responded under simple fixed-ratio schedules. Under each schedule, rate of reinforcement varied across four successive conditions. In Experiment 3, 14 rats responded under either a multiple fixed-ratio schedule or a multiple fixed-interval schedule, each with two components that differed in rate of reinforcement. In Experiment 4, 7 pigeons responded under either a multiple fixed-ratio or a multiple fixed-interval schedule, each with three components that also differed in rate of reinforcement. Under each condition of each experiment, resistance to change was studied by measuring schedule-controlled performance under conditions with prefeeding, response-independent food during the schedule or during timeouts that separated components of the multiple schedules, and by measuring behavior under extinction. There were no consistent differences between rats and pigeons. There was no direct relationship between rates of reinforcement and resistance to change when rates of reinforcement varied across successive conditions in the simple schedules. By comparison, in the multiple schedules there was a direct relationship between rates of reinforcement and resistance to change during most tests of resistance to change. The major exception was delivering response-independent food during the schedule; this disrupted responding, but there was no direct relationship between rates of reinforcement and resistance to change in simple- or multiple-schedule contexts. The data suggest that rate of reinforcement determines resistance to change in multiple schedules, but that this relationship does not hold under simple schedules.     

Behavioral momentum: the effects of the temporal separation of rates of reinforcement.

In Part 1 of the experiment, rats responded under a variable-interval (VI) 30-s schedule and a VI 120-s schedule, with each in effect for a block of consecutive sessions. That is, the two VI schedules were presented in successive conditions. In Part 2 the VI schedules alternated each day, and in Part 3 the schedules alternated within the session as a multiple schedule. For half of the rats in Parts 1 and 2, the VI schedule alternated every few minutes within the session with a stimulus that signaled extinction. For each part, once response rates had stabilized, resistance to change was measured by prefeeding and extinction. When the schedules were examined in successive conditions (Part 1), resistance to extinction was greater under the VI 120-s schedule of reinforcement than under the VI 30-s schedule, but no consistent differences in resistance to prefeeding were observed between the two VI schedules. When the VI schedules alternated each day (Part 2), resistance to extinction was greater under the VI 120-s schedule. However, no consistent differences in resistance to prefeeding were observed between the VI schedules without extinction in Group A, but resistance to prefeeding was greater under the VI 30-s schedule for rats with the added extinction component in Group B. When the VI schedules alternated within the session as a multiple schedule (Part 3), resistance to extinction and resistance to prefeeding were greater under the VI 30-s schedule. The data suggest that different rates of reinforcement, and their accompanying discriminative stimuli, must be compared within the same session (or at least on alternate days) to produce data consistent with the behavioral momentum model.     

Response persistence under ratio and interval reinforcement schedules

In Experiment 1, rats were exposed to progressive-ratio schedules of food reinforcement while other rats were exposed simultaneously to yoked-interval schedules that arranged equivalent interreinforcer intervals but required only a single response at the end of the interval for food delivery. In Experiment 2, a within-subject yoked-control procedure was employed in which pigeons were exposed to alternating sessions (one per day) of progressive-ratio schedules and yoked-interval schedules as described above. In both experiments, responding under the yoked-interval schedule persisted beyond the point at which responding under the progressive-ratio schedule had ceased. The progressive-ratio schedules controlled break-and-run distributions, and the yoked-interval schedules controlled more even distributions of responses in time. Response rates decreased and postreinforcement pauses increased over time within individual sessions under both schedules. The results suggest that responding maintained by interval schedules is more persistent than that maintained by ratio schedules. The limitations and implications of this conclusion are discussed in the context of other investigations of response strength and behavioral momentum.     

An integrative model for the study of behavioral momentum.

Behavioral momentum is the product of response rate and resistance to change. The data on relative resistance to change are summarized for pigeons responding on single-key two-component multiple schedules, in the initial links of two-key multiple chained schedules, and in equivalent components of two-key serial schedules. For single-key procedures, the ratio of resistance to change in two schedule components is shown to depend on the ratio of reinforcer rates obtained in the presence of the component stimuli. For two-key procedures, the ratio of resistance to change in equivalent components is shown to depend on the ratio of reinforcer rates correlated with key locations. A model based on stimulus-reinforcer contingencies that combines the reinforcer rates in schedule components summed over key locations and reinforcer rates correlated with key locations summed over components, each expressed relative to the session average reinforcer rate, gives a good account of the data. An extension of the relative law of effect for multiple schedules fails to provide a complete account of resistance to change, but both approaches are needed for a comprehensive understanding of behavioral momentum.     

Variable-ratio versus variable-interval schedules: response rate, resistance to change, and preference

Two experiments asked whether resistance to change depended on variable-ratio as opposed to variable-interval contingencies of reinforcement and the different response rates they establish. In Experiment 1, pigeons were trained on multiple random-ratio random-interval schedules with equated reinforcer rates. Baseline response rates were disrupted by intercomponent food, extinction, and prefeeding. Resistance to change relative to baseline was greater in the interval component, and the difference was correlated with the extent to which baseline response rates were higher in the ratio component. In Experiment 2, pigeons were trained on multiple variable-ratio variable-interval schedules in one half of each session and on concurrent chains in the other half in which the terminal links corresponded to the multiple-schedule components. The schedules were varied over six conditions, including two with equated reinforcer rates. In concurrent chains, preference strongly overmatched the ratio of obtained reinforcer rates. In multiple schedules, relative resistance to response-independent food during intercomponent intervals, extinction, and intercomponent food plus extinction depended on the ratio of obtained reinforcer rates but was less sensitive than was preference. When reinforcer rates were similar, both preference and relative resistance were greater for the variable-interval schedule, and the differences were correlated with the extent to which baseline response rates were higher on the variable-ratio schedule, confirming the results of Experiment 1. These results demonstrate that resistance to change and preference depend in part on response rate as well as obtained reinforcer rate, and challenge the independence of resistance to change and preference with respect to response rate proposed by behavioral momentum theory.     

Paired baseline performance as a behavioral ideal

Several recent theories view performance under the constraints of a schedule as an attempt to approach the basepoint, the total amount of the instrumental response and the total amount of the contingent response seen in the absence of schedule constraint. Some new analyses of experiments on concurrent ratio schedules, and simple ratio schedules offering an optional magnitude of contingent reward, tested this view directly. In each of the five experiments examined the organism rejected the chance of a closer approach to the basepoint, and thereby failed in addition to maximize the rate of reinforcement.     

Effects of reinforcer probability, delay, and response requirements on the choices of rats and pigeons: possible species differences

In Experiment 1 with rats, a left lever press led to a 5-s delay and then a possible reinforcer. A right lever press led to an adjusting delay and then a certain reinforcer. This delay was adjusted over trials to estimate an indifference point, or a delay at which the two alternatives were chosen about equally often. Indifference points increased as the probability of reinforcement for the left lever decreased. In some conditions with a 20% chance of food, a light above the left lever was lit during the 5-s delay on all trials, but in other conditions, the light was only lit on those trials that ended with food. Unlike previous results with pigeons, the presence or absence of the delay light on no-food trials had no effect on the rats' indifference points. In other conditions, the rats showed less preference for the 20% alternative when the time between trials was longer. In Experiment 2 with rats, fixed-interval schedules were used instead of simple delays, and the presence or absence of the fixed-interval requirement on no-food trials had no effect on the indifference points. In Experiment 3 with rats and Experiment 4 with pigeons, the animals chose between a fixed-ratio 8 schedule that led to food on 33% of the trials and an adjusting-ratio schedule with food on 100% of the trials. Surprisingly, the rats showed less preference for the 33% alternative in conditions in which the ratio requirement was omitted on no-food trials. For the pigeons, the presence or absence of the ratio requirement on no-food trials had little effect. The results suggest that there may be differences between rats and pigeons in how they respond in choice situations involving delayed and probabilistic reinforcers.     

Fixed-interval performance: The dynamics of behavior and the interval length

Postreinforcement pauses from successive intervals under various fixed-interval schedules (ranging from 15 seconds to 480 seconds in length) were subjected to lag-1 autocorrelation analysis. Results from both rats and pigeons suggested that there was a consistent tendency for pause values in successive intervals to be weakly positively related. This tendency did not appear to change systematically with interval length and was exhibited both when the reinforcer magnitude was constant and when it was variable at different interval values. The findings do not support suggestions that the dynamic properties of performance under fixed-interval schedules vary systematically with interval length, and are in the opposite direction from some previous findings suggesting that measures of behavior (such as post-reinforcement pause length or number of responses) in successive intervals are inversely related.     

Differences in the effect of Pavlovian contingencies upon behavioral momentum using auditory versus visual stimuli.

We examined the role of Pavlovian and operant relations in behavioral momentum by arranging response-contingent alternative reinforcement in one component of a three-component multiple concurrent schedule with rats. This permitted the simultaneous arranging of different response-reinforcer (operant) and stimulus-reinforcer (Pavlovian) contingencies during three baseline conditions. Auditory or visual stimuli were used as discriminative stimuli within the multiple concurrent schedules. Resistance to change of a target response was assessed during a single session of extinction following each baseline condition. The rate of the target response during baseline varied inversely with the rate of response-contingent reinforcement derived from a concurrent source, regardless of whether the discriminative stimuli were auditory or visual. Resistance to change of the target response, however, did depend on the discriminative-stimulus modality. Resistance to change in the presence of visual stimuli was a positive function of the Pavlovian contingencies, whereas resistance to change was unrelated to either the operant or Pavlovian contingencies when the discriminative stimuli were auditory. Stimulus salience may be a factor in determining the differences in resistance to change across sensory modalities.     

Patterns of responding within sessions.

Rates of responding changed systematically across sessions for rats pressing levers and keys and for pigeons pressing treadles and pecking keys. A bitonic function in which response rates increased and then decreased across sessions was the most common finding, although an increase in responding also occurred alone. The change in response rate was usually large. The function relating responding to time in session had the following general characteristics: It appeared early in training, and further experience moved and reduced its peak; it was flatter for longer sessions; and it was flatter, more symmetrical, and peaked later for lower than for higher rates of reinforcement. Factors related to reinforcement exerted more control over the location of the peak rate of responding and the steepness of the decline in response rates than did factors related to responding. These within-session changes in response rates have fundamental theoretical and methodological implications.     

Resistance to reinforcement change in multiple and concurrent schedules assessed in transition and at steady state

Behavioral momentum theory relates resistance to change of responding in a multiple-schedule component to the total reinforcement obtained in that component, regardless of how the reinforcers are produced. Four pigeons responded in a series of multiple-schedule conditions in which a variable-interval 40-s schedule arranged reinforcers for pecking in one component and a variable-interval 360-s schedule arranged them in the other. In addition, responses on a second key were reinforced according to variable-interval schedules that were equal in the two components. In different parts of the experiment, responding was disrupted by changing the rate of reinforcement on the second key or by delivering response-independent food during a blackout separating the two components. Consistent with momentum theory, responding on the first key in Part 1 changed more in the component with the lower reinforcement total when it was disrupted by changes in the rate of reinforcement on the second key. However, responding on the second key changed more in the component with the higher reinforcement total. In Parts 2 and 3, responding was disrupted with free food presented during intercomponent blackouts, with extinction (Part 2) or variable-interval 80-s reinforcement (Part 3) arranged on the second key. Here, resistance to change was greater for the component with greater overall reinforcement. Failures of momentum theory to predict short-term differences in resistance to change occurred with disruptors that caused greater change between steady states for the richer component. Consistency of effects across disruptors may yet be found if short-term effects of disruptors are assessed relative to the extent of change observed after prolonged exposure.     

Response-independent food delivery and behavioral resistance to change

Response-independent food was delivered during a dark-key phase between two multiple-schedule components to explore its disruptive effects on responding. Responding in components was maintained by separate variable-interval 120-s schedules, with a 2-s reinforcer in Component 1 and a 6-s reinforcer in Component 2. Across conditions the rate and duration of response-independent food presentations were manipulated. The results showed that response rates in both components decreased as a function of the duration and the rate of response-independent food presentations; moreover, the decrease in response rate relative to the baseline level was larger in Component 1 than in Component 2. These findings were consistent with expectations from behavioral momentum theory, which predicts that if equal disruption (response-independent food in this case) is applied to responding in two components, then the ratio of response-rate change in Component 1 versus Component 2 should remain constant, irrespective of the magnitude of that disruption.     

Consumption-leisure tradeoffs in pigeons: Effects of changing marginal wage rates by varying amount of reinforcement

Pigeons' rates of responding and food reinforcement under simple random-ratio schedules were compared with those obtained under comparable ratio schedules in which free food deliveries were added, but the duration of each food delivery was halved. These ratio-with-free-food schedules were constructed so that, were the pigeon to maintain the same rate of responding as it had under the simple ratio schedule, total food obtained (earned plus free) would remain unchanged. However, any reduction in responding would reduce total food consumption below that under the simple ratio schedule. These “compensated wage decreases” led to decreases in responding and decreases in food consumption, as predicted by an economic model of labor supply. Moreover, the reductions in responding increased as the ratio value increased (i.e., as wage rates decreased). Pigeons, therefore, substituted leisure for consumption. The relationship between these procedures and negative-income-tax programs is noted.     

Preference and resistance to change with constant-duration schedule components

Previous research on preference between variable-interval terminal links in concurrent chains has most often used variable-duration terminal links ending with a single reinforcer. By contrast, most research on resistance to change in multiple schedules has used constant-duration components that include variable numbers of reinforcers in each presentation. Grace and Nevin (1997) examined both preference and resistance in variable-duration components; here, preference and resistance were examined in constant-duration components. Reinforcer rates were varied across eight conditions, and a generalized-matching-law analysis showed that initial-link preference strongly over-matched terminal-link reinforcer ratios. In multiple schedules, baseline response rates were unaffected by reinforcer rates, but resistance to intercomponent food, to extinction, and to intercomponent food plus extinction was greater in the richer component. The between-component difference in resistance to change exhibited additive effects for the three resistance tests, and was systematically related to reinforcer ratios. However, resistance was less sensitive to reinforcer ratios than was preference. Resistance to intercomponent food and to intercomponent food plus extinction was more sensitive to reinforcer ratios in the present study than in Grace and Nevin (1997). Thus, relative to variable-duration components, constant-duration components increased the sensitivity of both preference and relative resistance, supporting the proposition that these are independent and convergent measures of the effects of a history of reinforcement.     

Within-session Changes In Responding During Concurrent Variable-interval Schedules

Five rats and 4 pigeons responded for food delivered by several concurrent variable-interval schedules. The sum of the rates of reinforcement programmed for the two components varied from 15 to 480 reinforcers per hour in different conditions. Rates of responding usually changed within the experimental session in a similar manner for the two components of each concurrent schedule. The within-session changes were similar to previously reported changes during simple schedules that provided rates of reinforcement equal to the sum of all reinforcers obtained from the concurrent schedules. The number of changeovers also changed within sessions in a manner similar to the changes in instrumental responding. These results suggest that changeovers are governed by the same variables that govern instrumental responding. They also suggest that the within-session change in responding during each component of a concurrent schedule is determined by approximately the sum of the reinforcers obtained from both components when both components provide the same type of reinforcer.     

Performances on ratio and interval schedules of reinforcement: Data and theory

Two differences between ratio and interval performance are well known: (a) Higher rates occur on ratio schedules, and (b) ratio schedules are unable to maintain responding at low rates of reinforcement (ratio “strain”). A third phenomenon, a downturn in response rate at the highest rates of reinforcement, is well documented for ratio schedules and is predicted for interval schedules. Pigeons were exposed to multiple variable-ratio variable-interval schedules in which the intervals generated in the variable-ratio component were programmed in the variable-interval component, thereby “yoking” or approximately matching reinforcement in the two components. The full range of ratio performances was studied, from strained to continuous reinforcement. In addition to the expected phenomena, a new phenomenon was observed: an upturn in variable-interval response rate in the midrange of rates of reinforcement that brought response rates on the two schedules to equality before the downturn at the highest rates of reinforcement. When the average response rate was corrected by eliminating pausing after reinforcement, the downturn in response rate vanished, leaving a strictly monotonic performance curve. This apparent functional independence of the postreinforcement pause and the qualitative shift in response implied by the upturn in variable-interval response rate suggest that theoretical accounts will require thinking of behavior as partitioned among at least three categories, and probably four: postreinforcement activity, other unprogrammed activity, ratio-typical operant behavior, and interval-typical operant behavior.     

Choice performance in several concurrent key-peck treadle-press reinforcement schedules

Five pigeons were exposed to several concurrent variable-interval food reinforcement schedules. For three subjects, one component of the schedule required a key-pecking response, the other a treadle-pressing response. For the other two subjects, both schedule components required treadle-pressing responses. The relative probability of reinforcement associated with the manipulanda was varied from 0 to 1.0 in 13 experimental conditions for the Key-Treadle subjects and nine conditions for the Treadle-Treadle subjects. The results indicated that the logarithms of relative time spent responding, and the logarithms of relative number of responses emitted on a manipulandum, approximated direct linear functions of logarithms of the relative frequencies of reinforcement associated with that manipulandum. No systematic bias in favor of time spent key pecking over time spent treadle pressing was apparent for the Key-Treadle subjects. All subjects exhibited undermatching, in that the ratios of time and response allocation at the alternatives systematically differed from the ratios of reinforcers obtained from the alternatives in the direction of indifference. Key pecking appeared to have no special link to food beyond treadle pressing or what would be expected on the basis of the reinforcement dependencies alone.     

Within-session Response Patterns On Conjoint Variable-interval Variable-time Schedules

Operant responding often changes within sessions, even when factors such as rate of reinforcement remain constant. The present study was designed to determine whether within-session response patterns are determined by the total number of reinforcers delivered during the session or only by the reinforcers earned by the operant response. Four rats pressed a lever and 3 pigeons pecked a key for food reinforcers delivered by a conjoint variable-interval variable-time schedule. The overall rate of reinforcement of the conjoint schedule varied across conditions from 15 to 480 reinforcers per hour. When fewer than 120 reinforcers were delivered per hour, the within-session patterns of responding on conjoint schedules were similar to those previously observed when subjects received the same total number of reinforcers by responding on simple variable-interval schedules. Response patterns were less similar to those observed on simple variable-interval schedules when the overall rate of reinforcement exceeded 120 reinforcers per hour. These results suggest that response-independent reinforcers can affect the within-session pattern of responding on a response-dependent schedule. The results are incompatible with a response-based explanation of within-session changes in responding (e.g., fatigue), but are consistent with both reinforcer-based (e.g., satiation) and stimulus-based (e.g., habituation) explanations.