Pavlovian contingencies and resistance to change in a multiple schedule

According to theoretical accounts of behavioral momentum, the Pavlovian stimulus—reinforcer contingency determines resistance to change. To assess this prediction, 8 pigeons were exposed to an unsignaled delay-of-reinforcement schedule (a tandem variable-interval fixed-time schedule), a signaled delay-of-reinforcement schedule (a chain variable-interval fixed-time schedule), and an immediate, zero-delay schedule of reinforcement in a three-component multiple schedule. The unsignaled delay and signaled delay schedules employed equal fixed-time delays, with the only difference being a stimulus change in the signaled delay schedule. Overall rates of reinforcement were equated for the three schedules. The Pavlovian contingency was identical for the unsignaled and immediate schedules, and response—reinforcer contiguity was degraded for the unsignaled schedule. Results from two disruption procedures (prefeeding subjects prior to experimental sessions and adding a variable-time schedule to timeout periods separating baseline components) demonstrated that response—reinforcer contiguity does play a role in determining resistance to change. The results from the extinction manipulation were not as clear. Responding in the unsignaled delay component was consistently less resistant to change than was responding in both the immediate and presignaled segments of the signaled delay components, contrary to the view that Pavlovian contingencies determine resistance to change. Probe tests further supported the resistance-to-change results, indicating consistency between resistance to change and preference, both of which are putative measures of response strength.     

Preference,resistance to change,and the cumulative decision model

According to behavioral momentum theory (Nevin & Grace, 2000a), preference in concurrent chains and resistance to change in multiple schedules are independent measures of a common construct representing reinforcement history. Here I review the original studies on preference and resistance to change in which reinforcement variables were manipulated parametrically, conducted by Nevin, Grace and colleagues between 1997 and 2002, as well as more recent research. The cumulative decision model proposed by Grace and colleagues for concurrent chains is shown to provide a good account of both preference and resistance to change, and is able to predict the increased sensitivity to reinforcer rate and magnitude observed with constant‐duration components. Residuals from fits of the cumulative decision model to preference and resistance to change data were positively correlated, supporting the prediction of behavioral momentum theory. Although some questions remain, the learning process assumed by the cumulative decision model, in which outcomes are compared against a criterion that represents the average outcome value in the current context, may provide a plausible model for the acquisition of differential resistance to change.     

Response rate, latency, and resistance to change

Pigeons were trained on a multiple variable-interval/variable-interval schedule with pacing contingencies that generated high response rates in one component and low response rates in the other. Timeout periods separated the schedule components. During resistance-to-change tests, response-independent food was presented during the timeout periods, and the duration of that food presentation was varied among test sessions. Response rates in the schedule components decreased and latencies to the first response increased as a function of the duration of food presentations during the timeout. Both dependent measures changed about the same amount relative to their own baseline levels. The conclusions are that baseline response rates controlled by pacing contingencies are equally resistant to change, given equal reinforcement densities, and latency is a sensitive measure of resistance to change.     

Resistance to reinforcement change in multiple and concurrent schedules assessed in transition and at steady state

Behavioral momentum theory relates resistance to change of responding in a multiple-schedule component to the total reinforcement obtained in that component, regardless of how the reinforcers are produced. Four pigeons responded in a series of multiple-schedule conditions in which a variable-interval 40-s schedule arranged reinforcers for pecking in one component and a variable-interval 360-s schedule arranged them in the other. In addition, responses on a second key were reinforced according to variable-interval schedules that were equal in the two components. In different parts of the experiment, responding was disrupted by changing the rate of reinforcement on the second key or by delivering response-independent food during a blackout separating the two components. Consistent with momentum theory, responding on the first key in Part 1 changed more in the component with the lower reinforcement total when it was disrupted by changes in the rate of reinforcement on the second key. However, responding on the second key changed more in the component with the higher reinforcement total. In Parts 2 and 3, responding was disrupted with free food presented during intercomponent blackouts, with extinction (Part 2) or variable-interval 80-s reinforcement (Part 3) arranged on the second key. Here, resistance to change was greater for the component with greater overall reinforcement. Failures of momentum theory to predict short-term differences in resistance to change occurred with disruptors that caused greater change between steady states for the richer component. Consistency of effects across disruptors may yet be found if short-term effects of disruptors are assessed relative to the extent of change observed after prolonged exposure.     

Resistance to change and the law of effect

Three experiments using multiple schedules of reinforcement explored the implications of resistance-to-change findings for the response-reinforcer relation described by the law of effect, using both steady-state responding and responding recorded in the first few sessions of conditions. In Experiment 1, when response-independent reinforcement was increased during a third component, response rate in Components 1 and 2 decreased. This response-rate reduction was proportionately greater in a component in which reinforcer magnitude was small (2-s access to wheat) than in the component in which it was large (6-s access to wheat). However, when reinforcer rates in the two components were varied together in Experiments 2 and 3, response-rate change was the same regardless of the magnitude of reinforcers used in the two components, so that sensitivity of response rates to reinforcer rates (Experiment 2) and of response-rate ratios to reinforcer-rate ratios (Experiment 3) was unaffected by the magnitude of the reinforcers. Therefore, the principles determining resistance to change, described by behavioral momentum theory, seem not to apply when the source of behavior change is the variation of reinforcement contingencies that maintain the behavior. The use of extinction as a manipulation to study resistance to change is questioned.     

Preference and resistance to change with constant-duration schedule components

Previous research on preference between variable-interval terminal links in concurrent chains has most often used variable-duration terminal links ending with a single reinforcer. By contrast, most research on resistance to change in multiple schedules has used constant-duration components that include variable numbers of reinforcers in each presentation. Grace and Nevin (1997) examined both preference and resistance in variable-duration components; here, preference and resistance were examined in constant-duration components. Reinforcer rates were varied across eight conditions, and a generalized-matching-law analysis showed that initial-link preference strongly over-matched terminal-link reinforcer ratios. In multiple schedules, baseline response rates were unaffected by reinforcer rates, but resistance to intercomponent food, to extinction, and to intercomponent food plus extinction was greater in the richer component. The between-component difference in resistance to change exhibited additive effects for the three resistance tests, and was systematically related to reinforcer ratios. However, resistance was less sensitive to reinforcer ratios than was preference. Resistance to intercomponent food and to intercomponent food plus extinction was more sensitive to reinforcer ratios in the present study than in Grace and Nevin (1997). Thus, relative to variable-duration components, constant-duration components increased the sensitivity of both preference and relative resistance, supporting the proposition that these are independent and convergent measures of the effects of a history of reinforcement.     

On the relation between preference and resistance to change

Nevin (1979) noted that preference in concurrent chains and resistance to change in multiple schedules were correlated, in that both measures were affected similarly by variations in parameters of reinforcement such as rate, immediacy, and magnitude. To investigate the relationship between preference and resistance to change directly, we used a within-session procedure that arranged concurrent chains in one half of the session and a multiple schedule in the other half. The same variable-interval schedules served as terminal links in concurrent chains and as the components of the multiple schedule, and were signaled by the same stimuli. After performances had stabilized, responding in the multiple schedule was disrupted by delivering response-independent reinforcement during the blackout periods between components. Both preference in concurrent chains and relative resistance to change of multiple-schedule responding were well described as power functions of relative reinforcement rate, as predicted by current quantitative models (Grace, 1994; Nevin, 1992b). In addition, unsystematic variation in preference and resistance to change was positively correlated, which suggests that preference and resistance to change are independent measures of a single construct. That construct could be described as the learning that occurs regarding the prevailing conditions of reinforcement in a distinctive stimulus situation.     

Comparing preference and resistance to change in constant- and variable-duration schedule components

Two experiments explored preference and resistance to change in concurrent chains in which the terminal links were variable-interval schedules that ended either after a single reinforcer had been delivered (variable duration) or after a fixed period of access to the schedule (constant duration). In Experiment 1, pigeons' preference between the same pair of terminal links overmatched relative reinforcement rate when the terminal links were of constant duration, but not when they were of variable duration. Responding during the richer terminal link decreased less, relative to baseline, when response-independent food was presented during the initial links according to a variable-time schedule. In Experiment 2, all subjects consistently preferred a terminal link that consisted of 20-s access to a variable-interval 20-s schedule over a terminal link that ended after one reinforcer had been delivered by the same schedule. Results of resistance-to-change tests corresponded to preference, as responding during the constant-duration terminal link decreased less, relative to baseline, when disrupted by both response-independent food during the initial links and prefeeding. Overall, these data extend the general covariation of preference and resistance to change seen in previous studies. However, they suggest that reinforcement numerosity, including variability in the number of reinforcers per terminal-link entry, may sometimes affect preference and resistance to change in ways that are difficult to explain in terms of current models.     

Accuracy of discrimination,rate of responding,and resistance to change

Pigeons were trained on multiple schedules in which responding on a center key produced matching-to-sample trials according to the same variable-interval 30-s schedules in both components. Matching trials consisted of a vertical or tilted line sample on the center key followed by vertical and tilted comparisons on the side keys. Correct responses to comparison stimuli were reinforced with probability .80 in the rich component and .20 in the lean component. Baseline response rates and matching accuracies generally were higher in the rich component, consistent with previous research. When performance was disrupted by prefeeding, response-independent food during intercomponent intervals, intrusion of a delay between sample and comparison stimuli, or extinction, both response rates and matching accuracies generally decreased. Proportions of baseline response rate were greater in the rich component for all disrupters except delay, which had relatively small and inconsistent effects on response rate. By contrast, delay had large and consistent effects on matching accuracy, and proportions of baseline matching accuracy were greater in the rich component for all four disrupters. The dissociation of response rate and accuracy with delay reflects the localized impact of delay on matching performance. The similarity of the data for response rate and accuracy with prefeeding, response-independent food, and extinction shows that matching performance, like response rate, is more resistant to change in a rich than in a lean component. This result extends resistance to change analyses from the frequency of response emission to the degree of stimulus control, and suggests that the strength of discriminating, like the strength of responding, is positively related to rate of reinforcement.     

Effects of unsignaled delay of reinforcement on preference and resistance to change

In Phase 1, pigeons were trained on a concurrent chain in which a 3-s unsignaled delay of reinforcement was imposed on responding in a terminal link in some conditions. Preference for that terminal link was always reduced in comparison with conditions in which there was no delay, substantially so for 3 of the 4 pigeons. In Phase 2, pigeons responded in a two-component multiple schedule. The scheduled rates of reinforcement were equal, but a 3-s unsignaled delay was imposed in one component. Resistance of responding to prefeeding and extinction was reduced in the delay component for the same 3 subjects for which the data had shown strong effects of delay on preference. Systematic observation revealed differences in response topography. In the delay component, subjects oriented more closely to the key and responses were less forceful compared with the no-delay component. Our results give further evidence that preference and resistance to change covary within subjects. However, they challenge the premise that the critical determiners of preference (i.e., terminal-link value) and resistance to change (behavioral mass) may be quantified purely in terms of stimulus—reinforcer relations.     

Preference and resistance to change with constant- and variable-duration terminal links: independence of reinforcement rate and magnitude

Pigeons responded in a three-component multiple concurrent-chains procedure in which the variable-interval reinforcement schedules were the same across components but magnitudes differed across components. The terminal links were arranged either as a variable delay followed by presentation of a reinforcer ("variable duration") or as a fixed period of access to the schedule during which a variable number of reinforcers could be earned ("constant duration"). Relative reinforcement rate was varied parametrically across both types of conditions. After baseline training in each condition, resistance to change of terminal-link responding was assessed by delivering food during the initial links according to a variable-time schedule. Both preference and resistance to change were more sensitive to reinforcement-rate differences in the constant-duration conditions. Sensitivities of preference and resistance to change to relative reinforcement rate did not change depending on relative reinforcement magnitude. Taken together, these results confirm and extend those of prior studies, and suggest that reinforcement rate and magnitude combine additively to determine preference and resistance to change. A single structural relation linking preference and resistance to change describes all the data from this and several related studies.     

Response-independent milk delivery enhances persistence of pellet-reinforced lever pressing by rats

If, during training, one stimulus is correlated with a higher rate of reinforcement than another, responding will be more resistant to extinction in the presence of that higher rate signal, even if many of the reinforcers have been presented independently of responding. For the present study we asked if the response-independent reinforcers must be the same as the response-dependent reinforcers to enhance the response's persistence. Twelve Long-Evans hooded rats obtained 45-mg food pellets by lever pressing (variable-interval 100-s schedules) in the presence of two discriminative stimuli (blinking vs. steady lights) that alternated every minute during daily sessions. Also, in the presence of one of the stimuli (counterbalanced across rats), the rats received additional response-independent deliveries of sweetened condensed milk (a variable-time schedule). Extinction sessions were exactly like training sessions except that neither pellets nor milk were presented. Lever pressing was more resistant to extinction in the presence of the milk-correlated stimulus when (a) the size of the milk deliveries during training (under a variable-time 30 s schedule) was 0.04 ml (vs. 0.01 ml) and (b) 120-s or 240-s blackouts separated components. Response-independent reinforcers do not have to be the same as the response-dependent reinforcers to enhance persistence.     

Resistance to change of responding maintained by unsignaled delays to reinforcement: a response-bout analysis

Previous experiments have shown that unsignaled delayed reinforcement decreases response rates and resistance to change. However, the effects of different delays to reinforcement on underlying response structure have not been investigated in conjunction with tests of resistance to change. In the present experiment, pigeons responded on a three-component multiple variable-interval schedule for food presented immediately, following brief (0.5 s), or following long (3 s) unsignaled delays of reinforcement. Baseline response rates were lowest in the component with the longest delay; they were about equal with immediate and briefly delayed reinforcers. Resistance to disruption by presession feeding, response-independent food during the intercomponent interval, and extinction was slightly but consistently lower as delays increased. Because log survivor functions of interresponse times (IRTs) deviated from simple modes of bout initiations and within-bout responding, an IRT-cutoff method was used to examine underlying response structure. These analyses suggested that baseline rates of initiating bouts of responding decreased as scheduled delays increased, and within-bout response rates tended to be lower in the component with immediate reinforcers. The number of responses per bout was not reliably affected by reinforcer delay, but tended to be highest with brief delays when total response rates were higher in that component. Consistent with previous findings, resistance to change of overall response rate was highly correlated with resistance to change of bout-initiation rates but not with within-bout responding. These results suggest that unsignaled delays to reinforcement affect resistance to change through changes in the probability of initiating a response bout rather than through changes in the underlying response structure.     

Conditioned reinforcement value and resistance to change

Three experiments examined the effects of conditioned reinforcement value and primary reinforcement rate on resistance to change using a multiple schedule of observing-response procedures with pigeons. In the absence of observing responses in both components, unsignaled periods of variable-interval (VI) schedule food reinforcement alternated with extinction. Observing responses in both components intermittently produced 15 s of a stimulus associated with the VI schedule (i.e., S+). In the first experiment, a lower-valued conditioned reinforcer and a higher rate of primary reinforcement were arranged in one component by adding response-independent food deliveries uncorrelated with S+. In the second experiment, one component arranged a lower valued conditioned reinforcer but a higher rate of primary reinforcement by increasing the probability of VI schedule periods relative to extinction periods. In the third experiment, the two observing-response components provided similar rates of primary reinforcement but arranged different valued conditioned reinforcers. Across the three experiments, observing-response rates were typically higher in the component associated with the higher valued conditioned reinforcer. Resistance to change was not affected by conditioned reinforcement value, but was an orderly function of the rate of primary reinforcement obtained in the two components. One interpretation of these results is that S+ value does not affect response strength and that S+ deliveries increase response rates through a mechanism other than reinforcement. Alternatively, because resistance to change depends on the discriminative stimulus-reinforcer relation, the failure of S+ value to impact resistance to change could have resulted from a lack of transfer of S+ value to the broader discriminative context.     

Resistance to extinction versus extinction as discrimination

The hypothesis that response strength might be measured by persistence of responding in the face of extinction was discredited in the 1960s because experiments showed that responding persists longer following intermittent reinforcers than following continuous reinforcers. Instead, researchers proposed that the longer persistence following intermittent reinforcers arises because intermittent reinforcement more closely resembles extinction—a discrimination theory. Attention to resistance to extinction revived because one observation seemed to support the persistence hypothesis: Following training on a multiple schedule with unequal components, responding usually persisted longer in the formerly richer component than in the formerly lean component. This observation represents an anomaly, however, because results with single schedules and concurrent schedules contradict it. We suggest that the difference in results arises because the multiple-schedule procedure, while including extensive training on stimulus discrimination, includes no training on discrimination between food available and food unavailable, whereas comparable single- and concurrent-schedule procedures include such training with repeated extinction. In Experiment 1, we replicated the original result, and in Experiment 2 showed that when the multiple-schedule procedure includes training on food/no-food discrimination, extinction following multiple schedules contradicts behavioral momentum theory and agrees with the discrimination theory and research with single and concurrent schedules.     

Variable-ratio versus variable-interval schedules: response rate, resistance to change, and preference

Two experiments asked whether resistance to change depended on variable-ratio as opposed to variable-interval contingencies of reinforcement and the different response rates they establish. In Experiment 1, pigeons were trained on multiple random-ratio random-interval schedules with equated reinforcer rates. Baseline response rates were disrupted by intercomponent food, extinction, and prefeeding. Resistance to change relative to baseline was greater in the interval component, and the difference was correlated with the extent to which baseline response rates were higher in the ratio component. In Experiment 2, pigeons were trained on multiple variable-ratio variable-interval schedules in one half of each session and on concurrent chains in the other half in which the terminal links corresponded to the multiple-schedule components. The schedules were varied over six conditions, including two with equated reinforcer rates. In concurrent chains, preference strongly overmatched the ratio of obtained reinforcer rates. In multiple schedules, relative resistance to response-independent food during intercomponent intervals, extinction, and intercomponent food plus extinction depended on the ratio of obtained reinforcer rates but was less sensitive than was preference. When reinforcer rates were similar, both preference and relative resistance were greater for the variable-interval schedule, and the differences were correlated with the extent to which baseline response rates were higher on the variable-ratio schedule, confirming the results of Experiment 1. These results demonstrate that resistance to change and preference depend in part on response rate as well as obtained reinforcer rate, and challenge the independence of resistance to change and preference with respect to response rate proposed by behavioral momentum theory.     

Baseline response rates affect resistance to change

The effect of response rates on resistance to change, measured as resistance to extinction, was examined in two experiments. In Experiment 1, responding in transition from a variable‐ratio schedule and its yoked‐interval counterpart to extinction was compared with pigeons. Following training on a multiple variable‐ratio yoked‐interval schedule of reinforcement, in which response rates were higher in the former component, reinforcement was removed from both components during a single extended extinction session. Resistance to extinction in the yoked‐interval component was always either greater or equal to that in the variable‐ratio component. In Experiment 2, resistance to extinction was compared for two groups of rats that exhibited either high or low response rates when maintained on identical variable‐interval schedules. Resistance to extinction was greater for the lower‐response‐rate group. These results suggest that baseline response rate can contribute to resistance to change. Such effects, however, can only be revealed when baseline response rate and reinforcement rate are disentangled (Experiments 1 and 2) from the more usual circumstance where the two covary. Furthermore, they are more cleanly revealed when the programmed contingencies controlling high and low response rates are identical, as in Experiment 2.     

Behavioral momentum: the effects of the temporal separation of rates of reinforcement.

In Part 1 of the experiment, rats responded under a variable-interval (VI) 30-s schedule and a VI 120-s schedule, with each in effect for a block of consecutive sessions. That is, the two VI schedules were presented in successive conditions. In Part 2 the VI schedules alternated each day, and in Part 3 the schedules alternated within the session as a multiple schedule. For half of the rats in Parts 1 and 2, the VI schedule alternated every few minutes within the session with a stimulus that signaled extinction. For each part, once response rates had stabilized, resistance to change was measured by prefeeding and extinction. When the schedules were examined in successive conditions (Part 1), resistance to extinction was greater under the VI 120-s schedule of reinforcement than under the VI 30-s schedule, but no consistent differences in resistance to prefeeding were observed between the two VI schedules. When the VI schedules alternated each day (Part 2), resistance to extinction was greater under the VI 120-s schedule. However, no consistent differences in resistance to prefeeding were observed between the VI schedules without extinction in Group A, but resistance to prefeeding was greater under the VI 30-s schedule for rats with the added extinction component in Group B. When the VI schedules alternated within the session as a multiple schedule (Part 3), resistance to extinction and resistance to prefeeding were greater under the VI 30-s schedule. The data suggest that different rates of reinforcement, and their accompanying discriminative stimuli, must be compared within the same session (or at least on alternate days) to produce data consistent with the behavioral momentum model.     

Resistance to change and preference for variable versus fixed response sequences

In Experiment 1, 4 pigeons were trained on a multiple chain schedule in which the initial link was a variable-interval (VI) 20-s schedule signalled by a red or green center key, and terminal links required four responses made to the left (L) and/or right (R) keys. In the REPEAT component, signalled by red keylights, only LRLR terminal-link response sequences were reinforced, while in the VARY component, signalled by green keylights, terminal-link response sequences were reinforced if they satisfied a variability criterion. The reinforcer rate for both components was equated by adjusting the reinforcer probability for correct REPEAT sequences across sessions. Results showed that initial- and terminal-link responding in the VARY component was generally more resistant to prefeeding, extinction, and response-independent food than responding in the REPEAT component. In Experiment 2, the REPEAT and VARY contingencies were arranged as terminal links of a concurrent chain and the relative reinforcer rate was manipulated across conditions. For all pigeons, initial-link response allocation was biased toward the alternative associated with the VARY terminal link. These results replicate previous reports that operant variation is more resistant to change than operant repetition (Doughty & Lattal, 2001), and show that variation is preferred to repetition with reinforcer-related variables controlled. Behavioral momentum theory (Nevin & Grace, 2000) predicts the covariation of preference and resistance to change in Experiments 1 and 2, but does not explain why these aspects of behavior should depend on contingencies that require repetition or variation.