Behavioral economics and behavioral momentum

Some relations between elasticity of demand and the conditions of reinforcement are reanalyzed in terms of resistance to change, in ways suggested by the metaphor of behavioral momentum; some relations between resistance to change and the conditions of reinforcement are reanalyzed in terms of elasticity of demand, in ways suggested by behavioral economics. In addition, some data on labor supply in relation to variable-ratio schedules and alternative reinforcement are reanalyzed in terms of resistance to change and compared with steady-state resistance data for performance on multiple and concurrent interval schedules. The results of these studies can be summarized by two functions based on the behavioral momentum approach, relating relative behavioral mass to relative reinforcement per response or per unit time. The former is a relation between relative unit price and relative behavioral mass, suggesting the possibility of convergent measurement of a theoretical construct common to both approaches. However, the momentum and economic approaches differ fundamentally on whether it is preferable to construe discriminated operant behavior as selected and strengthened by its consequences or as part of a behavior–consequence bundle that maximizes utility.     

Preference,resistance to change,and the cumulative decision model

According to behavioral momentum theory (Nevin & Grace, 2000a), preference in concurrent chains and resistance to change in multiple schedules are independent measures of a common construct representing reinforcement history. Here I review the original studies on preference and resistance to change in which reinforcement variables were manipulated parametrically, conducted by Nevin, Grace and colleagues between 1997 and 2002, as well as more recent research. The cumulative decision model proposed by Grace and colleagues for concurrent chains is shown to provide a good account of both preference and resistance to change, and is able to predict the increased sensitivity to reinforcer rate and magnitude observed with constant‐duration components. Residuals from fits of the cumulative decision model to preference and resistance to change data were positively correlated, supporting the prediction of behavioral momentum theory. Although some questions remain, the learning process assumed by the cumulative decision model, in which outcomes are compared against a criterion that represents the average outcome value in the current context, may provide a plausible model for the acquisition of differential resistance to change.     

On the relation between preference and resistance to change

Nevin (1979) noted that preference in concurrent chains and resistance to change in multiple schedules were correlated, in that both measures were affected similarly by variations in parameters of reinforcement such as rate, immediacy, and magnitude. To investigate the relationship between preference and resistance to change directly, we used a within-session procedure that arranged concurrent chains in one half of the session and a multiple schedule in the other half. The same variable-interval schedules served as terminal links in concurrent chains and as the components of the multiple schedule, and were signaled by the same stimuli. After performances had stabilized, responding in the multiple schedule was disrupted by delivering response-independent reinforcement during the blackout periods between components. Both preference in concurrent chains and relative resistance to change of multiple-schedule responding were well described as power functions of relative reinforcement rate, as predicted by current quantitative models (Grace, 1994; Nevin, 1992b). In addition, unsystematic variation in preference and resistance to change was positively correlated, which suggests that preference and resistance to change are independent measures of a single construct. That construct could be described as the learning that occurs regarding the prevailing conditions of reinforcement in a distinctive stimulus situation.     

Temporal context, preference, and resistance to change

According to behavioral momentum theory, preference and relative resistance to change in concurrent-chains schedules are correlated and reflect the relative conditioned value of discriminative stimuli. In the present study, we explore the generality of this relation by manipulating the temporal context within a concurrent-chains procedure through changes in the duration of the initial links. Consistent with previous findings, preference for a richer terminal link was less extreme with longer initial links across three experiments with pigeons. In Experiment 1, relative resistance to change and preference were related inversely when responding was disrupted with response-independent food presentations during initial links, replicating a previous finding with rats. However, more food was presented with longer initial links, confounding the disrupter and initial-link duration. In Experiment 2, presession feeding was used instead and eliminated the negative relation between relative resistance to change and preference, but relative resistance to change was not sensitive to relative terminal-link reinforcement rates. In Experiment 3, with more extreme relative terminal-link reinforcement rates, increasing initial-link duration similarly decreased preference and relative resistance to change for the richer terminal link. Thus, when conditions of disruption are equal and assessed under the appropriate reinforcement conditions, changes in temporal context impact relative resistance to change and preference similarly.     

Preference and resistance to change with constant- and variable-duration terminal links: independence of reinforcement rate and magnitude

Pigeons responded in a three-component multiple concurrent-chains procedure in which the variable-interval reinforcement schedules were the same across components but magnitudes differed across components. The terminal links were arranged either as a variable delay followed by presentation of a reinforcer ("variable duration") or as a fixed period of access to the schedule during which a variable number of reinforcers could be earned ("constant duration"). Relative reinforcement rate was varied parametrically across both types of conditions. After baseline training in each condition, resistance to change of terminal-link responding was assessed by delivering food during the initial links according to a variable-time schedule. Both preference and resistance to change were more sensitive to reinforcement-rate differences in the constant-duration conditions. Sensitivities of preference and resistance to change to relative reinforcement rate did not change depending on relative reinforcement magnitude. Taken together, these results confirm and extend those of prior studies, and suggest that reinforcement rate and magnitude combine additively to determine preference and resistance to change. A single structural relation linking preference and resistance to change describes all the data from this and several related studies.     

The analysis of behavioral momentum

Learned behavior varies in its resistance to change, depending on the rate of reinforcement. Resistance to change may be characterized as behavioral momentum, which in turn may be analyzed into terms corresponding to mass and velocity in classical physics. Behavioral mass may be inferred from changes in response rate when experimental conditions are altered. Relevant data were obtained by training pigeons to peck a key on two-component multiple variable-interval, variable-interval schedules. Six pigeons were studied on three pairs of variable-interval schedules in all possible orders. When performance stabilized, resistance to change was assessed by arranging response-independent food during periods between components and by extinction. For each operation, the data for all schedule performances converged onto a single function, permitting estimation of the ratio of behavioral masses for each pair of schedules. The response-independent food data suggested that the ratio of behavioral masses is a power function of the ratio of reinforcement rates and that behavioral mass may be measured on a ratio scale.     

Assessing potential reinforcement-like effects of brief stimuli unrelated to food reinforcers

It is widely assumed that reinforcers are biologically relevant stimuli, or stimuli that have been associated with biologically relevant stimuli. However, brief, arbitrary stimuli have also been reported to have reinforcement-like effects, despite being unrelated to biologically relevant stimuli like food. The present study explored the potential reinforcement-like effects of brief stimuli across 5 experiments. In Experiments 1 through 4, pigeon subjects responded for food reinforcement and brief stimulus presentations in a 2-component multiple schedule. Neither baseline response rates nor resistance to change during disruption tests were systematically greater in a component with versus without brief stimulus presentations. Increasing the rate and duration of brief stimulus presentations in Experiment 4 did not reveal reinforcement-like effects when compared directly with food. In Experiment 5, pigeons chose between independent terminal links in a concurrent-chains procedure. Across conditions, varying the location, duration, and rate of brief stimulus presentations in the terminal links had no systematic effects on preference. In contrast, varying rates of food reinforcers resulted in large and reliable shifts in preference. Therefore, the present study found no systematic evidence that brief stimuli unrelated to food reliably increase response rates, resistance to change, or preference. These data demonstrate the value of systematic replication, and a behavioral momentum approach to assessing potential reinforcement-like effects.     

Resistance to reinforcement change in multiple and concurrent schedules assessed in transition and at steady state

Behavioral momentum theory relates resistance to change of responding in a multiple-schedule component to the total reinforcement obtained in that component, regardless of how the reinforcers are produced. Four pigeons responded in a series of multiple-schedule conditions in which a variable-interval 40-s schedule arranged reinforcers for pecking in one component and a variable-interval 360-s schedule arranged them in the other. In addition, responses on a second key were reinforced according to variable-interval schedules that were equal in the two components. In different parts of the experiment, responding was disrupted by changing the rate of reinforcement on the second key or by delivering response-independent food during a blackout separating the two components. Consistent with momentum theory, responding on the first key in Part 1 changed more in the component with the lower reinforcement total when it was disrupted by changes in the rate of reinforcement on the second key. However, responding on the second key changed more in the component with the higher reinforcement total. In Parts 2 and 3, responding was disrupted with free food presented during intercomponent blackouts, with extinction (Part 2) or variable-interval 80-s reinforcement (Part 3) arranged on the second key. Here, resistance to change was greater for the component with greater overall reinforcement. Failures of momentum theory to predict short-term differences in resistance to change occurred with disruptors that caused greater change between steady states for the richer component. Consistency of effects across disruptors may yet be found if short-term effects of disruptors are assessed relative to the extent of change observed after prolonged exposure.     

A theory of attending and reinforcement in conditional discriminations

A model of conditional discrimination performance (Davison & Nevin, 1999) is combined with the notion that unmeasured attending to the sample and comparison stimuli, in the steady state and during disruption, depends on reinforcement in the same way as predicted for overt free-operant responding by behavioral momentum theory (Nevin & Grace, 2000). The rate of observing behavior, a measurable accompaniment of attending, is well described by an equation for steady-state responding derived from momentum theory, and the resistance to change of observing conforms to predictions of momentum theory, supporting a key assumption of the model. When probabilities of attending are less than 1.0, the model accounts for some aspects of conditional-discrimination performance that posed problems for the Davison-Nevin model: (a) the effects of differential reinforcement on the allocation of responses to the comparison stimuli and on accuracy in several matching-to-sample and signal-detection tasks where the differences between the stimuli or responses were varied across conditions, (b) the effects of overall reinforcer rate on the asymptotic level and resistance to change of both response rate and accuracy of matching to sample in multiple schedules, and (c) the effects of fixed-ratio reinforcement on accuracy. Some tests and extensions of the model are suggested, and the role of unmeasured events in behavior theory is considered.     

Variable-ratio versus variable-interval schedules: response rate, resistance to change, and preference

Two experiments asked whether resistance to change depended on variable-ratio as opposed to variable-interval contingencies of reinforcement and the different response rates they establish. In Experiment 1, pigeons were trained on multiple random-ratio random-interval schedules with equated reinforcer rates. Baseline response rates were disrupted by intercomponent food, extinction, and prefeeding. Resistance to change relative to baseline was greater in the interval component, and the difference was correlated with the extent to which baseline response rates were higher in the ratio component. In Experiment 2, pigeons were trained on multiple variable-ratio variable-interval schedules in one half of each session and on concurrent chains in the other half in which the terminal links corresponded to the multiple-schedule components. The schedules were varied over six conditions, including two with equated reinforcer rates. In concurrent chains, preference strongly overmatched the ratio of obtained reinforcer rates. In multiple schedules, relative resistance to response-independent food during intercomponent intervals, extinction, and intercomponent food plus extinction depended on the ratio of obtained reinforcer rates but was less sensitive than was preference. When reinforcer rates were similar, both preference and relative resistance were greater for the variable-interval schedule, and the differences were correlated with the extent to which baseline response rates were higher on the variable-ratio schedule, confirming the results of Experiment 1. These results demonstrate that resistance to change and preference depend in part on response rate as well as obtained reinforcer rate, and challenge the independence of resistance to change and preference with respect to response rate proposed by behavioral momentum theory.     

Effects of unsignaled delay of reinforcement on preference and resistance to change

In Phase 1, pigeons were trained on a concurrent chain in which a 3-s unsignaled delay of reinforcement was imposed on responding in a terminal link in some conditions. Preference for that terminal link was always reduced in comparison with conditions in which there was no delay, substantially so for 3 of the 4 pigeons. In Phase 2, pigeons responded in a two-component multiple schedule. The scheduled rates of reinforcement were equal, but a 3-s unsignaled delay was imposed in one component. Resistance of responding to prefeeding and extinction was reduced in the delay component for the same 3 subjects for which the data had shown strong effects of delay on preference. Systematic observation revealed differences in response topography. In the delay component, subjects oriented more closely to the key and responses were less forceful compared with the no-delay component. Our results give further evidence that preference and resistance to change covary within subjects. However, they challenge the premise that the critical determiners of preference (i.e., terminal-link value) and resistance to change (behavioral mass) may be quantified purely in terms of stimulus—reinforcer relations.     

Experience with dynamic reinforcement rates decreases resistance to extinction

The ability of organisms to detect reinforcer‐rate changes in choice preparations is positively related to two factors: the magnitude of the change in rate and the frequency with which rates change. Gallistel (2012) suggested similar rate‐detection processes are responsible for decreases in responding during operant extinction. Although effects of magnitude of change in reinforcer rate on resistance to extinction are well known (e.g., the partial‐reinforcement‐extinction effect), effects of frequency of changes in rate prior to extinction are unknown. Thus, the present experiments examined whether frequency of changes in baseline reinforcer rates impacts resistance to extinction. Pigeons pecked keys for variable‐interval food under conditions where reinforcer rates were stable and where they changed within and between sessions. Overall reinforcer rates between conditions were controlled. In Experiment 1, resistance to extinction was lower following exposure to dynamic reinforcement schedules than to static schedules. Experiment 2 showed that resistance to presession feeding, a disruptor that should not involve change‐detection processes, was unaffected by baseline‐schedule dynamics. These findings are consistent with the suggestion that change detection contributes to extinction. We discuss implications of change‐detection processes for extinction of simple and discriminated operant behavior and relate these processes to the behavioral‐momentum based approach to understanding extinction.     

Preference and resistance to change with constant-duration schedule components

Previous research on preference between variable-interval terminal links in concurrent chains has most often used variable-duration terminal links ending with a single reinforcer. By contrast, most research on resistance to change in multiple schedules has used constant-duration components that include variable numbers of reinforcers in each presentation. Grace and Nevin (1997) examined both preference and resistance in variable-duration components; here, preference and resistance were examined in constant-duration components. Reinforcer rates were varied across eight conditions, and a generalized-matching-law analysis showed that initial-link preference strongly over-matched terminal-link reinforcer ratios. In multiple schedules, baseline response rates were unaffected by reinforcer rates, but resistance to intercomponent food, to extinction, and to intercomponent food plus extinction was greater in the richer component. The between-component difference in resistance to change exhibited additive effects for the three resistance tests, and was systematically related to reinforcer ratios. However, resistance was less sensitive to reinforcer ratios than was preference. Resistance to intercomponent food and to intercomponent food plus extinction was more sensitive to reinforcer ratios in the present study than in Grace and Nevin (1997). Thus, relative to variable-duration components, constant-duration components increased the sensitivity of both preference and relative resistance, supporting the proposition that these are independent and convergent measures of the effects of a history of reinforcement.     

Behavioral momentum theory: equations and applications

Behavioral momentum theory provides a quantitative account of how reinforcers experienced within a discriminative stimulus context govern the persistence of behavior that occurs in that context. The theory suggests that all reinforcers obtained in the presence of a discriminative stimulus increase resistance to change, regardless of whether those reinforcers are contingent on the target behavior, are noncontingent, or are even contingent on an alternative behavior. In this paper, we describe the equations that constitute the theory and address their application to issues of particular importance in applied settings. The theory provides a framework within which to consider the effects of interventions such as extinction, noncontingent reinforcement, differential reinforcement of alternative behavior, and other phenomena (e.g., resurgence). Finally, the theory predicts some counterintuitive and potentially counterproductive effects of alternative reinforcement, and can serve as an integrative guide for intervention when its terms are identified with the relevant conditions of applied settings.     

Pavlovian contingencies and resistance to change in a multiple schedule

According to theoretical accounts of behavioral momentum, the Pavlovian stimulus—reinforcer contingency determines resistance to change. To assess this prediction, 8 pigeons were exposed to an unsignaled delay-of-reinforcement schedule (a tandem variable-interval fixed-time schedule), a signaled delay-of-reinforcement schedule (a chain variable-interval fixed-time schedule), and an immediate, zero-delay schedule of reinforcement in a three-component multiple schedule. The unsignaled delay and signaled delay schedules employed equal fixed-time delays, with the only difference being a stimulus change in the signaled delay schedule. Overall rates of reinforcement were equated for the three schedules. The Pavlovian contingency was identical for the unsignaled and immediate schedules, and response—reinforcer contiguity was degraded for the unsignaled schedule. Results from two disruption procedures (prefeeding subjects prior to experimental sessions and adding a variable-time schedule to timeout periods separating baseline components) demonstrated that response—reinforcer contiguity does play a role in determining resistance to change. The results from the extinction manipulation were not as clear. Responding in the unsignaled delay component was consistently less resistant to change than was responding in both the immediate and presignaled segments of the signaled delay components, contrary to the view that Pavlovian contingencies determine resistance to change. Probe tests further supported the resistance-to-change results, indicating consistency between resistance to change and preference, both of which are putative measures of response strength.     

ALTERNATIVE REINFORCEMENT INCREASES RESISTANCE TO CHANGE: PAVLOVIAN OR OPERANT CONTINGENCIES?

Two multiple-schedule experiments with pigeons examined the effect of adding food reinforcement from an alternative source on the resistance of the reinforced response (target response) to the decremental effects of satiation and extinction. In Experiment 1, key pecks were reinforced by food in two components according to variable-interval schedules and, in some conditions, food was delivered according to variable-time schedules in one of the components. The rate of key pecking in a component was negatively related to the proportion of reinforcers from the alternative (variable-time) source. Resistance to satiation and extinction, in contrast, was positively related to the overall rate of reinforcement in the component. Experiment 2 was conceptually similar except that the alternative reinforcers were contingent on a specific concurrent response. Again, the rate of the target response varied as a function of its relative reinforcement, but its resistance to satiation and extinction varied directly with the overall rate of reinforcement in the component stimulus regardless of its relative reinforcement. Together the results of the two experiments suggest that the relative reinforcement of a response (the operant contingency) determines its rate, whereas the stimulus-reinforcement contingency (a Pavlovian contingency) determines its resistance to change.     

Persistence of stereotypic behavior: examining the effects of external reinforcers

Basic research has shown that behavioral persistence is often positively related to rate of reinforcement. This relation, expressed in the metaphor of behavioral momentum, has potentially important implications for clinical application. The current study examined one prediction of the momentum metaphor for automatically reinforced behavior. Participants were 3 children who had been diagnosed with an autism spectrum disorder and who engaged in stereotypic behavior maintained by automatic reinforcement. Results suggested that stereotypic behavior was more resistant to disruption following periods of access to preferred stimuli delivered on a variable-time schedule than following periods without access to preferred stimuli. The implications of these findings for the treatment of automatically reinforced behavior are discussed.     

Behavioral momentum in computer-presented discriminations in individuals with severe mental retardation

Behavioral momentum was examined in 2 individuals with severe mental retardation via within-subject manipulations of obtained reinforcer rates. Subjects performed self-paced discrimination problems presented on a touch screen computer monitor. Two different problems, Tasks A and B, alternated in blocks of 15 trials on a multiple schedule. Reinforcers were snack foods. The reinforcement schedule for Task A was continuous (fixed-ratio 1) and the schedule for Task B was continuous in some conditions and variable ratio in other conditions. Behavioral momentum was assessed in test sessions by prefeeding, presenting response-independent food, and making available alternatives to the tasks. When the obtained reinforcer rate for Task A was at least twice that for Task B, resistance to change was greater for Task A. When both reinforcer rates and response rates were a pproximately equal for the two tasks, resistance to change was approximately equal. These results are consistent with behavioral momentum effects. They extend previous findings with humans by examining momentum in self-initiated discrete-trial discrimination tasks with ratio schedules, and by isolating relative reinforcer rates as a controlling variable via within-subject manipulations.     

Informational and motivational components of social reinforcement

Thirty-six four-year-old children of each sex were tested in a two-choice marble dropping task. There were three Ss in each cell of a 3 × 2 × 2 × 2 factorial design. The factors investigated were: reinforcement condition (Contingent, Yoked, Nonreinforcement), sex, base preference level (strong vs weak), Base Rate Level (high vs low). The dependent variables were: base preference ratio, base rate, preference ratio change, rate change. The contingently reinforced Ss had significantly higher preference change scores than Ss in the other two reinforcement conditions but only at the high base preference level. The Ss in both the contingent and yoked groups had higher rate change scores than Ss in the nonreinforcement group. The results were interpreted as indicating social reinforcement may have two effects, one a cue function and the other an effect on S's motivational system. The results indicate that regression effects do not operate in the two-choice task and that crossing baseline levels of performance is an effective way to control baseline differences in analyses of change in the two-choice task.