Reinforcement magnitude: an evaluation of preference and reinforcer efficacy

Consideration of reinforcer magnitude may be important for maximizing the efficacy of treatment for problem behavior. Nonetheless, relatively little is known about children's preferences for different magnitudes of social reinforcement or the extent to which preference is related to differences in reinforcer efficacy. The purpose of the current study was to evaluate the relations among reinforcer magnitude, preference, and efficacy by drawing on the procedures and results of basic experimentation in this area. Three children who engaged in problem behavior that was maintained by social positive reinforcement (attention, access to tangible items) participated. Results indicated that preference for different magnitudes of social reinforcement may predict reinforcer efficacy and that magnitude effects may be mediated by the schedule requirement.     

Multiple determinants of the effects of reinforcement magnitude on free-operant response rates

Four experiments examined the effects of increasing the number of food pellets given to hungry rats for a lever-press response. On a simple variable-interval 60-s schedule, increased number of pellets depressed response rates (Experiment 1). In Experiment 2, the decrease in response rate as a function of increased reinforcement magnitude was demonstrated on a variable-interval 30-s schedule, but enhanced rates of response were obtained with the same increase in reinforcement magnitude on a variable-ratio 30 schedule. In Experiment 3, higher rates of responding were maintained by the component of a concurrent variable-interval 60-s variable-interval 60-s schedule associated with a higher reinforcement magnitude. In Experiment 4, higher rates of response were produced in the component of a multiple variable-interval 60-s variable-interval 60-s schedule associated with the higher reinforcement magnitude. It is suggested that on simple schedules greater reinforcer magnitudes shape the reinforced pattern of responding more effectively than do smaller reinforcement magnitudes. This effect is, however, overridden by another process, such a contrast, when two magnitudes are presented within a single session on two-component schedules.     

Frequency versus magnitude of reinforcement: New data with a different procedure

Two pigeons, with previous exposure to concurrent schedules, were submitted to 29 sessions of 8 hours each with concurrent variable-interval variable-interval schedules in which reinforcement parameters changed from session to session. In the first nine sessions reinforcement durations were equal in both schedules while reinforcement frequencies varied; in Sessions 10 through 18, both frequency and duration of reinforcement were varied; in Sessions 19 through 29, only reinforcement duration was varied. Results with this different procedure confirm previous findings that behavior is more sensitive to changes in reinforcement frequency than to reinforcement magnitude.     

Reinforcement magnitude and pausing on progressive-ratio schedules

Rats responded under progressive-ratio schedules for sweetened milk reinforcers; each session ended when responding ceased for 10 min. Experiment 1 varied the concentration of milk and the duration of postreinforcement timeouts. Postreinforcement pausing increased as a positively accelerated function of the size of the ratio, and the rate of increase was reduced as a function of concentration and by timeouts of 10 s or longer. Experiment 2 varied reinforcement magnitude within sessions (number of dipper operations per reinforcer) in conjunction with stimuli correlated with the upcoming magnitude. In the absence of discriminative stimuli, pausing was longer following a large reinforcer than following a small one. Pauses were reduced by a stimulus signaling a large upcoming reinforcer, particularly at the highest ratios, and the animals tended to quit responding when the past reinforcer was large and the stimulus signaled that the next one would be small. Results of both experiments revealed parallels between responding under progressive-ratio schedules and other schedules containing ratio contingencies. Relationships between pausing and magnitude suggest that ratio pausing is under the joint control of inhibitory properties of the past reinforcer and excitatory properties of stimuli correlated with the upcoming reinforcer, rather than under the exclusive control of either factor alone.     

Negative reinforcer magnitude manipulations for treating escape-maintained problem behavior

The purpose of the current study was to evaluate the effects of different magnitudes of escape for compliance relative to the magnitudes of escape for problem behavior in a concurrent-schedule arrangement. Three individuals who exhibited escape-maintained problem behavior participated. A large differential magnitude condition (240-s escape for compliance, 10-s escape for problem behavior) was compared to equal (30-s escape for compliance and problem behavior) and moderate differential magnitude (90-s escape for compliance, 10-s escape for problem behavior) conditions. The authors also evaluated the impact of correcting for reinforcer access time (i.e., time on escape intervals) on intervention interpretation. For all participants, problem behavior decreased during only the large differential magnitude condition, and including reinforcer access time in the overall session time did not affect interpretation of treatment outcomes. Providing larger escape magnitudes for compliance relative to problem behavior may facilitate treatment involving concurrent-reinforcement schedules for escape-maintained problem behavior.     

Reinforcement magnitude and responding during treatment with differential reinforcement

Basic findings indicate that the amount or magnitude of reinforcement can influence free-operant responding prior to and during extinction. In this study, the relation between reinforcement magnitude and adaptive behavior was evaluated with 3 children as part of treatment with differential reinforcement. In the first experiment, a communicative response was shaped and maintained by the same reinforcer that was found to maintain problem behavior. Two reinforcement magnitudes (20-s or 60-s access to toys or escape from demands) were compared and found to be associated with similar levels of resistance to extinction. The relation between reinforcement magnitude and response maintenance was further evaluated in the second experiment by exposing the communicative response to 20-s or 300-s access to toys or escape. Results for 2 participants suggested that this factor may alter the duration of postreinforcement pauses.     

Relative reinforcer magnitude under a nonindependent concurrent schedule of cocaine reinforcement in rhesus monkeys.

Lever pressing by three rhesus monkeys was maintained under a two-lever concurrent schedule of cocaine reinforcement. Responding on one lever (constant-dose lever) produced a constant dose of 0.05 or 0.1 mg/kg/injection arranged according to a variable-interval 1-min schedule. Responding on the other lever (variable-dose lever) produced a comparison dose of cocaine (0.013 to 0.8 mg/kg/injection), also under a variable-interval 1-min schedule. The two variable-interval schedules were made nonindependent by arranging that the assignment of a reinforcer by one schedule inactivated the second schedule until the assigned reinforcer had been obtained. This modification ensured that the two cocaine doses were obtained with approximately equal frequency, regardless of the distribution of the subject's responding. Preference, indicated by relative response frequency on the variable-dose lever, was almost always for the larger of the doses and was a monotonic function of the comparison dose, except at the highest doses. Preferences at the highest comparison doses may have resulted from the low overall response rates exhibited at these doses. Relative response frequencies on the variable-dose lever roughly matched relative reinforcer magnitude (mg/kg/injection available on the variable-dose lever divided by the sum of mg/kg/injections available on each lever).     

The effects of magnitude and quality of reinforcement on choice responding during play activities

Three boys with autism participated in a study of the effects of magnitude and quality of reinforcement on choice responding. Two concurrent response alternatives were arranged: (a) to play in an area where a peer or sibling was located, or (b) to play in an area where there was no peer or sibling. During one condition, the magnitude (i.e., duration of access to toys) or quality (level of preference) of reinforcement provided for both responses was equal. During the other condition, the magnitude or quality of reinforcement was relatively greater for choosing the play area where the peer or sibling was located than the area where the peer or sibling was not located. Results showed that after repeated exposure to the unequal magnitude or quality condition, the participant increasingly allocated his responses to the play area where the peer or sibling was located. For 2 participants, this pattern of responding was maintained in the subsequent equal magnitude or quality condition. Overall, the analysis suggests that the dimensions of magnitude and quality of reinforcement can be arranged to influence choice responding in favor of playing near a peer or sibling rather than playing alone.     

Preference and resistance to change with constant- and variable-duration terminal links: independence of reinforcement rate and magnitude

Pigeons responded in a three-component multiple concurrent-chains procedure in which the variable-interval reinforcement schedules were the same across components but magnitudes differed across components. The terminal links were arranged either as a variable delay followed by presentation of a reinforcer ("variable duration") or as a fixed period of access to the schedule during which a variable number of reinforcers could be earned ("constant duration"). Relative reinforcement rate was varied parametrically across both types of conditions. After baseline training in each condition, resistance to change of terminal-link responding was assessed by delivering food during the initial links according to a variable-time schedule. Both preference and resistance to change were more sensitive to reinforcement-rate differences in the constant-duration conditions. Sensitivities of preference and resistance to change to relative reinforcement rate did not change depending on relative reinforcement magnitude. Taken together, these results confirm and extend those of prior studies, and suggest that reinforcement rate and magnitude combine additively to determine preference and resistance to change. A single structural relation linking preference and resistance to change describes all the data from this and several related studies.     

Independence of reinforcement delay and magnitude in concurrent chains

A three-component concurrent-chains procedure was used to investigate preference between terminal-link schedules that differed in delay and magnitude of reinforcement. Response and time allocation data were well described by a generalized matching model. Sensitivity to delay appeared to be lower when reinforcement magnitudes were unequal than when they were equal, but when obtained rather than programmed time spent responding in the initial links was used in the model, the difference vanished. The results support independence of delay and magnitude as separate dimensions of reinforcement value, as required by the matching law, and the assumption of the contextual choice model (Grace, 1994) that sensitivities to delay and magnitude are affected similarly by temporal context. Although there was statistical evidence for interaction between successive components, the effects were small and transient. The multiple-component concurrent-chains procedure should prove useful in future research on multidimensional preference, although it may be necessary to control obtained initial-link time more precisely.     

Duration and rate of reinforcement as determinants of concurrent responding

The duration and frequency of food presentation were varied in concurrent variable-interval variable-interval schedules of reinforcement. In the first experiment, in which pigeons were exposed to a succession of eight different schedules, neither relative duration nor relative frequency of reinforcement had as great an effect on response distribution as they have when they are manipulated separately. These results supported those previously reported by Todorov (1973) and Schneider (1973). In a second experiment, each of seven pigeons was exposed to only one concurrent schedule in which the frequency and/or duration of reinforcement differed on the two keys. Under these conditions, each pigeon's relative rate of response closely matched the relative total access to food that each schedule provided. This result suggests that previous failures to obtain matching may be due to factors such as an insufficient length of exposure to each schedule or to the pigeons' repeated exposure to different concurrent schedules.     

Noncontingent reinforcement without extinction plus differential reinforcement of alternative behavior during treatment of problem behavior

The effects of noncontingent reinforcement (NCR) without extinction during treatment of problem behavior maintained by social positive reinforcement were evaluated for five individuals diagnosed with autism spectrum disorder. A continuous NCR schedule was gradually thinned to a fixed‐time 5‐min schedule. If problem behavior increased during NCR schedule thinning, a continuous NCR schedule was reinstated and NCR schedule thinning was repeated with differential reinforcement of alternative behavior (DRA) included. Results showed an immediate decrease in all participants’ problem behavior during continuous NCR, and problem behavior maintained at low levels during NCR schedule thinning for three participants. Problem behavior increased and maintained at higher rates during NCR schedule thinning for two other participants; however, the addition of DRA to the intervention resulted in decreased problem behavior and increased mands.     

Effects on choice of reinforcement delay and conditioned reinforcement

Pigeons chose between fixed-interval schedules of different durations presented in the terminal links of concurrent-chains schedules. The pair of schedules was always in the ratio of 2:1, but the absolute duration of the fixed intervals varied. In one set of conditions, the different terminal-link schedules were associated with different keylight stimuli (cued conditions). In a second set of conditions, the different terminal-link schedules were associated with the same stimulus (uncued conditions). Results from the cued conditions replicated previous findings that preference for the shorter fixed-interval schedule increased with fixed-interval duration. Preferences in the uncued conditions were lower than in the corresponding cued conditions but also increased with fixed-interval length. In addition, the degree of control under the uncued conditions was correlated with the extent to which the schedule during the terminal link was discriminated immediately upon entry into the terminal link. The pattern of results in both conditions was inconsistent with the notion that choice behavior matches relative immediacy of reinforcement. Reanalysis of previous evidence for matching (Chung and Herrnstein, 1967) showed that matching in fact did not occur, as the preferences of their subjects for the shorter of two delays also increased with the absolute size of the delays.     

Fixed-ratio pausing: Joint effects of past reinforcer magnitude and stimuli correlated with upcoming magnitude

Pigeons responded on fixed-ratio schedules ending in small or large reinforcers (grain presentations of different duration) interspersed within each session. In mixed-schedule conditions, the response key was lit with a single color throughout the session, and pausing was directly related to the past reinforcer (longer pauses after large reinforcers than after small ones). In multiple-schedule conditions, different colors accompanied the ratios ending in small and large reinforcers, and pausing was affected by the upcoming reinforcer as well as the past one. Pauses were shorter before large reinforcers than before small ones, but they continued to be longer after large reinforcers than after small ones. The influence of the past reinforcer was modulated by the magnitude of the upcoming reinforcer; in the presence of the stimulus before the small reinforcer, the effect of the past reinforcer was enhanced relative to its effect in the stimulus before the large reinforcer. These results show that pausing between ratios is jointly determined by two competing factors: past conditions of reinforcement and stimuli correlated with upcoming conditions.     

Effects of primary reinforcement on pigeons' initial-link responding under a concurrent chains schedule with nondifferntial terminal links

The effect of primary reinforcement on initial-link responding under concurrent-chains schedules with nondifferential terminal links was assessed in 12 pigeons. The iniitial and terminal links were variable-interval schedules (always the same for both alternatives). The positions (left or right key) of the initial-link stimuli (red or green) were randomized while the correlation between color and food amount remained constant within each condition. The terminal-link stimuli were always presented on the center key. Except in two control groups and conditions, the terminal-link stimuli were the same color (nondifferential, blue or yellow). Over six conditions, the differences in food amont and the durations of the initial- and terminal-link schedules were manipulated. In 57 of 60 cases, birds generated choice proportions above .50 in favor of the initial-link stimlus that was correlated with the larger reinforcer. There was some indication that preference increased with shortened terminal-link durations. Because the terminal-link stimuli were nondifferential, differential responding in the initial links cannot be explained easily by conditioned reinforcement represented by the terminal-link stimuli. Thus, primiary reinforcement has a direct effect on initial-link responding in concurrent-chains schedules.     

A molar theory of reinforcement schedules

Behavior of subjects exposed to concurrent and individual interval and ratio schedules of reinforcement may be described in terms of a set of expressions relating the value of responses to their durations, a feedback equation relating reinforcement to response duration, and the assumption that subjects allocate their time among various responses so as to maximize value.     

Choice and conditioned reinforcement.

A potential weakness of one formulation of delay-reduction theory is its failure to include a term for rate of conditioned reinforcement, that is, the rate at which the terminal-link stimuli occur in concurrent-chains schedules. The present studies assessed whether or not rate of conditioned reinforcement has an independent effect upon choice. Pigeons responded on either modified concurrent-chains schedules or on comparable concurrent-tandem schedules. The initial link was shortened on only one of two concurrent-chains schedules and on only one of two corresponding concurrent-tandem schedules. This manipulation increased rate of conditioned reinforcement sharply in the chain but not in the tandem schedule. According to a formulation of delay-reduction theory, when the outcomes chosen (the terminal links) are equal, as in Experiment 1, choice should depend only on rate of primary reinforcement; thus, choice should be equivalent for the tandem and chain schedules despite a large difference in rate of conditioned reinforcement. When the outcomes chosen are unequal, however, as in Experiment 2, choice should depend upon both rate of primary reinforcement and relative signaled delay reduction; thus, larger preferences should occur in the chain than in the tandem schedules. These predictions were confirmed, suggesting that increasing the rate of conditioned reinforcement on concurrent-chains schedules may have no independent effect on choice.     

Observing behavior: effects of rate and magnitude of primary reinforcement

Four experiments examined the free-operant observing behavior of rats. In Experiment 1, observing was a bitonic function of random-ratio schedule requirements for the primary reinforcer. In Experiment 2, decreases in the magnitude of the primary reinforcer decreased observing. Experiment 3 examined observing when a random-ratio schedule or a yoked random-time schedule of primary reinforcement was in effect across conditions. Removing the response requirement for the primary reinforcer increased observing, suggesting that the effects of the random-ratio schedule in Experiment 1 likely were due to an interaction between observing and responding for the primary reinforcer. In Experiment 4, decreasing the rate of primary reinforcement by increasing the duration of a random-time schedule decreased observing monotonically. Overall, these results suggest that observing decreases with decreases in the rate or magnitude of the primary reinforcer, but that behavior related to the primary reinforcer can affect observing and potentially affect measurement of conditioned reinforcing value.     

Sensitivity to reinforcement in concurrent arithmetic and exponential schedules

The generalized matching law states that the logarithm of the ratio of responses emitted or time spent responding in concurrent variable-interval schedules is a linear function of the logarithm of the ratio of reinforcements obtained. The slope of this relation, sensitivity to reinforcement, varies about 1.0 but has been shown to be different when obtained in different laboratories. The present paper analyzed the results from 18 experiments on concurrent variable-interval schedule performance and showed that response-allocation sensitivity to reinforcement was significantly smaller when arithmetic, rather than exponential, progressions were used to produce variable-interval schedules. There were no differences in time-allocation sensitivity between the two methods of constructing variable-interval schedules. Since the two laboratories have consistently used different methods for constructing variable-interval schedules, the differences in obtained sensitivities to reinforcement are explained. The reanalysis suggests that animals may be sensitive to differences in the distribution of reinforcements in time.     

Effects of reinforcer consumption and magnitude on response rates during noncontingent reinforcement

Results of previous research on the effects of noncontingent reinforcement (NCR) have been inconsistent when magnitude of reinforcement was manipulated. We attempted to clarify the influence of NCR magnitude by including additional controls. In Study 1, we examined the effects of reinforcer consumption time by comparing the same magnitude of NCR when session time was and was not corrected to account for reinforcer consumption. Lower response rates were observed when session time was not corrected, indicating that reinforcer consumption can suppress response rates. In Study 2, we first selected varying reinforcer magnitudes (small, medium, and large) on the basis of corrected response rates observed during a contingent reinforcement condition and then compared the effects of these magnitudes during NCR. One participant exhibited lower response rates when large-magnitude reinforcers were delivered; the other ceased responding altogether even when small-magnitude reinforcers were delivered. We also compared the effects of the same NCR magnitude (medium) during 10-min and 30-min sessions. Lower response rates were observed during 30-min sessions, indicating that the number of reinforcers consumed across a session can have the same effect as the number consumed per reinforcer delivery. These findings indicate that, even when response rate is corrected to account for reinforcer consumption, larger magnitudes of NCR (defined on either a per-delivery or per-session basis) result in lower response rates than do smaller magnitudes.