Beyond the relational principle of reinforcement

Behavior under baseline conditions in which the contingency is absent can shed some light on the individual's performance under a schedule, but is insufficient as a basis for prediction of performance. This insufficiency of the baseline data runs counter to a recent formalization of the relational principle of reinforcement (Donahoe, 1977). A more satisfactory predictive model must incorporate not only the baseline level of the instrumental response and that of the contingent response, but also the schedule requirements, the character of each response in relation to the other, and the behavior required in simply switching from each to the other.     

Increasing and signaling background reinforcement: effect on the foreground response-reinforcer relation.

Herrnstein's (1970) hyperbolic matching equation describes the relationship between response rate and reinforcement rate. It has two estimated parameters, k and Re. According to one interpretation, k measures motor performance and Re measures the efficacy of the reinforcer maintaining responding relative to background sources of reinforcement. Experiment 1 tested this interpretation of the Re parameter by observing the effect of adding and removing an additional source of reinforcement to the context. Using a within-session procedure, estimates of Re were obtained from the response-reinforcer relation over a series of seven variable-interval schedules. A second, concurrently available variable-interval schedule of reinforcement was added and then removed from the context. Results showed that when the alternative was added to the context, the value of Re increased by 107 reinforcers per hour; this approximated the 91 reinforcers per hour obtained from this schedule. Experiment 2 investigated the effects of signaling background reinforcement on k and Re. The signal decreased Re, but did not have a systematic effect on k. In general, the results supported Herrnstein's interpretation that in settings with one experimenter-controlled reinforcement source, Re indexes the strength of the reinforcer maintaining responding relative to uncontrolled background sources of reinforcement.     

The general matching law describes choice on concurrent variable-interval schedules of wheel-running reinforcement.

Six male Wistar rats were exposed to concurrent variable-interval schedules of wheel-running reinforcement. The reinforcer associated with each alternative was the opportunity to run for 15 s, and the duration of the changeover delay was 1 s. Results suggested that time allocation was more sensitive to relative reinforcement rate than was response allocation. For time allocation, the mean slopes and intercepts were 0.82 and 0.008, respectively. In contrast, for response allocation, mean slopes and intercepts were 0.60 and 0.03, respectively. Correction for low response rates and high rates of changing over, however, increased slopes for response allocation to about equal those for time allocation. The results of the present study suggest that the two-operant form of the matching law can be extended to wheel-running reinforcement. 'I'he effects of a low overall response rate, a short Changeover delay, and long postreinforcement pausing on the assessment of matching in the present study are discussed.     

Studies of wheel-running reinforcement: parameters of Herrnstein's (1970) response-strength equation vary with schedule order

Six male Wistar rats were exposed to different orders of reinforcement schedules to investigate if estimates from Herrnstein's (1970) single-operant matching law equation would vary systematically with schedule order. Reinforcement schedules were arranged in orders of increasing and decreasing reinforcement rate. Subsequently, all rats were exposed to a single reinforcement schedule within a session to determine within-session changes in responding. For each condition, the operant was lever pressing and the reinforcing consequence was the opportunity to run for 15 s. Estimates of k and R(O) were higher when reinforcement schedules were arranged in order of increasing reinforcement rate. Within a session on a single reinforcement schedule, response rates increased between the beginning and the end of a session. A positive correlation between the difference in parameters between schedule orders and the difference in response rates within a session suggests that the within-session change in response rates may be related to the difference in the asymptotes. These results call into question the validity of parameter estimates from Herrnstein's (1970) equation when reinforcer efficacy changes within a session.     

Resurgence and downshifts in alternative reinforcement rate

Resurgence refers to an increase in a previously suppressed target behavior with a relative worsening of conditions for a more recently reinforced alternative behavior. This experiment examined the relation between resurgence and the magnitude of a reduction in the rate of reinforcement for the alternative behavior. Groups of both male and female rats initially pressed a target lever for food on a variable-interval (VI) 30-s schedule. In a second phase, responding to the target lever was extinguished for all groups and pressing an alternative lever was reinforced on a VI 10-s schedule. Next, the rate of reinforcement for alternative behavior was reduced differentially across groups by arranging extinction, VI 80-s, VI 40-s, VI 20-s, or continued VI 10-s reinforcement. Target responding increased as an exponential function of the magnitude of the reduction in alternative reinforcement rates. With the exception that males appeared to show higher rates of target responding in baseline and higher rates of alternative responding in other phases, the overall pattern of responding across phases was not meaningfully different between sexes. The pattern of both target and alternative response rates across sessions and phases was well described quantitatively by the Resurgence as Choice in Context model.     

Aftereffects of reinforcement on variable-ratio schedules

On each of variable-ratio 10, 40, and 80 schedules of reinforcement, when rats' lever-pressing rates were stable, the concentration of a liquid reinforcer was varied within sessions. The duration of the postreinforcement pause was an increasing function of the reinforcer concentration, this effect being more marked the higher the schedule parameter. The running rate, calculated by excluding the postreinforcement pause, was unaffected by concentration. The duration of the postreinforcement pause increased with the schedule parameter, but the proportion of the interreinforcement interval taken up by the pause decreased. Consequently, the overall response rate was an increasing function of the schedule parameter; i.e., it was inversely related to reinforcement frequency, contrary to the law of effect. The running rate, however, decreased with the reinforcement frequency, in accord with the law of effect. When 50% of reinforcements were randomly omitted, the postomission pause was shorter than the postreinforcement pause, but the running rate of responses was not affected.     

ACHIEVING AND MAINTAINING COMPLIANCE WITH THE KETOGENIC DIET

The ketogenic diet, a treatment for intractable epilepsy, is rarely initiated because it requires strict compliance with a diet that is perceived to be unpalatable. In a case study of a 15-year-old girl with uncontrolled epilepsy, we used a stimulus-choice procedure to assess relative preferences of 33 foods from the diet and to develop two treatments based on Premack's principle. The results of a multielement analysis showed that both treatments increased dietary compliance. Compliance was maintained with generalization of treatment across settings, and was associated with a 40% reduction in seizures.     

Behavior of humans in variable-interval schedules of reinforcement

During Phase I, human subjects pressed a button for monetary reinforcement in five variable-interval schedules, each of which specified a different frequency of reinforcement. The rate of responding was an increasing, negatively accelerated function of reinforcement frequency; the data conformed closely to Herrnstein's equation. During Phase II, the same five schedules were in operation, but in addition a concurrent variable-interval schedule (B) was introduced, responses on which were always reinforced at the same frequency. Response rate in component A increased while the response rate in B decreased, as a function of the reinforcement frequency in component A. Relative response rates in the two component schedules matched the relative frequencies of reinforcement. Comparing the absolute response rates in component A during Phase I and Phase II it was found that introduction of the concurrent schedule did not affect the value of the theoretical maximum response rate, but did increase the value of the reinforcement frequency needed to obtain any particular submaximal response rate.     

Effects of adding a second reinforcement alternative: implications for Herrnstein's interpretation of r(e)

Herrnstein's hyperbola describes the relation between response rate and reinforcer rate on variable-interval (VI) schedules. According to Herrnstein's (1970) interpretation, the parameter r(e) represents the reinforcer rate extraneous to the alternative to which the equation is fitted (the target alternative). The hyperbola is based on an assumption that extraneous reinforcer rate remains constant with changes in reinforcer rate on the target alternative (the constant-r(e) assumption) and that matching with no bias and perfect sensitivity occurs between response and reinforcer ratios. In the present experiment, 12 rats pressed levers for food on a series of 10 VI schedules arranged on the target alternative. Across conditions, six VI values and extinction were arranged on a second alternative. Reinforcer rate on the second alternative, r2, negatively covaried with reinforcer rate on the target alternative for five of the six VI values on the second alternative, and significant degrees of bias and undermatching occurred in response ratios. Given covariation of reinforcer rate on the second and target alternatives, the constant-r(e) assumption can be maintained only by assuming that reinforcer rate from unmeasured background sources, rb, covaries with reinforcer rate on the second alternative such that their sum, r(e), remains constant. In a single-schedule arrangement, however, r(e) equals rb and thus rb is assumed to remain constant, forcing a conceptual inconsistency between single- and concurrent-schedule arrangements. Furthermore, although an alternative formulation of the hyperbola can account for variations in bias and sensitivity, the modified equation also is based on the constant-r(e) assumption and therefore suffers from the same logical problem as the hyperbola when reinforcer rate on the second alternative covaries with reinforcer rate on the target alternative.     

Resurgence as Choice in Context: Treatment duration and on/off alternative reinforcement

Resurgence as Choice (RaC) is a quantitative theory suggesting that an increase in an extinguished target behavior with subsequent extinction of an alternative behavior (i.e., resurgence) is governed by the same processes as choice more generally. We present data from an experiment with rats examining a range of treatment durations with alternative reinforcement plus extinction and demonstrate that increases in treatment duration produce small but reliable decreases in resurgence. Although RaC predicted the relation between target responding and treatment duration, the model failed in other respects. First, contrary to predictions, the present experiment also replicated previous findings that exposure to cycling on/off alternative reinforcement reduces resurgence. Second, RaC did a poor job simultaneously accounting for target and alternative behaviors across conditions. We present a revised model incorporating a role for more local signaling effects of reinforcer deliveries or their absence on response allocation. Such signaling effects are suggested to impact response allocation above and beyond the values of the target and alternative behaviors as longer-term repositories of experience. The new model provides an excellent account of the data and can be viewed as an integration of RaC and a quantitative approximation of some aspects of Context Theory.     

On the classic and modern theories of matching

Classic matching theory, which is based on Herrnstein's (1961) original matching equation and includes the well-known quantitative law of effect, is almost certainly false. The theory is logically inconsistent with known experimental findings, and experiments have shown that its central constant-k assumption is not tenable. Modern matching theory, which is based on the power function version of the original matching equation, remains tenable, although it has not been discussed or studied extensively. The modern theory is logically consistent with known experimental findings, it predicts the fact and details of the violation of the classic theory's constant-k assumption, and it accurately describes at least some data that are inconsistent with the classic theory.     

Stimulus and reinforcer relativity in multiple schedules: Local and dimensional effects on sensitivity to reinforcement

Pigeons' responses in two successive components of multiple schedules were reinforced according to variable-interval schedules of reinforcement that varied over five different conditions. Within each session of all conditions, line orientations of 0°, 30°, or 45° in Component 1 alternated with orientations of 45°, 60°, or 90° in Component 2. Response rates were recorded in three successive subintervals of each component. Ratios were taken between the response rate in each Component 1 line orientation and the response rate in each Component 2 orientation. These ratios were found to be power functions of the corresponding ratios of obtained reinforcement rates. Sensitivity of response ratios to changes in reinforcer ratios, given by the value of the exponent of the power function, increased systematically with increasing disparity between the dimensional values of orientation stimuli. In addition, sensitivity decreased systematically over successive subintervals of components, that is, with increasing time since component alternation. Dimensional and local (subinterval) effects interacted in that sensitivity increased with stimulus disparity to a far greater extent in the first subinterval than later in components. The data could be described by a combination of rectangular hyperbolae which attributed the interaction between local and dimensional effects to limits set by local effects on the extent that stimulus differences could affect sensitivity.     

Limited matching on concurrent-schedule reinforcement of academic behavior

Three adolescent students with special educational needs were given a choice between completing one of two available sets of math problems. Reinforcers (nickels) across these alternatives were arranged systematically in separate experimental phases according to three different concurrent variable-interval schedules (reinforcement ratios of 2:1, 6:1, and 12:1). Time allocated to the two stacks of math problems stood in linear relationship to the reinforcement rate obtained from each stack, although substantial undermatching and bias were observed for all subjects. However, changes in the schedules were not followed by changes in allocation patterns until adjunct procedures (e.g., changeover delays, limited holds, timers, and demonstrations) were introduced. The necessity of adjunct procedures in establishing matching in applied situations is discussed as a limitation to quantitative applications of the matching law in applied behavior analysis.     

Relationship between response rate and reinforcement frequency in variable-interval schedules: the effect of the concentration of sucrose reinforcement

Four rats were exposed to variable-interval schedules specifying a range of different reinforcement frequencies, using sucrose of two different concentrations and distilled water as the reinforcer. With sucrose, the rates of responding of all four rats were increasing negatively accelerated functions of reinforcement frequency, the data conforming closely to Herrnstein's equation; this was also true of the data from three of the four rats when distilled water was used as the reinforcer. The values of both constants in Herrnstein's equation were related to the sucrose concentration: the asymptotic response rate decreased, and the reinforcement frequency corresponding to the half-maximal response rate increased, with decreasing sucrose concentration.     

Relationship between response rate and reinforcement frequency in variable-interval schedules: II. Effect of the volume of sucrose reinforcement

Three rats were exposed to variable-interval schedules specifying a range of different reinforcement frequencies, using three different volumes of .32 molar sucrose (.10, .05, and .02 milliliters) as the reinforcer. With each of the three volumes, the rates of responding of all three rats were increasing, negatively accelerated functions of reinforcement frequency, the data conforming closely to Herrnstein's equation. In each rat the value of the constant K_H, which expresses the reinforcement frequency needed to obtain the half-maximal response rate, increased with decreasing reinforcer volume, the values obtained with .02 milliliters being significantly greater than the values obtained with .10 milliliters. The values of the constant R_max, which expresses the theoretical maximum response rate, were not systematically related to reinforcer volume. The effect of reinforcer volume upon the relationship between response rate and reinforcement frequency is thus different from the effect of the concentration of sucrose reinforcement: In a previous experiment (Bradshaw, Szabadi, & Bevan, 1978) it was found that sucrose concentration influenced the values of both constants, R_max increasing and K_H decreasing with increasing sucrose concentration.     

Matching, contrast, and equalizing in the concurrent lever-press responding of rats

Five rats pressed levers for food reinforces delivered by several concurrent variable-interval variable-interval schedules. The rate of reinforcement available for responding on one component schedule was held constant at 60 reinforcers per hour. The rate of reinforcement available for responding on the other schedule varied from 30 to 240 reinforcers per hour. The behavior of the rats resembled the behavior of pigeons pecking keys for food reinforcers. The ratio of the overall rates of responding emitted under, and the ratio of the time spent responding under, the two components of each concurrent schedule were approximately equal to the ratio of the overall rates of reinforcement obtained from the components. The overall rate of responding emitted under, and the time spent responding under, the variable component schedule varied directly with the overall rate of reinforcement from that schedule. The overall rate of responding emitted under, and the time spent responding under, the constant component schedule varied inversely with the overall rate of reinforcement obtained from the variable component. The local rates of responding emitted under, and the local rates of reinforcement obtained from, the two components did not differ consistently across subjects. But they were not exactly equal either.     

Choice as a function of local versus molar reinforcement contingencies.

Rats were trained on a discrete-trial probability learning task. In Experiment 1, the molar reinforcement probabilities for the two response alternatives were equal, and the local contingencies of reinforcement differentially reinforced a win-stay, lose-shift response pattern. The win-stay portion was learned substantially more easily and appeared from the outset of training, suggesting that its occurrence did not depend upon discrimination of the local contingencies but rather only upon simple strengthening effects of individual reinforcements. Control by both types of local contingencies decreased with increases in the intertrial interval, although some control remained with intertrial intervals as long as 30 s. In Experiment 2, the local contingencies always favored win-shift and lose-shift response patterns but were asymmetrical for the two responses, causing the molar reinforcement rates for the two responses to differ. Some learning of the alternation pattern occurred with short intertrial intervals, although win-stay behavior occurred for some subjects. The local reinforcement contingencies were discriminated poorly with longer intertrial intervals. In the absence of control by the local contingencies, choice proportion was determined by the molar contingencies, as indicated by high exponent values for the generalized matching law with long intertrial intervals, and lower values with short intertrial intervals. The results show that when molar contingencies of reinforcement and local contingencies are in opposition, both may have independent roles. Control by molar contingencies cannot generally be explained by local contingencies.     

Choice and reinforcement delay

Previous studies of choice between two delayed reinforcers have indicated that the relative immediacy of the reinforcer is a major determinant of the relative frequency of responding. Parallel studies of choice between two interresponse times have found exceptions to this generality. The present study looked at the choice by pigeons between two delays, one of which was always four times longer than the other, but whose absolute durations were varied across conditions. The results indicated that choice is not uniquely determined by the relative immediacy of reinforcement, but that absolute delays are also involved. Models for concurrent chained schedules appear to be more applicable to the present data than the matching relation; however, these too failed to predict choice for long delays.