The law of effect and avoidance: a quantitative relationship between response rate and shock-frequency reduction

Two experiments were conducted to investigate the quantitative relationship between response rate and reinforcement frequency in single and multiple variable-interval avoidance schedules. Responses cancelled delivery of shocks that were scheduled by variable-interval schedules. When shock-frequency reduction was taken as the measure of reinforcement, the relationship between response rate and reinforcement frequency on single variable-interval avoidance schedules was accurately described by Herrnstein's (1970) equation for responding on single variable-interval schedules of positive reinforcement. On multiple variable-interval avoidance schedules with brief components, asymptotic relative response rate matched relative shock-frequency reduction. The results suggest that many interactions between response rates and shock-frequency reduction in avoidance can be understood within the framework of the generalized matching relation, as applied by Herrnstein (1970) to positive reinforcement.     

Two-key concurrent paced variable-interval paced variable-interval schedules of reinforcement

Nine pigeons were used in two experiments in which a response was reinforced if a variable-interval schedule had assigned a reinforcement and if the response terminated an interresponse time within a certain interval, or class, of interresponse times. One such class was scheduled on one key, and a second class was scheduled on a second key. The procedure was, therefore, a two-key concurrent paced variable-interval paced variable-interval schedule. In Exp. I, the lengths of the two reinforced interresponse times were varied. The relative frequency of responding on a key approximately equalled the relative reciprocal of the length of the interresponse time reinforced on that key. In Exp. II, the relative frequency and relative magnitude of reinforcement were varied. The relative frequency of responding on the key for which the shorter interresponse time was reinforced was a monotonically increasing, negatively accelerated function of the relative frequency of reinforcement on that key. The relative frequency of responding depended on the relative magnitude of reinforcement in approximately the same way as it depended on the relative frequency of reinforcement. The relative frequency of responding on the key for which the shorter interresponse time was reinforced depended on the lengths of the two reinforced interresponse times and on the relative frequency and relative magnitude of reinforcement in the same way as the relative frequency of the shorter interresponse time depended on these variables in previous one-key concurrent schedules of reinforcement for two interresponse times.     

Effects of reinforcement rate and reinforcer magnitude on choice behavior of humans

Two experiments with human subjects investigated the effects of rate of reinforcement and reinforcer magnitude upon choice. In Experiment 1, each of five subjects responded on four concurrent variable-interval schedules. In contrast to previous studies using non-human organisms, relative response rate did not closely match relative rate of reinforcement. Discrepancies ranged from 0.03 to 0.43 (mean equal to 0.19). Similar discrepancies were found between relative amount of time spent responding on each schedule and the corresponding relative rates of reinforcement. In Experiment 2, in which reinforcer magnitude was varied for each of five subjects, similar discrepancies ranging from 0.05 to 0.50 (mean equal to 0.21), were found between relative response rate and relative proportion of reinforcers received. In both experiments, changeover rates were lower on the long-interval concurrent schedules than on the short-interval ones. The results suggest that simple application of previous generalizations regarding the effects of reinforcement rate and reinforcer magnitude on choice for variable-interval schedules does not accurately describe human behavior in a simple laboratory situation.     

Schedule-induced licking during multiple schedules

Schedule-induced polydipsia was studied in rats bar pressing under two-component multiple schedules of food reinforcement. The first component of the multiple schedule was a variable-interval 1-min schedule throughout the experiment. The schedule comprising the second component was varied over blocks of sessions in terms of rate and magnitude of reinforcement, and was either variable-interval 3-min (one pellet), variable-interval 3-min (three pellets), variable-interval 1-min (one pellet), or extinction. Water intake per session varied with the rate of reinforcement in the schedule comprising the second component and was highest when the schedule was variable-interval 1-min. Both bar-pressing behavior and licking behavior showed behavioral interactions between the two components of the multiple schedules. With magnitude of reinforcement held constant, a matching relationship was observed between lick rate and reinforcement rate; the relative frequency of licks in the constant component matched the relative frequency of reinforcement in that component. Bar pressing, however, showed only a moderate degree of relativity matching. During the schedule-induced licking, a burst of licking followed each delivery of a pellet (post-prandial drinking). The duration of these bursts of licking was observed to be a function of the inter-reinforcement interval.     

Duration and rate of reinforcement as determinants of concurrent responding

The duration and frequency of food presentation were varied in concurrent variable-interval variable-interval schedules of reinforcement. In the first experiment, in which pigeons were exposed to a succession of eight different schedules, neither relative duration nor relative frequency of reinforcement had as great an effect on response distribution as they have when they are manipulated separately. These results supported those previously reported by Todorov (1973) and Schneider (1973). In a second experiment, each of seven pigeons was exposed to only one concurrent schedule in which the frequency and/or duration of reinforcement differed on the two keys. Under these conditions, each pigeon's relative rate of response closely matched the relative total access to food that each schedule provided. This result suggests that previous failures to obtain matching may be due to factors such as an insufficient length of exposure to each schedule or to the pigeons' repeated exposure to different concurrent schedules.     

Behavior of humans in variable-interval schedules of reinforcement

During Phase I, human subjects pressed a button for monetary reinforcement in five variable-interval schedules, each of which specified a different frequency of reinforcement. The rate of responding was an increasing, negatively accelerated function of reinforcement frequency; the data conformed closely to Herrnstein's equation. During Phase II, the same five schedules were in operation, but in addition a concurrent variable-interval schedule (B) was introduced, responses on which were always reinforced at the same frequency. Response rate in component A increased while the response rate in B decreased, as a function of the reinforcement frequency in component A. Relative response rates in the two component schedules matched the relative frequencies of reinforcement. Comparing the absolute response rates in component A during Phase I and Phase II it was found that introduction of the concurrent schedule did not affect the value of the theoretical maximum response rate, but did increase the value of the reinforcement frequency needed to obtain any particular submaximal response rate.     

Multiple determinants of the effects of reinforcement magnitude on free-operant response rates

Four experiments examined the effects of increasing the number of food pellets given to hungry rats for a lever-press response. On a simple variable-interval 60-s schedule, increased number of pellets depressed response rates (Experiment 1). In Experiment 2, the decrease in response rate as a function of increased reinforcement magnitude was demonstrated on a variable-interval 30-s schedule, but enhanced rates of response were obtained with the same increase in reinforcement magnitude on a variable-ratio 30 schedule. In Experiment 3, higher rates of responding were maintained by the component of a concurrent variable-interval 60-s variable-interval 60-s schedule associated with a higher reinforcement magnitude. In Experiment 4, higher rates of response were produced in the component of a multiple variable-interval 60-s variable-interval 60-s schedule associated with the higher reinforcement magnitude. It is suggested that on simple schedules greater reinforcer magnitudes shape the reinforced pattern of responding more effectively than do smaller reinforcement magnitudes. This effect is, however, overridden by another process, such a contrast, when two magnitudes are presented within a single session on two-component schedules.     

Performance of humans in variable-interval avoidance schedules programmed singly, and concurrently with variable-interval schedules of positive reinforcement

Performance maintained under single variable-interval avoidance schedules, single variable-interval schedules of positive reinforcement, and concurrent schedules consisting of a variable-interval avoidance component and a variable-interval positive reinforcement component, was studied in three human subjects, using points exchangeable for money as the reinforcer. Response rate in the single variable-interval avoidance schedules was an increasing function of the frequency of monetary loss avoidance. Response rate in the single variable-interval positive reinforcement schedules was an increasing function of the frequency of obtained monetary reinforcement. In the concurrent avoidance/reinforcement schedules, the rate of responding in the avoidance component increased, and the rate of responding in the positive reinforcement schedule decreased (with one exception) as a function of the frequency of loss avoidance in the avoidance component. The logarithms of the ratios of the response rates in the two components, and the logarithms of the ratios of the times spent in the two components, were linearly related to the logarithms of the ratios of the frequency of loss avoidance in the avoidance component to the frequency of reinforcement in the positive reinforcement component. All three subjects exhibited marked undermatching of response rate ratios to reinforcement frequency ratios. The results are discussed in the context of Herrnstein's quantitative model of operant performance.     

Frequency versus magnitude of reinforcement: New data with a different procedure

Two pigeons, with previous exposure to concurrent schedules, were submitted to 29 sessions of 8 hours each with concurrent variable-interval variable-interval schedules in which reinforcement parameters changed from session to session. In the first nine sessions reinforcement durations were equal in both schedules while reinforcement frequencies varied; in Sessions 10 through 18, both frequency and duration of reinforcement were varied; in Sessions 19 through 29, only reinforcement duration was varied. Results with this different procedure confirm previous findings that behavior is more sensitive to changes in reinforcement frequency than to reinforcement magnitude.     

Time allocation in concurrent schedules: the effect of signalled reinforcement

The responses of five pigeons were reinforced on concurrent variable-interval variable-interval reinforcement schedules in which changeover key responses changed the stimulus and reinforcement schedules associated with the food key. While the reinforcement availability in one component remained unchanged throughout the experiment, the reinforcement availability in the other component was, during several conditions, signalled by the onset of an additional discriminative stimulus. During unsignalled conditions, both the relative frequency of responding and the relative time spent in each component approximated the obtained relative reinforcement frequency in each component. The effect of signalling reinforcer availability in one component was to (1) reduce responding in the signalled component to near-zero levels, and (2) increase the relative time in the unsignalled component, without a corresponding increase in the obtained relative reinforcement frequency. The magnitude of the increase in relative time in the unsignalled component decreased as the overall frequency of reinforcement increased. This deviation in the matching relation between relative time and the obtained relative reinforcement frequency was eliminated if the overall reinforcement frequency was increased before the signal was introduced and then, without removing the signal, gradually reduced.     

Magnitude and frequency of reinforcement and frequencies of interresponse times

The relative magnitude and relative frequency of reinforcement for two concurrent interresponse times (1.5 to 2.5 sec and 3.5 to 4.5 sec) were simultaneously varied in an experiment in which pigeons obtained grain by pecking on a single key. Visual discriminative stimuli accompanied the two time intervals in which reinforcements were arranged by a one-minute variable-interval schedule. The resulting interresponse times of each of three pigeons fell into two groups; "short" (1.0 to 2.5 sec) and "long" (3.0 to 4.5 sec). Steady-state relative frequencies of these interresponse times were orderly functions of both reinforcement variables. The combined effects of both independent variables were well summarized by a linear function of one variable, relative access to food. Unlike corresponding two-key concurrent variable-interval schedules, the present schedule did not produce an equality between the relative frequency of an operant and either the relative magnitude or the relative frequency of reinforcement of that operant. A tentative account is provided for this difference between one-key and two-key functions.     

Fixed-ratio punishment by timeout of concurrent variable-interval behavior

Pigeons' responding was maintained by two concurrently available variable-interval reinforcement schedules. A fixed-ratio punishment schedule of timeout periods from the concurrent reinforcement schedules was arranged for responding during one of the variable-interval schedules. The greater the probability of a timeout after a response on the punished variable-interval schedule (the smaller the fixed ratio that produced timeout), the greater the decline in the relative punished response rates. Relative reinforcement rates remained invariant when relative response rates declined. Both behavioral contrast and induction effects were observed on the unpunished variable-interval schedule as a function of timeout punishment of the other schedule.     

Matching with a trio of concurrent variable-interval schedules of reinforcement

A trio of concurrent variable-interval schedules of reinforcement was arranged according to a changeover-key procedure, including a changeover delay of 1.5 sec. The three schedules provided a combined maximum reinforcement rate of 45 reinforcements per hour. With that restriction, the nine experimental conditions included several combinations of variable-interval schedules, sometimes including extinction. The pigeons matched relative response rate and relative time to relative reinforcement rate. Relative time appeared to match some-what better than relative response rate. Performance adjusted rapidly from one experimental condition to the next, whether the change involved two or all three schedules of the concurrent trio.     

Choice behavior and the accessibility of the reinforcer

In Experiment 1, matching of relative response rates to relative rates of reinforcement was obtained in concurrent variable-interval schedules when the absolute values of the two concurrent variable-interval schedules varied from 6 sec and 12 sec to 600 sec and 1200 sec. Increases in the duration of the changeover delay, however, produced decreases in the relative response rates and, consequently, some deviation from matching. In Experiment 2, matching of relative response rates to the relative duration of the reinforcer failed to occur when the equal variable-interval schedules arranging access to the two different reinforcer durations (1.5 and 6 sec) were varied in size from concurrent variable-interval 10-sec schedules to concurrent variable-interval 600-sec schedules.     

Matching to relative reinforcement frequency in multiple schedules with a short component duration

Three pigeons performed on two-component multiple variable-interval variable-interval schedules of reinforcement. There were two independent variables: component duration and the relative frequency of reinforcement in a component. The component duration, which was always the same in both components, was varied over experimental conditions from 2 to 180 sec. Over these conditions, the relative frequency of reinforcement in a component was either 0.2 or 0.8 (±0.03). As the component duration was shortened, the relative frequency of responding in a component approached a value equal to the relative frequency of reinforcement in that component. When the relative frequency of reinforcement was varied over conditions in which the component duration was fixed at 5 sec, the relative frequency of responding in a component closely approximated the relative frequency of reinforcement in that component. That is, the familiar matching relationship, obtained previously only with concurrent schedules, was obtained in multiple schedules with a short component duration.     

Relative frequency of reinforcement and rate of punished behavior

After training on a multiple variable-interval variable-interval schedule of reinforcement, each response in one component of the schedule was followed by a brief electric shock. When the rate of punished responses stabilized, the frequency of reinforcement in the other component was first decreased and then increased from the baseline frequency. The effects of these manipulations were consistent with reports of interactions in multiple schedules involving only unpunished behavior, i.e., the rate of punished responses increased with a higher relative frequency of reinforcement in the punishment component and decreased with a lower relative frequency of reinforcement in that component. The relevance of such findings to a further generality of behavioral contrast effects is discussed.     

Behavioral contrast and relative reinforcement frequency in two multiple schedules

After preliminary variable-interval training, one group of pigeons was trained on a series of multiple variable-interval low-rate reinforcement schedules, while another group was trained on a series of multiple variable-interval fixed-ratio reinforcement schedules. Contrast effects were observed as variable-interval baseline rate changed in a direction away from the change in reinforcement frequency in the other component. The effects of the variable-interval component on performance in the low-rate and fixed-ratio reinforcement components in the multiple schedules were assessed by comparing the birds' performances on each of these schedules alone. Fixed-ratio reinforcement schedules showed a susceptibility to contrast effects, low-rate reinforcement schedules did not. The rate of reinforcement in fixed-ratio schedules at which no interaction occurred in the multiple schedules was higher than that in variable-interval 1-min schedules, suggesting that pigeons may prefer time-based, rather than response-based, reinforcement.     

Rates and patterns of responding with concurrent fixed-interval and variable-interval reinforcement

Pigeons were exposed to concurrent fixed-interval and variable-interval schedules of food reinforcement on two keys. The times between reinforcement were varied systematically on both keys. The overall relative frequency of responding on the fixed-interval key depended on the relative frequency of reinforcement, but did not match it. Instead, the ratio of responses on the fixed-interval key to responses on the variable-interval key was a power function of the ratio of reinforcements, with an exponent of 0.5. Patterns of responding between reinforcements on the fixed-interval key depended on both relative and absolute values of the reinforcement schedules. Similar overall relative responding was obtained at different absolute schedule values with equal relative reinforcement, despite some differences in patterns of responding.     

The concurrent reinforcement of two interresponse times: the relative frequency of an interresponse time equals its relative harmonic length

The relative lengths of two concurrently reinforced interresponse times were varied in an experiment in which three pigeons obtained food by pecking on a single key. Visual discriminative stimuli accompanied the two time intervals in which reinforcements were scheduled according to a one-minute variable-interval. The steady-state relative frequency of an interresponse time approximately equalled the complement of its relative length, that is, its relative harmonic length. Thus, lengths of interresponse times and delays of reinforcement have the same effect on the relative frequencies of interresponse times and choices in one-key and two-key concurrent variable-interval schedules, respectively. A second experiment generalized further the functional equivalence between the effects of these one-key and two-key concurrent schedules by revealing that the usual matching-to-relative-immediacy in two-key concurrent schedules is undisturbed if reinforcement depends upon the occurrence of a response at the end of the delay interval, as it does in the one-key schedules. The results of both experiments are consistent with a quantitative theory of concurrent operant behavior.     

Molar And Molecular Control In Variable-interval And Variable-ratio Schedules

Response rates are typically higher under variable-ratio than under variable-interval schedules of reinforcement, perhaps because of differences in the dependence of reinforcement rate on response rate or because of differences in the reinforcement of long interresponse times. A variable-interval-with-added-linear-feedback schedule is a variable-interval schedule that provides a response rate/reinforcement rate correlation by permitting the minimum interfood interval to decrease with rapid responding. Four rats were exposed to variable-ratio 15, 30, and 60 food reinforcement schedules, variable-interval 15-, 30-, and 60-s food reinforcement schedules, and two versions of variable-interval-with-added-linear-feedback 15-, 30-, and 60-s food reinforcement schedules. Response rates on the variable-interval-with-added-linear-feedback schedule were similar to those on the variable-interval schedule; all three schedules led to lower response rates than those on the variable-ratio schedules, especially when the schedule values were 30. Also, reinforced interresponse times on the variable-interval-with-added-linear-feedback schedule were similar to those on variable interval and much longer than those produced by variable ratio. The results were interpreted as supporting the hypothesis that response rates on variable-interval schedules in rats are lower than those on comparable variable-ratio schedules, primarily because the former schedules reinforce long interresponse times.