Local patterns of responding maintained by concurrent and multiple schedules

Local patterns of responding were studied when pigeons pecked for food in concurrent variable-interval schedules (Experiment I) and in multiple variable-interval schedules (Experiment II). In Experiment I, similarities in the distribution of interresponse times on the two keys provided further evidence that responding on concurrent schedules is determined more by allocation of time than by changes in local pattern of responding. Relative responding in local intervals since a preceding reinforcement showed consistent deviations from matching between relative responding and relative reinforcement in various postreinforcement intervals. Response rates in local intervals since a preceding changeover showed that rate of responding is not the same on both keys in all postchangeover intervals. The relative amount of time consumed by interchangeover times of a given duration approximately matched relative frequency of reinforced interchangeover times of that duration. However, computer simulation showed that this matching was probably a necessary artifact of concurrent schedules. In Experiment II, when component durations were 180 sec, the relationship between distribution of interresponse times and rate of reinforcement in the component showed that responding was determined by local pattern of responding in the components. Since responding on concurrent schedules appears to be determined by time allocation, this result would establish a behavioral difference between multiple and concurrent schedules. However, when component durations were 5 sec, local pattern of responding in a component (defined by interresponse times) was less important in determining responding than was amount of time spent responding in a component (defined by latencies). In fact, with 5-sec component durations, the relative amount of time spent responding in a component approximately matched relative frequency of reinforcement in the component. Thus, as component durations in multiple schedules decrease, multiple schedules become more like concurrent schedules, in the sense that responding is affected by allocation of time rather than by local pattern of responding.     

Matching and contrast on several concurrent treadle-press schedules

Four White King pigeons pressed treadles for food reinforcement on several concurrent variable-interval variable-interval schedules. The rate of reinforcement available for responding in one of the two component schedules was held constant at 30 reinforcers per hour. The rate of reinforcement available for responding in the other was varied from 120 to 60 to 15, and then to 30 reinforcers per hour. The relative rate of responding in each component schedule equalled the relative rate of reinforcement that the component provided. And, behavioral contrast, defined as an inverse relationship between the rate of responding in the constant component and the rate of reinforcement obtained by responding in the other component, occurred for all schedules.     

On the relation between preference and resistance to change

Nevin (1979) noted that preference in concurrent chains and resistance to change in multiple schedules were correlated, in that both measures were affected similarly by variations in parameters of reinforcement such as rate, immediacy, and magnitude. To investigate the relationship between preference and resistance to change directly, we used a within-session procedure that arranged concurrent chains in one half of the session and a multiple schedule in the other half. The same variable-interval schedules served as terminal links in concurrent chains and as the components of the multiple schedule, and were signaled by the same stimuli. After performances had stabilized, responding in the multiple schedule was disrupted by delivering response-independent reinforcement during the blackout periods between components. Both preference in concurrent chains and relative resistance to change of multiple-schedule responding were well described as power functions of relative reinforcement rate, as predicted by current quantitative models (Grace, 1994; Nevin, 1992b). In addition, unsystematic variation in preference and resistance to change was positively correlated, which suggests that preference and resistance to change are independent measures of a single construct. That construct could be described as the learning that occurs regarding the prevailing conditions of reinforcement in a distinctive stimulus situation.     

Schedule-induced mirror responding in the pigeon

Two pigeons that were previously exposed to a multiple schedule of reinforcement in the presence of a stuffed and a live pigeon, and two of three naive pigeons, responded on a mirror during exposure to multiple fixed-ratio, fixed-ratio schedules of reinforcement for key pecking. Both the topography and temporal pattern of mirror responding were comparable to schedule-induced “attack” on live and stuffed targets. Rate of target responding was reduced when either the mirror was covered with paper or when the multiple schedule was removed. A reversal in the relationship between reinforcement schedules and discriminative stimuli demonstrated that mirror responding was controlled by the stimulus correlated with the higher fixed-ratio schedule. With one component of the multiple schedule held constant at fixed ratio 25 and the ratio requirement of the other component varying from 25 to 150, there was an inverted U-shaped relationship between rate of mirror responding and fixed-ratio schedule in the varied component. As in Flory's study (1969b) there was an inverted U-shaped relationship between target responding and inter-food intervals. The combined results of these studies suggest that the relationship between rate of target responding and reinforcement schedules is controlled primarily by the inter-food intervals resulting from the schedules.     

The law of effect and avoidance: a quantitative relationship between response rate and shock-frequency reduction

Two experiments were conducted to investigate the quantitative relationship between response rate and reinforcement frequency in single and multiple variable-interval avoidance schedules. Responses cancelled delivery of shocks that were scheduled by variable-interval schedules. When shock-frequency reduction was taken as the measure of reinforcement, the relationship between response rate and reinforcement frequency on single variable-interval avoidance schedules was accurately described by Herrnstein's (1970) equation for responding on single variable-interval schedules of positive reinforcement. On multiple variable-interval avoidance schedules with brief components, asymptotic relative response rate matched relative shock-frequency reduction. The results suggest that many interactions between response rates and shock-frequency reduction in avoidance can be understood within the framework of the generalized matching relation, as applied by Herrnstein (1970) to positive reinforcement.     

The role of discriminative stimuli in concurrent performances

Key pecking in pigeons was examined under concurrent and parallel arrangements of two independent and simultaneously available variable-interval schedules. Pecks on the changeover key alternated the schedule of reinforcement for responses on the main key. Under concurrent schedules, discriminative stimuli were paired with the reinforcement schedule arranged in each component and changeover responses also alternated these stimuli. Under parallel schedules, changeover responses alternated the effective reinforcement schedule, but did not change the discriminative stimulus. On concurrent procedures, changeover response rate was inversely related to the difference in reinforcement rate between the two components, whereas on parallel schedules no consistent relationship was found. With both schedules, absolute response and reinforcement rates were positively related, although for a given set of reinforcement frequencies, rates were often higher on the concurrent schedules. On concurrent schedules, relative response rates and relative times were equal to relative reinforcement rates. On parallel schedules these ratios were positively related, but response and time ratios were much smaller than were obtained with comparable concurrent schedules. This inequality was most pronounced when absolute reinforcement frequencies were lowest.     

Concurrent schedules: a quantitative relation between changeover behavior and its consequences

Data from several published experiments on concurrent variable-interval schedules were analyzed with respect to the effects of changeover delay on the time spent responding on a schedule before changing to an alternate schedule: i.e., the interchangeover time. Interchangeover time increases as the duration of the changeover delay increases, and the present analysis shows that a power function describes the relation. The power relation applied in spite of numerous differences in the experiments: different variable-interval schedules for the concurrent pairs; equal or unequal reinforcement rates for the schedules of the concurrent pairs; different durations of the changeover delay; response-dependent or response-independent reinforcers; pigeons or rats as subjects; different reinforcers. A power function also described the data in experiments where the changeover incurred a timeout, where a fixed ratio was required to changeover, and also when asymmetrical changeover delays were used.     

Effects of reinforcement rate and reinforcer magnitude on choice behavior of humans

Two experiments with human subjects investigated the effects of rate of reinforcement and reinforcer magnitude upon choice. In Experiment 1, each of five subjects responded on four concurrent variable-interval schedules. In contrast to previous studies using non-human organisms, relative response rate did not closely match relative rate of reinforcement. Discrepancies ranged from 0.03 to 0.43 (mean equal to 0.19). Similar discrepancies were found between relative amount of time spent responding on each schedule and the corresponding relative rates of reinforcement. In Experiment 2, in which reinforcer magnitude was varied for each of five subjects, similar discrepancies ranging from 0.05 to 0.50 (mean equal to 0.21), were found between relative response rate and relative proportion of reinforcers received. In both experiments, changeover rates were lower on the long-interval concurrent schedules than on the short-interval ones. The results suggest that simple application of previous generalizations regarding the effects of reinforcement rate and reinforcer magnitude on choice for variable-interval schedules does not accurately describe human behavior in a simple laboratory situation.     

An analytic comparison of Herrnstein's equations and a multivariate rate equation

Herrnstein's equations are approximations of the multivariate rate equation at ordinary rates of reinforcement and responding. The rate equation is the result of a linear system analysis of variable-interval performance. Rate equation matching is more comprehensive than ordinary matching because it predicts and specifies the nature of concurrent bias, and predicts a tendency toward undermatching, which is sometimes observed in concurrent situations. The rate equation contradicts one feature of Herrnstein's hyperbola, viz., the theoretically required constancy of k. According to the rate equation, Herrnstein's k should vary directly with parameters of reinforcement such as amount or immediacy. Because of this prediction, the rate equation asserts that the conceptual framework of matching does not apply to single alternative responding. The issue of the constancy of k provides empirical grounds for distinguishing between Herrnstein's account and a linear system analysis of single alternative variable-interval responding.     

The influence of upcoming food-pellet delivery on subjects' responding for 1% sucrose reinforcement delivered by concurrent random-interval schedules

Researchers have demonstrated that rats' rates of operant responding that are maintained by 1% liquid-sucrose reinforcement will increase if food-pellet reinforcement is upcoming within the same session. The authors investigated whether a similar induction effect would be observed when rats pressed a lever for 1% sucrose that was delivered by concurrent random-interval schedules of reinforcement. Results demonstrated that upcoming noncontingent food-pellet delivery increased absolute response rates on the concurrent schedules in 10 of 12 possible instances. Upcoming food-pellet delivery also increased subjects' sensitivity to reinforcement on the concurrent schedules, as measured by the generalized matching law (W. M. Baum, 1974), in 5 of 6 possible instances. The present results extended the finding of induction to responding on concurrent schedules. They also added to evidence suggesting that the effect occurs because the reinforcing value of the weak reinforcer (i.e., the 1% sucrose) has been increased.     

Matching to relative reinforcement frequency in multiple schedules with a short component duration

Three pigeons performed on two-component multiple variable-interval variable-interval schedules of reinforcement. There were two independent variables: component duration and the relative frequency of reinforcement in a component. The component duration, which was always the same in both components, was varied over experimental conditions from 2 to 180 sec. Over these conditions, the relative frequency of reinforcement in a component was either 0.2 or 0.8 (±0.03). As the component duration was shortened, the relative frequency of responding in a component approached a value equal to the relative frequency of reinforcement in that component. When the relative frequency of reinforcement was varied over conditions in which the component duration was fixed at 5 sec, the relative frequency of responding in a component closely approximated the relative frequency of reinforcement in that component. That is, the familiar matching relationship, obtained previously only with concurrent schedules, was obtained in multiple schedules with a short component duration.     

Performance of humans in variable-interval avoidance schedules programmed singly, and concurrently with variable-interval schedules of positive reinforcement

Performance maintained under single variable-interval avoidance schedules, single variable-interval schedules of positive reinforcement, and concurrent schedules consisting of a variable-interval avoidance component and a variable-interval positive reinforcement component, was studied in three human subjects, using points exchangeable for money as the reinforcer. Response rate in the single variable-interval avoidance schedules was an increasing function of the frequency of monetary loss avoidance. Response rate in the single variable-interval positive reinforcement schedules was an increasing function of the frequency of obtained monetary reinforcement. In the concurrent avoidance/reinforcement schedules, the rate of responding in the avoidance component increased, and the rate of responding in the positive reinforcement schedule decreased (with one exception) as a function of the frequency of loss avoidance in the avoidance component. The logarithms of the ratios of the response rates in the two components, and the logarithms of the ratios of the times spent in the two components, were linearly related to the logarithms of the ratios of the frequency of loss avoidance in the avoidance component to the frequency of reinforcement in the positive reinforcement component. All three subjects exhibited marked undermatching of response rate ratios to reinforcement frequency ratios. The results are discussed in the context of Herrnstein's quantitative model of operant performance.     

Relative frequency of reinforcement and rate of punished behavior

After training on a multiple variable-interval variable-interval schedule of reinforcement, each response in one component of the schedule was followed by a brief electric shock. When the rate of punished responses stabilized, the frequency of reinforcement in the other component was first decreased and then increased from the baseline frequency. The effects of these manipulations were consistent with reports of interactions in multiple schedules involving only unpunished behavior, i.e., the rate of punished responses increased with a higher relative frequency of reinforcement in the punishment component and decreased with a lower relative frequency of reinforcement in that component. The relevance of such findings to a further generality of behavioral contrast effects is discussed.     

Effects of reinforcement context on choice

Two experiments investigated the effects of successive reinforcement contexts on choice. In the first, concurrent variable-interval schedules of primary reinforcement operated during the initial links of concurrent chains. The rate of this reinforcement arranged by the concurrent schedules was decreased across conditions: When it was higher than the terminal-link rate, preference for the higher frequency initial-link schedule increased relative to baseline. (During baseline, a standard concurrent-schedule procedure was in effect). When the initial-link reinforcement rate was lower than the terminal-link rate, preference converged toward indifference. In the second experiment, a chain schedule was available on a third key while a concurrent schedule was in effect on the side keys. When the terminal link of the chain schedule was produced, the side keys became inoperative. Availability of the chain schedule did not affect choice between the concurrent schedules. These results show that only when successive reinforcement contexts are produced by choice responding do those successive contexts affect choice in concurrent schedules.     

Reinforcement of least-frequent sequences of choices

When a pigeon's choices between two keys are probabilistically reinforced, as in discrete trial probability learning procedures and in concurrent variable-interval schedules, the bird tends to maximize, or to choose the alternative with the higher probability of reinforcement. In concurrent variable-interval schedules, steady-state matching, which is an approximate equality between the relative frequency of a response and the relative frequency of reinforcement of that response, has previously been obtained only as a consequence of maximizing. In the present experiment, maximizing was impossible. A choice of one of two keys was reinforced only if it formed, together with the three preceding choices, the sequence of four successive choices that had occurred least often. This sequence was determined by a Bernoulli-trials process with parameter p. Each of three pigeons matched when p was ½ or ¼. Therefore, steady-state matching by individual birds is not always a consequence of maximizing. Choice probability varied between successive reinforcements, and sequential statistics revealed dependencies which were adequately described by a Bernoulli-trials process with p depending on the time since the preceding reinforcement.     

Temporal constraint on choice: Sensitivity and bias in multiple schedules

In multiple schedules of reinforcement, ratios of responses in successive components are relatively insensitive to ratios of obtained reinforcers. An analysis is proposed that attributes changes in absolute response rates to concurrent interactions between programmed reinforcement and extraneous reinforcement in other components. The analysis predicts that ratios of responses in successive components vary with reinforcer ratios, qualified by a term describing the reinforcement context, that is, programmed and extraneous reinforcers. Two main predictions from the analysis were confirmed in an experiment in which pigeons' responses were reinforced in the components of a multiple schedule and analog extraneous reinforcement was scheduled for an alternative response in each component. Sensitivity of response and time ratios to reinforcer ratios in the multiple schedules varied as a function of the rate of extraneous reinforcers. Bias towards responding in one component of the multiple schedule varied as an inverse function of the ratios of extraneous reinforcer rate in the two components. The data from this and previous studies of multiple-concurrent performance were accurately predicted by our analysis and supported our contention that the allocation of behavior in multiple-schedule components depends on the relative values of concurrently-available reinforcers within each component.     

Resistance to extinction versus extinction as discrimination

The hypothesis that response strength might be measured by persistence of responding in the face of extinction was discredited in the 1960s because experiments showed that responding persists longer following intermittent reinforcers than following continuous reinforcers. Instead, researchers proposed that the longer persistence following intermittent reinforcers arises because intermittent reinforcement more closely resembles extinction—a discrimination theory. Attention to resistance to extinction revived because one observation seemed to support the persistence hypothesis: Following training on a multiple schedule with unequal components, responding usually persisted longer in the formerly richer component than in the formerly lean component. This observation represents an anomaly, however, because results with single schedules and concurrent schedules contradict it. We suggest that the difference in results arises because the multiple-schedule procedure, while including extensive training on stimulus discrimination, includes no training on discrimination between food available and food unavailable, whereas comparable single- and concurrent-schedule procedures include such training with repeated extinction. In Experiment 1, we replicated the original result, and in Experiment 2 showed that when the multiple-schedule procedure includes training on food/no-food discrimination, extinction following multiple schedules contradicts behavioral momentum theory and agrees with the discrimination theory and research with single and concurrent schedules.     

Two-key concurrent responding: response-reinforcement dependencies and blackouts

Two-key concurrent responding was maintained for three pigeons by a single variable-interval 1-minute schedule of reinforcement in conjunction with a random number generator that assigned feeder operations between keys with equal probability. The duration of blackouts was varied between keys when each response initiated a blackout, and grain arranged by the variable-interval schedule was automatically presented after a blackout (Exp. I). In Exp. II every key peck, except for those that produced grain, initiated a blackout, and grain was dependent upon a response following a blackout. For each pigeon in Exp. I and for one pigeon in Exp. II, the relative frequency of responding on a key approximated, i.e., matched, the relative reciprocal of the duration of the blackout interval on that key. In a third experiment, blackouts scheduled on a variable-interval were of equal duration on the two keys. For one key, grain automatically followed each blackout; for the other key, grain was dependent upon a response and never followed a blackout. The relative frequency of responding on the former key, i.e., the delay key, better approximated the negative exponential function obtained by Chung (1965) than the matching function predicted by Chung and Herrnstein (1967).     

Behavior-dependent reinforcer-rate changes in concurrent schedules: A further analysis

Six pigeons were trained on concurrent variable-interval schedules in three different procedures. The first procedure was a standard concurrent schedule, and the relative reinforcer frequency for responding was varied. The second was a schedule in which a relative left-key response rate (over a fixed period of time) exceeding .75 produced, in the next identical time period a higher reinforcer rate on the right key. If this criterion was not exceeded, equal reinforcer rates were arranged on the two keys in this period. This was the dependent procedure. In the third (independent) procedure, the periods of higher right-key reinforcer rates occurred with the same probability as in the second procedure, but occurred independently of behavior. In the second and third procedures, the fixed-time period (window) was varied from 5 s to 60 s, and to 240 s in the second procedure only. Performance on the two keys was similar in the concurrent and independent procedures. The procedure used in the dependent conditions generally affected performance when the windows were shorter than about 30 s. Models of performance that assume that subjects do not discriminate changes in local relative reinforcer rates cannot account for the data. Moreover, existing models are inherently unable to account for the effects of contingencies of reinforcement between responding on one alternative and gaining reinforcers on another that are arranged or that emerge as a result of time allocated to alternative schedules. Undermatching on concurrent variable-interval schedules may result from such emergent contingencies.     

Matching under nonindependent variable-ratio schedules of drug reinforcement.

Response-contingent deliveries of oral pentobarbital maintained responding of 3 rhesus monkeys during daily 3-hr sessions. Deliveries of pentobarbital were arranged under nonindependent concurrent variable-ratio variable-ratio schedules. Responses to either schedule counted toward completion of both variable-ratio schedule requirements. This schedule is similar in some respects to conventional concurrent variable-interval variable-interval schedules, in which passage of time counts toward completion of the interval value on both schedules. Restricted nonindependent concurrent variable-ratio variable-ratio schedules were also studied. On that schedule, when a drug delivery was assigned to one spout, it had to be collected before responses on the opposite spout again counted toward completion of the schedule requirements. Relative reinforcer magnitude was varied by changing the drug concentration on one schedule while keeping the drug concentration constant on the other variable-ratio schedule. Under both types of concurrent variable-ratio schedules, the relative rate of responding corresponded to the relative drug intake. Unlike earlier studies of concurrent variable-interval variable-interval intravenous cocaine reinforcement, preference was proportionate to concentration, and exclusive preferences did not develop. The relationship between relative rate of responding and relative drug intake was well described by the generalized matching law.