Independence of response force and reinforcement rate on concurrent variable-interval schedule performance

Five pigeons were trained over 43 experimental conditions on a variety of concurrent variable-interval schedules on which the forces required on the response keys were varied. The results were well described by the generalized matching law with log reinforcement ratios and log force ratios exerting independent (noninteractive) effects on preference. A further analysis using the Akaike criterion, an information-theoretic measure of the efficiency of a model, showed that overall reinforcement rate and overall force requirement did not affect preference. Unlike reinforcement rate changes, force requirement increases did not change the response rate on the alternate key, and an extension of Herrnstein's absolute response rate function for force variation on a single variable-interval schedule is suggested.     

Choice dynamics in concurrent ratio schedules of reinforcement

The generalized matching law predicts performance on concurrent schedules when variable-interval schedules are programmed but is trivially applicable when independent ratio schedules are used. Responding usually is exclusive to the schedule with the lowest response requirement. Determining a method to program concurrent ratio schedules such that matching analyses can be usefully employed would extend the generality of matching research and lead to new avenues of research. In the present experiments, ratio schedules were programmed dependently such that responses to either of the two options progressed the requirement on both schedules. Responding is not exclusive because the probability of reinforcement increases on both schedules as responses are allocated to either schedule. In Experiment 1, performance on concurrent variable-ratio schedules was assessed, and reinforcer ratios were varied across conditions to investigate changes in sensitivity. Additionally, the length of a changeover delay was manipulated. In Experiment 2, performance was compared under concurrently available, dependently programmed variable-ratio and fixed-ratio schedules. Performance was well described by the generalized matching law. Increases in the changeover delay decreased sensitivity, whereas sensitivity was higher when variable-ratio schedules were employed, compared with fixed-ratio schedules. Concurrent ratio schedules can be a viable approach to studying functional differences between ratio and interval schedules.     

Molar versus local reinforcement probability as determinants of stimulus value.

During one component of a multiple schedule, pigeons were trained on a discrete-trial concurrent variable-interval variable-interval schedule in which one alternative had a high scheduled rate of reinforcement and the other a low scheduled rate of reinforcement. When the choice proportion between the alternatives matched their respective relative reinforcement frequencies, the obtained probabilities of reinforcement (reinforcer per peck) were approximately equal. In alternate components of the multiple schedule, a single response alternative was presented with an intermediate scheduled rate of reinforcement. During probe trials, each alternative of the concurrent schedule was paired with the constant alternative. The stimulus correlated with the high reinforcement rate was preferred over that with the intermediate rate, whereas the stimulus correlated with the intermediate rate of reinforcement was preferred over that correlated with the low rate of reinforcement. Preference on probe tests was thus determined by the scheduled rate of reinforcement. Other subjects were presented all three alternatives individually, but with a distribution of trial frequency and reinforcement probability similar to that produced by the choice patterns of the original subjects. Here, preferences on probe tests were determined by the obtained probabilities of reinforcement. Comparison of the two sets of results indicates that the availability of a choice alternative, even when not responded to, affects the preference for that alternative. The results imply that models of choice that invoke only obtained probability of reinforcement as the controlling variable (e.g., melioration) are inadequate.     

The effects of concurrent responding and reinforcement on behavioral output

Four birds key pecked on concurrent variable-interval one-minute variable-interval four-minute schedules with a two-second changeover delay. Response rates to the variable-interval one-minute key were then reduced by signaling its reinforcer availability and later by providing its reinforcers independently of responding. Each manipulation increased response rates to the variable-interval four-minute key even though relative reinforcement rates were unchanged. In a final phase, eliminating the variable-interval one-minute key and its schedule produced the highest rates of all to the variable-interval four-minute key. These results show that both reinforcement and response rates to one schedule influence response rates to another schedule. These results join those of Guilkey, Shull, & Brownstein (1975) in failing to replicate Catania (1963). Moreover, they violate the predictions of the equation for simple action (de Villiers & Herrnstein, 1976). In terms of a median-rate measure (reciprocal of the median interresponse time), rates to the variable-interval four-minute key were high when responding was not reduced to the variable-interval one-minute key and were low when it was reduced. This rate difference suggests a process difference between concurrent-schedule procedures that maintain high concurrent response rates versus those that do not. This process difference jeopardizes attempts to integrate single- and concurrent-operant performances within a single formulation.     

Reporting contingencies of reinforcement in concurrent schedules

Five pigeons were trained on concurrent variable-interval schedules in which two intensities of yellow light served as discriminative stimuli in a switching-key procedure. A conditional discrimination involving a simultaneous choice between red and green keys followed every reinforcer obtained from both alternatives. A response to the red side key was occasionally reinforced if the prior reinforcer had been obtained from the bright alternative, and a response to the green side key was occasionally reinforced if the prior reinforcer had been obtained from the dim alternative. Measures of the discriminability between the concurrent-schedule alternatives were obtained by varying the reinforcer ratio for correct red and correct green responses across conditions in two parts. Part 1 arranged equal rates of reinforcement in the concurrent schedule, and Part 2 provided a 9:1 concurrent-schedule reinforcer ratio. Part 3 arranged a 1:9 reinforcer ratio in the conditional discrimination, and the concurrent-schedule reinforcer ratio was varied across conditions. Varying the conditional discrimination reinforcer ratio did not affect response allocation in the concurrent schedule, but varying the concurrent-schedule reinforcer ratio did affect conditional discrimination performance. These effects were incompatible with a contingency-discriminability model of concurrent-schedule performance (Davison & Jenkins, 1985), which implies a constant discriminability parameter that is independent of the obtained reinforcer ratio. However, a more detailed analysis of conditional discrimination performance showed that the discriminability between the concurrent-schedule alternatives decreased with time since changing over to an alternative. This effect, combined with aspects of the temporal distribution of reinforcers obtained in the concurrent schedules, qualitatively predicted the molar results and identified the conditions that operate whenever contingency discriminability remains constant.     

Effects of concurrent response-independent reinforcement on fixed-interval schedule performance

In three experiments, behavior maintained by fixed-interval schedules changed when response-independent reinforcement was delivered concurrently according to fixed- or variable-time schedules. In Experiment I, a pattern of positively accelerated responding during fixed interval was changed to a linear pattern when response-independent reinforcement occurred under a variable-time schedule. Overall response rates (total responses/total time) decreased as the frequency of response-independent reinforcement increased. Experiment II showed that the response-rate changes in the first experiment were controlled by the response-reinforcer relation, but the changes in patterns of responding were similar whether concurrently available reinforcement at varying times was response-dependent or response-independent. In the final experiment, the addition of response-independent reinforcement at fixed times to a fixed-interval schedule resulted in changes in both local and overall response rates and in the occurrence of positively accelerated responding between reinforcements. These results suggest that the temporal distribution of reinforcers determines response patterns and that both the response-reinforcement dependency and the schedule of reinforcement determine overall response rates during concurrently scheduled response-dependent and response-independent reinforcement.     

Contrast and induction in multiple schedules of discrete-trial concurrent reinforcement

Three pigeons were exposed to two-key discrete-trial concurrent schedules of reinforcement. Red and white key colors alternated irregularly and the assignment of reinforcers depended on key color. The red-key schedules were held constant, with the scheduled relative frequency of reinforcement for left-key pecks set at 0.75, while the white-key schedules varied. When the location of white-key reinforcement was changed from one side to the other, while its overall frequency was constant, red-key choices shifted in the same direction as white-key choices, an induction effect. When the overall frequency of white-key reinforcement was changed while its location remained constant, red key choices shifted in a direction opposite to white-key choices, a contrast effect. Both induction and contrast effects were clearer when the overall frequency of red-key reinforcement was reduced. These data demonstrate that the allocation of responding may exhibit schedule interaction effects similar to those commonly reported for response rate.     

Local rates of responding and reinforcement during concurrent schedules

The literature was searched for information about the local rates of responding and reinforcement during concurrent schedules. The local rates of reinforcement obtained from the two components of a concurrent schedule were equal when a long-duration changeover delay was used and when many sessions were conducted, except when the two components provided different simple schedules. The local rates of responding were equal under some conditions, but they differed when one component provided a ratio and the other an interval schedule. Across schedules, local rates of reinforcement changed with changes in the schedule of reinforcement. Local rates of responding did not change with changes in change-over-delay duration but did with changes in the changeover ratio and with changes in the programmed rates of reinforcement. The results generally conform to the Equalizing and Melioration Principles and help to clarify current statements of the Matching Law. The results also suggest that changes in the local rates of responding and reinforcement may be orderly across schedules.     

Effects of primary reinforcement on pigeons' initial-link responding under a concurrent chains schedule with nondifferntial terminal links

The effect of primary reinforcement on initial-link responding under concurrent-chains schedules with nondifferential terminal links was assessed in 12 pigeons. The iniitial and terminal links were variable-interval schedules (always the same for both alternatives). The positions (left or right key) of the initial-link stimuli (red or green) were randomized while the correlation between color and food amount remained constant within each condition. The terminal-link stimuli were always presented on the center key. Except in two control groups and conditions, the terminal-link stimuli were the same color (nondifferential, blue or yellow). Over six conditions, the differences in food amont and the durations of the initial- and terminal-link schedules were manipulated. In 57 of 60 cases, birds generated choice proportions above .50 in favor of the initial-link stimlus that was correlated with the larger reinforcer. There was some indication that preference increased with shortened terminal-link durations. Because the terminal-link stimuli were nondifferential, differential responding in the initial links cannot be explained easily by conditioned reinforcement represented by the terminal-link stimuli. Thus, primiary reinforcement has a direct effect on initial-link responding in concurrent-chains schedules.     

Effects of reinforcement context on choice

Two experiments investigated the effects of successive reinforcement contexts on choice. In the first, concurrent variable-interval schedules of primary reinforcement operated during the initial links of concurrent chains. The rate of this reinforcement arranged by the concurrent schedules was decreased across conditions: When it was higher than the terminal-link rate, preference for the higher frequency initial-link schedule increased relative to baseline. (During baseline, a standard concurrent-schedule procedure was in effect). When the initial-link reinforcement rate was lower than the terminal-link rate, preference converged toward indifference. In the second experiment, a chain schedule was available on a third key while a concurrent schedule was in effect on the side keys. When the terminal link of the chain schedule was produced, the side keys became inoperative. Availability of the chain schedule did not affect choice between the concurrent schedules. These results show that only when successive reinforcement contexts are produced by choice responding do those successive contexts affect choice in concurrent schedules.     

Effects on choice of reinforcement delay and conditioned reinforcement

Pigeons chose between fixed-interval schedules of different durations presented in the terminal links of concurrent-chains schedules. The pair of schedules was always in the ratio of 2:1, but the absolute duration of the fixed intervals varied. In one set of conditions, the different terminal-link schedules were associated with different keylight stimuli (cued conditions). In a second set of conditions, the different terminal-link schedules were associated with the same stimulus (uncued conditions). Results from the cued conditions replicated previous findings that preference for the shorter fixed-interval schedule increased with fixed-interval duration. Preferences in the uncued conditions were lower than in the corresponding cued conditions but also increased with fixed-interval length. In addition, the degree of control under the uncued conditions was correlated with the extent to which the schedule during the terminal link was discriminated immediately upon entry into the terminal link. The pattern of results in both conditions was inconsistent with the notion that choice behavior matches relative immediacy of reinforcement. Reanalysis of previous evidence for matching (Chung and Herrnstein, 1967) showed that matching in fact did not occur, as the preferences of their subjects for the shorter of two delays also increased with the absolute size of the delays.     

Choice and conditioned reinforcement.

A potential weakness of one formulation of delay-reduction theory is its failure to include a term for rate of conditioned reinforcement, that is, the rate at which the terminal-link stimuli occur in concurrent-chains schedules. The present studies assessed whether or not rate of conditioned reinforcement has an independent effect upon choice. Pigeons responded on either modified concurrent-chains schedules or on comparable concurrent-tandem schedules. The initial link was shortened on only one of two concurrent-chains schedules and on only one of two corresponding concurrent-tandem schedules. This manipulation increased rate of conditioned reinforcement sharply in the chain but not in the tandem schedule. According to a formulation of delay-reduction theory, when the outcomes chosen (the terminal links) are equal, as in Experiment 1, choice should depend only on rate of primary reinforcement; thus, choice should be equivalent for the tandem and chain schedules despite a large difference in rate of conditioned reinforcement. When the outcomes chosen are unequal, however, as in Experiment 2, choice should depend upon both rate of primary reinforcement and relative signaled delay reduction; thus, larger preferences should occur in the chain than in the tandem schedules. These predictions were confirmed, suggesting that increasing the rate of conditioned reinforcement on concurrent-chains schedules may have no independent effect on choice.     

Timeout from concurrent schedules.

Response-contingent timeouts of equal duration and frequency were added to both alternatives of unequal concurrent schedules of reinforcement. For each of 4 pigeons in Experiment 1, relative response rates generally became less extreme as the frequency of timeout increased. In Experiment 2, relative response rates consistently approached indifference as the duration of timeout was increased. Variation in time allocation was less consistent in both experiments. Absolute response rates did not vary with the timeout contingency in either experiment. In a third experiment, neither measure of choice varied systematically when the duration of a postreinforcement blackout was varied. In contrast to the present results, preference has been shown to vary directly with the parameters of shock delivery in related procedures. The pattern of results in the first two experiments follows that obtained with other manipulations of the overall rate of reinforcement in concurrent schedules. The results of the third experiment suggest that an intertrial interval following reinforcement is not a critical feature of the overall rate of reinforcement.     

Response rate and changeover performance on concurrent variable-interval schedules

Six pigeons were exposed to variable-interval schedules arranged on one, two, three, and four response keys. The reinforcement rate was also varied across conditions. Numbers of responses, the time spent responding, the number of reinforcements, and the number of changeovers between keys were recorded. Response rates on each key were an increasing function of reinforcement rate on that key and a decreasing function of the reinforcement rate on other keys. Response and time-allocation ratios under-matched ratios of obtained reinforcements. Three sets of equations were developed to express changeover rate as a function of response rate, time allocation, and reinforcement rate respectively. These functions were then applied to a broad range of experiments in the literature in order to test their generality. Further expressions were developed to account for changeover rates reported in experiments where changeover delays were varied.     

Matching with a trio of concurrent variable-interval schedules of reinforcement

A trio of concurrent variable-interval schedules of reinforcement was arranged according to a changeover-key procedure, including a changeover delay of 1.5 sec. The three schedules provided a combined maximum reinforcement rate of 45 reinforcements per hour. With that restriction, the nine experimental conditions included several combinations of variable-interval schedules, sometimes including extinction. The pigeons matched relative response rate and relative time to relative reinforcement rate. Relative time appeared to match some-what better than relative response rate. Performance adjusted rapidly from one experimental condition to the next, whether the change involved two or all three schedules of the concurrent trio.     

Undermatching: a reappraisal of performance on concurrent variable-interval schedules of reinforcement

The extant data for pigeons' performance on concurrent variable-interval schedules were examined in detail. Least-squares lines relating relative pecks and time to the corresponding relative reinforcements were obtained for four studies. The between-study group slopes for time and pecks and five of seven within-study group slopes from individual studies were less than 1.00. This suggested the generality that pigeons respond less to the richer reinforcement schedule than predicted by matching. For pecks, a nonparametric test for distribution of points also supported this concept of undermatching (to the richer reinforcement schedule). In addition, using mean squared error as the criterion, a cubic curve fit the peck proportion data better than any line or other polynomial. This indicates that the relation between peck and reinforcement proportions may be nonlinear.     

The effects of delayed reinforcement on free-operant responding

In previous studies of delayed reinforcement, response rate has been found to vary inversely with the response-reinforcer interval. However, in all of these studies the independent variable, response-reinforcer time, was confounded with the number of reinforcers presented in a fixed period of time (reinforcer frequency). In the present study, the frequency of available reinforcers was held constant, while temporal separation between response and reinforcer was independently manipulated. A repeating time cycle, T, was divided into two alternating time periods, t^D and t^Δ. The first response in t^D was reinforced at the end of the prevailing T cycle and extinction prevailed in t^Δ. Two placements for t^D were defined, an early t^D placement in which t^D precedes t^Δ and a late t^D placement in which t^D follows t^Δ. The duration of the early and late t^D was systematically decreased from 30 seconds (i.e., t^D = T) to 0.1 second. Manipulation of t^D placement and duration controlled the temporal separation between response and reinforcement, but it did not affect the frequency of programmed reinforcers, which was 1/T. The results show that early and late t^D placements of equal duration have similar overall effects upon response rate, reinforcer frequency, responses per reinforcer, and obtained response-reinforcer temporal separation. A stepwise regression analysis using log response rate as the dependent variable showed that the obtained delay was a significant first-step variable for six of eight subjects, with obtained reinforcer frequency significant for the remaining two subjects.     

Allocation of complex, sequential operants on multiple and concurrent schedules of reinforcement

Pigeons could produce food by pecking exactly four times on each of two keys, in any order. In the first experiment, these response sequences were reinforced on a series of multiple schedules of variable-interval reinforcement. In the second experiment, these response sequences were reinforced on a series of concurrent schedules of reinforcement. In both experiments, highly stereotyped response sequences developed. If these response sequences were treated as individual responses, the resulting data conformed to what is typically reported in studies of multiple and concurrent schedules involving individual responses. For example, behavioral contrast was observed with the multiple schedules, and matching was observed with the concurrent schedules. However, schedule manipulation had no effect on within-sequence characteristics of responses like accuracy, stereotypy, or rate. These data constitute further evidence that response sequences can become functional behavioral units.     

How to teach a pigeon to maximize overall reinforcement rate

In two experiments deviations from matching earned higher overall reinforcement rates than did matching. In Experiment 1 response proportions were calculated over a 360-response moving average, updated with each response. Response proportions that differed from the nominal reinforcement proportions, by a criterion that was gradually increased, were eligible for reinforcement. Response proportions that did not differ from matching were not eligible for reinforcement. When the deviation requirement was relatively small, the contingency proved to be effective. However, there was a limit as to how far response proportions could be pushed from matching. Consequently, when the deviation requirement was large, overall reinforcement rate decreased and pecking was eventually extinguished. In Experiment 2 a discriminative stimulus was added to the procedure. The houselight was correlated with the relationship between response proportions and the nominal (programmed) reinforcement proportions. When the difference between response and reinforcement proportions met the deviation requirement, the light was white and responses were eligible for reinforcement. When the difference between response and reinforcement proportions failed to exceed the deviation requirement, the light was blue and responses were not eligible for reinforcement. With the addition of the light, it proved to be possible to shape deviations from matching without any apparent limit. Thus, in Experiment 2 overall reinforcement rate predicted choice proportions and relative reinforcement rate did not. In contrast, in previous experiments on the relationship between matching and overall reinforcement maximization, relative reinforcement rate was usually the better predictor of responding. The results show that whether overall or relative reinforcement rate better predicts choice proportions may in part be determined by stimulus conditions.