Local rates of responding and reinforcement during concurrent schedules

The literature was searched for information about the local rates of responding and reinforcement during concurrent schedules. The local rates of reinforcement obtained from the two components of a concurrent schedule were equal when a long-duration changeover delay was used and when many sessions were conducted, except when the two components provided different simple schedules. The local rates of responding were equal under some conditions, but they differed when one component provided a ratio and the other an interval schedule. Across schedules, local rates of reinforcement changed with changes in the schedule of reinforcement. Local rates of responding did not change with changes in change-over-delay duration but did with changes in the changeover ratio and with changes in the programmed rates of reinforcement. The results generally conform to the Equalizing and Melioration Principles and help to clarify current statements of the Matching Law. The results also suggest that changes in the local rates of responding and reinforcement may be orderly across schedules.     

Choice, changing over, and reinforcement delays

In three experiments, pigeons were used to examine the independent effects of two normally confounded delays to reinforcement associated with changing between concurrently available variable-interval schedules of reinforcement. In Experiments 1 and 2, combinations of changeover-delay durations and fixed-interval travel requirements were arranged in a changeover-key procedure. The delay from a changeover-produced stimulus change to a reinforcer was varied while the delay between the last response on one alternative and a reinforcer on the other (the total obtained delay) was held constant. Changeover rates decreased as a negative power function of the total obtained delay. The delay between a changeover-produced stimulus change had a small and inconsistent effect on changeover rates. In Experiment 3, changeover delays and fixed-interval travel requirements were arranged independently. Changeover rates decreased as a negative power function of the total obtained delay despite variations in the delay from a change in stimulus conditions to a reinforcer. Periods of high-rate responding following a changeover, however, were higher near the end of the delay from a change in stimulus conditions to a reinforcer. The results of these experiments suggest that the effects of changeover delays and travel requirements primarily result from changes in the delay between a response at one alternative and a reinforcer at the other, but the pattern of responding immediately after a changeover depends on the delay from a changeover-produced change in stimulus conditions to a reinforcer.     

Choice: Effects of changeover schedules on concurrent performance

The components of concurrent schedules were separated temporally by placing interval schedules on the changeover key. The rates of responding on both the main and changeover keys were examined as a function of the reinforcement rates. In the first experiment, the sensitivity of main-key performance to changing reinforcement rates was inversely related to the temporal separation of components, and changeover performance was monotonically related to the ratio of the reinforcement rates. In the second experiment, when the ratio of the reinforcement rates was scheduled to remain constant while the frequency of reinforcement was varied, changeover performance did not remain constant. A “sampling” interpretation of changeover responding was proposed and subsequently tested in a third experiment where extinction was always scheduled in one component and the frequency of reinforcement was varied in the second component. It was concluded that changeover performance can be interpreted using molar measures of reinforcement and that animals sample activities available to them at rates which are controlled by relative reinforcement rates.     

Response rate and changeover performance on concurrent variable-interval schedules

Six pigeons were exposed to variable-interval schedules arranged on one, two, three, and four response keys. The reinforcement rate was also varied across conditions. Numbers of responses, the time spent responding, the number of reinforcements, and the number of changeovers between keys were recorded. Response rates on each key were an increasing function of reinforcement rate on that key and a decreasing function of the reinforcement rate on other keys. Response and time-allocation ratios under-matched ratios of obtained reinforcements. Three sets of equations were developed to express changeover rate as a function of response rate, time allocation, and reinforcement rate respectively. These functions were then applied to a broad range of experiments in the literature in order to test their generality. Further expressions were developed to account for changeover rates reported in experiments where changeover delays were varied.     

Concurrent schedules: Effects of reinforcement rate and changeover delay on time allocation in a three-alternative procedure

Pigeons were exposed to a continuous choice procedure where three alternatives alternated in a fixed, recycling order (ABCABC, etc.). Responses were reinforced according to independent variable-interval schedules. For three birds, the reinforcement rate for responses on alternative C was varied. For three other birds, the duration of the changeover delay after the changeover to C was varied. For both groups, the reinforcement rates and changeover delay durations associated with A and B were constant throughout the experiment. The time proportion at A relative to B increased as a function of the reinforcement rate for responses on C and decreased as a function of the duration of the changeover delay during C. The results show that the proportion of time spent at a variable-interval alternative of a continuous choice procedure is not completely determined by the reinforcement rates provided by the alternatives. The results support the assumption that time allocation is governed by delayed reinforcement of changeover behaviour.     

On the functions of the changeover delay

The function of changeover delays in producing matching was examined with pigeons responding on concurrent variable-interval variable-interval schedules. In Experiment 1, no changeover delay was compared to two different types of changeover delay. One type, designated generically as response-response but in the present example as peck-peck, was timed from the first response on the switched-to key; the other, designated generically as pause-response but in the present example as pause-peck, was timed from the last response on the switched-from key. High changeover rates occurred with no changeover delay. Peck-peck and pause-peck changeover delays produced low and intermediate changeover rates, respectively. In Experiment 2, pause-peck and peck-peck changeover delays were compared across a range of relative reinforcement rates. Similar matching relations developed despite differences in the changeover rates and local response patterns as a function of the type of changeover delay. In Experiment 3, both types of changeover delay yielded similar changeover rates when their obtained durations were equal via yoking. The results suggest that changeover delays function to separate responses on one key from reinforcers on the other or to delay reinforcement for changing over. In addition, the distribution of responding during and after the changeover delay may vary considerably without affecting matching.     

Reinforcing staying and switching while using a changeover delay

Performance on concurrent schedules can be decomposed to run lengths (the number of responses before switching alternatives), or visit durations (time at an alternative before switching alternatives), that are a function of the ratio of the rates of reinforcement for staying and switching. From this analysis, a model of concurrent performance was developed and examined in two experiments. The first exposed rats to variable-interval schedules for staying and for switching, which included a changeover delay for reinforcers following a switch. With the changeover delay, run lengths and visit durations were functions of the ratios of the rates of reinforcement for staying and for switching, as found by previous research not using a changeover delay. The second directly assessed the effect of a changeover delay on run lengths and visit durations. Each component of a multiple schedule consisted of equivalent stay and switch schedules but only one component included a changeover delay. Run lengths and visit durations were longer when a changeover delay was used. Because visit duration is the reciprocal of changeover rate, these results are consistent with the established finding that a changeover delay reduces the frequency of switching. Together these results support the local model of concurrent performance as an alternative to the generalized matching law as a model of concurrent performance. The local model may be preferred when accounting for more molecular aspects of concurrent performance.     

Concurrent schedules: a quantitative relation between changeover behavior and its consequences

Data from several published experiments on concurrent variable-interval schedules were analyzed with respect to the effects of changeover delay on the time spent responding on a schedule before changing to an alternate schedule: i.e., the interchangeover time. Interchangeover time increases as the duration of the changeover delay increases, and the present analysis shows that a power function describes the relation. The power relation applied in spite of numerous differences in the experiments: different variable-interval schedules for the concurrent pairs; equal or unequal reinforcement rates for the schedules of the concurrent pairs; different durations of the changeover delay; response-dependent or response-independent reinforcers; pigeons or rats as subjects; different reinforcers. A power function also described the data in experiments where the changeover incurred a timeout, where a fixed ratio was required to changeover, and also when asymmetrical changeover delays were used.     

Concurrent responding with fixed relative rate of reinforcement

Responding by pigeons on one key of a two-key chamber alternated the color of the second key, on which responding produced food according to a variable-interval schedule of reinforcement. From time to time, reinforcement would be available for a response, but in the presence of a particular stimulus, either red or green light on the key. Red or green was chosen irregularly from reinforcement to reinforcement, so that a proportion of the total number of reinforcements could be specified for each color. Experimental manipulations involved variations of (1) the proportions for each color, (2) changeover delay, or, alternatively, (3) a fixed-ratio changeover requirement. The main findings were: (1) relative overall rates of responding and relative times in the presence of a key color approximated the proportions of reinforcements obtained in the presence of that color, while relative local rates of responding changed little; (2) changeover rate decreased as the proportions diverged from 0.50; (3) relative overall rate of responding and relative time remained constant as the changeover delay was increased from 2 to 32 sec, with reinforcement proportions for red and green of 0.75 and 0.25, but they increased above 0.90 when a fixed-ratio changeover of 20 responses replaced the changeover delay; (4) changeover rate decreased as the delay or fixed-ratio was increased.     

The role of discriminative stimuli in concurrent performances

Key pecking in pigeons was examined under concurrent and parallel arrangements of two independent and simultaneously available variable-interval schedules. Pecks on the changeover key alternated the schedule of reinforcement for responses on the main key. Under concurrent schedules, discriminative stimuli were paired with the reinforcement schedule arranged in each component and changeover responses also alternated these stimuli. Under parallel schedules, changeover responses alternated the effective reinforcement schedule, but did not change the discriminative stimulus. On concurrent procedures, changeover response rate was inversely related to the difference in reinforcement rate between the two components, whereas on parallel schedules no consistent relationship was found. With both schedules, absolute response and reinforcement rates were positively related, although for a given set of reinforcement frequencies, rates were often higher on the concurrent schedules. On concurrent schedules, relative response rates and relative times were equal to relative reinforcement rates. On parallel schedules these ratios were positively related, but response and time ratios were much smaller than were obtained with comparable concurrent schedules. This inequality was most pronounced when absolute reinforcement frequencies were lowest.     

Choice, changeover, and travel: A quantitative model

Six pigeons were trained on concurrent variable-interval schedules in which responding on fixed-interval schedules was required to give access to the alternate schedule. Responding on the concurrent schedules was not allowed, after changing over had commenced, until the changeover schedule had been completed. In Parts 1 to 3 of the experiment, the changeover fixed-interval schedules were equal and were 0 s, 10 s, and 20 s, respectively. In each part, the relative frequency of reinforcement obtained on the concurrent schedules was varied over at least five conditions. In Part 4, the concurrent schedules were equal, and one changeover fixed-interval schedule was twice the other. Under these conditions, the absolute sizes of the changeover schedules were varied. Increasing the changeover requirement from 0 s to 10 s (Parts 1 and 2) resulted in increases in the sensitivity of behavior allocation to reinforcers obtained, but no further increase was obtained when the changeover schedules were increased to 20 s (Part 3). In Part 4, performance was biased towards the concurrent schedule that took less time to enter. These results are consistent with a subtractive punishment model of travel in which the degree of punishment is measured by the number of reinforcers apparently lost from a schedule when the subject changes to that schedule. Absolute times spent on the main keys could be accurately described by a previous model of changeover performance.     

Analysis of group differences in processing speed: Where are the models of processing?

Recently, Myerson, Adams, Hale, and Jenkins (2003) replied to arguments advanced by Ratcliff, Spieler, and McKoon (2000) about interpretations of Brinley functions. Myerson et al. (2003) focused on methodological and terminological issues, arguing that (1) Brinley functions are not quantile-quantile (QQ) plots of distributions of mean reaction times (RTs) across conditions; that the fact that the slope of a Brinley function is the ratio of the standard deviations of the two distributions of means has no implications for the use of slope as a measure of processing speed; that the ratio of slopes of RT functions for older and young subjects plotted against independent variables equals the Brinley function slope; and that speed-accuracy criterion effects do not account for slowing with age. We reply by showing that Brinley functions are plots of quantiles against quantiles; that the slope is best estimated by the ratio of standard deviations because there is variability in the distributions of mean RTs for both older and young subjects; that the interpretation of equality of the slopes Brinley functions and plots of RTs against independent variables in terms of processing speed is model dependent; and that speed-accuracy effects in some, but not all, experiments are solely responsible for Brinley slopes greater than 1. We conclude by reiterating the point that was not addressed in Myerson et al. (2003), that the goal of research should be modelbased accounts of processing that deal with correct and error RT distributions and accuracy.     

Variables affecting preference for signaled shock in a symmetrical changeover design

In a signaled-shock situation, the signal's presence serves a warning function (prediction of shock) whereas the signal's absence serves a safety function (prediction of nonshock periods). The relative contribution to preference for signaled shock made by these two potential sources of reinforcement was assessed using a symmetrical changeover procedure in which rats were permitted to control the amount of time spent in one of two mutually exclusive conditions, the signaled and unsignaled states. The stimulus conditions that prevailed in the signaled state differed for several groups of animals. For some groups, the safety function of signals was degraded while the warning function was left intact. For other groups, the warning function was degraded and the safety function was not. The availability of the unaltered feature was then manipulated in both sets of groups to determine its effect on preference for the signaled state. Changeover behavior was strongly related to the availability of the signal's safety function and weakly related to its warning function, indicating that the two variables contribute differentially to preference for signaled shock. Other sources of behavioral control, having little to do with the safety or the warning function of signals, were also demonstrated, suggesting the need for careful control procedures when using changeover behavior as a measure of preference.     

The reinforcement of least-frequent interresponse times

A new schedule of reinforcement was used to maintain key-pecking by pigeons. The schedule reinforced only pecks terminating interresponse times which occurred least often relative to the exponential distribution of interresponse times to be expected from an ideal random generator. Two schedule parameters were varied: (1) the rate constant of the controlling exponential distribution and (2) the probability that a response would be reinforced, given that it met the interresponse-time contingency. Response rate changed quickly and markedly with changes in the rate constant; it changed only slightly with a fourfold change in the reinforcement probability. The schedule produced stable rates and high intra- and inter-subject reliability, yet interresponse time distributions were approximately exponential. Such local interresponse time variability in the context of good overall control suggests that the schedule may be used to generate stable, predictable, yet sensitive baseline rates. Implications for the measurement of rate are discussed.     

Concurrent schedules: Spatial separation of response alternatives

Four pigeons were exposed to independent concurrent variable-interval 20-second variable-interval 60-second schedules of reinforcement. A transparent partition was inserted midway between the two response keys. The length of the partition was systematically manipulated. Increasing partition length produced a decrease in changeover rate in Experiment 1. Over-matching was observed with a partition length of 20 centimeters. In Experiment 2 a four-second limited hold was added to the schedules. Increasing partition length produced a decrease in changeover rate that exceeded the decrease observed in Experiment 1. This manipulation produced nearly exclusive choice of the variable-interval 20-second component. The present results, together with results obtained in related research, suggest that deviation from matching is a function of procedural variables that determine the consequences of a changeover response.     

Concurrent-schedule performance in transition: changeover delays and signaled reinforcer ratios

Six pigeons were trained in experimental sessions that arranged six or seven components with various concurrent-schedule reinforcer ratios associated with each. The order of the components was determined randomly without replacement. Components lasted until the pigeons had received 10 reinforcers, and were separated by 10-s blackout periods. The component reinforcer ratios arranged in most conditions were 27:1, 9:1, 3:1, 1:1, 1:3, 1:9 and 1:27; in others, there were only six components, three of 27:1 and three of 1:27. In some conditions, each reinforcement ratio was signaled by a different red-yellow flash frequency, with the frequency perfectly correlated with the reinforcer ratio. Additionally, a changeover delay was arranged in some conditions, and no changeover delay in others. When component reinforcer ratios were signaled, sensitivity to reinforcement values increased from around 0.40 before the first reinforcer in a component to around 0.80 before the 10th reinforcer. When reinforcer ratios were not signaled, sensitivities typically increased from zero to around 0.40. Sensitivity to reinforcement was around 0.20 lower in no-changeover-delay conditions than in changeover-delay conditions, but increased in the former after exposure to changeover delays. Local analyses showed that preference was extreme towards the reinforced alternative for the first 25 s after reinforcement in changeover-delay conditions regardless of whether components were signaled or not. In no-changeover-delay conditions, preference following reinforcers was either absent, or, following exposure to changeover delays, small. Reinforcers have both local and long-term effects on preference. The former, but not the latter, is strongly affected by the presence of a changeover delay. Stimulus control may be more closely associated with longer-term, more molar, reinforcer effects.     

Changeover behavior under pairs of fixed-ratio and variable-ratio schedules of reinforcement

Three pigeons were studied under a pair of equal fixed-ratio schedules and a pair of equal variable-ratio schedules. Each pair was arranged as independent, concurrent schedules and also in a non-independent relation where each peck in a schedule counted toward the response requirement of both schedules. The non-independent pair of variable-ratio schedules maintained much higher changeover rates than any of the other three arrangements. Thus, two factors seemed necessary for generating high changeover rates. Responding on a schedule had to count toward the response requirement of both schedules, and the component schedules had to be variable. These data are consistent with the hypothesis that changeovers are at least partly controlled by the probability of reinforcement following a changeover.     

Reinforcement of eye movement with concurrent schedules

Human macrosaccadic eye movements to two areas of a four-dial display were conditioned by concurrent variable-interval schedules of signals. Reinforcers (signals) were delivered to the two right-hand dials on one schedule and to the two left-hand dials on another, independent schedule. The use of a changeover delay between crossover eye movements and reinforcement had the effect of changing the pattern of scanning from fixating four dials in succession or in a Z-shaped pattern to scanning vertically the dials on either side with fewer crossovers. In the presence of a changeover delay, subjects matched relative eye-movement rates and relative reinforcement rates on each schedule. Rate of crossover eye movements, with a changeover delay in effect, was also inversely related to the difference in reinforcements arranged by the concurrent schedules. The results suggest that for stimuli whose critical components are arranged spatially, conditioned eye movements play an important part in selective stimulus control.     

Concurrent fixed-interval variable-ratio schedules and the matching relation

Five rats responded under concurrent fixed-interval variable-ratio schedules of food reinforcement. Fixed-interval values ranged from 50-seconds to 300-seconds and variable-ratio values ranged from 30 to 360; a five-second changeover delay was in effect throughout the experiment. The relations between reinforcement ratios obtained from the two schedules and the ratios of responses and time spent on the schedules were described by Baum's (1974) generalized matching equation. All subjects undermatched both response and time ratios to reinforcement ratios, and all subjects displayed systematic bias in favor of the variable-ratio schedules. Response ratios undermatched reinforcement ratios less than did time ratios, but response ratios produced greater bias than did time ratios for every subject and for the group as a whole. Local rates of responding were generally higher on the variable-ratio than on the fixed-interval schedules. When responding was maintained by both schedules, a period of no responding on either schedule immediately after fixed-interval reinforcement typically was followed by high-rate responding on the variable-ratio schedule. At short fixed-interval values, when a changeover to the fixed-interval schedule was made, responding usually continued until fixed-interval reinforcement was obtained; at longer values, a changeover back to the variable-ratio schedule usually occurred when fixed-interval reinforcement was not forthcoming within a few seconds, and responding then alternated between the two schedules every few seconds until fixed-interval reinforcement finally was obtained.