Reporting contingencies of reinforcement in concurrent schedules

Five pigeons were trained on concurrent variable-interval schedules in which two intensities of yellow light served as discriminative stimuli in a switching-key procedure. A conditional discrimination involving a simultaneous choice between red and green keys followed every reinforcer obtained from both alternatives. A response to the red side key was occasionally reinforced if the prior reinforcer had been obtained from the bright alternative, and a response to the green side key was occasionally reinforced if the prior reinforcer had been obtained from the dim alternative. Measures of the discriminability between the concurrent-schedule alternatives were obtained by varying the reinforcer ratio for correct red and correct green responses across conditions in two parts. Part 1 arranged equal rates of reinforcement in the concurrent schedule, and Part 2 provided a 9:1 concurrent-schedule reinforcer ratio. Part 3 arranged a 1:9 reinforcer ratio in the conditional discrimination, and the concurrent-schedule reinforcer ratio was varied across conditions. Varying the conditional discrimination reinforcer ratio did not affect response allocation in the concurrent schedule, but varying the concurrent-schedule reinforcer ratio did affect conditional discrimination performance. These effects were incompatible with a contingency-discriminability model of concurrent-schedule performance (Davison & Jenkins, 1985), which implies a constant discriminability parameter that is independent of the obtained reinforcer ratio. However, a more detailed analysis of conditional discrimination performance showed that the discriminability between the concurrent-schedule alternatives decreased with time since changing over to an alternative. This effect, combined with aspects of the temporal distribution of reinforcers obtained in the concurrent schedules, qualitatively predicted the molar results and identified the conditions that operate whenever contingency discriminability remains constant.     

Choice: Effects of changeover schedules on concurrent performance

The components of concurrent schedules were separated temporally by placing interval schedules on the changeover key. The rates of responding on both the main and changeover keys were examined as a function of the reinforcement rates. In the first experiment, the sensitivity of main-key performance to changing reinforcement rates was inversely related to the temporal separation of components, and changeover performance was monotonically related to the ratio of the reinforcement rates. In the second experiment, when the ratio of the reinforcement rates was scheduled to remain constant while the frequency of reinforcement was varied, changeover performance did not remain constant. A “sampling” interpretation of changeover responding was proposed and subsequently tested in a third experiment where extinction was always scheduled in one component and the frequency of reinforcement was varied in the second component. It was concluded that changeover performance can be interpreted using molar measures of reinforcement and that animals sample activities available to them at rates which are controlled by relative reinforcement rates.     

Response rate and changeover performance on concurrent variable-interval schedules

Six pigeons were exposed to variable-interval schedules arranged on one, two, three, and four response keys. The reinforcement rate was also varied across conditions. Numbers of responses, the time spent responding, the number of reinforcements, and the number of changeovers between keys were recorded. Response rates on each key were an increasing function of reinforcement rate on that key and a decreasing function of the reinforcement rate on other keys. Response and time-allocation ratios under-matched ratios of obtained reinforcements. Three sets of equations were developed to express changeover rate as a function of response rate, time allocation, and reinforcement rate respectively. These functions were then applied to a broad range of experiments in the literature in order to test their generality. Further expressions were developed to account for changeover rates reported in experiments where changeover delays were varied.     

Arousal, changeover responses, and preference in concurrent schedules

Pigeons were trained on multiple schedules that provided concurrent reinforcement in each of two components. In Experiment 1, one component consisted of a variable-interval (VI) 40-s schedule presented with a VI 20-s schedule, and the other a VI 40-s schedule presented with a VI 80-s schedule. After extended training, probe tests measured preference between the stimuli associated with the two 40-s schedules. Probe tests replicated the results of Belke (1992) that showed preference for the 40-s schedule that had been paired with the 80-s schedule. In a second condition, the overall reinforcer rate provided by the two components was equated by adding a signaled VI schedule to the component with the lower reinforcer rate. Probe results were unchanged. In Experiment 2, pigeons were trained on alternating concurrent VI 30-s VI 60-s schedules. One schedule provided 2-s access to food and the other provided 6-s access. The larger reinforcer magnitude produced higher response rates and was preferred on probe trials. Rate of changeover responding, however, did not differ as a function of reinforcer magnitude. The present results demonstrate that preference on probe trials is not a simple reflection of the pattern of changeover behavior established during training.     

Some effects of relative reinforcement rate and changeover delay in response-independent concurrent schedules of reinforcement

Reinforcements were arranged independently of the pigeon's behavior by concurrent variable-interval schedules. The reinforcements arranged by one of the schedules occurred when the chamber was illuminated with amber light, and the reinforcements arranged by the other schedule occurred when the chamber was illuminated with blue light. Both schedules functioned concurrently, but reinforcers were delivered by each only in the presence of the appropriate stimulus condition. A response on a white key, the only key in the chamber, alternated the stimulus condition and the effective schedule. The results of this procedure were similar to those obtained with concurrent response-dependent variable-interval schedules of reinforcement. The proportion of the total session time spent in the presence of a schedule component approximated the proportion of the total number of reinforcements in the component. Changeover rate was a decreasing function of the changeover delay and of the difference between the relative rates of reinforcement for each pair of concurrent schedules.     

Concurrent schedules: Transient-state changeover behaviour

Ten pigeons responded in a two-alternative continuous-choice situation. Rein-forcers were programmed by a single variable-interval 30-sec schedule. The proportion of reinforcers assigned to one alternative was either 0.67 or 0.50 in different conditions. The changes in local changeover rates following changes of conditions were adequately described by an exponential function. This result can be explained by a model of changeover behaviour proposed by Myerson and Miezin (1980), if a simplifying assumption is made about reinforcement rates as controlling variables. A modification of the Myerson and Miezin model that accounts for the exponential function but does not make assumptions about reinforcement rates as controlling variables is presented.     

The effects of different component response requirements in multiple and concurrent schedules

Six pigeons were trained on multiple and concurrent schedules. The reinforcement rates were varied systematically (a) when lever pressing was required in one component and key pecking in the successive component; (b) when lever pressing was required in both multiple components; (c) when key pecking was required in both multiple components; and (d) when key pecking was required on one schedule and lever pressing was required on the concurrently-available schedule. Only the absolute level of responding was changed by different response requirements. Analyzed by the generalized matching law, performance under different response requirements resulted in a bias toward key pecking, and the measured response bias was the same in multiple and concurrent schedule arrangements. The bias in time measures obtained from concurrent schedule performance was reliably smaller than the obtained response biases. The sensitivity to reinforcement-rate changes was ordered: concurrent key-lever; multiple key-key; multiple lever-key; and, the least sensitive, multiple lever-lever. The results confirm that requirements of different topographical responses can be handled by the generalized matching law mainly in the bias parameter, but problems for this type of analysis may be caused by the changing sensitivity to reinforcement in multiple schedule performance as response requirements are changed.     

Response-rate differences in variable-interval and variable-ratio schedules: An old problem revisited

In Experiment 1, a variable-ratio 10 schedule became, successively, a variable-interval schedule with only the minimum interreinforcement intervals yoked to the variable ratio, or a variable-interval schedule with both interreinforcement intervals and reinforced interresponse times yoked to the variable ratio. Response rates in the variable-interval schedule with both interreinforcement interval and reinforced interresponse time yoking fell between the higher rates maintained by the variable-ratio schedule and the lower rates maintained by the variable-interval schedule with only interreinforcement interval yoking. In Experiment 2, a tandem variable-interval 15-s variable-ratio 5 schedule became a yoked tandem variable-ratio 5 variable-interval x-s schedule, and a tandem variable-interval 30-s variable-ratio 10 schedule became a yoked tandem variable-ratio 10 variable-interval x-s schedule. In the yoked tandem schedules, the minimum interreinforcement intervals in the variable-interval components were those that equated overall interreinforcement times in the two phases. Response rates did not decline in the yoked schedules even when the reinforced interresponse times became longer. Experiment 1 suggests that both reinforced interresponse times and response rate–reinforcement rate correlations determine response-rate differences in variable-ratio 10 and yoked variable-interval schedules in rats. Experiment 2 suggests a minimal role for the reinforced interresponse time in determining response rates on tandem variable-interval 30-s variable-ratio 10 and yoked tandem variable-ratio 10 variable-interval x-s schedules in rats.     

Concurrent schedules of reinforcement: effects of gradual and abrupt increases in changeover delay

Pigeons' pecks were maintained on concurrent variable-interval 1-min variable-interval 3-min schedules of reinforcement, with a changeover delay of 2 sec. When changeover delay was increased successively to 5.0, 7.5, and 12.5 sec (Exp. I) the actual relative rate of reinforcement for the variable-interval 3-min key decreased progressively for two birds, abruptly for two other birds, and the subjects devoted proportionately less of their time and responding to that key. However, the relative performance measures (relative time and relative responding) approximated the actual relative rate of reinforcement, with a maximum discrepancy of 11%, over all changeover delay values investigated. Experiment II attempted to lengthen response-run durations on the variable-interval 3-min key so that they were long enough to meet the changeover delay requirement at each new changeover delay value, by progressively increasing the changeover delay by 0.5-sec increments. With this procedure the actual relative rate of reinforcement approximated more closely the scheduled relative rate as changeover delay increased. As in Exp. I, relative performance measures approximated the actual relative reinforcement rate (maximum discrepancy 17%).     

Concurrent schedules: Maximization versus reinforcement of changeover behaviour

Pigeons responded on concurrent variable-interval 180-sec variable-interval 36-sec schedules during Conditions 1 and 3 of Experiment 1. Condition 2 arranged variable-interval 60-sec schedules for both response alternatives. The schedule assigned to the alternative that was associated with the variable-interval 36-sec schedule in Conditions 1 and 3 operated only when the subject responded on that alternative. The proportion of time spent responding on the alternative with the conventional variable-interval 60-sec schedule increased during Condition 2, but exclusive choice of that alternative did not develop. This result is inconsistent with maximization of the overall reinforcement rate and with maximization of the momentary probability of reinforcement (momentary maximizing). Increasing time proportions were also found in Experiment 2, which arranged similar conditions, except that reinforcement was provided on a variable-time basis. The time proportions were close to the momentary maximizing prediction in Experiment 2. The results of both experiments can be explained if it is assumed that time allocation is controlled by delayed reinforcement of changeovers between alternatives.     

Local preference in concurrent schedules: the effects of reinforcer sequences

We investigated the effects that sequences of reinforcers obtained from the same response key have on local preference in concurrent variable-interval schedules with pigeons as subjects. With an overall reinforcer rate of one every 27 s, on average, reinforcers were scheduled dependently, and the probability that a reinforcer would be arranged on the same alternative as the previous reinforcer was manipulated. Throughout the experiment, the overall reinforcer ratio was 1:1, but across conditions we varied the average lengths of same-key reinforcer sequences by varying this conditional probability from 0 to 1. Thus, in some conditions, reinforcer locations changed frequently, whereas in others there tended to be very long sequences of same-key reinforcers. Although there was a general tendency to stay at the just-reinforced alternative, this tendency was considerably decreased in conditions where same-key reinforcer sequences were short. Some effects of reinforcers are at least partly to be accounted for by their signaling subsequent reinforcer locations.     

Molecular contingencies in schedules of intermittent punishment

In two experiments, key pecking of pigeons was maintained by a variable-interval 180-s schedule of food presentation. Conjointly, a second schedule delivered response-dependent electric shock. In the first experiment, shocks were presented according to either a variable-interval or a nondifferential interval-percentile schedule. The variable-interval shock schedule differentially delivered shocks following long interresponse times. Although the nondifferential shock schedules delivered shocks less differentially with respect to interresponse times, the two shock schedules equally reduced the relative frequency of long interresponse times. The second experiment differentially shocked long or short interresponse times in different conditions, with resulting decreases in the relative frequency of the targeted interresponse times. These experiments highlight the importance of selecting the appropriate level of analysis for the interaction of behavior and environment. Orderly relations present at one level of analysis (e.g., interresponse times) may not be revealed at other levels of analysis (e.g., overall response rate).     

DRO contingencies: an analysis of variable-momentary schedules.

We conducted several comparative analyses to determine the relative effectiveness of variable-momentary differential-reinforcement-of-other-behavior (VM DRO) schedules. Three individuals who had been diagnosed with mental retardation participated. Results of functional analyses indicated that their self-injurious behavior (SIB) was maintained by social-positive reinforcement. Two individuals participated in a two-stage comparative analysis within multielement and multiple baseline designs. Fixed-interval (FI) and variable-interval (VI) DRO were compared in the first stage; VI DRO and VM DRO were compared in the second. All three schedules effectively reduced the participants' SIB. Treatment for the 3rd individual was conducted in a reversal design to examine the effects of VM DRO when it was implemented in isolation, and results indicated that the procedure was effective in reducing SIB. These findings suggest that VM DRO schedules may represent attractive alternatives to traditional FI schedules because momentary schedules do not require continuous monitoring and may result in higher rates of reinforcement.     

Ratio requirement and reinforcer effects in concurrent fixed-interval fixed-ratio schedules

The fixed-ratio requirement was varied in concurrent fixed-interval fixed-ratio schedules. Fixed-interval responding was reinforced by food. In different phases, fixed-ratio responding was reinforced by food or water. There was a direct relation between the ratio requirement and interval response rates when both responses were reinforced with food, but essentially no relation when the reinforcers were different. The role of reinforcers in concurrent schedules merits detailed study.     

Reinforcement and punishment effects in concurrent schedules: A test of two models

The joint effects of punishment and reinforcement on the pigeon's key-peck response were examined in three choice experiments conducted to compare predictions of Farley and Fantino's (1978) subtractive model with those made by Deluty's (1976) and Deluty and Church's (1978) model of punishment. In Experiment 1, the addition of equal punishment schedules to both alternatives of a concurrent reinforcement schedule enhanced the preference exhibited for the more frequent reinforcement alternative. Experiment 2 demonstrated decreases in the absolute response rate for each member of a concurrent reinforcement schedule when increasing frequencies of punishment were added to each alternative. Experiment 3 found that preference for the denser of two reinforcement schedules diminished when the absolute frequencies of reinforcement were increased by a constant factor and conditions of punishment for both alternatives were held constant. Diminished preferences were obtained regardless of whether the frequency of punishment associated with the denser reinforcement schedule was greater or less than that associated with the lean reinforcement alternative. The results from all three experiments uniquely supported Farley and Fantino's (1978) subtractive model of punishment and reinforcement.     

The effects of terminal-link fixed-interval and variable-interval schedules on responding under concurrent chained schedules

Previous work using variable-interval schedules in the terminal links of concurrent chained schedules suggested that relative choice proportion in the initial links equalled relative rate of reinforcement in the terminal links. With fixed-interval terminal-link schedules, however, matching was not obtained. The present study held pairs of fixed-interval terminal-link schedules in a constant ratio but varied absolute sizes. Relative choice for the smaller terminal-link fixed-interval schedule was a negatively accelerated, increasing function of absolute size of the fixed-interval pairs. Matching was found only with the fixed-interval pair of 5 and 10 sec. When pairs of variable-interval schedules were arranged so that the harmonic mean of the intervals equalled the fixed-interval parameter values, relative choice functions were like those for fixed-interval schedules.     

Performance in concurrent interval schedules

Four pigeons were trained under concurrent variable-interval variable-interval and fixed-interval variable-interval schedules in a two-key situation. Both response allocation and time allocation to the two schedules were measured when various reinforcement rates were arranged on each key. All animals showed an approximately constant proportional preference for the variable-interval schedule over the fixed-interval schedule. These results support Schneider's (1969) analysis of fixed-interval schedule performance.     

Examining the discriminative and strengthening effects of reinforcers in concurrent schedules

Reinforcers may increase operant responding via a response-strengthening mechanism whereby the probability of the preceding response increases, or via some discriminative process whereby the response more likely to provide subsequent reinforcement becomes, itself, more likely. We tested these two accounts. Six pigeons responded for food reinforcers in a two-alternative switching-key concurrent schedule. Within a session, equal numbers of reinforcers were arranged for responses to each alternative. Those reinforcers strictly alternated between the two alternatives in half the conditions, and were randomly allocated to the alternatives in half the conditions. We also varied, across conditions, the alternative that became available immediately after a reinforcer. Preference after a single reinforcer always favored the immediately available alternative, regardless of the local probability of a reinforcer on that alternative (0 or 1 in the strictly alternating conditions, .5 in the random conditions). Choice then reflected the local reinforcer probabilities, suggesting some discriminative properties of reinforcement. At a more extended level, successive same-alternative reinforcers from an alternative systematically shifted preference towards that alternative, regardless of which alternative was available immediately after a reinforcer. There was no similar shift when successive reinforcers came from alternating sources. These more temporally extended results may suggest a strengthening function of reinforcement, or an enhanced ability to respond appropriately to "win-stay" contingencies over "win-shift" contingencies.