Resistance to the response-decrementing effects of response-independent reinforcement produced by delay and non-delay schedules of reinforcement

In two experiments, animals were initially exposed to response-dependent schedules of food before exposure to response-independent reinforcement matched for overall rate and temporal distribution of reinforcers to the preceding condition. In Experiment I, response decrements during the response-independent phase were smaller after delayed reinforcement training than after a comparable immediate reinforcement schedule, for both doves and rats. In Experiment II variable-interval and variable-ratio schedules, both with either immediate or delayed reinforcement, were used with rats. Both the delayed reinforcement schedules produced resistance to subsequent response-independent reinforcement, but response decrements were larger after either of the immediate reinforcement conditions. It was concluded that the critical factor in response maintenance under response-independent reinforcement was the type of response-reinforcer contiguities permitted under the response-dependent schedule rather than perception of response-reinforcer “contingencies”. If the response-dependent schedule was arranged so that behaviours other than a designated operant (key pecking or lever pressing) could be contiguous with food, responding was maintained well under response-independent schedules.     

Concurrent schedules of reinforcement: effects of gradual and abrupt increases in changeover delay

Pigeons' pecks were maintained on concurrent variable-interval 1-min variable-interval 3-min schedules of reinforcement, with a changeover delay of 2 sec. When changeover delay was increased successively to 5.0, 7.5, and 12.5 sec (Exp. I) the actual relative rate of reinforcement for the variable-interval 3-min key decreased progressively for two birds, abruptly for two other birds, and the subjects devoted proportionately less of their time and responding to that key. However, the relative performance measures (relative time and relative responding) approximated the actual relative rate of reinforcement, with a maximum discrepancy of 11%, over all changeover delay values investigated. Experiment II attempted to lengthen response-run durations on the variable-interval 3-min key so that they were long enough to meet the changeover delay requirement at each new changeover delay value, by progressively increasing the changeover delay by 0.5-sec increments. With this procedure the actual relative rate of reinforcement approximated more closely the scheduled relative rate as changeover delay increased. As in Exp. I, relative performance measures approximated the actual relative reinforcement rate (maximum discrepancy 17%).     

Response rate and changeover performance on concurrent variable-interval schedules

Six pigeons were exposed to variable-interval schedules arranged on one, two, three, and four response keys. The reinforcement rate was also varied across conditions. Numbers of responses, the time spent responding, the number of reinforcements, and the number of changeovers between keys were recorded. Response rates on each key were an increasing function of reinforcement rate on that key and a decreasing function of the reinforcement rate on other keys. Response and time-allocation ratios under-matched ratios of obtained reinforcements. Three sets of equations were developed to express changeover rate as a function of response rate, time allocation, and reinforcement rate respectively. These functions were then applied to a broad range of experiments in the literature in order to test their generality. Further expressions were developed to account for changeover rates reported in experiments where changeover delays were varied.     

Concurrent schedules: Effects of reinforcement rate and changeover delay on time allocation in a three-alternative procedure

Pigeons were exposed to a continuous choice procedure where three alternatives alternated in a fixed, recycling order (ABCABC, etc.). Responses were reinforced according to independent variable-interval schedules. For three birds, the reinforcement rate for responses on alternative C was varied. For three other birds, the duration of the changeover delay after the changeover to C was varied. For both groups, the reinforcement rates and changeover delay durations associated with A and B were constant throughout the experiment. The time proportion at A relative to B increased as a function of the reinforcement rate for responses on C and decreased as a function of the duration of the changeover delay during C. The results show that the proportion of time spent at a variable-interval alternative of a continuous choice procedure is not completely determined by the reinforcement rates provided by the alternatives. The results support the assumption that time allocation is governed by delayed reinforcement of changeover behaviour.     

Conjunctive schedules of response-dependent and response-independent reinforcement

Pigeons received food when they emitted the number of responses specified by a fixed-ratio schedule, and the time specified by a fixed-time schedule had elapsed. The order of meeting the response and time requirements was irrelevant. In different conditions, stimuli signalled completion of one, both, or neither requirement. Ratio size interacted with stimulus condition to determine performance. When a stimulus signalled the end of the fixed-time period, under all ratios the birds tended to respond after the stimulus appeared. When stimuli followed both components, small ratios produced responding during the fixed-time period, and other ratios resulted in responses after the time period had elapsed. With either no stimulus changes, or with a stimulus correlated with completion of the ratio alone, responding first increased and then decreased as the ratio increased. Low and high ratios produced stable response frequencies and patterns in successive intervals. Intermediate ratios resulted in two types of performance. Intervals with long initial pauses and few responses during the fixed-time period were followed by intervals with short pauses and numerous responses and vice versa. The source of these dynamic effects was hypothesized to be number of responses per reinforcer in one condition and response-reinforcer contiguity in the other.     

Fixed and variable schedules of response-independent reinforcement

After response-dependent reinforcement established key-pecking as the predominant response, pigeons received schedules in which reinforcements occurred without reference to responding. These response-independent schedules involved either a reinforcement every 5 min, or reinforcements at irregular intervals that averaged 5 min. The response-independent schedules generated characteristic patterns of responding. The fixed schedule produced positively accelerated responding between reinforcements, and the variable schedule produced either steady rates, erratic, or negatively accelerated patterns. The patterns developed independent of the distribution of responses existing when the schedule was first imposed. The rate of responding varied for the three birds, but, for all, response-independent schedules decreased the rates below the level maintained by response-dependent reinforcement. Although the rate of responding was affected primarily by the events contiguous with reinforcement, the pattern of responding appeared to be determined mainly by the presentation of reinforcements in relation to time.     

Changeover delay and concurrent schedules: some effects on relative performance measures

The pigeon and the rat partition total response output between both schedules of a concurrent variable-interval pair. The quantitative nature of a partition seems critically dependent on the relative rates with which the two schedules provide reinforcements for responding, in addition to the changeover delay. The manner in which the changeover delay controls the partition was studied by varying the duration of the changeover delay from 0 to 20 sec with each of two pairs of concurrent variable-interval schedules, viz., Conc VI 1.5-min VI 1.5-min and Conc VI 1-min VI 3-min. Rats served as the subjects and brain stimulation was employed as the reinforcer. When the schedules were Conc VI 1.5-min VI 1.5-min, relative response rate approximated 0.50 at all values of the changeover delay. When the schedules were Conc VI 1-min VI 3-min, relative response rate, computed with respect to the VI 1-min schedule, increased when the duration of the changeover delay increased. Changeover rate decreased when the duration of the changeover delay increased. The decrease was the same for both VI schedules of the Conc VI 1.5-min VI 1.5-min pair but was more rapid for the VI 3-min schedule of the Conc VI 1-min VI 3-min pair.     

Choice, rate of reinforcement, and the changeover delay

Pigeons distribute their responses on concurrently available variable-interval schedules in the same proportion as reinforcements are distributed on the two schedules only when a changeover delay is used. The present study shows that this equality between proportions of responses and proportions of reinforcements (“matching”) is obtained when the value of the changeover delay is varied. When responses are partitioned into the set of rapid response bursts occurring during the delay interval and the set of responses occurring subsequently, the proportion of neither set of responses matches the proportion of reinforcements. Instead, each set deviates from matching but in opposite directions. Matching on the gross level results from the interaction of two patterns evident in the local response rates: (I) the lengthening of the changeover delay response burst is accompanied by a commensurate decrease in the number of changeovers; (2) the changeover delay response burst is longer than the scheduled delay duration. When delay responses are eliminated by introducing a blackout during the delay interval, response matching is eliminated; the pigeon, however, continues to match the proportion of time spent responding on a key to the proportion of reinforcements obtained on that key.     

Response-rate invariance in concurrent schedules: effects of different changeover contingencies

In a two-key chamber, one key (the food key) was either red or green with different variable-interval schedules operating concurrently in each color and a second key (the changeover key) served to change the food-key color. Three pigeons were trained with either a 2-sec changeover delay or a 0-sec changeover delay and three birds with a fixed-ratio 2 on the changeover key instead of a changeover delay. The proportion of time spent in red approximated the proportion of reinforcers delivered in red for all birds. When the procedure was changed so that reinforcers were signalled in the green schedule, rates of reinforcement were unaltered, but the pigeons spent virtually the whole session in red. Changeovers to green were allowed only when a reinforcer was assigned by the schedule associated with green. For all pigeons with the fixed–ratio requirement on the changeover key or with a 0-sec changeover delay, the overall rate of red-key responses was higher during the signalling condition than during unsignalled, or baseline, condition. The present data question the generality of previous reports that the rate of one response is independent of the amount of time allocated to the alternative response.     

Patterning with fixed-time schedules of response-independent reinforcement

Pigeons were first exposed to a schedule providing food when the time between successive key pecks (the interresponse time) exceeded a specified duration. When food then was presented at regular intervals independent of responding (fixed-time schedule), responses typically occurred at a steady rate in the periods between successive food presentations. Once the birds had been exposed to a fixed-ratio schedule, however, response rate under fixed-time schedules was positively accelerated. Variations in the sequence of conditions given different subjects indicated that the changes in patterning were due to the fixed-ratio schedule, rather than to the number of transitions from a response-dependent to the response-independent fixed-time schedule, to changed parameter values, or to prolonged experience with the fixed-time schedule. The effects of fixed-time schedules on patterning depended upon experimental history.     

Preference and discrimination between response-dependent and response-independent schedules of reinforcement

Four Asian quail (Coturnix coturnix japonica) were exposed to concurrent-chain schedules, the terminal links of which were either variable-interval 30 sec and variable-time 30 sec, or fixed-interval 30 sec and fixed-time 30 sec. Except for one bird that exhibited a preference for the variable-interval schedule over the variable-time schedule, no consistent preferences were demonstrated for response-dependent or response-independent schedules. However, response rates were three times greater on response-dependent than on response-independent schedules. The discrimination between terminal-link schedules was rapidly recovered after the schedule positions were reversed. Casual observations revealed that the birds engaged in stereotypic circling and pecking while the response-independent schedules were operative.     

Arousal, changeover responses, and preference in concurrent schedules

Pigeons were trained on multiple schedules that provided concurrent reinforcement in each of two components. In Experiment 1, one component consisted of a variable-interval (VI) 40-s schedule presented with a VI 20-s schedule, and the other a VI 40-s schedule presented with a VI 80-s schedule. After extended training, probe tests measured preference between the stimuli associated with the two 40-s schedules. Probe tests replicated the results of Belke (1992) that showed preference for the 40-s schedule that had been paired with the 80-s schedule. In a second condition, the overall reinforcer rate provided by the two components was equated by adding a signaled VI schedule to the component with the lower reinforcer rate. Probe results were unchanged. In Experiment 2, pigeons were trained on alternating concurrent VI 30-s VI 60-s schedules. One schedule provided 2-s access to food and the other provided 6-s access. The larger reinforcer magnitude produced higher response rates and was preferred on probe trials. Rate of changeover responding, however, did not differ as a function of reinforcer magnitude. The present results demonstrate that preference on probe trials is not a simple reflection of the pattern of changeover behavior established during training.     

Choice: Effects of changeover schedules on concurrent performance

The components of concurrent schedules were separated temporally by placing interval schedules on the changeover key. The rates of responding on both the main and changeover keys were examined as a function of the reinforcement rates. In the first experiment, the sensitivity of main-key performance to changing reinforcement rates was inversely related to the temporal separation of components, and changeover performance was monotonically related to the ratio of the reinforcement rates. In the second experiment, when the ratio of the reinforcement rates was scheduled to remain constant while the frequency of reinforcement was varied, changeover performance did not remain constant. A “sampling” interpretation of changeover responding was proposed and subsequently tested in a third experiment where extinction was always scheduled in one component and the frequency of reinforcement was varied in the second component. It was concluded that changeover performance can be interpreted using molar measures of reinforcement and that animals sample activities available to them at rates which are controlled by relative reinforcement rates.     

Some effects of response-independent reinforcement on auditory generalization gradients.

Two groups of six rats received discrimination training with two auditory stimuli differing in intensity. During one stimulus, the schedule was variable interval; during the other, it was either variable time or extinction. Both the variable time and extinction schedules resulted in differential rates of responding in the presence of the two stimuli. Extinction resulted in an earlier and more stable difference. Stimulus generalization gradients obtained along the noise-intensity dimension revealed peak shift with both procedures. In addition, a secondary peak to stimuli in between the two training stimuli occurred with the variable-time schedule.     

Effects of concurrent response-independent reinforcement on fixed-interval schedule performance

In three experiments, behavior maintained by fixed-interval schedules changed when response-independent reinforcement was delivered concurrently according to fixed- or variable-time schedules. In Experiment I, a pattern of positively accelerated responding during fixed interval was changed to a linear pattern when response-independent reinforcement occurred under a variable-time schedule. Overall response rates (total responses/total time) decreased as the frequency of response-independent reinforcement increased. Experiment II showed that the response-rate changes in the first experiment were controlled by the response-reinforcer relation, but the changes in patterns of responding were similar whether concurrently available reinforcement at varying times was response-dependent or response-independent. In the final experiment, the addition of response-independent reinforcement at fixed times to a fixed-interval schedule resulted in changes in both local and overall response rates and in the occurrence of positively accelerated responding between reinforcements. These results suggest that the temporal distribution of reinforcers determines response patterns and that both the response-reinforcement dependency and the schedule of reinforcement determine overall response rates during concurrently scheduled response-dependent and response-independent reinforcement.     

Variable-time reinforcement in multiple and concurrent schedules

Experiment I examined the role of a reduced rate of responding in the occurrence of behavioral contrast. Four rats and a pigeon were exposed to a two-component multiple schedule in which one component was always a variable-interval schedule. The second component was, at different times, either a variable-time schedule in which food was delivered independently of responding, or extinction. Both extinction and the variable-time schedule reduced the rate of responding in the second component. Behavioral contrast was observed, however, only when extinction was scheduled in the second component. Experiment II examined preference, as measured by time allocation in concurrent schedules for a variable-interval schedule relative to a variable-time schedule. Two rats displayed a lack of preference between the two schedules. The results of these experiments support a preference interpretation of behavioral contrast, which holds that behavioral contrast is the result of the introduction of a less-preferred condition in one component of a multiple schedule.     

Concurrent schedules: Maximization versus reinforcement of changeover behaviour

Pigeons responded on concurrent variable-interval 180-sec variable-interval 36-sec schedules during Conditions 1 and 3 of Experiment 1. Condition 2 arranged variable-interval 60-sec schedules for both response alternatives. The schedule assigned to the alternative that was associated with the variable-interval 36-sec schedule in Conditions 1 and 3 operated only when the subject responded on that alternative. The proportion of time spent responding on the alternative with the conventional variable-interval 60-sec schedule increased during Condition 2, but exclusive choice of that alternative did not develop. This result is inconsistent with maximization of the overall reinforcement rate and with maximization of the momentary probability of reinforcement (momentary maximizing). Increasing time proportions were also found in Experiment 2, which arranged similar conditions, except that reinforcement was provided on a variable-time basis. The time proportions were close to the momentary maximizing prediction in Experiment 2. The results of both experiments can be explained if it is assumed that time allocation is controlled by delayed reinforcement of changeovers between alternatives.     

Signalled reinforcement in multiple and concurrent schedules

Five pigeons were exposed to multiple and concurrent variable-interval, variable-interval reinforcement schedules in which reinforcement availability in one component was never signalled. During certain phases of the experiment, reinforcement availability in the other component was signalled. Behavioral contrast was observed in seven of eight instances when reinforcement availability in the multiple schedules was signalled. Under the concurrent schedules in which reinforcement availability was signalled, the subjects did not always allocate more time to (prefer) the component containing non-signalled reinforcement, as would be predicted by an account of behavioral contrast holding that contrast results from the introduction of a less-preferred condition in one component of a multiple schedule.     

Relative rate of response and relative magnitude of reinforcement in multiple schedules

Pigeons were trained on a multiple schedule in which the duration of access to grain reinforcement was varied independently in the two components. The relative response rate in one component was an increasing function of the relative duration of reinforcement in that component. The similarity of this interaction to that found in multiple schedules of different reinforcement frequency is discussed. Extinction data were also similar to those obtained after training on multiple schedules of different reinforcement frequency.