Chained schedules of avoidance: Reinforcement within and by avoidance situations

Four rats were exposed to chained schedules with variable-cycle avoidance in both links. Responding in the initial link cancelled shocks scheduled once per minute and, according to a conjoint fixed-ratio schedule, produced a terminal link where scheduled shock rates varied from 0 to 8 shocks per minute in different conditions of the experiment. Response rates in the terminal link increased as a function of the scheduled shock rate. Response rates in the initial link, on the other hand, decreased as a function of the shock rate actually received (rather than scheduled) in the terminal link. While consistent with other studies of aversive control, these results differ from those obtained in chained schedules of positive reinforcement in that increases in reinforcement within the terminal link of the chain did not systematically increase the reinforcing value of that link.     

Concurrent random-interval schedules and the matching law

In Experiment I, a group of eight pigeons performed on concurrent random-interval schedules constructed by holding probability equal and varying cycle time to produce ratios of reinforcer densities of 1:1, 3:1, and 5:1 for key pecking. Schedules for a second group of seven were constructed with equal cycle times and unequal probabilities. Both groups deviated from simple matching, but the two forms of the schedules appeared to produce no consistent patterns of deviation. The data were found to be consistent with those obtained in concurrent variable-interval situations. The parameters of the matching equation in the form of Y=k X^a were estimated; the value of k was unity and a was 0.84. In Experiment II, six pigeons were exposed to two conc RI RI schedules in which one component increasingly approximated an FI schedule. The value of k was not 1.0. Concurrent RI RI schedules were shown to represent a continuum from conc FI VI to conc VI VI schedules. The use of the exponential equation in testing “matching laws” suggests that a<1 will continue to be observed, and this will set limits on the form of new laws and the assumed or rational values of the component variables in these laws.     

Behavioral contrast during multiple avoidance schedules

Changes were observed in the rate of avoidance responding in both components of a multiple schedule of Sidman-avoidance after the shock frequency was changed in only one component. The rate change in each component was positively correlated, in direction and magnitude, with the change in the relative rate of reinforcement (percentage of total shocks) in that component.     

Relative rate of response and relative magnitude of reinforcement in multiple schedules

Pigeons were trained on a multiple schedule in which the duration of access to grain reinforcement was varied independently in the two components. The relative response rate in one component was an increasing function of the relative duration of reinforcement in that component. The similarity of this interaction to that found in multiple schedules of different reinforcement frequency is discussed. Extinction data were also similar to those obtained after training on multiple schedules of different reinforcement frequency.     

Reinforcement probability and ordinal position of response in fixed-interval schedules

Four rats pressed levers and received food pellets under fixed-interval reinforcement schedules of 20, 60, and 180 seconds. The number of responses in each interval was recorded. From these data, the probability of reinforcement was determined as a function of response count. These functions were generally increasing. This finding is consistent with previous suggestions that increasing response rates within fixed intervals may be a function of response count in addition to or instead of elapsed or remaining time.     

Component duration and relative response rates in multiple schedules

Pigeons were trained on a multiple variable-interval 30-sec, variable-interval 90-sec schedule with each component presented alternately for an equal (on the average) duration. This average duration of exposure to each component was varied from 5 to 300 sec. The main concern was with rate of response in the variable-interval 30-sec component relative to rate of response in the variable-interval 90-sec component. In all cases, rate of response was higher in the variable-interval 30 sec component, but the discrepancy in the rate produced by the two schedules tended to be greatest when the duration of component presentation was brief. The mean proportion of responses emitted during the variable-interval 30-sec component (responses in variable-interval 30-sec component divided by total responses) varied from about 0.60 to 0.71, where 0.75 would be expected on the basis of a matching rule, and 0.59 was that obtained by Lander and Irwin (1968). These results are in agreement with data reported by Shimp and Wheatley (1971) from a similar experiment.     

Contrast and autoshaping in multiple schedules varying reinforcer rate and duration

Thirteen master pigeons were exposed to multiple schedules in which reinforcement frequency (Experiment I) or duration (Experiment II) was varied. In Phases 1 and 3 of Experiment I, the values of the first and second components' random-interval schedules were 33 and 99 seconds, respectively. In Phase 2, these values were 99 seconds for both components. In Experiment II, a random-interval 33-second schedule was associated with each component. During Phases 1 and 3, the first and second components had hopper durations of 7.5 and 2.5 seconds respectively. During Phase 2, both components' hopper durations were 2.5 seconds. In each experiment, positive contrast obtained for about half the master subjects. The rest showed a rate increase in both components (positive induction). Each master subject's key colors and reinforcers were synchronously presented on a response-independent basis to a yoked control. Richer component key-pecking occurred during each experiment's Phases 1 and 3 among half these subjects. However, none responded during the contrast condition (unchanged component of each experiment's Phase 2). From this it is inferred that autoshaping did not contribute to the contrast and induction findings among master birds. Little evidence of local contrast (highest rate at beginning of richer component) was found in any subject. These data show that (a) contrast can occur independently from autoshaping, (b) contrast assays during equal-valued components may produce induction, (c) local contrast in multiple schedules often does not occur, and (d) differential hopper durations can produce autoshaping and contrast.     

Some effects of response independent reinforcers in multiple schedules

In Exp. I, rats' lever presses were conditioned on multiple variable-interval variable-interval schedules. Changing one of the multiple schedule components to variable time reduced responding in that component. Further reductions in responding occurred in both components when the schedule was changed to multiple variable-time variable-time. After reinstating the multiple variable-interval variable-time schedule, lower response rates were maintained in the variable-time component during a series of stimulus reversals. In Exp. II, replacement of extinction components of multiple variable-interval extinction or multiple extinction extinction with variable-time schedules for single sessions (probe) resulted in response rate increments in those components. In the former schedule these increases were concomitant with response decreases during the variable-interval components. Response increases in the variable-time probes were related to conditioning history and, as a result, to response probability at the time of the probe.     

Matching in concurrent variable-interval avoidance schedules

After pretraining with multiple variable-interval avoidance schedules, two rats were exposed to a series of concurrent variable-interval avoidance schedules. Responses on two levers cancelled delivery of electric shocks arranged according to two independent variable-interval schedules. The ratio of responses and time spent on the two levers approximately matched the ratio of shocks avoided on each. Matching to the number of shocks received was not obtained. Concurrent variable-interval avoidance can therefore be added to the group of positive and negative reinforcement schedules that can be expressed in the quantitative framework of the matching law.     

Positive interaction (induction) in multiple variable-interval, differential-reinforcement-of-high-rate schedules

The effect of increases in the rate of responding in one component of a multiple schedule upon the rate of responding in a second component was investigated. Pigeons were exposed to a multiple schedule where both components were initially variable-interval schedules having the same parameter value. After rate of key pecking stabilized, one component was changed to a schedule that differentially reinforced high rates of responding. Rate of reinforcement in this varied component was adjusted to remain equal to rate of reinforcement in the constant (variable-interval) component. Four of five pigeons showed a maintained increase in rate of responding during both the constant and varied components, even though rates of reinforcement did not change.     

The effect of response force on avoidance rate

In a Sidman-avoidance schedule of counter losses for two human subjects, the loss-to-loss and response-to-loss intervals were 20 sec. The avoidance response was a vocal response that was louder than a minimum vocal requirement. This requirement was set at 80 db, 95 db, or 110 db. In addition to vocal responses meeting the minimum requirement, all responses exceeding a threshold of 75 db or louder were recorded. The rate of both above-threshold and avoidance responses decreased as the response-force requirement increased. Thus, high response-force requirements produced an effect on avoidance responding similar to its effect on positively reinforced responding.     

The effects of different component response requirements in multiple and concurrent schedules

Six pigeons were trained on multiple and concurrent schedules. The reinforcement rates were varied systematically (a) when lever pressing was required in one component and key pecking in the successive component; (b) when lever pressing was required in both multiple components; (c) when key pecking was required in both multiple components; and (d) when key pecking was required on one schedule and lever pressing was required on the concurrently-available schedule. Only the absolute level of responding was changed by different response requirements. Analyzed by the generalized matching law, performance under different response requirements resulted in a bias toward key pecking, and the measured response bias was the same in multiple and concurrent schedule arrangements. The bias in time measures obtained from concurrent schedule performance was reliably smaller than the obtained response biases. The sensitivity to reinforcement-rate changes was ordered: concurrent key-lever; multiple key-key; multiple lever-key; and, the least sensitive, multiple lever-lever. The results confirm that requirements of different topographical responses can be handled by the generalized matching law mainly in the bias parameter, but problems for this type of analysis may be caused by the changing sensitivity to reinforcement in multiple schedule performance as response requirements are changed.     

Effects of variations in local reinforcement rate on local response rate in variable interval schedules

Rats trained to lever press for sucrose were exposed to variable-interval schedules in which (i) the probability of reinforcement in each unit of time was a constant, (ii) the probability was high in the first ten seconds after reinforcement and low thereafter, (iii) the probability was low for ten seconds and high thereafter, (iv) the probability increased with time since reinforcement, or (v) the probability was initially zero and then increased with time since reinforcement. All schedules generated similar overall reinforcement rates. A peak in local response rate occurred several seconds after reinforcement under those schedules where reinforcement rate at this time was moderate or high ([i], [ii], and [iv]). Later in the inter-reinforcement interval, local response rate was roughly constant under those schedules with a constant local reinforcement rate ([i], [ii], and [iii]), but increased steadily when local reinforcement rate increased with time since reinforcement ([iv] and [v]). Postreinforcement pauses occurred on all schedules, but were much longer when local reinforcement rate was very low in the ten seconds after reinforcement ([iii]). The interresponse time distribution was highly correlated with the distribution of reinforced interresponse times, and the distribution of postreinforcement pauses was highly correlated with the distribution of reinforced postreinforcement pauses on some schedules. However, there was no direct evidence that these correlations resulted from selective reinforcement of classes of interresponse times and pauses.     

Reinforcement magnitude and pausing on progressive-ratio schedules

Rats responded under progressive-ratio schedules for sweetened milk reinforcers; each session ended when responding ceased for 10 min. Experiment 1 varied the concentration of milk and the duration of postreinforcement timeouts. Postreinforcement pausing increased as a positively accelerated function of the size of the ratio, and the rate of increase was reduced as a function of concentration and by timeouts of 10 s or longer. Experiment 2 varied reinforcement magnitude within sessions (number of dipper operations per reinforcer) in conjunction with stimuli correlated with the upcoming magnitude. In the absence of discriminative stimuli, pausing was longer following a large reinforcer than following a small one. Pauses were reduced by a stimulus signaling a large upcoming reinforcer, particularly at the highest ratios, and the animals tended to quit responding when the past reinforcer was large and the stimulus signaled that the next one would be small. Results of both experiments revealed parallels between responding under progressive-ratio schedules and other schedules containing ratio contingencies. Relationships between pausing and magnitude suggest that ratio pausing is under the joint control of inhibitory properties of the past reinforcer and excitatory properties of stimuli correlated with the upcoming reinforcer, rather than under the exclusive control of either factor alone.     

Performance in multiple fixed-interval schedules

The performances of five pigeons were studied under a variety of multiple fixed-interval schedules in which both component duration and reinforcement rate were varied. The three series of experimental conditions were: (a) when the ratio of component durations equalled the reciprocal of the ratio of component reinforcement rates; (b) when the component durations were equal; and (c) when the ratio of component durations equalled the ratio of component reinforcement rates. Relative response rates were related to relative reinforcement rates in the same manner as in multiple variable-interval schedules, but no effect of component duration was found.     

Response strength in multiple schedules

In several different experiments, pigeons were trained with one schedule or condition of food reinforcement for pecking in the presence of one key color, and a different schedule or condition in the presence of a second key color. After responding in both of these multiple schedule components stabilized, response-independent food was presented during dark-key periods between components, and the rates of pecking in both schedule components decreased. The decrease in responding relative to baseline depended on the frequency, magnitude, delay, or response-rate contingencies of reinforcement prevailing in that component. When reinforcement was terminated, decreases in responding relative to baseline rates were ordered in the same way as with response-independent food. The relations between component response rates were power functions. Internal consistencies in the data, in conjunction with parallel findings in the literature, suggest that the concept of response strength summarizes the effects of diverse procedures, where response strength is identified with relative resistance to change. The exponent of the power function relating response rates may provide the basis for scaling response strength.     

Signalled reinforcement and multiple schedules

The responses of four pigeons were first reinforced in the presence of two different wave-lengths (green and red) on a two-ply multiple schedule with identical variable-interval 3-min schedules of reinforcement associated with each component. While the constant-component reinforcement schedule remained unchanged during the experiment, the schedule associated with the variable component was changed to (1) signalled variable time, (2) unsignalled variable time, or (3) signalled variable interval. The probability with which the availability of the reinforcer was signalled in the variable-interval schedules was either 0.5 or 1.0. Positive contrast occurred in both signalled variable-interval and variable-time schedules, but only when the availability of all the variable-component reinforcers was signalled. Signalling the availability of only 50% of the reinforcers in signalled variable-interval schedules resulted in negative induction. The present data suggest that positive behavioral contrast resulting from signalled reinforcer availability is due to the presence of an extinction-correlated stimulus.     

Rats' performance on variable-interval schedules with a linear feedback loop between response rate and reinforcement rate

Three experiments investigated whether rats are sensitive to the molar properties of a variable-interval (VI) schedule with a positive relation between response rate and reinforcement rate (i.e., a VI+ schedule). In Experiment 1, rats responded faster on a variable ratio (VR) schedule than on a VI+ schedule with an equivalent feedback function. Reinforced interresponse times (IRTs) were shorter on the VR as compared to the VI+ schedule. In Experiments 2 and 3, there was no systematic difference in response rates maintained by a VI+ schedule and a VI schedule yoked in terms of reinforcement rate. This was found both when the yoking procedure was between-subject (Experiment 2) and within-subject (Experiment 3). Mean reinforced IRTs were similar on both the VI+ and yoked VI schedules, but these values were more variable on the VI+ schedule. These results provided no evidence that rats are sensitive to the feedback function relating response rate to reinforcement rate on a VI+ schedule.