Performance of humans in concurrent avoidance/positive-reinforcement schedules

Performance maintained under concurrent schedules consisting of a variable-interval avoidance component and a variable-interval positive-reinforcement component was studied in three human subjects using points exchangeable for money as the reinforcer. The rate of responding in the avoidance component increased, and the rate of responding in the positive-reinforcement component declined, as a function of the frequency of point-losses avoided in the avoidance component. The performance of all three subjects conformed to equations proposed by Herrnstein to describe behavior in concurrent schedules. The logarithms of the ratios of the response rates in the two components, and the logarithms of the ratios of the times spent in the two components, were linearly related to the logarithms of the ratios of the frequency of loss avoidance in the avoidance component to the frequency of reinforcement in the positive-reinforcement component. When a changeover delay of 5.0 sec was imposed, the slopes of the linear functions were close to 1.0 in the case of two subjects, whereas the third subject exhibited significant undermatching. For two subjects the changeover delay was then reduced to 2.0 sec; in both cases the slopes of the linear functions were lower than under the 5.0-sec condition. One subject participated in a third phase, in which no changeover delay was imposed; there was a further reduction in the slopes of the linear functions.     

Interval and ratio reinforcement of a complex sequential operant in pigeons

Pigeons were required to produce exactly four pecks on each of two keys in any order for reinforcement. Correct response sequences were reinforced on either fixed-interval two-minute or fixed-ratio four schedules, with each correct sequence treated as a single response. Each pigeon developed a particular dominant sequence that accounted for more than 80% of all sequences. Sequence stereotypy was relatively unaffected by the temporal properties of the fixed-interval and fixed-ratio schedules. Response time (time from the first response in each sequence to the last) was also relatively unaffected by the temporal properties of the schedules. In contrast, response latency (time from end of one sequence to the beginning of the next) was markedly affected by the schedules. Latencies were long early in the interreinforcement interval and got shorter as the interreinforcement interval progressed. These data suggest that stereotyped response sequences become functional behavioral units, resistant to disruption or alteration by reinforcement variables that ordinarily influence the temporal spacing of individual responses.     

Choice and multiple reinforcers

Pigeons chose between equivalent two-component mixed and multiple terminal-link schedules of reinforcement in the concurrent-chains procedure. The pigeons preferred the multiple schedule over the mixed when the components of the compound schedules were differentiated in terms of density of reinforcement, but the pigeons were indifferent when the components were differentiated in terms of number of reinforcers per cycle. Taken together, these results indicate that a local variable, the interval to the first reinforcer, but not a molar variable, the number of reinforcers, was sufficient to differentiate the components and thereby evoke preference.     

Stimulus discriminability in free-operant and discrete-trial detection procedures.

Six pigeons were trained to discriminate different light intensities in four experimental procedures. Experiment 1 compared stimulus discriminability in a yes-no signal-detection task with discriminability measures obtained from two free-operant procedures. Discriminability estimates were significantly lower in the detection procedure. Experiment 2 showed this lowered discriminability to be a function of the delay between stimulus presentation and the availability of the choice-response keys in the standard detection task. In addition, reinforcement sensitivity was lowest when correct choice responses were intermittently, rather than continuously, reinforced.     

Human performance on conjunctive fixed-interval fixed-ratio schedules

Eighteen young adults performed a lever-pulling task for money. Subjects were initially exposed to a fixed-interval 80-second schedule and subsequently to one of three conjunctive schedules in which the added fixed-ratio requirement was set at either 10, 80, or 120 responses. Three fixed-interval response patterns emerged: high constant rate, intermediate rate, or low rate, with most subjects displaying the last. Conjunctive performance was related to the subjects' prior fixed-interval patterns and the conjunctive ratio requirements. Low-rate subjects tended to optimize reinforcement (maximum reinforcers for minimum responses) on conjunctive schedules. Response rate was directly related to ratio requirements. Subjects' performance closely corresponded to their verbal statements of the contingencies.     

A comparison of responding maintained under second-order schedules of intramuscular cocaine injection or food presentation in squirrel monkeys.

Key pressing by squirrel monkeys was maintained under second-order schedules of either intramuscular cocaine injection or food presentation. Under one schedule, each completion of a 10-response fixed-ratio unit produced a brief visual stimulus; the first fixed-ratio unit completed after 30 minutes elapsed produced the stimulus paired with either cocaine injection or food presentation. Generally, short pauses followed by high rates of responding were maintained within the fixed-ratio units, and responding was positively accelerated over the 30-minute interval. Under another schedule, each completion of a 3-minute fixed-interval unit produced the brief stimulus; completion of the 10th fixed-interval unit produced the stimulus paired with either cocaine injection or food presentation. Generally, short pauses followed by high rates of responding were maintained within the fixed-ratio units, and responding was positively accelerated over the 30-minute interval. Under another schedule, each completion of a 3-minute fixed-interval unit produced the brief stimulus; completion of the 10th fixed-interval unit produced the stimulus paired with either cocaine injection or food presentation. Rates of responding increased within the fixed-interval units, and to a greater extent over the entire 10 fixed-interval units. Patterns of responding depended more on the schedule of reinforcement than on whether cocaine or food maintained responding. Omitting the brief stimuli following all but the last fixed-ratio or fixed-interval units decreased average rates and altered the patterns of responding. Substituting a visual stimulus that was never paired with cocaine or food following all but the last fixed-ratio or fixed-interval units decreased response rates to a lesser extent and did not substantially alter patterns of responding. When the duration of the paired stimulus was varied from .3 to 30.0 seconds, the highest response rates occurred at intermediate durations (1.0 to 10.0 seconds). The manner in which the stimulus changes affected performances depended more on the schedule of reinforcement than on whether cocaine injection or food presentation maintained responding.     

Concurrent schedule assessment of food preference in cows

Six dairy cows (Bos taurus) were trained on several pairs of concurrent variable-interval schedules with different types of food available on each alternative. The required response was a plate press made by the animal's muzzle. Performance generally replicated that found with other species. The generalized matching law accounted for the preference data, showing that food preference could be quantitatively analyzed as a special case of response bias. The preference functions showed that the response- and time-allocation ratios were not as extreme as obtained reinforcement rate ratios (undermatching).     

Reinforcement magnitude and the inhibiting effect of reinforcement

In a two-key concurrent variable-interval schedule (using pigeons), if the reinforcement frequency for one response is held constant while that for the other is increased, the rate of response on the constant key decreases. The immediate reinforcement for key pecking can usually be conceptualized as the change from a condition in which the key light is on and the food hopper light is off to one in which the key light is off and the hopper light is on. The prechange condition is associated with a delay to food of one-half the average interreinforcement interval in effect during this condition. The postchange condition is associated with a delay to food of about .5 seconds. The programming of additional reinforcement results in a decrease in the delay to food associated with the prechange stimulus condition, and thus a decrease in the value of the improvement that results from the change. This would appear to be analogous to a decrease in the amount of reinforcement, and thus sufficient explanation for the decrease in the rate of the response.     

Concurrent schedules: undermatching and control by previous experimental conditions.

Five pigeons were trained on concurrent variable-interval schedules. A series of conditions in which the ratio of reinforcement rates on two keys was progressively increased and then decreased was arranged twice. The birds were then exposed to an irregular sequence of conditions. Each condition in which reinforcement was available on both keys lasted six sessions. Performance in the first, third, and sixth sessions after a condition change was analyzed. Following a condition change, preference was biased toward the preference in the last condition, but this effect largely disappeared before the sixth session of training. The birds' preferences also appeared less sensitive to reinforcement rates in early sessions after a transition. Preference in a session was a function of both the reinforcements in that session and the reinforcements obtained in as many as four or five previous sessions. The effects of reinforcements in previous sessions could be summarized by the performance in the immediately preceding session, giving a relatively simple relation between present performance and a combination of present reinforcement and prior session performance. While such hysteresis could cause undermatching when only a small number of sessions are arranged in a condition, undermatching in a stable-state performance probably arises elsewhere.     

Elimination of reinforced behavior: intermittent schedules of not-responding

Pigeons' key pecking resulted in food according to either a variable-ratio or a variable-interval schedule. At the same time, food was available for not pecking for a specified time. The required time of not-pecking was segmented into not-responding units, and these units were followed by food according to a fixed-ratio schedule. Both unit duration and the number required were varied. In general, the shorter the time unit or the smaller the ratio, the lower was response rate. When total required not-responding time was constant, but changes in unit duration and the number required altered how the total was achieved, shorter units produced lower rates. Other conditions involved substitution of food delivered independent of responding for the not-responding schedule. With low and moderate total times to food presentation, the not-responding schedule produced lower rates; with the longest times, the response-independent schedule generated less responding. When considered in terms of relative frequency of food presentation available from a source other than pecking, the not-responding schedule reduced rate more effectively than did the response-independent schedule. Comparisons with other research suggested that food presented dependent on not responding compared favorably with punishment as a procedure for reducing response rate. Transient effects differed. Although punishment temporarily depresses rate when first imposed and temporarily enhances it when first removed, food given for not responding quickly generated steady-state rates.