Conditioned reinforcement and discrimination in second-order schedules

Pigeons were exposed to multiple second-order schedules in which responding on the “main key” was reinforced according to either a variable-interval or fixed-interval schedule by production of a brief stimulus on the “brief-stimulus key”. A response was required to the brief stimulus during its fourth (final) presentation to produce food; responses to the earlier brief stimuli indicated the extent to which the final brief stimulus was discriminated from preceding ones. Main-key response rates were higher in early components of paired brief-stimulus schedules, in which each brief stimulus was the same as that paired with reinforcement, than in comparable unpaired brief-stimulus or tandem schedules. Poor discrimination occurred between paired brief stimuli (Experiment I). When chain stimuli on the main key induced a discrimination between the first two and second two brief stimuli, the response-rate enhancement in the paired brief-stimulus schedule persisted (Experiment II). Rate enhancement diminished when the initial link of the chain included the first three components (Experiment IV). Eliminating the contingency between responding and brief-stimulus production also diminished rate enhancement (Experiment III). The results show that the discriminative and conditioned reinforcing effects of food-paired brief stimuli may be selectively manipulated and suggest that the reinforcing effects are modulated by other reinforcers in the situation.     

Preference and discrimination between response-dependent and response-independent schedules of reinforcement

Four Asian quail (Coturnix coturnix japonica) were exposed to concurrent-chain schedules, the terminal links of which were either variable-interval 30 sec and variable-time 30 sec, or fixed-interval 30 sec and fixed-time 30 sec. Except for one bird that exhibited a preference for the variable-interval schedule over the variable-time schedule, no consistent preferences were demonstrated for response-dependent or response-independent schedules. However, response rates were three times greater on response-dependent than on response-independent schedules. The discrimination between terminal-link schedules was rapidly recovered after the schedule positions were reversed. Casual observations revealed that the birds engaged in stereotypic circling and pecking while the response-independent schedules were operative.     

Effects of ratio reinforcement schedules on discrimination performance by Japanese monkeys

In Experiment 1, Japanese monkeys were trained on three conditional position-discrimination problems with colors as the conditional cues. Within each session, each problem was presented for two blocks of ten reinforcements; correct responses were reinforced under continuous-reinforcement, fixed-ratio 5, and variable-ratio 5 schedules, each assigned to one of the three problems. The assignment of schedules to problems was rotated a total of three times (15 sessions per assignment) after 30 sessions of acquisition training. Accuracy of discrimination increased to a moderate level with fewer trials under CRF than under ratio schedules. In contrast, the two ratio schedules, fixed and variable, were more effective in maintaining accurate discrimination than was CRF. With further training, as asymptotes were reached, accuracy was less affected by the schedule differences. These results demonstrated an interaction between the effects of reinforcement schedules and the level of acquisition. In Experiment 2, ratio sizes were gradually increased to 30. Discrimination accuracy was maintained until the ratio reached 20; ratio 30 strained the performance. Under FR conditions, accuracy increased as correct choice responses cumulated after reinforcement.     

Color alternation learning in the pigeon under fixed-ratio schedules of reinforcement

Pigeons were trained on a non-spatial delayed alternation task in which the correct stimulus was that color not responded to on the preceding trial. Subjects required to emit either 15 or 30 pecks to the correct stimulus within a trial learned the task, those required to emit only one or five pecks did not. Also, alternation was learned more easily after an incorrect than after a correct trial. Later experiments showed that a minimum fixed-ratio value was required for successful color alternation to occur, even though no fixed-ratio requirement was necessary when a position cue was available. The mechanism of the fixed-ratio effects derived from the pigeons' tendency to repeat their response in the presence of the color reinforced on the last trial. Whereas subjects trained on larger fixed-ratios corrected this error tendency within a trial, subjects trained on smaller fixed ratios did not.     

A mathematical model of learning under schedules of interresponse time reinforcement

A model is proposed for free responding under schedules of interresponse time reinforcement. Each response generates a pattern of stimuli that changes in an orderly way over time. At any time the changing pattern may be sampled, and the state of conditioning of the sampled pattern determines whether or not a response actually occurs. The prediction of this model for the asymptotic distribution of interresponse times is derived. The predictions are tested against data from ratio, interval, and DRL schedules of reinforcement. The fit of the model is sufficiently good to make it worth-while continuing investigation of the model.     

Second-order schedules: discrimination of components

Pigeons were exposed to a series of second-order schedules in which the completion of a fixed number of fixed-interval components produced food. In Experiment 1, brief (2 sec) stimulus presentations occurred as each fixed-interval component was completed. During the brief-stimulus presentation terminating the last fixed-interval component, a response was required on a second key, the brief-stimulus key, to produce food. Responses on the brief-stimulus key before the last brief-stimulus presentation had no scheduled consequences, but served as a measure of the extent to which the final component was discriminated from preceding components. Whether there were one, two, four, or eight fixed-interval components, responses on the brief-stimulus key occurred during virtually every brief-stimulus presentation. In Experiment 2, an attempt was made to punish unnecessary responses on the brief-stimulus key, i.e., responses on the brief-stimulus key that occurred before the last component. None of the pigeons learned to withhold these responses, even though they produced a 15-sec timeout and loss of primary reinforcement. In Experiment 3, different key colors were associated with each component of a second-order schedule (a chain schedule). In contrast to Experiment 1, brief-stimulus key responses were confined to the last component. It was concluded that pigeons do not discriminate well between components of second-order schedules unless a unique exteroceptive cue is provided for each component. The relative discriminability of the components may account for the observed differences in initial-component response rates between comparable brief-stimulus, tandem, and chain schedules.     

Sequence schedules of reinforcement

The performance of pigeons was studied under a second-order schedule composed of fixed-interval components, each of which was associated with a different discriminative stimulus, the stimuli occurring in a fixed order. In one condition, food presentation followed the completion of the fourth component. This was designated a fixed-ratio sequence schedule. In another condition, responses in the first component completed after a fixed time were reinforced. This was designated a fixed-interval sequence schedule. Although the stimulus order and maximum reinforcement frequency were identical under the two schedules, considerably more responding occurred under the fixed-interval sequence schedule in all components. Relatively few food presentations occurred after responding during any but the terminal components of the fixed-interval sequence schedule, a feature independent of the parameter values investigated. In addition, while a pattern of increased responding between food presentations prevailed under both schedules, under the fixed-interval sequence schedule the rate in the terminal component was frequently less than in the penultimate component. The fixed-interval sequence schedule appeared to have several properties of simple fixed-interval schedules.     

Patterning with fixed-time schedules of response-independent reinforcement

Pigeons were first exposed to a schedule providing food when the time between successive key pecks (the interresponse time) exceeded a specified duration. When food then was presented at regular intervals independent of responding (fixed-time schedule), responses typically occurred at a steady rate in the periods between successive food presentations. Once the birds had been exposed to a fixed-ratio schedule, however, response rate under fixed-time schedules was positively accelerated. Variations in the sequence of conditions given different subjects indicated that the changes in patterning were due to the fixed-ratio schedule, rather than to the number of transitions from a response-dependent to the response-independent fixed-time schedule, to changed parameter values, or to prolonged experience with the fixed-time schedule. The effects of fixed-time schedules on patterning depended upon experimental history.     

Interlocking schedules of reinforcement

Four male pigmented rats were exposed to a procedure designed to investigate the relation between several performance measures and a schedule continuum ranging from FR 36 to FI 2 through four intermediate interlocking schedules. On all schedules, each subject developed a stable performance that was generally break-and-run. Analysis of the cumulative records, post-S^R breaks, and running rates showed a continuum of performance related to the schedule continuum.     

Second-order schedules of token reinforcement with pigeons: effects of fixed- and variable-ratio exchange schedules

Pigeons' key pecks produced food under second-order schedules of token reinforcement, with light-emitting diodes serving as token reinforcers. In Experiment 1, tokens were earned according to a fixed-ratio 50 schedule and were exchanged for food according to either fixed-ratio or variable-ratio exchange schedules, with schedule type varied across conditions. In Experiment 2, schedule type was varied within sessions using a multiple schedule. In one component, tokens were earned according to a fixed-ratio 50 schedule and exchanged according to a variable-ratio schedule. In the other component, tokens were earned according to a variable-ratio 50 schedule and exchanged according to a fixed-ratio schedule. In both experiments, the number of responses per exchange was varied parametrically across conditions, ranging from 50 to 400 responses. Response rates decreased systematically with increases in the fixed-ratio exchange schedules, but were much less affected by changes in the variable-ratio exchange schedules. Response rates were consistently higher under variable-ratio exchange schedules than tinder comparable fixed-ratio exchange schedules, especially at higher exchange ratios. These response-rate differences were due both to greater pre-ratio pausing and to lower local rates tinder the fixed-ratio exchange schedules. Local response rates increased with proximity to food under the higher fixed-ratio exchange schedules, indicative of discriminative control by the tokens.