Choice and reinforcement delay

Previous studies of choice between two delayed reinforcers have indicated that the relative immediacy of the reinforcer is a major determinant of the relative frequency of responding. Parallel studies of choice between two interresponse times have found exceptions to this generality. The present study looked at the choice by pigeons between two delays, one of which was always four times longer than the other, but whose absolute durations were varied across conditions. The results indicated that choice is not uniquely determined by the relative immediacy of reinforcement, but that absolute delays are also involved. Models for concurrent chained schedules appear to be more applicable to the present data than the matching relation; however, these too failed to predict choice for long delays.     

Context dependent changes in the reinforcing strength of schedule-induced drinking.

Previous experiments show that the opportunity to engage in schedule-induced responding is reinforcing. In this experiment, the reinforcing strength of schedule-induced drinking was measured. Four rats were trained on a concurrent-chain schedule. The two terminal links provided food pellets on identical fixed-time schedules. In addition, one terminal link also provided the opportunity to press a button that operated a water dipper. In this link, the rats showed polydipsic drinking. Button-pressing rate for polydipsic drinking was a bitonic function of pellet rate, and it was possible to describe the relationship with a slightly modified version of the matching equation for primary reinforcement. This equation also closely fit the data from other studies. Initial-link response rates, however, did not appear to be influenced by the availability of water in the terminal links. Control conditions suggested that the reinforcing strength of polydipsia was strongly bound to the context provided by periodic food reinforcement.     

The following schedule of reinforcement as a fundamental determinant of steady state contrast in multiple schedules

Two experiments investigated whether steady-state interactions in multiple schedules depend exclusively on the following schedule of reinforcement. Experiment 1 used a four-component multiple schedule in which two components were associated with the same constant schedule of reinforcement, and where rate of reinforcement was varied in the component that followed one of these. Contrast effects were reliable only in the component that preceded the point of reinforcement variation, although some contrast did occur otherwise. In those instances where contrast other than the following-schedule effect did occur, it was accounted for by the effect of the preceding schedule, an effect for which there were consistent individual differences among subjects, and which varied with component duration. Experiment 2 used a three-component schedule, in which reinforcement rate was varied in the middle component. The results were consistent with Experiment 1, as the following-schedule effect was the only consistent effect that occurred, although an effect of the preceding schedule did occur for some subjects under some conditions, and was especially evident early in training. The conclusion from both experiments is that there is no general effect of relative rate of reinforcement apart from the sum of the effects of the preceding and following schedules, and that the following-schedule effect is the fundamental cause of steady-state interactions.     

How to maximize reward rate on two variable-interval paradigms

Without assuming any constraints on behavior, we derive the policy that maximizes overall reward rate on two variable-interval paradigms. The first paradigm is concurrent variable time-variable time with changeover delay. It is shown that for nearly all parameter values, a switch to the schedule with the longer interval should be followed immediately by a switch back to the schedule with the shorter interval. The matching law does not hold at the optimum and does not uniquely specify the obtained reward rate. The second paradigm is discrete trial concurrent variable interval-variable interval. For given schedule parameters, the optimal policy involves a cycle of a fixed number of choices of the schedule with the shorter interval followed by one choice of the schedule with the longer interval. Molecular maximization sometimes results in optimal behavior.     

Concurrent fixed-interval variable-ratio schedules and the matching relation

Five rats responded under concurrent fixed-interval variable-ratio schedules of food reinforcement. Fixed-interval values ranged from 50-seconds to 300-seconds and variable-ratio values ranged from 30 to 360; a five-second changeover delay was in effect throughout the experiment. The relations between reinforcement ratios obtained from the two schedules and the ratios of responses and time spent on the schedules were described by Baum's (1974) generalized matching equation. All subjects undermatched both response and time ratios to reinforcement ratios, and all subjects displayed systematic bias in favor of the variable-ratio schedules. Response ratios undermatched reinforcement ratios less than did time ratios, but response ratios produced greater bias than did time ratios for every subject and for the group as a whole. Local rates of responding were generally higher on the variable-ratio than on the fixed-interval schedules. When responding was maintained by both schedules, a period of no responding on either schedule immediately after fixed-interval reinforcement typically was followed by high-rate responding on the variable-ratio schedule. At short fixed-interval values, when a changeover to the fixed-interval schedule was made, responding usually continued until fixed-interval reinforcement was obtained; at longer values, a changeover back to the variable-ratio schedule usually occurred when fixed-interval reinforcement was not forthcoming within a few seconds, and responding then alternated between the two schedules every few seconds until fixed-interval reinforcement finally was obtained.     

Effect of delay-interval stimuli on delayed symbolic matching to sample in the pigeon

In Experiment 1, food-deprived pigeons received delayed symbolic matching to sample training in a darkened Skinner box. Trials began with the illumination of the grain feeder lamp (no food sample), or illumination of this lamp, accompanied by the raising of the feeder tray (food sample). After a delay of a few seconds, the two side response keys were illuminated, one with red and one with green light, with positions counterbalanced over trials. Pecking the red (green) comparison produced grain reinforcement if the trial had started with food (no food); pecking red after a no-food sample or green after a food sample was not reinforced. Once matching performance was stable, four stimuli were presented during the delay interval, and their effects on matching accuracy were evaluated. Both illumination of the houselight and the center key with white geometric forms decreased matching accuracy, whereas presentation of a tone and vibration of the test chamber did not. In Experiment 2, pecking the red center key was reinforced with food according to a variable interval schedule. The effects of occasional brief presentations of the four stimuli used in the first experiment on ongoing pecking were assessed. The houselight and form disturbed key pecking, but the tone and vibration did not. Thus, stimuli that interfered with delayed matching also interfered with simple operant behavior. Implications of these results for theories of remembering are discussed.     

The development of derived stimulus relations through training in arbitrary-matching sequences

Five-year-old children were taught three-stage sequences of arbitrary matching: A-C, B-C, A-D; A-C, B-D, B-C; or A-C, A-D, B-C. Each stage refers to a sample-comparison relation between stimuli. Unreinforced test probes revealed untrained arbitrary matches (B-D, A-D, and B-D, respectively), derivable by substitution of stimuli with a common sample or comparison function. Additional probes revealed further untrained sample-comparison relations derivable by substitution and identity, including the commuted relations D-B, D-A, and D-B, respectively. These processes may have relevance to conceptual and verbal behavior.     

Reaction times of younger and older men and temporal contingencies of reinforcement

Influences of extended training and temporal contingencies on reaction time were studied in relation to developmental differences. Older and younger men were trained on a chained schedule in which completion of a variable interval produced a terminal link in which reaction time was measured. The reaction-time procedure involved a conditional discrimination with matching to sample in one component and oddity matching in the other. During baseline training, no time limit was placed on the response to the discrimination choice stimuli. Subsequently, increasingly severe time limits were imposed over a series of sessions. Older and younger men showed increased speeds (decreased reaction times) when temporal contingencies were imposed, and these changes were maintained during post-training baseline sessions when there was unlimited time in which to respond. The younger men generally responded faster than the older ones, and age differences were not appreciably reduced during the course of the experiment. The results indicated the feasibility of studying reaction time in human subjects using operant procedures analogous to those developed for the study of nonverbal organisms.     

Concurrent schedules: Spatial separation of response alternatives

Four pigeons were exposed to independent concurrent variable-interval 20-second variable-interval 60-second schedules of reinforcement. A transparent partition was inserted midway between the two response keys. The length of the partition was systematically manipulated. Increasing partition length produced a decrease in changeover rate in Experiment 1. Over-matching was observed with a partition length of 20 centimeters. In Experiment 2 a four-second limited hold was added to the schedules. Increasing partition length produced a decrease in changeover rate that exceeded the decrease observed in Experiment 1. This manipulation produced nearly exclusive choice of the variable-interval 20-second component. The present results, together with results obtained in related research, suggest that deviation from matching is a function of procedural variables that determine the consequences of a changeover response.     

Choice as a function of local versus molar reinforcement contingencies.

Rats were trained on a discrete-trial probability learning task. In Experiment 1, the molar reinforcement probabilities for the two response alternatives were equal, and the local contingencies of reinforcement differentially reinforced a win-stay, lose-shift response pattern. The win-stay portion was learned substantially more easily and appeared from the outset of training, suggesting that its occurrence did not depend upon discrimination of the local contingencies but rather only upon simple strengthening effects of individual reinforcements. Control by both types of local contingencies decreased with increases in the intertrial interval, although some control remained with intertrial intervals as long as 30 s. In Experiment 2, the local contingencies always favored win-shift and lose-shift response patterns but were asymmetrical for the two responses, causing the molar reinforcement rates for the two responses to differ. Some learning of the alternation pattern occurred with short intertrial intervals, although win-stay behavior occurred for some subjects. The local reinforcement contingencies were discriminated poorly with longer intertrial intervals. In the absence of control by the local contingencies, choice proportion was determined by the molar contingencies, as indicated by high exponent values for the generalized matching law with long intertrial intervals, and lower values with short intertrial intervals. The results show that when molar contingencies of reinforcement and local contingencies are in opposition, both may have independent roles. Control by molar contingencies cannot generally be explained by local contingencies.