Choice dynamics in concurrent ratio schedules of reinforcement

The generalized matching law predicts performance on concurrent schedules when variable-interval schedules are programmed but is trivially applicable when independent ratio schedules are used. Responding usually is exclusive to the schedule with the lowest response requirement. Determining a method to program concurrent ratio schedules such that matching analyses can be usefully employed would extend the generality of matching research and lead to new avenues of research. In the present experiments, ratio schedules were programmed dependently such that responses to either of the two options progressed the requirement on both schedules. Responding is not exclusive because the probability of reinforcement increases on both schedules as responses are allocated to either schedule. In Experiment 1, performance on concurrent variable-ratio schedules was assessed, and reinforcer ratios were varied across conditions to investigate changes in sensitivity. Additionally, the length of a changeover delay was manipulated. In Experiment 2, performance was compared under concurrently available, dependently programmed variable-ratio and fixed-ratio schedules. Performance was well described by the generalized matching law. Increases in the changeover delay decreased sensitivity, whereas sensitivity was higher when variable-ratio schedules were employed, compared with fixed-ratio schedules. Concurrent ratio schedules can be a viable approach to studying functional differences between ratio and interval schedules.     

Drug discrimination under a concurrent schedule.

Three pigeons were trained to discriminate a 5.0 mg/kg dose of pentobarbital from saline under a two-key concurrent schedule with responding on the key associated with the presession injection, under both stimulus conditions, producing four times as many reinforcers as responding on the other key. This concurrent schedule resulted in approximately 70% responding to the higher reinforcement key under the pentobarbital stimulus and approximately 30% responding to that key under the saline stimulus. During testing, then, the pigeons were able to dose-dependently emit higher (>70%) or lower (<30%) values than were established under the control conditions. Dose-response curves were determined for pentobarbital (twice), methamphetamine, phencyclidine, chlordiazepoxide, and the combination of pentobarbital and the barbiturate antagonist bemegride. The results obtained with pentobarbital and chlordiazepoxide showed that, as the dose increased, pentobarbital-appropriate responding also increased. Methamphetamine produced relatively flat dose--response curves, whereas phencyclidine administration produced inconsistent effects on responding. The combination of the training dose of pentobarbital with increasing doses of bemegride produced a decrease in pentobarbital-appropriate responding. The results also showed that the dose-response curves for pentobarbital and chlordiazepoxide, instead of being all or none, were graded functions of the drug dose.     

Commitment using punishment.

Experimental parameters were adjusted so that pigeons' pairwise choices among three alternatives reflected the following order of preference: (a) a smaller-sooner reinforcer, (b) a larger-later reinforcer, and (c) the smaller-sooner reinforcer followed by a punishment (consisting of an extended blackout period). After this order of preference was established, the pigeons were exposed to a two-link, concurrent-chain-like choice procedure. One terminal link consisted of a choice between the smaller-sooner and the larger-later reinforcer; the other terminal link was identical to the first except that the smaller-sooner reinforcer was followed by blackout punishment. The pigeons' preference (in their initial-link choice) for the terminal link with the punished smaller-sooner alternative increased as the delay between the initial and terminal links increased. By choosing this terminal link, the pigeons are said to have "committed" themselves to obtaining the larger-later reinforcer. However, unlike prior studies of commitment (e.g., Rachlin & Green, 1972), it was still possible after making the commitment for the pigeons to choose the smaller-sooner reinforcer and undergo the punishment. The pigeons did in fact occasionally make this highly deleterious choice.     

Response-dependent prechoice effects on foraging-related choice

Three experiments explored the influence of prechoice events on pigeons' preference. In two of three studies, a fixed-interval 200-second prechoice period preceded the initial links of a concurrent chain in which outcomes differed either (a) in terms of the delay to food or (b) in terms of amount of food and delay to food. In Experiment 3, the prechoice period preceded the initial links that provided a choice between a small single food presentation and two identical, more delayed food presentations. In all three cases, obtained choice proportions did not vary as a function of prechoice duration. These results suggest that a local-contextual view adequately describes the foraging context; they also have implications for the appropriate formulation of the delay-reduction theory of conditioned reinforcement and rate-maximizing views of optimal foraging theory.     

Concurrent schedules: Quantifying the aversiveness of noise

Four hens worked under independent multiple concurrent variable-interval schedules with an overlaid aversive stimulus (sound of hens in a poultry shed at 100dBA) activated by the first peck on a key. The sound remained on until a response was made on the other key. The key that activated the sound in each component was varied over a series of conditions. When the sound was activated by the left (or right) key in one component, it was activated by the right (or left) key in the other component. Bias was examined under a range of different variable-interval schedules, and the applicability of the generalized matching law was examined. It was found that the hens' behavior was biased away from the sound independently of the schedule in effect and that this bias could be quantified using a modified version of the generalized matching law. Behavior during the changeover delays was not affected by the presence of the noise or by changes in reinforcement rate, even though the total response measures were. Insensitivity shown during the delay suggests that behavior after the changeover delay may be more appropriate as a measure of preference (or aversiveness) of stimuli than are overall behavior measures.     

ALTERNATIVE REINFORCEMENT INCREASES RESISTANCE TO CHANGE: PAVLOVIAN OR OPERANT CONTINGENCIES?

Two multiple-schedule experiments with pigeons examined the effect of adding food reinforcement from an alternative source on the resistance of the reinforced response (target response) to the decremental effects of satiation and extinction. In Experiment 1, key pecks were reinforced by food in two components according to variable-interval schedules and, in some conditions, food was delivered according to variable-time schedules in one of the components. The rate of key pecking in a component was negatively related to the proportion of reinforcers from the alternative (variable-time) source. Resistance to satiation and extinction, in contrast, was positively related to the overall rate of reinforcement in the component. Experiment 2 was conceptually similar except that the alternative reinforcers were contingent on a specific concurrent response. Again, the rate of the target response varied as a function of its relative reinforcement, but its resistance to satiation and extinction varied directly with the overall rate of reinforcement in the component stimulus regardless of its relative reinforcement. Together the results of the two experiments suggest that the relative reinforcement of a response (the operant contingency) determines its rate, whereas the stimulus-reinforcement contingency (a Pavlovian contingency) determines its resistance to change.     

Choice with uncertain outcomes: conditioned reinforcement effects.

Pigeons responded on concurrent chains with equal initial- and terminal-link durations. In all conditions, the terminal links of one chain ended reliably in reinforcement; the terminal links on the alternative chain ended in either food or blackout. In Experiment 1, the terminal-link stimuli were correlated with (signaled) the outcome, and the durations of the initial and terminal links were varied across conditions. Preference did not vary systematically across conditions. In Experiment 2, terminal-link durations were varied under different stimulus conditions. The initial links were variable-interval 80-s schedules. Preference for the reliable alternative was generally higher in unsignaled than in signaled conditions. Preference increased with terminal-link durations only in the unsignaled conditions. There were no consistent differences between conditions with and without a common signal for reinforcement on the two chains. In the first series of conditions in Experiment 3, a single response was required in the initial links, and the stimulus conditions during 50-s terminal links were varied. Preference for the reliable outcome approached 1.0 in unsignaled conditions and was considerably lower (below .50 for 3 of 5 subjects) in signaled conditions. In a final series of signaled conditions with relatively long terminal links, preference varied with duration of the initial links. The results extend previous findings and are discussed in terms of the delay reduction signaled by terminal-link stimuli.     

Short-term and long-term effects of reinforcers on choice

The relation between molar and molecular aspects of time allocation was studied in pigeons on concurrent variable-time variable-time schedules of reinforcement. Fifteen-minute reinforcer-free periods were inserted in the middle of every third session. Generalized molar matching of time ratios to reinforcer ratios was observed during concurrent reinforcement. Contrary to melioration theory, preference was unchanged during the reinforcer-free periods as well as in extinction. In addition to this long-term effect of reinforcement, short-term effects were observed: Reinforcers increased the duration of the stays during which they were delivered but had little consistent effect either on the immediately following stay in the same schedule or on the immediately following stay in the alternative schedule. Thus, an orderly effect of reinforcer delivery on molecular aspects of time allocation was observed, but because of its short-term nature, this effect cannot account for the matching observed at the molar level.     

Psychological distance to reward: Segmentation of aperiodic schedules of reinforcement

College students responded for monetary rewards in two experiments on choice between differentially segmented aperiodic schedules of reinforcement. On a microcomputer, the concurrent chains were simulated as an air-defense video game in which subjects used two radars for detecting and destroying enemy aircraft. To earn more cash-exchangeable points, subjects had to shoot down as many planes as possible within a given period of time. For both experiments, access to one of two radar systems (terminal link) was controlled by a pair of independent concurrent variable-interval 60-s schedules (initial link) with a 4-s changeover delay always in effect. In Experiment 1, the appearance of an enemy aircraft in the terminal link was determined by a variable-interval (15 s or 60 s) schedule or a two-component chained variable-interval schedule of equal duration. Experiment 2 was similar to Experiment 1 except for the segmented schedule, which had three components. Subjects preferred the unsegmented schedule over its segmented counterpart in the conditions with variable-interval 60 s, and preference tended to be more pronounced with more components in the segmented schedule. These findings are compatible with those from previous studies of periodic and aperiodic schedules with pigeons or humans as subjects.     

Determinants of human performance on concurrent schedules.

Six experiments, each with 5 human adults, were conducted to investigate the determinants of human performance on multiple concurrent variable-interval schedules. A two-key procedure was employed in which subjects' key presses produced points exchangeable for money. Variables manipulated across experiments were (a) changeover delay (Experiments 2, 4, and 6), (b) ordinal cues related to scheduled reinforcement frequencies (Experiments 3 and 4), and (c) instructions describing the ordinal relations between schedule-correlated stimuli and scheduled reinforcement frequency (Experiments 5 and 6). The performances of only 13 of the 30 subjects could be described by the generalized matching equation and were within a range of values typical of those reported in the animal literature. Eight subjects showed indifference, 9 undermatched, 7 approximated matching, 3 overmatched, and a further 3 responded exclusively to the richer component of the concurrent schedules. These differing modes of responding were closely related to the different types of performance rules reported by subjects in postexperimental questionnaires. The results are in good agreement with those from studies of human performance on single schedules, suggesting that rule-governed behavior, in interaction with contingencies, may be an important determinant of human choice.