Conditioned reinforcement value and choice.

The delay-reduction hypothesis of conditioned reinforcement states that the reinforcing value of a food-associated stimulus is determined by the delay to primary reinforcement signaled by the onset of the stimulus relative to the average delay to primary reinforcement in the conditioning situation. In contrast, most contemporary models of conditioned reinforcement strength posit that the reinforcing strength of a stimulus is some simple function only of the delay to primary reinforcement in the presence of stimulus. The delay-reduction hypothesis diverges from other conditioned reinforcement models in that it predicts that a fixed-duration food-paired stimulus will have different reinforcing values depending on the frequency of its presentation. In Experiment 1, pigeons' key pecks were reinforced according to concurrent-chains schedules with variable-interval 10-second and variable-interval 20-second terminal-link schedules. The initial-link schedule preceding the shorter terminal link was always variable-interval 60 seconds, and the initial-link schedule requirement preceding the longer terminal link was varied between 1 second and 60 seconds across conditions. In Experiment 2, the initial-link schedule preceding the longer of two terminal links was varied for each of three groups of pigeons. The terminal links of the concurrent chains for the three groups were variable-interval 10 seconds and 20 seconds, variable-interval 10 seconds and 30 seconds, and variable-interval 30 seconds and 50 seconds. In both experiments, preference for the shorter terminal link was either a bitonic function or an inverse function of the initial-link schedule preceding the longer terminal-link schedule. Consistent with the predictions of the delay-reduction hypothesis, the relative values of the terminal-link stimuli changed as a function of the overall frequency of primary reinforcement. Vaughan's (1985) melioration model, which was shown to be formally similar to Squires and Fantino's (1971) delay-reduction model, can be modified so as to predict these results without changing its underlying assumptions.     

An eight-alternative concurrent schedule: foraging in a radial maze.

In two experiments conducted in an eight-arm radial maze, food pellets were delivered when a photocell beam was broken at the end of each arm via a nose poke, according to either fixed-interval or random-interval schedules of reinforcement, with each arm providing a different frequency of reinforcement. The behavior of rats exposed to these procedures was well described by the generalized matching law; that is, the relationships between log behavior ratios and log pellet ratios were approximated by linear functions. The slopes of these log-log functions, an index of sensitivity to reinforcement frequency, were greatest for nose pokes, intermediate for time spent in an arm, and least for arm entries. Similar results were obtained with both fixed-interval and random-interval schedules. Addition of a 10-s changeover delay in both experiments eliminated the slope differentials between nose pokes and time spent in an arm by reducing the slopes of the nose-poke functions. These results suggest that different aspects of foraging may be differentially sensitive to reinforcement frequency. With concurrent fixed-interval schedules, the degree of temporal control exerted by individual fixed-interval schedules was directly related to reinforcement frequency.     

Delay and amount of reward in a concurrent chain

Eight pigeons responded under a concurrent-chain schedule for rewards differing in both delay and amount, the larger reward being associated with the longer delay. Preference was examined as the absolute durations of the terminal-link delays were increased at four different delay ratios. Difficulties with other experiments of this type were controlled for by the use of (a) a single-tape initial link to equalize terminal-link entries, (b) a blackout following the more immediate reward to equalize terminal-link length, and (c) a photocell to measure reinforcer duration more accurately. Preference for the larger reward changed systematically as delays increased in all conditions, decreasing for the 6:1, 3:1, and 3:2 ratios, and increasing for the 1:1 ratio. These results were similar to, but significantly different from, those of previous investigations. The implications of these results for various models of concurrent-chain behavior are discussed.     

Sensitivity to reinforcement in concurrent arithmetic and exponential schedules

The generalized matching law states that the logarithm of the ratio of responses emitted or time spent responding in concurrent variable-interval schedules is a linear function of the logarithm of the ratio of reinforcements obtained. The slope of this relation, sensitivity to reinforcement, varies about 1.0 but has been shown to be different when obtained in different laboratories. The present paper analyzed the results from 18 experiments on concurrent variable-interval schedule performance and showed that response-allocation sensitivity to reinforcement was significantly smaller when arithmetic, rather than exponential, progressions were used to produce variable-interval schedules. There were no differences in time-allocation sensitivity between the two methods of constructing variable-interval schedules. Since the two laboratories have consistently used different methods for constructing variable-interval schedules, the differences in obtained sensitivities to reinforcement are explained. The reanalysis suggests that animals may be sensitive to differences in the distribution of reinforcements in time.     

Matching, maximizing, and hill-climbing

In simple situations, animals consistently choose the better of two alternatives. On concurrent variable-interval variable-interval and variable-interval variable-ratio schedules, they approximately match aggregate choice and reinforcement ratios. The matching law attempts to explain the latter result but does not address the former. Hill-climbing rules such as momentary maximizing can account for both. We show that momentary maximizing constrains molar choice to approximate matching; that molar choice covaries with pigeons' momentary-maximizing estimate; and that the “generalized matching law” follows from almost any hill-climbing rule.     

Melioration, matching, and maximization

Pigeons were studied in an experiment involving two concurrently available response keys. Conditions were such that in the first condition the predictions of melioration (Herrnstein & Vaughan, 1980), minimization of deviation from matching, and maximization were identical: relative time on the right key should have fallen between .125 and .25, which in fact occurred. In the second condition, melioration predicted a shift in relative time on the right to between .75 and .875, which would involve a transient deviation from matching as well as a substantial drop in rate of reinforcement. All three birds eventually shifted their distribution of behavior to within the range predicted by melioration.     

Effects of varying stimulus disparity and the reinforcer ratio in concurrent-schedule and signal-detection procedures.

The present study measured the effects of stimulus and reinforcer variations on pigeons' behavior in two different choice procedures. Two intensities of white light were presented as the stimuli on the main key in a switching-key concurrent schedule and as the sample stimuli in a signal-detection procedure. Under both procedures, the scheduled rate of reinforcement was varied across conditions to produce various ratios of obtained reinforcement. These ratios were obtained for seven pairs of light intensities. In the concurrent schedules, the effects of reinforcer-ratio variations were positively correlated with the physical disparity between the two light intensities. In the signal-detection procedure, changes in the reinforcer ratio produced greater effects on performance when stimulus disparity was very low or very high compared to those found at intermediate levels of stimulus disparity. This discrepancy creates a dilemma for existing behavioral models of signal-detection performance.     

Titration of schedule parameters by pigeons

Pigeons were tested in a computer-controlled two-key chamber. A standard (nonchanging) schedule of reinforcement was in force on one key, and an adjusting schedule on the other. The schedules were available concurrently after each reinforcement, but after the first peck on either key (the choice peck), the schedule on the other key was made inoperative. The parameter of the adjusting schedule was decreased when the standard schedule was chosen and increased when the adjusting schedule was chosen. The standard schedule was changed only between sessions. Fixed intervals and fixed ratios were used as standard schedules, and intervals and ratios were used as adjusting schedules. When standard and adjusting schedules were of the same type, median parameters on the adjusting key equalled those of the standard schedules, at four values of each standard schedule. For four of five birds, and for the group median, similar curves could be plotted through the indifference points obtained from a standard ratio with an adjusting interval, and from a standard interval with an adjusting ratio. These points showed consistent individual differences, but they could be predicted by assuming that the median time from the choice peck to reinforcement should be the same on both keys. This is equivalent to treating the schedule as a concurrent chain and assuming that Herrnstein's quantitative law of effect applies.     

Matching-based hedonic scaling in the pigeon

Four slightly hungry pigeons chose between pairs of grains in a Findley concurrent choice procedure. For Condition I, choice involved hemp versus buckwheat; for Condition II, wheat versus buckwheat; and for Condition III, hemp versus wheat. In all conditions, frequency of reinforcement was arranged according to concurrent variable-interval variable-interval schedules. On the assumption that subjects matched their behavior and time distributions to those of reinforcer value, the choice functions obtained in Conditions I and II were transformed to yield estimates of values of hemp and wheat relative to buckwheat. These, in turn, provided predictions about behavior and time allocation in Condition III. In general, the predicted outcomes were close to those actually obtained. The results evidence the effectiveness of matching-based hedonic scales in the prediction of choice between qualitatively different reinforcers.