Tests of transitivity in choices between fixed and variable reinforcer delays.

This experiment tested for transitivity in pigeons' choices between variable-time (VT) and fixed-time (FT) schedules. In a discrete-trials procedure, a subject chose between two alternatives by making a single key peck. Each choice was between a "standard alternative," which was the same schedule throughout a condition, and an "adjusting alternative," in which the delay to reinforcement was systematically increased or decreased many times a session. These adjustments enabled an approximate indifference point to be identified--the value of the adjusting delay at which the subject chose each alternative about equally often. Each test of transitivity involved four conditions. In one, the standard alternative was a variable-time schedule with a 2-s reinforcer, and the adjusting alternative also delivered a 2-s reinforcer. A second condition was similar except that the adjusting alternative delivered a 5-s reinforcer. The indifference point from each of these conditions was then converted to a fixed-time schedule for subsequent comparisons in the third and fourth conditions, respectively. Each of these last two conditions compared one of the fixed-time schedules (based upon the previous conditions and including their different reinforcer durations) with an adjusting schedule that delivered the alternative reinforcer duration, to determine whether the obtained indifference points would be those predicted from the prior alternative-duration comparisons with the VT schedule. There was little evidence for intransitivity of choice: Averaged across subjects and replications, the obtained indifference points deviated from perfect transitivity by less than 8%, and these deviations were not statistically significant. These results contrast with those of Navarick and Fantino (1972), who found frequent violations of transitivity between periodic and aperiodic schedules using a concurrent-chains procedure with variable-interval schedules in the initial links.     

A comparison of delays and ratio requirements in self-control choice.

In a discrete-trial procedure, pigeons could choose between 2-s and 6-s access to grain by making a single key peck. In Phase 1, the pigeons obtained both reinforcers by responding on fixed-ratio schedules. In Phase 2, they received both reinforcers after simple delays, arranged by fixed-time schedules, during which no responses were required. In Phase 3, the 2-s reinforcer was available through a fixed-time schedule and the 6-s reinforcer was available through a fixed-ratio schedule. In all conditions, the size of the delay or ratio leading to the 6-s reinforcer was systematically increased or decreased several times each session, permitting estimation of an "indifference point," the schedule size at which a subject chose each alternative equally often. By varying the size of the schedule for the 2-s reinforcer across conditions, several such indifference points were obtained from both fixed-time conditions and fixed-ratio conditions. The resulting "indifference curves" from fixed-time conditions and from fixed-ratio conditions were similar in shape, and they suggested that a hyperbolic equation describes the relation between ratio size and reinforcement value as well as the relation between reinforcer delay and its reinforcement value. The results from Phase 3 showed that subjects chose fixed-time schedules over fixed-ratio schedules that generated the same average times between a choice response and reinforcement.     

Effects of reinforcer probability, delay, and response requirements on the choices of rats and pigeons: possible species differences

In Experiment 1 with rats, a left lever press led to a 5-s delay and then a possible reinforcer. A right lever press led to an adjusting delay and then a certain reinforcer. This delay was adjusted over trials to estimate an indifference point, or a delay at which the two alternatives were chosen about equally often. Indifference points increased as the probability of reinforcement for the left lever decreased. In some conditions with a 20% chance of food, a light above the left lever was lit during the 5-s delay on all trials, but in other conditions, the light was only lit on those trials that ended with food. Unlike previous results with pigeons, the presence or absence of the delay light on no-food trials had no effect on the rats' indifference points. In other conditions, the rats showed less preference for the 20% alternative when the time between trials was longer. In Experiment 2 with rats, fixed-interval schedules were used instead of simple delays, and the presence or absence of the fixed-interval requirement on no-food trials had no effect on the indifference points. In Experiment 3 with rats and Experiment 4 with pigeons, the animals chose between a fixed-ratio 8 schedule that led to food on 33% of the trials and an adjusting-ratio schedule with food on 100% of the trials. Surprisingly, the rats showed less preference for the 33% alternative in conditions in which the ratio requirement was omitted on no-food trials. For the pigeons, the presence or absence of the ratio requirement on no-food trials had little effect. The results suggest that there may be differences between rats and pigeons in how they respond in choice situations involving delayed and probabilistic reinforcers.     

Variable-time reinforcement in multiple and concurrent schedules

Experiment I examined the role of a reduced rate of responding in the occurrence of behavioral contrast. Four rats and a pigeon were exposed to a two-component multiple schedule in which one component was always a variable-interval schedule. The second component was, at different times, either a variable-time schedule in which food was delivered independently of responding, or extinction. Both extinction and the variable-time schedule reduced the rate of responding in the second component. Behavioral contrast was observed, however, only when extinction was scheduled in the second component. Experiment II examined preference, as measured by time allocation in concurrent schedules for a variable-interval schedule relative to a variable-time schedule. Two rats displayed a lack of preference between the two schedules. The results of these experiments support a preference interpretation of behavioral contrast, which holds that behavioral contrast is the result of the introduction of a less-preferred condition in one component of a multiple schedule.     

Relative sensitivity to reinforcer amount and delay in a self-control choice situation.

Rats were exposed to concurrent-chains schedules in which a single variable-interval schedule arranged entry into one of two terminal-link delay periods (fixed-interval schedules). The shorter delay ended with the delivery of a single food pellet; the longer day ended with a larger number of food pellets (two under some conditions and six under others). In Experiment 1, the terminal-link delays were selected so that under all conditions the ratio of delays would exactly equal the ratio of the number of pellets. But the absolute duration of the delays differed across conditions. In one condition, for example, rats chose between one pellet delayed 5 s and six pellets delayed 30 s; in another condition rats chose between one pellet delayed 10 s and six pellets delayed 60 s. The generalized matching law predicts indifference between the two alternatives, assuming that the sensitivity parameters for amount and delay of reinforcement are equal. The rats' choices were, in fact, close to indifference except when the choice was between one pellet delayed 5 s and six pellets delayed 30 s. That deviation from indifference suggests that the sensitivities to amount and delay differ from each other depending on the durations of the delays. In Experiment 2, rats chose between one pellet following a 5-s delay and six pellets following a delay that was systematically increased over sessions to find a point of indifference. Indifference was achieved when the delay to the six pellets was approximately 55 s. These results are consistent with the possibility that the relative sensitivities to amount and delay differ as a function of the delays.     

Negative reinforcement and choice in humans

Participants chose between reinforcement schedules differing in delay and/or duration of noise offset. In Experiment 1 it was found that (1) immediate reinforcement was preferred to delayed reinforcement when amounts (durations) of reinforcement were equal; (2) a relatively large reinforcer was preferred to a smaller one when both reinforcers were obtained immediately; and (3) preference for an immediate, small reinforcer versus a delayed, large reinforcer increased as the delay preceding the large reinforcer increased, a sign of “impulsivity”. In Experiment 2, the schedules differed in amount or delay and equal intervals were added either to the constant parameter or the varied parameter. A shift from virtually exclusive preference to indifference occurred in the latter case but not the former, a result supporting a model of self-control that assumes that the value of a schedule depends on the ratio of amount and delay, and that choice between schedules depends on the ratio of these values.     

Molar optimization versus delayed reinforcement as explanations of choice between fixed-ratio and progressive-ratio schedules.

In a discrete-trials procedure, pigeons chose between a fixed-ratio 81 schedule and a progressive-ratio schedule by making a single peck at the key correlated with one or the other of these schedules. The response requirement on the progressive-ratio schedule began at 1 and increased by 10 each time the progressive-ratio schedule was chosen. Each time the fixed-ratio schedule was chosen, the requirement on the progressive-ratio schedule was reset to 1 response. In conditions where there was no intertrial interval, subjects chose the progressive-ratio schedule for an average of about five consecutive trials (during which the response requirement increased to 41), and then chose the fixed-ratio schedule. This ratio was larger than that predicted by an optimality analysis that assumes that subjects respond in a pattern that minimizes the response-reinforcer ratio or one that assumes that subjects respond in a pattern that maximizes the overall rate of reinforcement. In conditions with a 25-s or 50-s intertrial interval, subjects chose the progressive-ratio schedule for an average of about eight consecutive trials before choosing the fixed-ratio schedule. This change in performance with the addition of an intertrial interval was also not predicted by an optimality analysis. On the other hand, the results were consistent with the theory that choice is determined by the delays to the reinforcers delivered on the present trial and on subsequent trials.     

Procrastination by pigeons: preference for larger, more delayed work requirements.

In three experiments, pigeons chose between alternatives that required the completion of a small ratio schedule early in the trial or a larger ratio schedule later in the trial. Completion of the ratio requirement did not lead to an immediate reinforcer, but simply allowed the events of the trial to continue. In Experiment 1, the ratio requirements interrupted periods in which food was delivered on a variable-time schedule. In Experiments 2 and 3, each ratio requirement was preceded and followed by a delay, and only one reinforcer was delivered, at the end of each trial. Two of the experiments used an adjusting-ratio procedure in which the ratio requirement was increased and decreased over trials so as to estimate an indifference point--a ratio size at which the two alternatives were chosen about equally often. These experiments found clear evidence for "procrastination"--the choice of a larger but more delayed response requirement. In some cases, subjects chose the more delayed ratio schedule even when it was larger than the more immediate alternative by a factor of four or more. The results suggest that as the delay to the start of a ratio requirement is increased, it has progressively less effect on choice behavior, in much the same way that delaying a positive reinforcer reduces it effect on choice.     

Fixed and variable ratios and delays: further tests of an equivalence rule

A discrete-trial procedure was used to measure pigeons' choices between fixed and variable ratio schedules and between fixed and variable delays before reinforcement. A peck at a green key produced a reinforcement schedule that was constant within a condition but varied across conditions. A peck at a red key produced a ratio schedule (or, in other conditions, a simple delay) whose size was increased or decreased many times a session, depending on the subject's previous choices. The purpose of these adjustments was to estimate an indifference point--a ratio size (or delay duration) at which the subject chose each key about equally often. The results were used to test a simple "equivalence rule" for choices between fixed and variable schedules (Mazur, 1984). This rule, which was applied without using free parameters, predicted the major trends in the obtained indifference points from both ratio and delay conditions. However, some small but consistent deviations from the predictions were apparent. Better predictions were generated with a more complex equation, which included parameters reflecting the subjects' sensitivities to delay of reinforcement and to events of different probabilities. It was concluded that a successful equivalence rule must include parameters that can be adjusted to describe the effects of delay and probability in a given experimental setting. Once these parameters are estimated, however, choices involving both fixed and variable delays and fixed and variable ratios can be accurately predicted with the same equation.     

Unit price and choice in a token-reinforcement context

Pigeons were exposed to multiple and concurrent second-order schedules of token reinforcement, with stimulus lights serving as token reinforcers. Tokens were produced and exchanged for food according to various fixed-ratio schedules, yielding equal and unequal unit prices (responses per unit food delivery). On one schedule (termed the standard schedule), the unit price was held constant across conditions. On a second schedule (the alternative schedule), the unit price was either the same or different from the standard. Under conditions with unequal unit prices, near-exclusive preference for the lower unit price was obtained. Under conditions with equal unit prices, the direction and degree of preference depended on ratio size (number of responses per exchange period). When this ratio differed, strong preferences for the smaller ratio were observed. When this ratio was equal, preferences were nearer indifference. Response rates on the multiple schedule were generally consistent with the preference data in showing sensitivity to ratio size. Results are discussed in terms of a unit-price model that includes handling and reinforcer immediacy as additional costs. On the whole, results show that preferences were determined primarily by delay to the exchange period.     

Pigeons' choices in situations of diminishing returns: fixed- versus progressive-ratio schedules.

In two different discrete-trial procedures, pigeons were faced with choices between fixed-ratio and progressive-ratio schedules. The latter schedules entail diminishing returns, a feature analogous to foraging situations in the wild. In the first condition (no reset), subjects chose between a progressive-ratio schedule that increased in increments of 20 throughout a session and a fixed-ratio schedule that was constant across blocks of sessions. The size of the fixed ratio was varied parametrically through an ascending and then a descending series. In the reset condition, the same fixed-ratio values were used, but each selection (and completion) of the fixed ratio reset the progressive-ratio schedule back to its minimal value. In the no-reset procedure, the pigeons tended to cease selecting the progressive ratio when it equaled or slightly exceeded the fixed-ratio value, whereas in reset, they chose the fixed ratio well in advance of that equality point. These results indicate sensitivity to molar as well as to molecular reinforcement rates, and those molar relationships are similar to predictions based on the marginal value theorem of optimal foraging theory (e.g., Charnov, 1976). However, although previous results with monkeys (Hineline & Sodetz, 1987) appeared to minimize responses per reinforcement, the present results corresponded more closely to predictions based on sums-of-reciprocals of distance from point of choice to each of the next four reinforcers. Results obtained by Hodos and Trumbule (1967) with chimpanzees in a similar procedure were intermediate between these two relationships. Variability of choices, as well as median choice points, differed between the reset and no-reset conditions.(ABSTRACT TRUNCATED AT 250 WORDS)     

Choice for aperiodic versus periodic ratio schedules: A comparison of concurrent and concurrent-chain procedures

Choice between mixed-ratio schedules, consisting of equiprobable ratios of 1 and 99 responses per reinforcement, and fixed-ratio schedules of food reinforcement was assessed by two commonly used procedures: concurrent schedules and concurrent-chains schedules. Rats were trained under concurrent fixed-ratio mixed-ratio schedules, in which both ratio schedules were simultaneously available, and under a concurrent-chains schedule, in which access to one of the mutually exclusive ratio schedules comprising the terminal links was contingent on a single “choice” response. The distribution of responses between the two ratio schedules was taken as the choice proportion under the concurrent procedure, and the distribution of “choice” responses was taken as the choice proportion under the concurrent-chains procedure. Seven of eight rats displayed systematic choice; of those, each displayed nearly exclusive choice for fixed-ratio 35 to the mixed-ratio schedule under the concurrent procedure, but each displayed nearly exclusive choice for the mixed-ratio schedule to fixed-ratio 35 under the concurrent-chains procedure. Thus, preference for a fixed or a mixed schedule of reinforcement depended on the procedure used to assess preference.     

Preference and discrimination between response-dependent and response-independent schedules of reinforcement

Four Asian quail (Coturnix coturnix japonica) were exposed to concurrent-chain schedules, the terminal links of which were either variable-interval 30 sec and variable-time 30 sec, or fixed-interval 30 sec and fixed-time 30 sec. Except for one bird that exhibited a preference for the variable-interval schedule over the variable-time schedule, no consistent preferences were demonstrated for response-dependent or response-independent schedules. However, response rates were three times greater on response-dependent than on response-independent schedules. The discrimination between terminal-link schedules was rapidly recovered after the schedule positions were reversed. Casual observations revealed that the birds engaged in stereotypic circling and pecking while the response-independent schedules were operative.     

Effects of intertrial reinforcers on self-control choice.

In three experiments, pigeons chose between a small amount of food delivered after a short delay and a larger amount delivered after a longer delay. A discrete-trial adjusting-delay procedure was used to estimate indifference points--pairs of delay-amount combinations that were chosen about equally often. In Experiment 1, when additional reinforcers were available during intertrial intervals on a variable-interval schedule, preference for the smaller, more immediate reinforcer increased. Experiment 2 found that this shift in preference occurred partly because the variable-interval schedule started sooner after the smaller, more immediate reinforcer, but there was still a small shift in preference when the durations and temporal locations of the variable-interval schedules were identical for both alternatives. Experiment 3 found greater increases in preference for the smaller, more immediate reinforcer with a variable-interval 15-s schedule than with a variable-interval 90-s schedule. The results were generally consistent with a model that states that the impact of any event that follows a choice response declines according to a hyperbolic function with increasing time since the moment of choice.     

Soft commitment: self-control achieved by response persistence.

With reinforcement contingent on a single peck on either of two available keys (concurrent continuous reinforcement schedules) 4 pigeons, at 80% of free-feeding weights, preferred a smaller-sooner reinforcer (2.5 s of mixed grain preceded by a 0.5-s delay) to a larger-later reinforcer (4.5 s of mixed grain preceded by a 3.5-s delay). However, when the smaller-sooner and larger-later reinforcers were contingent on a concurrent fixed-ratio 31 schedule (the first 30 pecks distributed in any way on the two keys), all pigeons obtained the larger-later reinforcer much more often than they did when only a single peck was required. This "self-control" was achieved by beginning to peck the key leading to the larger-later reinforcer and persisting on that key until reinforcement occurred. We call this persistence "soft commitment" to distinguish it from strict commitment, in which self-control is achieved by preventing changeovers. Soft commitment also effectively achieved self-control when a brief (1-s) signal was inserted between the 30th and 31st response of the ratio and with concurrent fixed-interval 30-s schedules (rather than ratio schedules) of reinforcement. In a second experiment with the same subjects, the fixed ratio was interrupted by darkening both keys and lighting a third (center) key on which pecking was required for various fractions of the fixed-ratio count. The interruption significantly reduced self-control. When interruption was complete (30 responses on the center key followed by a single choice response), pigeons chose the smaller-sooner reinforcer as frequently as they did when only a single choice response was required.     

Steady-state performance on fixed-, mixed-, and random-ratio schedules

Three groups of rats pressed a lever for milk reinforcers on various simple reinforcement schedules (one schedule per condition). In Group M, each pair of conditions included a mixed-ratio schedule and a fixed-ratio schedule with equal average response:reinforcer ratios. On mixed-ratio schedules, reinforcement occurred with equal probability after a small or a large response requirement was met. In Group R, fixed-ratio and random-ratio schedules were compared in each pair of conditions. For all subjects in these two groups, the frequency distributions of interresponse times of less than one second were very similar on all ratio schedules, exhibiting a peak at about .2 seconds. For comparison, subjects in Group V responded on variable-interval schedules, and few interresponse times as short as .2 seconds were recorded. The results suggest that the rate of continuous responding is the same on all ratio schedules, and what varies among ratio schedules is the frequency, location, and duration of pauses. Preratio pauses were longer on fixed-ratio schedules than on mixed-ratio or random-ratio schedules, but there was more within-ratio pausing on mixed-ratio and random-ratio schedules. Across a single trial, the probability of an interruption in responding decreased on fixed-ratio schedules, was roughly constant on random-ratio schedules, and often increased and then decreased on mixed-ratio schedules. These response patterns provided partial support for Mazur's (1982) theory that the probability of instrumental responding is directly related to the probability of reinforcement and the proximity of reinforcement.     

Pavlovian contingencies and resistance to change in a multiple schedule

According to theoretical accounts of behavioral momentum, the Pavlovian stimulus—reinforcer contingency determines resistance to change. To assess this prediction, 8 pigeons were exposed to an unsignaled delay-of-reinforcement schedule (a tandem variable-interval fixed-time schedule), a signaled delay-of-reinforcement schedule (a chain variable-interval fixed-time schedule), and an immediate, zero-delay schedule of reinforcement in a three-component multiple schedule. The unsignaled delay and signaled delay schedules employed equal fixed-time delays, with the only difference being a stimulus change in the signaled delay schedule. Overall rates of reinforcement were equated for the three schedules. The Pavlovian contingency was identical for the unsignaled and immediate schedules, and response—reinforcer contiguity was degraded for the unsignaled schedule. Results from two disruption procedures (prefeeding subjects prior to experimental sessions and adding a variable-time schedule to timeout periods separating baseline components) demonstrated that response—reinforcer contiguity does play a role in determining resistance to change. The results from the extinction manipulation were not as clear. Responding in the unsignaled delay component was consistently less resistant to change than was responding in both the immediate and presignaled segments of the signaled delay components, contrary to the view that Pavlovian contingencies determine resistance to change. Probe tests further supported the resistance-to-change results, indicating consistency between resistance to change and preference, both of which are putative measures of response strength.     

Concurrent ratio schedules: Fixed vs. variable response requirements

Rats were trained on concurrent fixed-ratio variable-ratio or concurrent fixed-ratio mixed-ratio schedules of food reinforcement. The variable-ratio schedule was composed of an arithmetic sequence of 11 ratios that averaged 50; the mixed-ratio schedule consisted of equiprobable ratios of 1 and 99. Fixed-ratio values, varied over experimental conditions, included 25, 35, 50, 60, and 99. The proportion of responses and time allocated to the variable- or mixed-ratio schedule increased as the size of the fixed ratio increased. For most subjects, higher proportions of responses and time were maintained on the fixed-ratio schedule at fixed-ratio values of 25 and 35; higher proportions of responses and time were maintained on the variable- or mixed-ratio schedule at fixed-ratio values of 50 or higher. On concurrent variable-ratio fixed-ratio schedules, the tendency for responding to be maintained exclusively by one schedule was related to the difference in local reinforcement rates obtained from those schedules. Exclusive responding was approximated when the difference in local reinforcement rates obtained from those schedules was large; responding was more evenly distributed between the schedules as the difference in the rates at which reinforcement was obtained from each decreased.     

Choosing between fixed time and variable time shock

Two experiments are reported dealing with choice of fixed or variable time shock schedules with and without signals. Response independent shock was presented on either a fixed-time or a variable-time schedule and subjects could change from one condition to the other for short periods after which the initial condition was automatically reinstated. In the first experiment shocks for both schedules were either unsignalled (6 subjects) or signalled (3 subjects) and all subjects changed from a schedule of variable-time shock to a fixed-time one. Subjects that were given the opportunity to change in the reverse direction did not do so. Opposite findings were obtained in the second experiment (7 subjects) comparing unsignalled fixed-time shock with signalled variable-time shock. Six of the seven subjects changed from the fixed-time schedule to the signalled variable-time one but not the reverse. The data were considered consistent with an analysis of choice based on the reinforcing value of discriminable shock-free periods.     

A temporal parameter influencing choice between signalled and unsignalled shock schedules

The present study investigated whether choice of a signalled variable-time shock schedule over an unsignalled one was influenced by the average intershock interval. Eight rats were given a choice between signalled and unsignalled shock schedules in a series of conditions with average intershock intervals of 510, 270, 150, 90, 60, and 45 sec. Each test condition was preceded by a training-baseline condition, and schedule values were arranged in an ascending (four subjects) or descending (four subjects) order. Choice of the signalled conditions was directly related to the average intershock interval of the variable-time schedule for six of the eight subjects. The per cent of time in the signalled condition was highest when the average intershock interval was 150 sec or longer and lowest when the average intershock interval was 45 sec. The findings were interpreted as being due to changes in the safety features of the signalled schedule, rather than to changes in the average intershock interval per se.