Estimation of indifference points with an adjusting-delay procedure.

In a series of conditions, pigeons chose between 1.5 s and 3 s of access to grain, each preceded by some delay. The delay that preceded the small reinforcer was constant throughout a condition. The delay that preceded the large reinforcer was increased or decreased a number of times each session in order to estimate an "indifference point," a delay at which the subject chose each alternative about equally often. The experiment was designed to determine whether variations in any of four features of this adjusting-delay procedure would systematically alter the estimated indifference points. The four features were the total trial duration, the number of center-key responses necessary to begin a trial, the number of choice trials that preceded each change in the adjusting delay, and step size--the size of each increment and decrement in the delay. Manipulation of the first three features had no systematic effects on the indifference points. As step size was increased from 0.5 s to 6 s, within-session variability of the adjusting delay steadily increased, and the 6-s step size produced larger indifference-point estimates for some subjects. The results suggest that, within certain limits, these procedural features can be altered without affecting the indifference-point estimates, but that the use of a large step size can distort the estimates. Some theoretical implications of the relative constancy of indifference points across these procedural variations are discussed.     

Probability and delay of reinforcement as factors in discrete-trial choice.

Pigeons chose between two alternatives that differed in the probability of reinforcement and the delay to reinforcement. A peck on the red key always produced a delay of 5 s and then a possible reinforcer. The probability of reinforcement for responding on this key varied from .05 to 1.0 in different conditions. A response on the green key produced a delay of adjustable duration and then a possible reinforcer, with the probability of reinforcement ranging from .25 to 1.0 in different conditions. The green-key delay was increased or decreased many times per session, depending on a subject's previous choices. The purpose of these adjustments was to estimate an indifference point, or a delay that resulted in a subject's choosing each alternative about equally often. In conditions where the probability of reinforcement was five times higher on the green key, the green-key delay averaged about 12 s at the indifference point. In conditions where the probability of reinforcement was twice as high on the green key, the green-key delay at the indifference point was about 8 s with high probabilities and about 6 s with low probabilities. An analysis based on these results and those from studies on delay of reinforcement suggests that pigeons' choices are relatively insensitive to variations in the probability of reinforcement between .2 and 1.0, but quite sensitive to variations in probability between .2 and 0.     

Choice between delayed reinforcers and fixed-ratio schedules requiring forceful responding.

This experiment measured pigeons' choices between delayed reinforcers and fixed-ratio schedules in which a force of approximately 0.48 N was needed to operate the response key. In ratio-delay conditions, subjects chose between a fixed-ratio schedule and an adjusting delay. The delay was increased or decreased several times a session in order to estimate an indifference point--a delay duration at which the two alternatives were chosen about equally often. Each ratio-delay condition was followed by a delay-delay condition in which subjects chose between the adjusting delay and a variable-time schedule, with the components of this schedule selected to match the ratio completion times of the preceding ratio-delay condition. The adjusting delays at the indifference point were longer when the alternative was a fixed-ratio schedule than when it was a matched variable-time schedule, which indicated a preference for the matched variable-time schedules over the fixed-ratio schedules. This preference increased in a nonlinear manner with increasing ratio size. This nonlinearity was inconsistent with a theory that states that indifference points for both time and ratio schedules can be predicted by multiplying the choice response-reinforcer intervals of the two types of schedules by different multiplicative constants. Two other theories, which predict nonlinear increases in preference for the matched variable-time schedules, are discussed.     

Token reinforcement, choice, and self-control in pigeons.

Pigeons were exposed to self-control procedures that involved illumination of light-emitting diodes (LEDs) as a form of token reinforcement. In a discrete-trials arrangement, subjects chose between one and three LEDs; each LED was exchangeable for 2-s access to food during distinct posttrial exchange periods. In Experiment 1, subjects generally preferred the immediate presentation of a single LED over the delayed presentation of three LEDs, but differences in the delay to the exchange period between the two options prevented a clear assessment of the relative influence of LED delay and exchange-period delay as determinants of choice. In Experiment 2, in which delays to the exchange period from either alternative were equal in most conditions, all subjects preferred the delayed three LEDs more often than in Experiment-1. In Experiment 3, subjects preferred the option that resulted in a greater amount of food more often if the choices also produced LEDs than if they did not. In Experiment 4, preference for the delayed three LEDs was obtained when delays to the exchange period were equal, but reversed in favor of an immediate single LED when the latter choice also resulted in quicker access to exchange periods. The overall pattern of results suggests that (a) delay to the exchange period is a more critical determinant of choice than is delay to token presentation; (b) tokens may function as conditioned reinforcers, although their discriminative properties may be responsible for the self-control that occurs under token reinforcer arrangements; and (c) previously reported differences in the self-control choices of humans and pigeons may have resulted at least in part from the procedural conventions of using token reinforcers with human subjects and food reinforcers with pigeon subjects.     

Foraging in a simulated natural environment: There's a rat loose in the lab

Rats were required to earn their food in a large room having nine boxes placed in it, each of which contained food buried in sand. In different phases of the experiment the amount of time allowed for foraging, the amount of food available in each food patch, and the location of the different available amounts were varied. The rats exhaustively sampled all patches each session but seemed to have fairly strong preferences for certain locations over others. If position preferences were for patches containing small amounts of food, the sensitivity to amount available was increased so that when location was compensated for, a pattern of optimal foraging was evident. The importance of environmental constraints in producing optimal behavior and the relation of the observed behavior to laboratory findings are discussed.     

OVERJUSTIFICATION EFFECTS IN TOKEN ECONOMIES

This study tested the relevance to clinical token economies of the overjustification hypothesis that tangible reward interferes with intrinsic interest in target behaviors and causes such behaviors to be less probable following a period of reinforcement than preceding such a period. The study was carried out in an ongoing token economy for chronic psychiatric patients. Alternated over an 8-week period were weeks of token and no-token reward for one of the program's target behaviors, toothbrushing. Two different amounts of token reward were employed in order to examine whether reward magnitude might influence the presence or extent of overjustification effects. Little evidence was found for the presence of overjustification effects in token economies. However, maintenance of toothbrushing was greater in no-token weeks following weeks of low amounts of token reward than in no-token weeks following weeks of higher amounts of reward. The importance of such complex functional relationships is discussed.     

Duration and rate of reinforcement as determinants of concurrent responding

The duration and frequency of food presentation were varied in concurrent variable-interval variable-interval schedules of reinforcement. In the first experiment, in which pigeons were exposed to a succession of eight different schedules, neither relative duration nor relative frequency of reinforcement had as great an effect on response distribution as they have when they are manipulated separately. These results supported those previously reported by Todorov (1973) and Schneider (1973). In a second experiment, each of seven pigeons was exposed to only one concurrent schedule in which the frequency and/or duration of reinforcement differed on the two keys. Under these conditions, each pigeon's relative rate of response closely matched the relative total access to food that each schedule provided. This result suggests that previous failures to obtain matching may be due to factors such as an insufficient length of exposure to each schedule or to the pigeons' repeated exposure to different concurrent schedules.     

Producing a change from competition to sharing: effects of large and adjusting response requirements

Pairs of high-school students matched-to-sample for money. On each trial, the first pair member to complete a fixed ratio of knob-pulling responses could work the matching problem on that trial. Competition occurred when both pair members responded for the problem. Sharing occurred when only one pair member responded on each trial, and the subjects alternated trials. Hence, sharing requires less responding and still allows a moderate number of reinforcers for each subject. Recent research has shown that increasing the response requirement to the point that it may have aversive properties will produce a change from competition to sharing. A related variable is an adjusting schedule that adjusts the subjects' response requirements so that their abilities to take reinforcers are equal. In this way, subjects might learn that competition requires more responding but produces no more reinforcers. However, recent research also suggests that competition decreases over sessions without experimental manipulations. Because of this possibility of a time-related variable, ratio size and an adjusting schedule were studied in a group design. Competition did decrease for all groups over sessions, but the large-ratio groups switched from competition to sharing sooner than the low-ratio groups. The adjusting schedule had a similar but smaller effect.     

MORPHINE TOLERANCE AS A FUNCTION OF RATIO SCHEDULE: RESPONSE REQUIREMENT OR UNIT PRICE?

Key pecking by 3 pigeons was maintained by a multiple fixed-ratio 10, fixed-ratio 30, fixed-ratio 90 schedule of food presentation. Components differed with respect to amount of reinforcement, such that the unit price was 10 responses per 1-s access to food. Acute administration of morphine, l-methadone, and cocaine dose-dependently decreased overall response rates in each of the components. When a rate decreasing dose of morphine was administered daily, tolerance, as measured by an increase in the dose that reduced response rates to 50% of control (i.e., the ED50 value), developed in each of the components; however, the degree of tolerance was smallest in the fixed-ratio 90 component (i.e., the ED50 value increased the least). When the l-methadone dose-effect curve was redetermined during the chronic morphine phase, the degree of cross-tolerance conferred to l-methadone was similar across components, suggesting that behavioral variables may not influence the degree of cross-tolerance between opioids. During the chronic phase, the cocaine dose-effect curve shifted to the right for 2 pigeons and to the left for 1 pigeon, which is consistent with predictions based on the lack of pharmacological similarity between morphine and cocaine. When the morphine, l-methadone, and cocaine dose-effect curves were redetermined after chronic morphine administration ended, the morphine and l-methadone ED50s replicated those obtained prior to chronic morphine administration. The morphine data suggest that the fixed-ratio value (i.e., the absolute output) determines the degree of tolerance and not the unit price.     

Transitional and steady-state choice behavior under an adjusting-delay schedule

Twelve rats made repeated choices on an adjusting-delay schedule between a smaller reinforcer (A) that was delivered immediately after a response and a larger reinforcer (B) that was delivered after a delay which increased or decreased by 20% depending on the subject's choices in successive blocks of trials. In two phases of the experiment (100 sessions and 40 sessions), reinforcer sizes were selected which enabled theoretical parameters expressing the rate of delay discounting and sensitivity to reinforcer size to be estimated from the ratio of the indifference delays obtained in the two phases. Indifference delays, calculated from adjusting delays in the last 10 sessions of each phase, were shorter when the sizes of A and B were 14 and 25 μl of a 0.6 M sucrose solution than when they were 25 and 100 μl of the same solution. The ratio of the indifference delays was significantly smaller than that predicted on the basis of an assumed linear relation between reinforcer size and instantaneous reinforcer value, consistent with a previous proposal that this relation may be hyperbolic in form. Estimates of the rate of delay discounting based on the ratio of the two indifference delays (mean, 0.08 s(-1)) were similar to values obtained previously using different intertemporal choice protocols. Estimates of the size-sensitivity parameter (mean 113 μl) were similar to estimates recently derived from performance on progressive-ratio schedules. In both phases of the experiment, adjusting delays in successive blocks of trials were analyzed using the Fourier transform. The power spectrum obtained from individual rats had a dominant frequency that corresponded to a period of oscillation of the adjusting delay between 30 and 100 trial blocks (mean, 78). Power in the dominant frequency band was highest in the early sessions of the first phase and declined with extended training. It is suggested that this experimental protocol may have utility in neurobehavioral studies of intertemporal choice.