Estimation of indifference points with an adjusting-delay procedure.

In a series of conditions, pigeons chose between 1.5 s and 3 s of access to grain, each preceded by some delay. The delay that preceded the small reinforcer was constant throughout a condition. The delay that preceded the large reinforcer was increased or decreased a number of times each session in order to estimate an "indifference point," a delay at which the subject chose each alternative about equally often. The experiment was designed to determine whether variations in any of four features of this adjusting-delay procedure would systematically alter the estimated indifference points. The four features were the total trial duration, the number of center-key responses necessary to begin a trial, the number of choice trials that preceded each change in the adjusting delay, and step size--the size of each increment and decrement in the delay. Manipulation of the first three features had no systematic effects on the indifference points. As step size was increased from 0.5 s to 6 s, within-session variability of the adjusting delay steadily increased, and the 6-s step size produced larger indifference-point estimates for some subjects. The results suggest that, within certain limits, these procedural features can be altered without affecting the indifference-point estimates, but that the use of a large step size can distort the estimates. Some theoretical implications of the relative constancy of indifference points across these procedural variations are discussed.     

Token reinforcement, choice, and self-control in pigeons.

Pigeons were exposed to self-control procedures that involved illumination of light-emitting diodes (LEDs) as a form of token reinforcement. In a discrete-trials arrangement, subjects chose between one and three LEDs; each LED was exchangeable for 2-s access to food during distinct posttrial exchange periods. In Experiment 1, subjects generally preferred the immediate presentation of a single LED over the delayed presentation of three LEDs, but differences in the delay to the exchange period between the two options prevented a clear assessment of the relative influence of LED delay and exchange-period delay as determinants of choice. In Experiment 2, in which delays to the exchange period from either alternative were equal in most conditions, all subjects preferred the delayed three LEDs more often than in Experiment-1. In Experiment 3, subjects preferred the option that resulted in a greater amount of food more often if the choices also produced LEDs than if they did not. In Experiment 4, preference for the delayed three LEDs was obtained when delays to the exchange period were equal, but reversed in favor of an immediate single LED when the latter choice also resulted in quicker access to exchange periods. The overall pattern of results suggests that (a) delay to the exchange period is a more critical determinant of choice than is delay to token presentation; (b) tokens may function as conditioned reinforcers, although their discriminative properties may be responsible for the self-control that occurs under token reinforcer arrangements; and (c) previously reported differences in the self-control choices of humans and pigeons may have resulted at least in part from the procedural conventions of using token reinforcers with human subjects and food reinforcers with pigeon subjects.     

Foraging in a simulated natural environment: There's a rat loose in the lab

Rats were required to earn their food in a large room having nine boxes placed in it, each of which contained food buried in sand. In different phases of the experiment the amount of time allowed for foraging, the amount of food available in each food patch, and the location of the different available amounts were varied. The rats exhaustively sampled all patches each session but seemed to have fairly strong preferences for certain locations over others. If position preferences were for patches containing small amounts of food, the sensitivity to amount available was increased so that when location was compensated for, a pattern of optimal foraging was evident. The importance of environmental constraints in producing optimal behavior and the relation of the observed behavior to laboratory findings are discussed.     

OVERJUSTIFICATION EFFECTS IN TOKEN ECONOMIES

This study tested the relevance to clinical token economies of the overjustification hypothesis that tangible reward interferes with intrinsic interest in target behaviors and causes such behaviors to be less probable following a period of reinforcement than preceding such a period. The study was carried out in an ongoing token economy for chronic psychiatric patients. Alternated over an 8-week period were weeks of token and no-token reward for one of the program's target behaviors, toothbrushing. Two different amounts of token reward were employed in order to examine whether reward magnitude might influence the presence or extent of overjustification effects. Little evidence was found for the presence of overjustification effects in token economies. However, maintenance of toothbrushing was greater in no-token weeks following weeks of low amounts of token reward than in no-token weeks following weeks of higher amounts of reward. The importance of such complex functional relationships is discussed.     

Duration and rate of reinforcement as determinants of concurrent responding

The duration and frequency of food presentation were varied in concurrent variable-interval variable-interval schedules of reinforcement. In the first experiment, in which pigeons were exposed to a succession of eight different schedules, neither relative duration nor relative frequency of reinforcement had as great an effect on response distribution as they have when they are manipulated separately. These results supported those previously reported by Todorov (1973) and Schneider (1973). In a second experiment, each of seven pigeons was exposed to only one concurrent schedule in which the frequency and/or duration of reinforcement differed on the two keys. Under these conditions, each pigeon's relative rate of response closely matched the relative total access to food that each schedule provided. This result suggests that previous failures to obtain matching may be due to factors such as an insufficient length of exposure to each schedule or to the pigeons' repeated exposure to different concurrent schedules.     

MORPHINE TOLERANCE AS A FUNCTION OF RATIO SCHEDULE: RESPONSE REQUIREMENT OR UNIT PRICE?

Key pecking by 3 pigeons was maintained by a multiple fixed-ratio 10, fixed-ratio 30, fixed-ratio 90 schedule of food presentation. Components differed with respect to amount of reinforcement, such that the unit price was 10 responses per 1-s access to food. Acute administration of morphine, l-methadone, and cocaine dose-dependently decreased overall response rates in each of the components. When a rate decreasing dose of morphine was administered daily, tolerance, as measured by an increase in the dose that reduced response rates to 50% of control (i.e., the ED50 value), developed in each of the components; however, the degree of tolerance was smallest in the fixed-ratio 90 component (i.e., the ED50 value increased the least). When the l-methadone dose-effect curve was redetermined during the chronic morphine phase, the degree of cross-tolerance conferred to l-methadone was similar across components, suggesting that behavioral variables may not influence the degree of cross-tolerance between opioids. During the chronic phase, the cocaine dose-effect curve shifted to the right for 2 pigeons and to the left for 1 pigeon, which is consistent with predictions based on the lack of pharmacological similarity between morphine and cocaine. When the morphine, l-methadone, and cocaine dose-effect curves were redetermined after chronic morphine administration ended, the morphine and l-methadone ED50s replicated those obtained prior to chronic morphine administration. The morphine data suggest that the fixed-ratio value (i.e., the absolute output) determines the degree of tolerance and not the unit price.     

Modeling the effect of reward amount on probability discounting

The present study with college students examined the effect of amount on the discounting of probabilistic monetary rewards. A hyperboloid function accurately described the discounting of hypothetical rewards ranging in amount from $20 to $10,000,000. The degree of discounting increased continuously with amount of probabilistic reward. This effect of amount was not due to changes in the rate parameter of the discounting function, but rather was due to increases in the exponent. These results stand in contrast to those observed with the discounting of delayed monetary rewards, in which the degree of discounting decreases with reward amount due to amount-dependent decreases in the rate parameter. Taken together, this pattern of results suggests that delay and probability discounting reflect different underlying mechanisms. That is, the fact that the exponent in the delay discounting function is independent of amount is consistent with a psychophysical scaling interpretation, whereas the finding that the exponent of the probability-discounting function is amount-dependent is inconsistent with such an interpretation. Instead, the present results are consistent with the idea that the probability-discounting function is itself the product of a value function and a weighting function. This idea was first suggested by Kahneman and Tversky (1979), although their prospect theory does not predict amount effects like those observed. The effect of amount on probability discounting was parsimoniously incorporated into our hyperboloid discounting function by assuming that the exponent was proportional to the amount raised to a power. The amount-dependent exponent of the probability-discounting function may be viewed as reflecting the effect of amount on the weighting of the probability with which the reward will be received.     

Stimulus Equivalence: Effects Of A Default-response Option On Emergence Of Untrained Stimulus Relations

Human subjects were exposed to a concurrent-chains schedule in which reinforcer amounts, delays, or both were varied in the terminal links, and consummatory responses were required to receive points that were later exchangeable for money. Two independent variable-interval 30-s schedules were in effect during the initial links, and delay periods were defined by fixed-time schedules. In Experiment 1, subjects were exposed to three different pairs of reinforcer amounts and delays, and sensitivity to reinforcer amount and delay was determined based on the generalized matching law. The relative responding (choice) of most subjects was more sensitive to reinforcer amount than to reinforcer delay. In Experiment 2, subjects chose between immediate smaller reinforcers and delayed larger reinforcers in five conditions with and without timeout periods that followed a shorter delay, in which reinforcer amounts and delays were combined to make different predictions based on local reinforcement density (i.e., points per delay) or overall reinforcement density (i.e., points per total time). In most conditions, subjects' choices were qualitatively in accord with the predictions from the overall reinforcement density calculated by the ratio of reinforcer amount and total time. Therefore, the overall reinforcement density appears to influence the preference of humans in the present self-control choice situation.     

Limits to preference and the sensitivity of choice to rate and amount of food

Studies of choice holding food‐amount ratio constant while varying food‐rate ratio within sessions showed that local changes in preference depend on relative amount of food. The present study investigated whether sensitivity of choice to food‐rate ratio and sensitivity to food‐amount ratio are independent of one another when food‐rate ratios are varied across sessions and food‐amount ratios are varied within sessions. Food deliveries for rats’ presses on the left and right levers were scheduled according to three different food‐rate ratios of 1:1, 9:1, and 1:9; each food‐rate ratio lasted for 106 sessions and was arranged independently of seven food‐amount ratios (7:1, 6:2, 5:3, 4:4, 3:5, 2:6, and 1:7 food pellets) occurring within sessions in random sequence. Each amount ratio lasted for 10 food deliveries and was separated from another by a 60‐s blackout. Sensitivity to rate ratio was high (1.0) across food deliveries. Sensitivity to amount ratio was low when food rates were equal across alternatives, but was high when rate ratio and amount ratio opposed one another. When rate ratio and amount ratio went in the same direction, choice ratio reached an elevenfold limit which reduced sensitivity to approximately zero. We conclude that three factors affect sensitivity to amount: (1) the limit to preference, (2) the equal effect on preference of amounts greater than four pellets, and (3) the absence of differential effects of switches in amount in the equal‐rates (1:1) condition. Taken together, these findings indicate that rate and amount only sometimes combine independently as additive variables to determine preference when amount ratios vary frequently within sessions.