Human choice in "counterintuitive" situations: fixed- versus progressive-ratio schedules.

College undergraduates were given repeated opportunities to choose between a fixed-ratio and a progressive-ratio schedule of reinforcement. Completions of a progressive-ratio schedule produced points (exchangeable for money) and incremented that response requirement by 20 responses with each consecutive choice. In the reset condition, completion of a fixed ratio produced the same number of points and also reset the progressive ratio back to its initial value. In the no-reset condition, the progressive ratio continued to increase by increments of 20 throughout the session with each successive selection of this schedule, irrespective of fixed-ratio choices. Subjects' schedule choices were sensitive to parametric manipulations of the size of the fixed-ratio schedule and were consistent with predictions made on the basis of minimizing the number of responses emitted per point earned, which is a principle of most optimality theories. Also, the present results suggest that if data from human performances are to be compared with results for other species, humans should be exposed to schedules of reinforcement for long periods of time, as is commonly done with nonhuman subjects.     

Choice in situations of time-based diminishing returns: immediate versus delayed consequences of action.

Pigeons chose between two schedules of food presentation, a fixed-interval schedule and a progressive-interval schedule that began at 0 s and increased by 20 s with each food delivery provided by that schedule. Choosing one schedule disabled the alternate schedule and stimuli until the requirements of the chosen schedule were satisfied, at which point both schedules were again made available. Fixed-interval duration remained constant within individual sessions but varied across conditions. Under reset conditions, completing the fixed-interval schedule not only produced food but also reset the progressive interval to its minimum. Blocks of sessions under the reset procedure were interspersed with sessions under a no-reset procedure, in which the progressive schedule value increased independent of fixed-interval choices. Median points of switching from the progressive to the fixed schedule varied systematically with fixed-interval value, and were consistently lower during reset than during no-reset conditions. Under the latter, each subject's choices of the progressive-interval schedule persisted beyond the point at which its requirements equaled those of the fixed-interval schedule at all but the highest fixed-interval value. Under the reset procedure, switching occurred at or prior to that equality point. These results qualitatively confirm molar analyses of schedule preference and some versions of optimality theory, but they are more adequately characterized by a model of schedule preference based on the cumulated values of multiple reinforcers, weighted in inverse proportion to the delay between the choice and each successive reinforcer.     

Pigeons' choices in situations of diminishing returns: fixed- versus progressive-ratio schedules.

In two different discrete-trial procedures, pigeons were faced with choices between fixed-ratio and progressive-ratio schedules. The latter schedules entail diminishing returns, a feature analogous to foraging situations in the wild. In the first condition (no reset), subjects chose between a progressive-ratio schedule that increased in increments of 20 throughout a session and a fixed-ratio schedule that was constant across blocks of sessions. The size of the fixed ratio was varied parametrically through an ascending and then a descending series. In the reset condition, the same fixed-ratio values were used, but each selection (and completion) of the fixed ratio reset the progressive-ratio schedule back to its minimal value. In the no-reset procedure, the pigeons tended to cease selecting the progressive ratio when it equaled or slightly exceeded the fixed-ratio value, whereas in reset, they chose the fixed ratio well in advance of that equality point. These results indicate sensitivity to molar as well as to molecular reinforcement rates, and those molar relationships are similar to predictions based on the marginal value theorem of optimal foraging theory (e.g., Charnov, 1976). However, although previous results with monkeys (Hineline & Sodetz, 1987) appeared to minimize responses per reinforcement, the present results corresponded more closely to predictions based on sums-of-reciprocals of distance from point of choice to each of the next four reinforcers. Results obtained by Hodos and Trumbule (1967) with chimpanzees in a similar procedure were intermediate between these two relationships. Variability of choices, as well as median choice points, differed between the reset and no-reset conditions.(ABSTRACT TRUNCATED AT 250 WORDS)     

Instructions, multiple schedules, and extinction: Distinguishing rule-governed from schedule-controlled behavior

Schedule sensitivity has usually been examined either through a multiple schedule or through changes in schedules after steady-state responding has been established. This study compared the effects of these two procedures when various instructions were given. Fifty-five college students responded in two 32-min sessions under a multiple fixed-ratio 18/differential-reinforcement-of-low-rate 6-s schedule, followed by one session of extinction. Some subjects received no instructions regarding the appropriate rates of responding, whereas others received instructions to respond slowly, rapidly, or both. Relative to the schedule in operation, the instructions were minimal, partially inaccurate, or accurate. When there was little schedule sensitivity in the multiple schedule, there was little in extinction. When apparently schedule-sensitive responding occurred in the multiple schedule, however, sensitivity in extinction occurred only if differential responding in the multiple schedule could not be due to rules supplied by the experimenter. This evidence shows that rule-governed behavior that occurs in the form of schedule-sensitive behavior may not in fact become schedule-sensitive even though it makes contact with the scheduled reinforcers.     

Base rates versus sample accuracy: competition for control in human matching to sample

People often place undue weight on specific sources of information (case cues) and insufficient weight on more global sources (base rates) even when the latter are highly predictive, a phenomenon termed base-rate neglect. This phenomenon was first demonstrated with paper-and-pencil tasks, and also occurs in a matching-to-sample procedure in which subjects directly experience case sample (cue) accuracy and base rates, and in which discrete, nonverbal choices are made. In two nonverbal experiments, subjects were exposed to hundreds of trials in which they chose between two response options that were both probabilistically reinforced. In Experiment 1, following one of two possible samples (the unpredictive sample), either response was reinforced with a .5 probability. The other sample (predictive) provided reinforcement for matching on 80% of the trials in one condition but in only 20% of the trials in another condition. Subjects' choices following the unpredictive sample were determined primarily by the contingencies in effect for the predictive sample: If matching was reinforced following the predictive sample, subjects tended to match the unpredictive sample as well; if countermatching the predictive sample was generally reinforced, subjects tended to countermatch the unpredictive sample. These results demonstrate only weak control by base rates. In Experiment 2, base rates and sample accuracy were simultaneously varied in opposite directions to keep one set of conditional probabilities constant. Subjects' choices were determined primarily by the overall accuracy of the sample, again demonstrating only weak control by base rates. The same pattern of choice occurred whether this pattern increased or decreased rate of reinforcement. Together, the results of the two experiments provide a clear empirical demonstration of base-rate neglect.     

Effects of self-generated rules on the development of schedule-controlled behavior.

College students responded under a multiple differential-reinforcement-of-low-rate 5-s fixed-ratio 8 schedule, with components alternating every 2 min. After 40 programmed minutes of acquisition and 12 min of maintenance, without notice, both schedules changed to extinction for 28 min. During acquisition, between alternations of the multiple schedule, some subjects were asked to develop rules describing the schedule contingencies. Other subjects were given these same rules between alternations, and a third group neither received nor were asked to develop rules. By the end of the acquisition phase, self-generated-rule subjects were more likely to show schedule-typical behavior than were subjects not asked to generate rules. The behavior of those given rules was similar to those asked to generate rules at the end of acquisition, but yoked-rule subjects acquired schedule-typical behavior at a quicker rate. By the end of extinction, during the period corresponding to the previous fixed-ratio interval, all no-rule subjects who had earned points during acquisition and maintenance were responding at a rate of less than 30 responses per minute. Only 3 of the 9 self-generated-rule subjects and 2 of the 5 yoked-rule subjects were similarly responding at this low rate. Results suggest that asking subjects to develop self-rules facilitates acquisition, but can retard extinction. Results also suggest that self-generated rules function similarly to external rules.     

Effects of social context, reinforcer probability, and reinforcer magnitude on humans' choices to compete or not to compete.

In the first two experiments, subjects' choices to earn points (exchangeable for money) either by competing with a fictitious opponent or by not competing were studied. Buskist, Barry, Morgan, and Rossi's (1984) competitive fixed-interval schedule was modified to include a second response option, a noncompetitive fixed-interval schedule. After choosing to enter either option, the opportunity for reinforcers became available after the fixed-interval's duration had elapsed. Under the no-competition condition, points were always available after the interval had elapsed. Under the competition condition, points were available based on a predetermined probability of delivery. Experiments 1 and 2 examined how reinforcer probabilities and reinforcer magnitudes affected subjects' choices to compete. Several general conclusions can be made about the results: (a) Strong preferences to compete were observed at high and moderate reinforcer probabilities; (b) competing was observed even at very low reinforcer probabilities; (c) response rates were always higher in the competition component than in the no-competition component; and (d) response rates and choices to compete were insensitive to reinforcer-magnitude manipulations. In Experiment 3, the social context of this choice schedule was removed to determine whether the high levels of competing observed in the first two experiments were due to a response preference engendered by the social context provided by the experimenters through instructions. In contrast to the first two experiments, these subjects preferred the 60-s fixed-interval schedule (formerly the no-competition option), indicating that the instructions themselves were responsible for the preference to compete. This choice paradigm may be useful to future researchers interested in the effects of other independent variables (e.g., drugs, social context, instructions) on competitive behavior.     

Clock control of human performance on avoidance and fixed-interval schedules

The avoidance and fixed-interval performances of human subjects were studied in two experiments. Addition of time-correlated stimuli (added clock) improved behavioral efficiency, since response rates decreased without decreases in reinforcement rates. Response-dependent display of the clock maintained a second, observing response and reductions in clock duration weakened such observing behavior. Generally, the reinforcing properties of the clock were more apparent with the avoidance than with the fixed-interval schedule, a finding attributed to temporal cues already provided by delivery of the fixed-interval reinforcers. Reduced rates of the main response when the clock was dependent on an observing response were more than offset by rates of the observing response in the majority of subjects. Thus, the results do not support an interpretation of the reinforcing properties of added clocks simply in terms of work reduction.     

Risk-sensitive choice in humans as a function of an earnings budget

Risky choice in 3 adult humans was investigated across procedural manipulations designed to model energy-budget manipulations conducted with nonhumans. Subjects were presented with repeated choices between a fixed and a variable number of points. An energy budget was simulated by use of an earnings budget, defined as the number of points needed within a block of trials for points to be exchanged for money. During positive earnings-budget conditions, exclusive preference for the fixed option met the earnings requirement. During negative earnings-budget conditions, exclusive preference for the certain option did not meet the earnings requirement, but choice for the variable option met the requirement probabilistically. Choice was generally risk averse (the fixed option was preferred) when the earnings budget was positive and risk prone (the variable option was preferred) when the earnings budget was negative. Furthermore, choice was most risk prone during negative earnings-budget conditions in which the earnings requirement was most stringent. Local choice patterns were also frequently consistent with the predictions of a dynamic optimization model, indicating that choice was simultaneously sensitive to short-term choice contingencies, current point earnings, and the earnings requirement. Overall, these results show that the patterns of risky choice generated by energy-budget variables can also be produced by choice contingencies that do not involve immediate survival, and that risky choice in humans may be similar to that shown in nonhumans when choice is studied under analogous experimental conditions.     

Determinants of pigeons' choices in token-based self-control procedures

Four pigeons were exposed to a token-based self-control procedure with stimulus lights serving as token reinforcers. Smaller-reinforcer choices produced one token immediately; larger-reinforcer choices produced three tokens following a delay. Each token could be exchanged for 2-s access to food during a signaled exchange period each trial. The main variables of interest were the exchange delays (delays from the choice to the exchange stimulus) and the food delays (also timed from the choice), which were varied separately and together across blocks of sessions. When exchange delays and food delays were shorter following smaller-reinforcer choices, strong preference for the smaller reinforcer was observed. When exchange delays and food delays were equal for both options, strong preference for the larger reinforcer was observed. When food delays were equal for both options but exchange delays were shorter for smaller-reinforcer choices, preference for the larger reinforcer generally was less extreme than under conditions in which both exchange and food delays were equal. When exchange delays were equal for both options but food delays were shorter for smaller-reinforcer choices, preference for the smaller reinforcer generally was less extreme than under conditions in which both exchange and food delays favored smaller-reinforcer choices. On the whole, the results were consistent with prior research on token-based self-control procedures in showing that choices are governed by reinforcer immediacy when exchange and food delays are unequal and by reinforcer amount when exchange and food delays are equal. Further, by decoupling the exchange delays from food delays, the results tentatively support a role for the exchange stimulus as a conditioned reinforcer.     

Choice for aperiodic versus periodic ratio schedules: A comparison of concurrent and concurrent-chain procedures

Choice between mixed-ratio schedules, consisting of equiprobable ratios of 1 and 99 responses per reinforcement, and fixed-ratio schedules of food reinforcement was assessed by two commonly used procedures: concurrent schedules and concurrent-chains schedules. Rats were trained under concurrent fixed-ratio mixed-ratio schedules, in which both ratio schedules were simultaneously available, and under a concurrent-chains schedule, in which access to one of the mutually exclusive ratio schedules comprising the terminal links was contingent on a single “choice” response. The distribution of responses between the two ratio schedules was taken as the choice proportion under the concurrent procedure, and the distribution of “choice” responses was taken as the choice proportion under the concurrent-chains procedure. Seven of eight rats displayed systematic choice; of those, each displayed nearly exclusive choice for fixed-ratio 35 to the mixed-ratio schedule under the concurrent procedure, but each displayed nearly exclusive choice for the mixed-ratio schedule to fixed-ratio 35 under the concurrent-chains procedure. Thus, preference for a fixed or a mixed schedule of reinforcement depended on the procedure used to assess preference.     

Determinants of human performance on concurrent schedules.

Six experiments, each with 5 human adults, were conducted to investigate the determinants of human performance on multiple concurrent variable-interval schedules. A two-key procedure was employed in which subjects' key presses produced points exchangeable for money. Variables manipulated across experiments were (a) changeover delay (Experiments 2, 4, and 6), (b) ordinal cues related to scheduled reinforcement frequencies (Experiments 3 and 4), and (c) instructions describing the ordinal relations between schedule-correlated stimuli and scheduled reinforcement frequency (Experiments 5 and 6). The performances of only 13 of the 30 subjects could be described by the generalized matching equation and were within a range of values typical of those reported in the animal literature. Eight subjects showed indifference, 9 undermatched, 7 approximated matching, 3 overmatched, and a further 3 responded exclusively to the richer component of the concurrent schedules. These differing modes of responding were closely related to the different types of performance rules reported by subjects in postexperimental questionnaires. The results are in good agreement with those from studies of human performance on single schedules, suggesting that rule-governed behavior, in interaction with contingencies, may be an important determinant of human choice.     

Rule-governed behavior and sensitivity to changing consequences of responding

Humans were presented with a task that required moving a light through a matrix. Button presses could produce light movements according to a multiple fixed-ratio 18/differential-reinforcement-of-low-rate 6-s schedule, with components alternating every 2 min. Moving the light through the maze earned points worth chances on money prizes. In Experiment 1 four conditions were assessed through between-subject comparisons: minimal instructions, instructions to press rapidly, instructions to press slowly, and instructions that sometimes rapid responding would work while at other times a slow rate would work best. Subjects responded in three successive sessions of 32 min each. The results suggested that instructions affected the nature of the contact made with the programmed consequences and thus subsequent performance. In some cases, responding seemed to result from added contingencies introduced by stating rules. In Experiment 2 the relative contribution of these two effects was assessed by presenting and then withdrawing two lights that had been paired with two specific instructions: “Go Fast” or “Go Slow.” There were three conditions. In one condition, only the Go Fast light was on; in a second, only the Go Slow light was on; and in a third, the lights alternated each minute. In each condition, half the subjects had all instruction lights turned off after the first session. The results once again showed an effect of instructions on contact with the programmed consequences. However, responding sometimes continued in a manner consistent with added contingencies for rule-following even when the programmed consequences had been contacted and would have controlled a different type of responding in the absence of instructions. The relevance of added contingencies for rule-following in determining the effects of explicitly programmed consequences is emphasized.     

Choice behavior of rats in a concurrent-chains schedule: Amount and delay of reinforcement

Rats were exposed to concurrent-chains schedules in which the terminal links were equal fixed-interval schedules terminating in one or three food pellets. Choice proportions for large reward increased with increases in delay intervals programmed on fixed-interval schedules and supported the predictions derived from a general choice model originally formulated by Fantino and later developed by Navarick and Fantino. In addition, a functional equivalence of two alternatives was established by increasing delay intervals with large reward, whereas delay intervals for small reward were held constant. Functionally equivalent delay intervals with large reward, for each delay interval with small reward, can be described by a power function with exponent smaller than 1.0. A better prediction of choice proportions resulted when this function was used to derive predicted choice proportions.     

Tests of transitivity in choices between fixed and variable reinforcer delays.

This experiment tested for transitivity in pigeons' choices between variable-time (VT) and fixed-time (FT) schedules. In a discrete-trials procedure, a subject chose between two alternatives by making a single key peck. Each choice was between a "standard alternative," which was the same schedule throughout a condition, and an "adjusting alternative," in which the delay to reinforcement was systematically increased or decreased many times a session. These adjustments enabled an approximate indifference point to be identified--the value of the adjusting delay at which the subject chose each alternative about equally often. Each test of transitivity involved four conditions. In one, the standard alternative was a variable-time schedule with a 2-s reinforcer, and the adjusting alternative also delivered a 2-s reinforcer. A second condition was similar except that the adjusting alternative delivered a 5-s reinforcer. The indifference point from each of these conditions was then converted to a fixed-time schedule for subsequent comparisons in the third and fourth conditions, respectively. Each of these last two conditions compared one of the fixed-time schedules (based upon the previous conditions and including their different reinforcer durations) with an adjusting schedule that delivered the alternative reinforcer duration, to determine whether the obtained indifference points would be those predicted from the prior alternative-duration comparisons with the VT schedule. There was little evidence for intransitivity of choice: Averaged across subjects and replications, the obtained indifference points deviated from perfect transitivity by less than 8%, and these deviations were not statistically significant. These results contrast with those of Navarick and Fantino (1972), who found frequent violations of transitivity between periodic and aperiodic schedules using a concurrent-chains procedure with variable-interval schedules in the initial links.     

Instructional effects on performance in a matching-to-sample study

Conducting studies using an undergraduate participant pool is fraught with difficulties. Among them are problems with adequately motivating subjects both to come to the study, and once there, to actively engage the experimental task. Thirty-one college students participated in a matching-to-sample (MTS) study involving substantial training, testing, retraining, and retesting of conditional discriminations and equivalence relations among four 4-member classes of nonsensical words. The study was conducted during the end of the semester, when performance often had been observed to be poorer than at other points in the semester. Eleven of the participants, in addition to standard instructions about the task, received additional instructions specifying molar consequences for high rates of "correct" responses throughout the procedure. This subset of participants displayed markedly improved performance as compared to those who did not receive the additional instructions. Results suggest that specification of molar contingencies improves participants' sensitivity to molecular contingencies within the study. Instructions that specify and increase the consequential functions of feedback provided during MTS trials may be one means of reducing unwanted variability in human MTS performance.