Responding during reinforcement delay in a self-control paradigm.

Eight pigeons chose between a small, immediate reinforcer and a large, increasingly delayed reinforcer. Responding during the large-reinforcer delays was examined. During large-reinforcer delays, pecks on one key produced the small, immediate reinforcer; pecks on the other key had no effect. Thus, a pigeon could reverse its initial choice of the large, delayed reinforcer, or it could maintain its original choice. Pigeons that made a relatively high number of initial large-reinforcer choices tended to maintain these choices, and those pigeons that actually received a relatively high number of large reinforcers, tended to respond more frequently on the ineffective key during the delay periods. The findings suggest that some previous studies of self-control training in pigeons may have resulted in increased self-control partially due to a lack of opportunity for the pigeons to change their choices.     

Self-control and impulsiveness in children and adults: Effects of food preferences

Experiment 1 used 6 preschool boys and Experiment 2 used 6 adult women to explore the effects of food preference on humans' choice in self-control paradigms. The boys showed a higher proportion of responses for more delayed, larger reinforcers (a measure of self-control) when those choices resulted in receipt of the most preferred food compared to when those choices resulted in the least preferred food. Further, the boys chose the less delayed, smaller reinforcers significantly more often when only those choices, as opposed to both choices, resulted in the most preferred food. Conversely, they chose the more delayed, larger reinforcers significantly more often when only those choices, as opposed to both choices, resulted in the most preferred food. Finally, the women demonstrated significantly less sensitivity to reinforcer amount relative to sensitivity to reinforcer delay (another measure of self-control) when they had a higher preference for the juice received as the less delayed, smaller reinforcer than for the juice received as the more delayed, larger reinforcer. Together, the results show that subjects' food preferences can influence self-control for food reinforcers.     

Choice in a self-control paradigm: Quantification of experience-based differences

Previous quantitative models of choice in a self-control paradigm (choice between a larger, more-delayed reinforcer and a smaller, less-delayed reinforcer) have not described individual differences. Two experiments are reported that provide additional quantitative data on experience-based differences in choice between reinforcers of varying sizes and delays. In Experiment 1, seven pigeons in a self-control paradigm were exposed to a fading procedure that increased choices of the larger, more-delayed reinforcer through gradually decreasing the delay to the smaller of two equally delayed reinforcers. Three control subjects, exposed to each of the small-reinforcer delays to which the experimental subjects were exposed, but for fewer sessions, demonstrated that lengthy exposure to each of the conditions in the fading procedure may be necessary in order for the increase to occur. In Experiment 2, pigeons with and without fading-procedure exposure chose between reinforcers of varying sizes and delays scheduled according to a concurrent variable-interval variable-interval schedule. In both experiments, pigeons with fading-procedure exposure were more sensitive to variations in reinforcer size than reinforcer delay when compared with pigeons without this exposure. The data were described by the generalized matching law when the relative size of its exponents, representing subjects' relative sensitivity to reinforcer size and delay, were grouped according to subjects' experience.     

Choice and number of reinforcers

Pigeons were exposed to the concurrent-chains procedure in two experiments designed to investigate the effects of unequal numbers of reinforcers on choice. In Experiment 1, the pigeons were indifferent between long and short durations of access to variable-interval schedules of equal reinforcement density, but preferred a short high-density terminal link over a longer, lower density terminal link, even though in both sets of comparisons there were many more reinforcers per cycle in the longer terminal link. In Experiment 2, the pigeons preferred five reinforcers, the first of which was available after 30 sec, over a single reinforcer available at 30 sec, but only when the local interval between successive reinforcers was short. The pigeons were indifferent when this local interval was sufficiently long. The pigeons' behavior appeared to be under the control of local terminal-link variables, such as the intervals to the first reinforcer and between successive reinforcers, and was not well described in terms of transformed delays of reinforcement or reductions in average delay to reinforcement.     

Token reinforcement, choice, and self-control in pigeons.

Pigeons were exposed to self-control procedures that involved illumination of light-emitting diodes (LEDs) as a form of token reinforcement. In a discrete-trials arrangement, subjects chose between one and three LEDs; each LED was exchangeable for 2-s access to food during distinct posttrial exchange periods. In Experiment 1, subjects generally preferred the immediate presentation of a single LED over the delayed presentation of three LEDs, but differences in the delay to the exchange period between the two options prevented a clear assessment of the relative influence of LED delay and exchange-period delay as determinants of choice. In Experiment 2, in which delays to the exchange period from either alternative were equal in most conditions, all subjects preferred the delayed three LEDs more often than in Experiment-1. In Experiment 3, subjects preferred the option that resulted in a greater amount of food more often if the choices also produced LEDs than if they did not. In Experiment 4, preference for the delayed three LEDs was obtained when delays to the exchange period were equal, but reversed in favor of an immediate single LED when the latter choice also resulted in quicker access to exchange periods. The overall pattern of results suggests that (a) delay to the exchange period is a more critical determinant of choice than is delay to token presentation; (b) tokens may function as conditioned reinforcers, although their discriminative properties may be responsible for the self-control that occurs under token reinforcer arrangements; and (c) previously reported differences in the self-control choices of humans and pigeons may have resulted at least in part from the procedural conventions of using token reinforcers with human subjects and food reinforcers with pigeon subjects.     

Determinants of pigeons' choices in token-based self-control procedures

Four pigeons were exposed to a token-based self-control procedure with stimulus lights serving as token reinforcers. Smaller-reinforcer choices produced one token immediately; larger-reinforcer choices produced three tokens following a delay. Each token could be exchanged for 2-s access to food during a signaled exchange period each trial. The main variables of interest were the exchange delays (delays from the choice to the exchange stimulus) and the food delays (also timed from the choice), which were varied separately and together across blocks of sessions. When exchange delays and food delays were shorter following smaller-reinforcer choices, strong preference for the smaller reinforcer was observed. When exchange delays and food delays were equal for both options, strong preference for the larger reinforcer was observed. When food delays were equal for both options but exchange delays were shorter for smaller-reinforcer choices, preference for the larger reinforcer generally was less extreme than under conditions in which both exchange and food delays were equal. When exchange delays were equal for both options but food delays were shorter for smaller-reinforcer choices, preference for the smaller reinforcer generally was less extreme than under conditions in which both exchange and food delays favored smaller-reinforcer choices. On the whole, the results were consistent with prior research on token-based self-control procedures in showing that choices are governed by reinforcer immediacy when exchange and food delays are unequal and by reinforcer amount when exchange and food delays are equal. Further, by decoupling the exchange delays from food delays, the results tentatively support a role for the exchange stimulus as a conditioned reinforcer.     

Failure to produce response variability with reinforcement

Two experiments attempted to train pigeons to produce variable response sequences. In the first, naive pigeons were exposed to a procedure requiring four pecks on each of two keys in any order, with a reinforcer delivered only if a given sequence was different from the preceding one. In the second experiment, the same pigeons were exposed to this procedure after having been trained successfully to alternate between two specific response sequences. In neither case did any pigeon produce more than a few different sequences or obtain more than 50% of the possible reinforcers. Stereotyped sequences developed even though stereotypy was not reinforced. It is suggested that reinforcers have both hedonic and informative properties and that the hedonic properties are responsible for sterotyped repetition of reinforced responses, even when stereotypy is negatively related to reinforcer delivery.     

Choice with delayed and probabilistic reinforcers: effects of variability, time between trials, and conditioned reinforcers.

In a discrete-trials procedure with pigeons, a response on a green key led to a 4-s delay (during which green houselights were lit) and then a reinforcer might or might not be delivered. A response on a red key led to a delay of adjustable duration (during which red houselights were lit) and then a certain reinforcer. The delay was adjusted so as to estimate an indifference point--a duration for which the two alternatives were equally preferred. Once the green key was chosen, a subject had to continue to respond on the green key until a reinforcer was delivered. Each response on the green key, plus the 4-s delay that followed every response, was called one "link" of the green-key schedule. Subjects showed much greater preference for the green key when the number of links before reinforcement was variable (averaging four) than when it was fixed (always exactly four). These findings are consistent with the view that probabilistic reinforcers are analogous to reinforcers delivered after variable delays. When successive links were separated by 4-s or 8-s "interlink intervals" with white houselights, preference for the probabilistic alternative decreased somewhat for 2 subjects but was unaffected for the other 2 subjects. When the interlink intervals had the same green houselights that were present during the 4-s delays, preference for the green key decreased substantially for all subjects. These results provided mixed support for the view that preference for a probabilistic reinforcer is inversely related to the duration of conditioned reinforcers that precede the delivery of food.     

Choice in a "self-control" paradigm: effects of a fading procedure

Pigeons chose between an immediate 2-second reinforcer (access to grain) and a 6-second reinforcer delayed 6 seconds. The four pigeons in the control group were exposed to this condition initially. The four experimental subjects first received a condition where both reinforcers were delayed 6 seconds. The small reinforcer delay was then gradually reduced to zero over more than 11,000 trials. Control subjects almost never chose the large delayed reinforcer. Experimental subjects chose the large delayed reinforcer significantly more often. Two experimental subjects showed preference for the large reinforcer even when the consequences for pecking the two keys were switched. The results indicate that fading procedures can lead to increased “self-control” in pigeons in a choice between a large delayed reinforcer and a small immediate reinforcer.     

SAVING THE BEST FOR LAST? A CROSS-SPECIES ANALYSIS OF CHOICES BETWEEN REINFORCER SEQUENCES

Two experiments were conducted to compare choices between sequences of reinforcers in pigeon (Experiment 1) and human (Experiment 2) subjects, using functionally analogous procedures. The subjects made pairwise choices among 3 sequence types, all of which provided the same overall reinforcerment rate, but differed in their temporal patterning. Token reinforcement schedules were used in both experiments and the type of exchange schedule varied across blocks of sessions. Some conditions permitted immediate exchange of tokens for consumable reinforcers (food for pigeons, video access for humans); in other conditions, tokens accumulated and were exchanged for consumable reinforcers only at the end of the sequence. Choice patterns in the immediate-exchange conditions were generally similar across species, with both pigeons and humans preferring sequences with the shortest delay to the initial reinforcer in the series. The results are broadly consistent with models of temporal discounting expanded to include the impact of sequences of delayed reinforcers acting in parallel from the time of the choice. Preferences were less consistent with discounting models in the delayed exchange conditions. Questionnaire data gathered at the end of the experiment were consistent with prior results of questionnaire studies, but showed no straightforward relation to the observed choice patterns, urging caution in the extrapolation of results from one decision-making domain to the other.     

Soft commitment: self-control achieved by response persistence.

With reinforcement contingent on a single peck on either of two available keys (concurrent continuous reinforcement schedules) 4 pigeons, at 80% of free-feeding weights, preferred a smaller-sooner reinforcer (2.5 s of mixed grain preceded by a 0.5-s delay) to a larger-later reinforcer (4.5 s of mixed grain preceded by a 3.5-s delay). However, when the smaller-sooner and larger-later reinforcers were contingent on a concurrent fixed-ratio 31 schedule (the first 30 pecks distributed in any way on the two keys), all pigeons obtained the larger-later reinforcer much more often than they did when only a single peck was required. This "self-control" was achieved by beginning to peck the key leading to the larger-later reinforcer and persisting on that key until reinforcement occurred. We call this persistence "soft commitment" to distinguish it from strict commitment, in which self-control is achieved by preventing changeovers. Soft commitment also effectively achieved self-control when a brief (1-s) signal was inserted between the 30th and 31st response of the ratio and with concurrent fixed-interval 30-s schedules (rather than ratio schedules) of reinforcement. In a second experiment with the same subjects, the fixed ratio was interrupted by darkening both keys and lighting a third (center) key on which pecking was required for various fractions of the fixed-ratio count. The interruption significantly reduced self-control. When interruption was complete (30 responses on the center key followed by a single choice response), pigeons chose the smaller-sooner reinforcer as frequently as they did when only a single choice response was required.     

On the effects of signaling reinforcer probability and magnitude in delayed matching to sample

Two experiments examined whether postsample signals of reinforcer probability or magnitude affected the accuracy of delayed matching to sample in pigeons. On each trial, red or green choice responses that matched red or green stimuli seen shortly before a variable retention interval were reinforced with wheat access. In Experiment 1, the reinforcer probability was either 0.2 or 1.0 for both red and green responses. Reinforcer probability was signaled by line or cross symbols that appeared after the sample had been presented. In Experiment 2, all correct responses were reinforced, and the signaled reinforcer durations were 1.0 s and 4.5 s. Matching was more accurate when larger or more probable reinforcers were signaled, independently of retention interval duration. Because signals were presented postsample, the effects were not the result of differential attention to the sample.     

AN ANALYSIS OF REINFORCER SUBSTITUTABILITY USING OBJECT MANIPULATION AND SELF-INJURY AS COMPETING RESPONSES

The concept of reinforcer substitutability proposes a continuum of interactions among reinforcers in a given situation. At one end of this continuum, reinforcers are substitutable, with one reinforcer being readily traded for another. We conducted an analysis of reinforcers that were substitutable with those produced by self-injurious behavior (SIB). Three individuals with profound developmental disabilities, whose SIB appeared to be maintained by automatic reinforcement, participated. Results of three experiments showed that (a) object manipulation and SIB were inversely related when leisure materials and SIB were concurrently available, with participants showing almost complete preference for object manipulation; (b) attempts to reduce SIB using the preferred objects as reinforcers in differential reinforcement contingencies were unsuccessful for all 3 participants; and (c) participants' preferences for SIB or object manipulation systematically changed when reinforcer cost (the amount of effort required to obtain the object) was varied. Results of the three experiments illustrate the importance of examining interactions among concurrently available reinforcers when conducting reinforcer assessments.     

Self-control: teaching tolerance for delay in impulsive children.

We investigated a procedure to increase the selection of larger, more delayed reinforcers (i.e., more advantageous in the long run) over smaller immediate reinforcers, in an effort to increase a key aspect of self-control in children. Six preschoolers, including one comparison subject, identified by their teachers as impulsive, were preassessed and found consistently to select smaller immediate reinforcers over larger, more delayed ones. The teaching procedure consisted of gradually increasing the durations of the delay interval over many sessions. The follow-up assessments showed that 5 of these children increased the proportion of their choices of the delayed reinforcers. Before training, indifference points ranged from 1.7 to 51.7 s; following treatment, points rose to a range of 37.5 to at least 90 s, with 3 children preferring the larger reinforcer at all delay intervals tested. The results demonstrated the feasibility of teaching young children to make choices more advantageous to them in the long run.     

Self-Control and Impulsiveness in Adult Human Females: Effects of Food Preferences

Two experiments explored the effects of food preferences on humans’ choice in a self-control paradigm. Reinforcers consisted of programmed periods of access time to drinking juice during the experimental session. Experiment 1 used 14 adult women and Experiment 2 used 9 adult women. In Experiment 1, the women demonstrated significantly less sensitivity to reinforcer amount relative to sensitivity to reinforcer delay (a measure of self-control) when they reported a higher preference for the juice received as the less delayed, smaller reinforcer than for the juice received as the more delayed, larger reinforcer. Conversely in Experiment 2, the women demonstrated significantly more sensitivity to reinforcer amount relative to sensitivity to reinforcer delay when they reported a higher preference for the juice received as the more delayed, larger reinforcer than for the juice received as the less delayed, smaller reinforcer. Together, the results show that participants’ food preferences can influence self-control for food reinforcers.     

Teaching self-control to small groups of dually diagnosed adults

The present study examined the use of a progressive delay procedure to teach self-control to two groups of dually diagnosed adults. When given a choice between an immediate smaller reinforcer and a larger delayed reinforcer, both groups chose the smaller reinforcer during baseline. During treatment, progressive increases in work requirements for gaining access to a larger reinforcer resulted in both groups selecting larger delayed reinforcers. The results are discussed with respect to increasing cooperative work behavior and self-control.     

Choice and multiple reinforcers

Pigeons chose between equivalent two-component mixed and multiple terminal-link schedules of reinforcement in the concurrent-chains procedure. The pigeons preferred the multiple schedule over the mixed when the components of the compound schedules were differentiated in terms of density of reinforcement, but the pigeons were indifferent when the components were differentiated in terms of number of reinforcers per cycle. Taken together, these results indicate that a local variable, the interval to the first reinforcer, but not a molar variable, the number of reinforcers, was sufficient to differentiate the components and thereby evoke preference.     

Fixed-interval performance and self-control in children.

Operant responses of 16 children (mean age 6 years and 1 month) were reinforced according to different fixed-interval schedules (with interreinforcer intervals of 20, 30, or 40 s) in which the reinforcers were either 20-s or 40-s presentations of a cartoon. In another procedure, they received training on a self-control paradigm in which both reinforcer delay (0.5 s or 40 s) and reinforcer duration (20 s or 40 s of cartoons) varied, and subjects were offered a choice between various combinations of delay and duration. Individual differences in behavior under the self-control procedure were precisely mirrored by individual differences under the fixed-interval schedule. Children who chose the smaller immediate reinforcer on the self-control procedure (impulsive) produced short postreinforcement pauses and high response rates in the fixed-interval conditions, and both measures changed little with changes in fixed-interval value. Conversely, children who chose the larger delayed reinforcer in the self-control condition (the self-controlled subjects) exhibited lower response rates and long postreinforcement pauses, which changed systematically with changes in the interval, in their fixed-interval performances.     

A comparison of delays and ratio requirements in self-control choice.

In a discrete-trial procedure, pigeons could choose between 2-s and 6-s access to grain by making a single key peck. In Phase 1, the pigeons obtained both reinforcers by responding on fixed-ratio schedules. In Phase 2, they received both reinforcers after simple delays, arranged by fixed-time schedules, during which no responses were required. In Phase 3, the 2-s reinforcer was available through a fixed-time schedule and the 6-s reinforcer was available through a fixed-ratio schedule. In all conditions, the size of the delay or ratio leading to the 6-s reinforcer was systematically increased or decreased several times each session, permitting estimation of an "indifference point," the schedule size at which a subject chose each alternative equally often. By varying the size of the schedule for the 2-s reinforcer across conditions, several such indifference points were obtained from both fixed-time conditions and fixed-ratio conditions. The resulting "indifference curves" from fixed-time conditions and from fixed-ratio conditions were similar in shape, and they suggested that a hyperbolic equation describes the relation between ratio size and reinforcement value as well as the relation between reinforcer delay and its reinforcement value. The results from Phase 3 showed that subjects chose fixed-time schedules over fixed-ratio schedules that generated the same average times between a choice response and reinforcement.