Enhancing tolerance to delayed reinforcers: the role of intervening activities

Three participants with moderate to profound mental retardation were exposed to choices between an immediate small amount and a delayed larger amount of a preferred reinforcer. All participants initially showed a relatively high preference for the smaller reinforcer, suggesting impulsive choice making. However this preference reversed, suggesting self-control, when the larger reinforcer was available immediately and, over time, its delay was gradually increased. Results highlight the potential utility of incorporating concurrent activities into self-control training paradigms.     

A procedure to teach self-control to children with attention deficit hyperactivity disorder

This study examined the use of a progressive delay procedure combined with verbal mediation to teach self-control to children with attention deficit disorder. Results showed that when participants were initially given the choice between an immediate smaller reinforcer and a larger delayed reinforcer, all participants chose the smaller reinforcer. When slight delays to obtain a larger reinforcer were instated in conjunction with intervening verbal activity, all participants demonstrated self-control regardless of the content of the verbal activity.     

THE EFFECTS OF CHOICE ON SELF‐CONTROL

Three adolescents with traumatic brain injury performed a physical therapy task in the absence of programmed consequences or duration requirements. Next, the experimenter gave the participants the options of a smaller immediate reinforcer with no response requirement or a larger delayed reinforcer with a response requirement. Self‐control training exposed participants to a procedure during which they chose between a smaller immediate reinforcer and a progressively increasing delayed reinforcer whose values varied and were determined by a die roll. The participants chose whether they or the experimenter rolled the die. All participants initially demonstrated low baseline durations of the physical therapy task, chose the smaller immediate reinforcer during the choice baseline, and changed their preference to the larger delayed reinforcer during self‐control training.     

Teaching self-control to small groups of dually diagnosed adults

The present study examined the use of a progressive delay procedure to teach self-control to two groups of dually diagnosed adults. When given a choice between an immediate smaller reinforcer and a larger delayed reinforcer, both groups chose the smaller reinforcer during baseline. During treatment, progressive increases in work requirements for gaining access to a larger reinforcer resulted in both groups selecting larger delayed reinforcers. The results are discussed with respect to increasing cooperative work behavior and self-control.     

Self-control in mentally retarded adolescents: choice as a function of amount and delay of reinforcement.

Three severely mentally retarded adolescents were studied under discrete-trial procedures in which a choice was arranged between edible reinforcers that differed in magnitude and, in some conditions, delay. In the absence of delays the larger reinforcer was consistently chosen. Under conditions in which the smaller reinforcer was not delayed, increasing the delay to delivery of the larger reinforcer decreased the percentage of trials in which that reinforcer was chosen. All subjects directed the majority of choice responses to the smaller reinforcer when the larger reinforcer was sufficiently delayed, although the value at which this occurred differed across subjects. Under conditions in which the larger reinforcer initially was sufficiently delayed to result in preference for the smaller one, progressively increasing in 5-s increments the delay to both reinforcers increased percentage of trials with the larger reinforcer chosen. At sufficiently long delays, 2 of the subjects consistently chose the larger, but more delayed, reinforcer, and the 3rd subject chose that reinforcer on half of the trials. These results are consistent with the findings of prior studies in which adult humans responded to terminate noise and pigeons responded to produce food.     

Self-control in children with autism: response allocation during delays to reinforcement.

This study examined the use of a progressive‐delay schedule of reinforcement to increase self‐control and decrease disruptive behavior in children with autism. When initially given the choice between an immediate smaller reinforcer and a larger delayed reinforcer, all participants chose the smaller reinforcer. When access to the larger reinforcer required either no activity or engaging in a concurrent task during the delay, all participants demonstrated both self‐control and preference for a response requirement. Disruptive behavior decreased during delays that required a concurrent task compared to sessions without an activity requirement.     

Self-Control and Impulsiveness in Adult Human Females: Effects of Food Preferences

Two experiments explored the effects of food preferences on humans’ choice in a self-control paradigm. Reinforcers consisted of programmed periods of access time to drinking juice during the experimental session. Experiment 1 used 14 adult women and Experiment 2 used 9 adult women. In Experiment 1, the women demonstrated significantly less sensitivity to reinforcer amount relative to sensitivity to reinforcer delay (a measure of self-control) when they reported a higher preference for the juice received as the less delayed, smaller reinforcer than for the juice received as the more delayed, larger reinforcer. Conversely in Experiment 2, the women demonstrated significantly more sensitivity to reinforcer amount relative to sensitivity to reinforcer delay when they reported a higher preference for the juice received as the more delayed, larger reinforcer than for the juice received as the less delayed, smaller reinforcer. Together, the results show that participants’ food preferences can influence self-control for food reinforcers.     

Choice in a self-control paradigm with human subjects: Effects of changeover delay duration

The present experiment examined the choices of human subjects as a function of changeover delay (COD) duration. A self-control paradigm was used; subjects chose between larger, more delayed and smaller, less delayed reinforcers. The COD durations were 1 s, 15 s, and 30 s. The results indicated that at the 1-s COD, the subjects distributed their responses approximately equally between the two response alternatives. However, at the 15-s and 30-s COD durations, the subjects tended to demonstrate virtually exclusive preference for the larger, more delayed reinforcer. Furthermore, increasing the COD duration significantly increased the subjects' sensitivity to variation in reinforcer delay. Increasing the COD duration also increased the subjects' sensitivity to reinforcer amount, but this effect was not significant. The results are qualitatively consistent with an interpretation that the subjects followed a strategy which attempted to maximize overall amount of reinforcement.     

Effects of methylphenidate and morphine on delay-discount functions obtained within sessions

Four rats responded under a "self-control" procedure designed to obtain delay-discount functions within sessions. Each session consisted of seven blocks, with seven trials within each block. Each block consisted of two initial forced-choice trials followed by five free-choice trials. On choice trials, the rats could press either of two retractable levers. A press on one lever was followed by presentation of a smaller reinforcer (a single dipper presentation of a sucrose solution); a press on the other lever was followed by presentation of a larger reinforcer (four consecutive dipper presentations). The delay associated with the smaller reinforcer always was 0 s, whereas the signaled delay associated with the larger reinforcer increased across blocks (from 0 to 50 s). Under these conditions, the percentage of choices of the larger reinforcer decreased across blocks, and relatively reliable delay-discount functions were obtained within sessions. Doses of methylphenidate (1.0 to 17.0 mg/kg) and morphine (0.3 to 17.0 mg/kg) were then administered prior to selected sessions. Typically, intermediate doses of methylphenidate shifted the discount functions to the right (increased choices of the larger reinforcer). For 2 of the rats, this effect was pronounced; for the other 2 rats, this effect occurred after the range of delays for the larger reinforcer was decreased (0 to 20 s). On the other hand, in most cases morphine produced a slight leftward shift in the discount function (decreased choices of the larger reinforcer). The present procedure appears to be a useful and efficient method to characterize drug effects on an entire delay-discount function. As with many procedures used to study self-control choices, however, sources of control other than reinforcement delay and amount may have been operating in the present study, and these sources must be considered when interpreting drug effects.     

Choice in a self-control paradigm: Quantification of experience-based differences

Previous quantitative models of choice in a self-control paradigm (choice between a larger, more-delayed reinforcer and a smaller, less-delayed reinforcer) have not described individual differences. Two experiments are reported that provide additional quantitative data on experience-based differences in choice between reinforcers of varying sizes and delays. In Experiment 1, seven pigeons in a self-control paradigm were exposed to a fading procedure that increased choices of the larger, more-delayed reinforcer through gradually decreasing the delay to the smaller of two equally delayed reinforcers. Three control subjects, exposed to each of the small-reinforcer delays to which the experimental subjects were exposed, but for fewer sessions, demonstrated that lengthy exposure to each of the conditions in the fading procedure may be necessary in order for the increase to occur. In Experiment 2, pigeons with and without fading-procedure exposure chose between reinforcers of varying sizes and delays scheduled according to a concurrent variable-interval variable-interval schedule. In both experiments, pigeons with fading-procedure exposure were more sensitive to variations in reinforcer size than reinforcer delay when compared with pigeons without this exposure. The data were described by the generalized matching law when the relative size of its exponents, representing subjects' relative sensitivity to reinforcer size and delay, were grouped according to subjects' experience.     

Self-control in rhesus macaques (Macaca mulatta): controlling for differential stimulus exposure

Previous research on self-control using macaques (Macaca fascicularis) showed these animals have a strong bias for a delayed, larger reinforcer (Self-control) over an immediate, smaller reinforcer (Impulsive). Typical studies of self-control have used a discrete trials methodology with a secondary discriminative stimulus during the delay periods. This results in a greater exposure to the stimulus representing the self-controlled option and may account for some of the early exclusive preference for self-control observed. The present experiment examined self-control bias in three rhesus macaques (Macaca mulatta) while controlling for differential durations of stimulus exposure. Subjects were presented stimuli via a computer monitor and made choices by touching the stimulus at which point both stimuli were removed for the delay periods. All three subjects displayed a nearly exclusive bias for the delayed, larger reinforcer self-control). These results are consistent with previous studies, despite the variations in methodology and species.     

Self-control and impulsiveness in children and adults: Effects of food preferences

Experiment 1 used 6 preschool boys and Experiment 2 used 6 adult women to explore the effects of food preference on humans' choice in self-control paradigms. The boys showed a higher proportion of responses for more delayed, larger reinforcers (a measure of self-control) when those choices resulted in receipt of the most preferred food compared to when those choices resulted in the least preferred food. Further, the boys chose the less delayed, smaller reinforcers significantly more often when only those choices, as opposed to both choices, resulted in the most preferred food. Conversely, they chose the more delayed, larger reinforcers significantly more often when only those choices, as opposed to both choices, resulted in the most preferred food. Finally, the women demonstrated significantly less sensitivity to reinforcer amount relative to sensitivity to reinforcer delay (another measure of self-control) when they had a higher preference for the juice received as the less delayed, smaller reinforcer than for the juice received as the more delayed, larger reinforcer. Together, the results show that subjects' food preferences can influence self-control for food reinforcers.     

Commitment and self-control in a prisoner's dilemma game

Humans often make seemingly irrational choices in situations of conflict between a particular smaller-sooner reinforcer and a more abstract, temporally extended, but larger reinforcer. In two experiments, the extent to which the availability of commitment responses-self-imposed restrictions on future choices-might improve self-control in such situations was investigated. Participants played a prisoner's dilemma game against a computer that played a tit-for-tat strategy-cooperating after a participant cooperated, defecting after a participant defected. Defecting produced a small-immediate reinforcer (consisting of points convertible to gift cards) whereas cooperating increased the amount of subsequent reinforcers, yielding a greater overall reinforcer rate. Participants were normally free to cooperate or defect on each trial. Additionally, they could choose to make a commitment response that forced their choice for the ensuing five trials. For some participants, the commitment response forced cooperation; for others, it forced defection. Most participants, with either commitment response available, chose to commit repeatedly despite a minor point loss for doing so. After extended exposure to these contingencies, the commit-to-cooperate group cooperated significantly more than a control group (with no commitment available). The commit-to-defect group cooperated significantly less than the control group. When both commitment alternatives were simultaneously available-one for cooperation and one for defection-cooperation commitment was strongly preferred. In Experiment 2, the commitment alternative was removed at the end of the session; gains in cooperation, relative to the control group, were not sustained in the absence of the self-imposed behavioral scaffold.     

Fixed-interval performance and self-control in children.

Operant responses of 16 children (mean age 6 years and 1 month) were reinforced according to different fixed-interval schedules (with interreinforcer intervals of 20, 30, or 40 s) in which the reinforcers were either 20-s or 40-s presentations of a cartoon. In another procedure, they received training on a self-control paradigm in which both reinforcer delay (0.5 s or 40 s) and reinforcer duration (20 s or 40 s of cartoons) varied, and subjects were offered a choice between various combinations of delay and duration. Individual differences in behavior under the self-control procedure were precisely mirrored by individual differences under the fixed-interval schedule. Children who chose the smaller immediate reinforcer on the self-control procedure (impulsive) produced short postreinforcement pauses and high response rates in the fixed-interval conditions, and both measures changed little with changes in fixed-interval value. Conversely, children who chose the larger delayed reinforcer in the self-control condition (the self-controlled subjects) exhibited lower response rates and long postreinforcement pauses, which changed systematically with changes in the interval, in their fixed-interval performances.     

Self-control: teaching tolerance for delay in impulsive children.

We investigated a procedure to increase the selection of larger, more delayed reinforcers (i.e., more advantageous in the long run) over smaller immediate reinforcers, in an effort to increase a key aspect of self-control in children. Six preschoolers, including one comparison subject, identified by their teachers as impulsive, were preassessed and found consistently to select smaller immediate reinforcers over larger, more delayed ones. The teaching procedure consisted of gradually increasing the durations of the delay interval over many sessions. The follow-up assessments showed that 5 of these children increased the proportion of their choices of the delayed reinforcers. Before training, indifference points ranged from 1.7 to 51.7 s; following treatment, points rose to a range of 37.5 to at least 90 s, with 3 children preferring the larger reinforcer at all delay intervals tested. The results demonstrated the feasibility of teaching young children to make choices more advantageous to them in the long run.     

Determinants of pigeons' choices in token-based self-control procedures

Four pigeons were exposed to a token-based self-control procedure with stimulus lights serving as token reinforcers. Smaller-reinforcer choices produced one token immediately; larger-reinforcer choices produced three tokens following a delay. Each token could be exchanged for 2-s access to food during a signaled exchange period each trial. The main variables of interest were the exchange delays (delays from the choice to the exchange stimulus) and the food delays (also timed from the choice), which were varied separately and together across blocks of sessions. When exchange delays and food delays were shorter following smaller-reinforcer choices, strong preference for the smaller reinforcer was observed. When exchange delays and food delays were equal for both options, strong preference for the larger reinforcer was observed. When food delays were equal for both options but exchange delays were shorter for smaller-reinforcer choices, preference for the larger reinforcer generally was less extreme than under conditions in which both exchange and food delays were equal. When exchange delays were equal for both options but food delays were shorter for smaller-reinforcer choices, preference for the smaller reinforcer generally was less extreme than under conditions in which both exchange and food delays favored smaller-reinforcer choices. On the whole, the results were consistent with prior research on token-based self-control procedures in showing that choices are governed by reinforcer immediacy when exchange and food delays are unequal and by reinforcer amount when exchange and food delays are equal. Further, by decoupling the exchange delays from food delays, the results tentatively support a role for the exchange stimulus as a conditioned reinforcer.     

Making choices in anticipation of similar future choices can increase self-control

Two experiments tested the efficacy of linking a current choice with similar future choices as a means of increasing self-control. Participants were offered choices between smaller and sooner vs. larger and later amounts of money (Experiment 1, n = 60) or food (Experiment 2, n = 34). After a small-large pair for which the participant preferred the smaller reward was found, a choice between the same pair was offered as the 1st of 5 such choices to be offered over a period of weeks. The majority of participants in both experiments who chose between all 5 smaller and all 5 larger rewards chose the larger rewards. One third of participants in Experiment 1 who could choose independently on each pair in the series reversed their previous preference and chose the larger reward in the context of the series. These results suggest that self-control can be enhanced by viewing one's current choice as predictive of future choices.     

The Matching Law And Amount-dependent Exponential Discounting As Accounts Of Self-control Choice

Studies with humans have found evidence for amount-dependent temporal discounting, that is, that the sensitivity of choice to reinforcer delay varies inversely with reinforcer magnitude. To test whether similar results could be obtained with nonhumans, pigeons were trained on a two-component concurrent-chains procedure in which the durations of food reinforcement in the terminal links were equal within components but unequal between components. Terminal-link schedules were varied over four conditions to allow separate estimates of sensitivity to delay to be obtained for the large and small reinforcer-magnitude components. Although sensitivity to delay was greater in the small-magnitude component for all subjects, the effect was quite small. The difference in sensitivity was generally less than one standard error, and t tests on parameter differences failed to reach significance. Several models for temporal discounting, including an amount-dependent exponential function, were fitted to the data from the first four conditions. The resulting parameter estimates were used to make predictions for a self-control condition in which one terminal link arranged a smaller, less delayed reinforcer and the other arranged a larger, more delayed reinforcer. For all models, predictions were considerably more accurate when sensitivity to delay was the same regardless of reinforcer magnitude. The results support the independence of delay and magnitude as required by a version of the matching law, and provide strong evidence against amount-dependent exponential discounting as an account of self-control choice. A new two-parameter discounting function, consistent with the matching law, is proposed that has wide empirical generality for both human and nonhuman data.     

Arranging response requirements and the distribution of reinforcers: A brief review of preference and performance outcomes

Recent research has demonstrated that some participants prefer to complete a larger series of responses in exchange for a longer duration of reinforcer access, rather than completing fewer tasks associated with smaller, but more frequent, reinforcer access. This review provides a summary of this line of research, examines variables contributing to participant preference and performance under different response‐reinforcer arrangements, and discusses limitations and areas for future research.