Choice in a self-control paradigm: Quantification of experience-based differences

Previous quantitative models of choice in a self-control paradigm (choice between a larger, more-delayed reinforcer and a smaller, less-delayed reinforcer) have not described individual differences. Two experiments are reported that provide additional quantitative data on experience-based differences in choice between reinforcers of varying sizes and delays. In Experiment 1, seven pigeons in a self-control paradigm were exposed to a fading procedure that increased choices of the larger, more-delayed reinforcer through gradually decreasing the delay to the smaller of two equally delayed reinforcers. Three control subjects, exposed to each of the small-reinforcer delays to which the experimental subjects were exposed, but for fewer sessions, demonstrated that lengthy exposure to each of the conditions in the fading procedure may be necessary in order for the increase to occur. In Experiment 2, pigeons with and without fading-procedure exposure chose between reinforcers of varying sizes and delays scheduled according to a concurrent variable-interval variable-interval schedule. In both experiments, pigeons with fading-procedure exposure were more sensitive to variations in reinforcer size than reinforcer delay when compared with pigeons without this exposure. The data were described by the generalized matching law when the relative size of its exponents, representing subjects' relative sensitivity to reinforcer size and delay, were grouped according to subjects' experience.     

Self‐control as generalized operant behavior by adults with autism spectrum disorder

The present investigation attempted to teach self‐control to 3 adults who had been diagnosed with autism spectrum disorder. Using a self‐control training procedure that coupled a gradual fading of delay to access a large delayed reinforcer with a concurrent work requirement, each participant was exposed to conditions in which baseline levels of choices for large delayed reinforcers and task engagement increased 10‐fold. Furthermore, generalization effects were partially demonstrated in a novel context.     

Assessment of impulsivity and the development of self-control in students with attention deficit hyperactivity disorder.

We examined a combined approach of manipulating reinforcer dimensions and delay fading to promote the development of self‐control with 3 students diagnosed with attention deficit hyperactivity disorder. First, we administered a brief computer‐based assessment to determine the relative influence of reinforcer rate (R), reinforcer quality (Q), reinforcer immediacy (I), and effort (E) on the students' choices between concurrently presented math problems. During each session, one of these dimensions was placed in direct competition with another dimension (e.g., RvI involving math problem alternatives associated with high‐rate delayed reinforcement vs. low‐rate immediate reinforcement), with all possible pairs of dimensions presented across the six assessment conditions (RvQ, RvI, RvE, QvI, QvE, IvE). The assessment revealed that the choices of all 3 students were most influenced by immediacy of reinforcement, reflecting impulsivity. We then implemented a self‐control training procedure in which reinforcer immediacy competed with another influential dimension (RvI or QvI), and the delay associated with the higher rate or quality reinforcer alternative was progressively increased. The students allocated the majority of their time to the math problem alternatives yielding more frequent (high‐rate) or preferred (high‐quality) reinforcement despite delays of up to 24 hr. Subsequent readministration of portions of the assessment showed that self‐control transferred across untrained dimensions of reinforcement.     

A COMPARISON OF THE EFFECTS OF BRIEF RULES, A TIMER, AND PREFERRED TOYS ON SELF-CONTROL

Some children make impulsive choices (i.e., choose a small but immediate reinforcer over a large but delayed reinforcer). Previous research has shown that delay fading, providing an alternative activity during the delay, teaching participants to repeat a rule during the delay, combining delay fading with an alternative activity, and combining delay fading with a countdown timer are effective for increasing self‐control (i.e., choosing the large but delayed reinforcer over the small but immediate reinforcer). The purpose of the current study was to compare the effects of various interventions in the absence of delay fading (i.e., providing brief rules, providing a countdown timer during the delay, or providing preferred toys during the delay) on self‐control. Results suggested that providing brief rules or a countdown timer during the delay was ineffective for enhancing self‐control. However, providing preferred toys during the delay effectively enhanced self‐control.     

Self-control in children with autism: response allocation during delays to reinforcement.

This study examined the use of a progressive‐delay schedule of reinforcement to increase self‐control and decrease disruptive behavior in children with autism. When initially given the choice between an immediate smaller reinforcer and a larger delayed reinforcer, all participants chose the smaller reinforcer. When access to the larger reinforcer required either no activity or engaging in a concurrent task during the delay, all participants demonstrated both self‐control and preference for a response requirement. Disruptive behavior decreased during delays that required a concurrent task compared to sessions without an activity requirement.     

Impulsive choice and pre‐exposure to delays: iv. effects of delay‐ and immediacy‐exposure training relative to maturational changes in impulsivity

Impulsive choice describes preference for smaller, sooner rewards over larger, later rewards. Excessive delay discounting (i.e., rapid devaluation of delayed rewards) underlies some impulsive choices, and is observed in many maladaptive behaviors (e.g., substance abuse, gambling). Interventions designed to reduce delay discounting may provide therapeutic gains. One such intervention provides rats with extended training with delayed reinforcers. When compared to a group given extended training with immediate reinforcers, delay‐exposed rats make significantly fewer impulsive choices. To what extent is this difference due to delay‐exposure training shifting preference toward self‐control or immediacy‐exposure training (the putative control group) shifting preference toward impulsivity? The current study compared the effects of delay‐ and immediacy‐exposure training to a no‐training control group and evaluated within‐subject changes in impulsive choice across 51 male Wistar rats. Delay‐exposed rats made significantly fewer impulsive choices than immediacy‐exposed and control rats. Between‐group differences in impulsive choice were not observed in the latter two groups. While delay‐exposed rats showed large, significant pre‐ to posttraining reductions in impulsive choice, immediacy‐exposed and control rats showed small reductions in impulsive choice. These results suggest that extended training with delayed reinforcers reduces impulsive choice, and that extended training with immediate reinforcers does not increase impulsive choice.     

TEACHING SELF‐CONTROL WITH QUALITATIVELY DIFFERENT REINFORCERS

This study examined the effectiveness of using qualitatively different reinforcers to teach self‐control to an adolescent boy who had been diagnosed with an intellectual disability. First, he was instructed to engage in an activity without programmed reinforcement. Next, he was instructed to engage in the activity under a two‐choice fixed‐duration schedule of reinforcement. Finally, he was exposed to self‐control training, during which the delay to a more preferred reinforcer was initially short and then increased incrementally relative to the delay to a less preferred reinforcer. Self‐control training effectively increased time on task to earn the delayed reinforcer.     

Providing noncontingent,alternative, functional reinforcers during delays following functional communication training

Functional Communication Training (FCT) involves arranging extinction for problem behavior and reinforcement for a more desirable, functionally equivalent, communicative response (FCR). Although effective under ideal arrangements, the introduction of delays to reinforcement following the FCR can result in increased problem behavior. Austin and Tiger (2015) showed that for individuals whose problem behavior was sensitive to multiple sources of reinforcement, providing access to alternative, functional reinforcers during delays mitigated this increase in problem behavior during delay fading. The current study replicated the procedures of Austin and Tiger with 2 individuals displaying multiply controlled problem behavior. Providing alternative functional reinforcers reduced problem behavior during 10-min delays for both participants without requiring delay fading.     

Self-control: teaching tolerance for delay in impulsive children.

We investigated a procedure to increase the selection of larger, more delayed reinforcers (i.e., more advantageous in the long run) over smaller immediate reinforcers, in an effort to increase a key aspect of self-control in children. Six preschoolers, including one comparison subject, identified by their teachers as impulsive, were preassessed and found consistently to select smaller immediate reinforcers over larger, more delayed ones. The teaching procedure consisted of gradually increasing the durations of the delay interval over many sessions. The follow-up assessments showed that 5 of these children increased the proportion of their choices of the delayed reinforcers. Before training, indifference points ranged from 1.7 to 51.7 s; following treatment, points rose to a range of 37.5 to at least 90 s, with 3 children preferring the larger reinforcer at all delay intervals tested. The results demonstrated the feasibility of teaching young children to make choices more advantageous to them in the long run.     

Stimulus Equivalence: Effects Of A Default-response Option On Emergence Of Untrained Stimulus Relations

Human subjects were exposed to a concurrent-chains schedule in which reinforcer amounts, delays, or both were varied in the terminal links, and consummatory responses were required to receive points that were later exchangeable for money. Two independent variable-interval 30-s schedules were in effect during the initial links, and delay periods were defined by fixed-time schedules. In Experiment 1, subjects were exposed to three different pairs of reinforcer amounts and delays, and sensitivity to reinforcer amount and delay was determined based on the generalized matching law. The relative responding (choice) of most subjects was more sensitive to reinforcer amount than to reinforcer delay. In Experiment 2, subjects chose between immediate smaller reinforcers and delayed larger reinforcers in five conditions with and without timeout periods that followed a shorter delay, in which reinforcer amounts and delays were combined to make different predictions based on local reinforcement density (i.e., points per delay) or overall reinforcement density (i.e., points per total time). In most conditions, subjects' choices were qualitatively in accord with the predictions from the overall reinforcement density calculated by the ratio of reinforcer amount and total time. Therefore, the overall reinforcement density appears to influence the preference of humans in the present self-control choice situation.     

Self-control in mentally retarded adolescents: choice as a function of amount and delay of reinforcement.

Three severely mentally retarded adolescents were studied under discrete-trial procedures in which a choice was arranged between edible reinforcers that differed in magnitude and, in some conditions, delay. In the absence of delays the larger reinforcer was consistently chosen. Under conditions in which the smaller reinforcer was not delayed, increasing the delay to delivery of the larger reinforcer decreased the percentage of trials in which that reinforcer was chosen. All subjects directed the majority of choice responses to the smaller reinforcer when the larger reinforcer was sufficiently delayed, although the value at which this occurred differed across subjects. Under conditions in which the larger reinforcer initially was sufficiently delayed to result in preference for the smaller one, progressively increasing in 5-s increments the delay to both reinforcers increased percentage of trials with the larger reinforcer chosen. At sufficiently long delays, 2 of the subjects consistently chose the larger, but more delayed, reinforcer, and the 3rd subject chose that reinforcer on half of the trials. These results are consistent with the findings of prior studies in which adult humans responded to terminate noise and pigeons responded to produce food.     

Token reinforcement, choice, and self-control in pigeons.

Pigeons were exposed to self-control procedures that involved illumination of light-emitting diodes (LEDs) as a form of token reinforcement. In a discrete-trials arrangement, subjects chose between one and three LEDs; each LED was exchangeable for 2-s access to food during distinct posttrial exchange periods. In Experiment 1, subjects generally preferred the immediate presentation of a single LED over the delayed presentation of three LEDs, but differences in the delay to the exchange period between the two options prevented a clear assessment of the relative influence of LED delay and exchange-period delay as determinants of choice. In Experiment 2, in which delays to the exchange period from either alternative were equal in most conditions, all subjects preferred the delayed three LEDs more often than in Experiment-1. In Experiment 3, subjects preferred the option that resulted in a greater amount of food more often if the choices also produced LEDs than if they did not. In Experiment 4, preference for the delayed three LEDs was obtained when delays to the exchange period were equal, but reversed in favor of an immediate single LED when the latter choice also resulted in quicker access to exchange periods. The overall pattern of results suggests that (a) delay to the exchange period is a more critical determinant of choice than is delay to token presentation; (b) tokens may function as conditioned reinforcers, although their discriminative properties may be responsible for the self-control that occurs under token reinforcer arrangements; and (c) previously reported differences in the self-control choices of humans and pigeons may have resulted at least in part from the procedural conventions of using token reinforcers with human subjects and food reinforcers with pigeon subjects.     

Worth the wait: Children trade off delay and reward in self‐ and other‐benefiting decisions

Human prosocial behaviors are supported by early‐emerging psychological processes that detect and fulfill the needs of others. However, little is known about the mechanisms that enable children to deliver benefits to others at costs to the self, which requires weighing other‐regarding and self‐serving preferences. We used an intertemporal choice paradigm to systematically study and compare these behaviors in 5‐year‐old children. Our results show that other‐benefiting and self‐benefiting behavior share a common decision‐making process that integrates delay and reward. Specifically, we found that children sought to minimize delay and maximize reward, and traded off delays against rewards, regardless of whether these rewards were for the children themselves or another child. However, we found that children were more willing to invest their time to benefit themselves than someone else. Together, these findings show that from childhood, other‐ and self‐serving decisions are supported by a general mechanism that flexibly integrates information about the magnitude of rewards, and the opportunity costs of pursuing them. A video abstract of this article can be viewed at: https://youtu.be/r8S0DGe7f8Q     

How quickly do children forget events? A systematic study of children's event reports as a function of delay

Sixty‐six children aged between 5 and 7 years participated in the event, ‘Visiting the Pirate’, and were interviewed about it at one of five delays, namely, no delay (immediate interview), or at a 1‐week, 1‐day, 1‐month or 6‐month delay. For open‐ended recall, a significant decrease in the amount of information reported was detected only at the 6‐month delay, although forgetting functions indicated forgetting was most rapid at the shorter delays. Accuracy of open‐ended recall was maintained over all delays. In contrast, in response to specific, leading and misleading questions, accuracy had decreased significantly by the 6‐month delay. The theoretical and practical implications of these findings regarding the effects of delays on children's event reports are discussed. Copyright © 2002 John Wiley & Sons, Ltd.     

Choice between single and multiple delayed reinforcers.

Pigeons chose between alternatives that differed in the number of reinforcers and in the delay to each reinforcer. A peck on a red key produced the same consequences on every trial within a condition, but between conditions the number of reinforcers varied from one to three and the reinforcer delays varied between 5 s and 30 s. A peck on a green key produced a delay of adjustable duration and then a single reinforcer. The green-key delay was increased or decreased many times per session, depending on a subject's previous choices, which permitted estimation of an indifference point, or a delay at which a subject chose each alternative about equally often. The indifference points decreased systematically with more red-key reinforcers and with shorter red-key delays. The results did not support the suggestion of Moore (1979) that multiple delayed reinforcers have no effect on preference unless they are closely grouped. The results were well described in quantitative detail by a simple model stating that each of a series of reinforcers increases preference, but that a reinforcer's effect is inversely related to its delay. The success of this model, which considers only delay of reinforcement, suggested that the overall rate of reinforcement for each alternative had no effect on choice between those alternatives.     

Developmental differences in the use of task goals in a cued version of the stroop task

The ability of children (M= 8.8 years) and adults (M= 25.6 years) to maintain task goals was examined by comparing their performance on a cued version of the Stroop colour‐word task. The experimental task presented a cue on each trial that instructed the participant to either read aloud the forthcoming word or name the colour of the word's lettering. Participants were tested with each of two cue‐stimulus delays (1,000 and 5,000 ms). Analysis of error rates in the colour‐naming condition revealed that children experienced greater interference than adults at each of the cue‐stimulus delays. In an effort to separate the relative contributions of colour‐naming and word‐reading processes, additional analyses were performed based on the process dissociation procedure of Lindsay and Jacoby (1994) . While colour‐naming process estimates did not vary with age group or cue‐stimulus delay, word‐reading process estimates were found to vary with age group and cue‐stimulus delay. Specifically, adults were superior to children in the inhibition of irrelevant word information only during a long cue‐stimulus delay. Collectively, these findings indicate that children have difficulty maintaining task goals in order to suppress stronger, goal‐irrelevant responses.     

Choice with delayed and probabilistic reinforcers: effects of variability, time between trials, and conditioned reinforcers.

In a discrete-trials procedure with pigeons, a response on a green key led to a 4-s delay (during which green houselights were lit) and then a reinforcer might or might not be delivered. A response on a red key led to a delay of adjustable duration (during which red houselights were lit) and then a certain reinforcer. The delay was adjusted so as to estimate an indifference point--a duration for which the two alternatives were equally preferred. Once the green key was chosen, a subject had to continue to respond on the green key until a reinforcer was delivered. Each response on the green key, plus the 4-s delay that followed every response, was called one "link" of the green-key schedule. Subjects showed much greater preference for the green key when the number of links before reinforcement was variable (averaging four) than when it was fixed (always exactly four). These findings are consistent with the view that probabilistic reinforcers are analogous to reinforcers delivered after variable delays. When successive links were separated by 4-s or 8-s "interlink intervals" with white houselights, preference for the probabilistic alternative decreased somewhat for 2 subjects but was unaffected for the other 2 subjects. When the interlink intervals had the same green houselights that were present during the 4-s delays, preference for the green key decreased substantially for all subjects. These results provided mixed support for the view that preference for a probabilistic reinforcer is inversely related to the duration of conditioned reinforcers that precede the delivery of food.     

IMPULSIVITY IN STUDENTS WITH SERIOUS EMOTIONAL DISTURBANCE: THE INTERACTIVE EFFECTS OF REINFORCER RATE,DELAY, AND QUALITY

We conducted two studies extending basic matching research on self-control and impulsivity to the investigation of choices of students diagnosed as seriously emotionally disturbed. In Study 1 we examined the interaction between unequal rates of reinforcement and equal versus unequal delays to reinforcer access on performance of concurrently available sets of math problems. The results of a reversal design showed that when delays to reinforcer access were the same for both response alternatives, the time allocated to each was approximately proportional to obtained reinforcement. When the delays to reinforcer access differed between the response alternatives, there was a bias toward the response alternative and schedule with the lower delays, suggesting impulsivity (i.e., immediate reinforcer access overrode the effects of rate of reinforcement). In Study 2 we examined the interactive effects of reinforcer rate, quality, and delay. Conditions involving delayed access to the high-quality reinforcers on the rich schedule (with immediate access to low-quality reinforcers earned on the lean schedule) were alternated with immediate access to low-quality reinforcers on the rich schedule (with delayed access to high-quality reinforcers on the lean schedule) using a reversal design. With 1 student, reinforcer quality overrode the effects of both reinforcer rate and delay to reinforcer access. The other student tended to respond exclusively to the alternative associated with immediate access to reinforcers. The studies demonstrate a methodology based on matching theory for determining influential dimensions of reinforcers governing individuals' choices.