Impulsive choice and pre‐exposure to delays: iv. effects of delay‐ and immediacy‐exposure training relative to maturational changes in impulsivity

Impulsive choice describes preference for smaller, sooner rewards over larger, later rewards. Excessive delay discounting (i.e., rapid devaluation of delayed rewards) underlies some impulsive choices, and is observed in many maladaptive behaviors (e.g., substance abuse, gambling). Interventions designed to reduce delay discounting may provide therapeutic gains. One such intervention provides rats with extended training with delayed reinforcers. When compared to a group given extended training with immediate reinforcers, delay‐exposed rats make significantly fewer impulsive choices. To what extent is this difference due to delay‐exposure training shifting preference toward self‐control or immediacy‐exposure training (the putative control group) shifting preference toward impulsivity? The current study compared the effects of delay‐ and immediacy‐exposure training to a no‐training control group and evaluated within‐subject changes in impulsive choice across 51 male Wistar rats. Delay‐exposed rats made significantly fewer impulsive choices than immediacy‐exposed and control rats. Between‐group differences in impulsive choice were not observed in the latter two groups. While delay‐exposed rats showed large, significant pre‐ to posttraining reductions in impulsive choice, immediacy‐exposed and control rats showed small reductions in impulsive choice. These results suggest that extended training with delayed reinforcers reduces impulsive choice, and that extended training with immediate reinforcers does not increase impulsive choice.     

An alternative approach to calculating Area‐Under‐the‐Curve (AUC) in delay discounting research

Applied to delay discounting data, Area‐Under‐the‐Curve (AUC) provides an atheoretical index of the rate of delay discounting. The conventional method of calculating AUC, by summing the areas of the trapezoids formed by successive delay‐indifference point pairings, does not account for the fact that most delay discounting tasks scale delay pseudoexponentially, that is, time intervals between delays typically get larger as delays get longer. This results in a disproportionate contribution of indifference points at long delays to the total AUC, with minimal contribution from indifference points at short delays. We propose two modifications that correct for this imbalance via a base‐10 logarithmic transformation and an ordinal scaling transformation of delays. These newly proposed indices of discounting, AUC_{log d} and AUC_{or d,} address the limitation of AUC while preserving a primary strength (remaining atheoretical). Re‐examination of previously published data provides empirical support for both AUC_{log d} and AUC_{or d} . Thus, we believe theoretical and empirical arguments favor these methods as the preferred atheoretical indices of delay discounting.     

Temporal framing and the hidden‐zero effect: rate‐dependent outcomes on delay discounting

Recent research suggests that presenting time intervals as units (e.g., days) or as specific dates, can modulate the degree to which humans discount delayed outcomes. Another framing effect involves explicitly stating that choosing a smaller–sooner reward is mutually exclusive to receiving a larger–later reward, thus presenting choices as an extended sequence. In Experiment 1, participants (N = 201) recruited from Amazon Mechanical Turk completed the Monetary Choice Questionnaire in a 2 (delay framing) by 2 (zero framing) design. Regression suggested a main effect of delay, but not zero, framing after accounting for other demographic variables and manipulations. We observed a rate‐dependent effect for the date‐framing group, such that those with initially steep discounting exhibited greater sensitivity to the manipulation than those with initially shallow discounting. Subsequent analyses suggest these effects cannot be explained by regression to the mean. Experiment 2 addressed the possibility that the null effect of zero framing was due to within‐subject exposure to the hidden‐ and explicit‐zero conditions. A new Amazon Mechanical Turk sample completed the Monetary Choice Questionnaire in either hidden‐ or explicit‐zero formats. Analyses revealed a main effect of reward magnitude, but not zero framing, suggesting potential limitations to the generality of the hidden‐zero effect.     

TEACHING SELF‐CONTROL WITH QUALITATIVELY DIFFERENT REINFORCERS

This study examined the effectiveness of using qualitatively different reinforcers to teach self‐control to an adolescent boy who had been diagnosed with an intellectual disability. First, he was instructed to engage in an activity without programmed reinforcement. Next, he was instructed to engage in the activity under a two‐choice fixed‐duration schedule of reinforcement. Finally, he was exposed to self‐control training, during which the delay to a more preferred reinforcer was initially short and then increased incrementally relative to the delay to a less preferred reinforcer. Self‐control training effectively increased time on task to earn the delayed reinforcer.     

Reinforcement value and fixed‐interval performance

The concept of reinforcement value summarizes the effect of different variables, such as reinforcement delay, reinforcement magnitude, and deprivation level, on behavior. In the present set of experiments, we evaluated the effect of reinforcement devaluation on performance under FI schedules. The literature on timing and reinforcement value suggests that devaluation generates longer expected times to reinforcement than the same intervals trained under control conditions. We devalued reinforcement with delay in Experiments 1A, 1B, and 2, and diminished deprivation in Experiments 3A and 3B. Devaluation reduced response rates, increased the number of one‐response intervals, and lengthened postreinforcement pauses, but had inconsistent effects on other timing measures such as quarter life and breakpoint. The results of delayed reinforcement and diminished deprivation manipulations are well summarized as reinforcement devaluation effects. These results suggest that devaluation may reduce stimulus control. In addition, we argue that the process by which delayed reinforcement affects behavior might also explain some effects observed in other devaluation procedures through the concept of reinforcement value.     

An overview of Bayesian reasoning in the analysis of delay‐discounting data

Statistical inference (including interval estimation and model selection) is increasingly used in the analysis of behavioral data. As with many other fields, statistical approaches for these analyses traditionally use classical (i.e., frequentist) methods. Interpreting classical intervals and p‐values correctly can be burdensome and counterintuitive. By contrast, Bayesian methods treat data, parameters, and hypotheses as random quantities and use rules of conditional probability to produce direct probabilistic statements about models and parameters given observed study data. In this work, we reanalyze two data sets using Bayesian procedures. We precede the analyses with an overview of the Bayesian paradigm. The first study reanalyzes data from a recent study of controls, heavy smokers, and individuals with alcohol and/or cocaine substance use disorder, and focuses on Bayesian hypothesis testing for covariates and interval estimation for discounting rates among various substance use disorder profiles. The second example analyzes hypothetical environmental delay‐discounting data. This example focuses on using historical data to establish prior distributions for parameters while allowing subjective expert opinion to govern the prior distribution on model preference. We review the subjective nature of specifying Bayesian prior distributions but also review established methods to standardize the generation of priors and remove subjective influence while still taking advantage of the interpretive advantages of Bayesian analyses. We present the Bayesian approach as an alternative paradigm for statistical inference and discuss its strengths and weaknesses.     

Effects of delay fading and signals on self‐control choices by children

The current study is a systematic replication and extension of work by Schweitzer & Sulzer‐Azaroff (1988). The effects of delay fading alone and in combination with signals on choices between larger, delayed reinforcers and smaller, immediate reinforcers by four children with language deficits were examined. For one of the two children exposed to delay fading alone, larger reinforcers were selected at longer delays relative to the initial self‐control assessment. For all four children, the delay‐fading‐plus‐signal condition resulted in selection of larger reinforcers at considerably longer delays relative to the self‐control assessment.     

RISKY CHOICE IN PIGEONS AND HUMANS: A CROSS‐SPECIES COMPARISON

Pigeon and human subjects were given repeated choices between variable and adjusting delays to token reinforcement that titrated in relation to a subject's recent choice patterns. Indifference curves were generated under two different procedures: immediate exchange, in which a token earned during each trial was exchanged immediately for access to the terminal reinforcer (food for pigeons, video clips for humans), and delayed exchange, in which tokens accumulated and were exchanged after 11 trials. The former was designed as an analogue of procedures typically used with nonhuman subjects, the latter as an analogue to procedures typically used with human participants. Under both procedure types, different variable‐delay schedules were manipulated systematically across conditions in ways that altered the reinforcer immediacy of the risky option. Under immediate‐exchange conditions, both humans and pigeons consistently preferred the variable delay, and indifference points were generally ordered in relation to relative reinforcer immediacies. Such risk sensitivity was greatly reduced under delayed‐exchange conditions. Choice and trial‐initiation response latencies varied directly with indifference points, suggesting that local analyses may provide useful ancillary measures of reinforcer value. On the whole, the results indicate that modifying procedural features brings choices of pigeons and humans into better accord, and that human—nonhuman differences on risky choice procedures reported in the literature may be at least partly a product of procedural differences.     

Effects of delayed time‐out on problem behavior of preschool children

Time‐out is a common negative punishment procedure in home and school settings. Although prior studies have shown time‐out is effective, more research is needed on its effects when implementation is imperfect. We evaluated delays to time‐out with 4 preschool children who engaged in some combination of aggression, property destruction, and rule breaking. Target behavior decreased for all subjects exposed to delayed time‐out, with 3 of 4 subjects displaying low levels of target behavior even when time‐out was delayed by 90‐120 s. These data suggest delayed time‐out might be effective in situations in which a caregiver or teacher cannot implement time‐out immediately.     

Evaluating the effect of delay spacing on delay discounting: carry‐over effects on steepness and the form of the discounting function

Most delay discounting studies use tasks that arrange delay progressions in which the spacing between consecutive delays becomes progressively larger. To date, little research has examined delay discounting using other progressions. The present study assessed whether the form or steepness of discounting varied across different delay progressions. Human participants completed three discounting tasks with delay progressions that varied in the time between consecutive delays: a standard (increasing duration between delays), linear (equal duration between delays), and an inverse progression (decreasing duration between delays). Steepness of discounting was generally reduced, and remained so, following experience with the inverse progression. Effects of the delay progression on the best fitting equation were order‐dependent. Overall the hyperbola model provided better fits, but the exponential model performed better with data from the inverse progression. Regardless, differences in which model fit best were often small. The finding that the best fitting model was dependent, in some cases, on the delay progression suggests that a single quantitative model of discounting may not be applicable to describe discounting across all procedural contexts. Ultimately, changes in steepness of discounting following experience with the inverse progression appeared similar to anchoring effects, whose mechanism will require further study to delineate.     

Reformative self‐control and discounting of reward value by delay or effort1

Abstract: The relation between reformative self‐control and discounting of reward value by delay or effort was examined. Thirty‐two Japanese undergraduates were selected based on their scores on a standardized pencil and paper test of self‐control, the Redressive–Reformative Self‐Control Scale ( Sugiwaka, 1995 ). The subjects indicated their preference (i) between a hypothetical 100 000 yen reward available with various delays and a certain reward of variable amount available immediately; or (ii) between a hypothetical 100 000 yen reward available with various efforts and a certain reward of variable amount available without effort. The discounting of delayed reward value was described well by a hyperbolic function proposed by Mazur (1987 ). The slopes of the delayed discounting curves were steeper for subjects who scored low on the Reformative Self‐Control Scale, which is a measure of reformative self‐control and a subscale of the Redressive–Reformative Self‐Control Scale. The discounting by effort was also described well by the hyperbolic function. The slopes of the effortful discounting curves did not correlate to those of the delayed discounting curves. The slopes were not systematically different across the groups of subjects with different scores on the reformative self‐control scale.     

Assessing the generality and durability of interview‐informed functional analyses and treatment

Due to the limited research demonstrating socially valid outcomes of function‐based treatments in ecologically relevant environments (Santiago, Hanley, Moore, & Jin, 2016), we replicated and extended the effects of the interview‐informed functional analysis and skill‐based treatment procedure described by Hanley, Jin, Vanselow, and Hanratty (2014) with two children diagnosed with autism in a home setting. The assessment and treatment was implemented by a home‐based service provider and treatment was extended to the participants’ parents. Following the interview‐informed functional analyses, we taught the participants functional communication responses and to engage in less‐preferred activities when functional communication outcomes were delayed. We observed large reductions in problem behavior following the introduction of the function‐based treatment. The effects extended to novel settings, stimuli, and caregivers and the results maintained at 6‐week follow‐ups.     

An evaluation of toy quality for increasing self‐control in typically developing preschool children

Children often make impulsive choices, and previous research has shown that access to activities during the delay may enhance self‐control (e.g., Newquist, Dozier, & Neidert, 2012). The purpose of the current study was to extend the results of Newquist et al. (2012) by comparing the effects of access to low‐preference, moderate‐preference, and high‐preference toys during delays. Results showed that (a) all toys increased self‐control for 2 participants when toys were available for all choice options and (b) high‐preference toys (and sometimes moderate‐preference toys) increased self‐control for 3 participants when the toys were available only for large delayed choices.