Disruption of responding maintained by conditioned reinforcement: alterations in response-conditioned-reinforcer relations

An observing procedure was used to investigate the effects of alterations in response-conditioned-reinforcer relations on observing. Pigeons responded to produce schedule-correlated stimuli paired with the availability of food or extinction. The contingency between observing responses and conditioned reinforcement was altered in three experiments. In Experiment 1, after a contingency was established in baseline between the observing response and conditioned reinforcement, it was removed and the schedule-correlated stimuli were presented independently of responding according to a variable-time schedule. The variable-time schedule was constructed such that the rate of stimulus presentations was yoked from baseline. The removal of the observing contingency reliably reduced rates of observing. In Experiment 2, resetting delays to conditioned reinforcement were imposed between observing responses and the schedule-correlated stimuli they produced. Delay values of 0, 0.5, 1, 5, and 10 s were examined. Rates of observing varied inversely as a function of delay value. In Experiment 3, signaled and unsignaled resetting delays between observing responses and schedule-correlated stimuli were compared. Baseline rates of observing were decreased less by signaled delays than by unsignaled delays. Disruptions in response-conditioned-reinforcer relations produce similar behavioral effects to those found with primary reinforcement.     

Responding of pigeons under variable-interval schedules of unsignaled, briefly signaled, and completely signaled delays to reinforcement

In Experiment 1, three pigeons' key pecking was maintained under a variable-interval 60-s schedule of food reinforcement. A 1-s unsignaled nonresetting delay to reinforcement was then added. Rates decreased and stabilized at values below those observed under immediate-reinforcement conditions. A brief stimulus change (key lit red for 0.5 s) was then arranged to follow immediately the peck that began the delay. Response rates quickly returned to baseline levels. Subsequently, rates near baseline levels were maintained with briefly signaled delays of 3 and 9 s. When a 27-s briefly signaled delay was instituted, response rates decreased to low levels. In Experiment 2, four pigeons' responding was first maintained under a multiple variable-interval 60-s (green key) variable-interval 60-s (red key) schedule. Response rates in both components fell to low levels when a 3-s unsignaled delay was added. In the first component delays were then briefly signaled in the same manner as Experiment 1, and in the second component they were signaled with a change in key color that remained until food was delivered. Response rates increased to near baseline levels in both components, and remained near baseline when the delays in both components were lengthened to 9 s. When delays were lengthened to 27 s, response rates fell to low levels in the briefly signaled delay component for three of four pigeons while remaining at or near baseline in the completely signaled delay component. In Experiment 3, low response rates under a 9-s unsignaled delay to reinforcement (tandem variable-interval 60 s fixed-time 9 s) increased when the delay was briefly signaled. The role of the brief stimulus as conditioned reinforcement may be a function of its temporal relation to food, and thus may be related to the eliciting function of the stimulus.     

Delay, Probability, and Social Discounting in A Public Goods Game

A human social discount function measures the value to a person of a reward to another person at a given social distance. Just as delay discounting is a hyperbolic function of delay, and probability discounting is a hyperbolic function of odds-against, social discounting is a hyperbolic function of social distance. Experiment 1 obtained individual social, delay, and probability discount functions for a hypothetical $75 reward; participants also indicated how much of an initial $100 endowment they would contribute to a common investment in a public good. Steepness of discounting correlated, across participants, among all three discount dimensions. However, only social and probability discounting were correlated with the public-good contribution; high public-good contributors were more altruistic and also less risk averse than low contributors. Experiment 2 obtained social discount functions with hypothetical $75 rewards and delay discount functions with hypothetical $1,000 rewards, as well as public-good contributions. The results replicated those of Experiment 1; steepness of the two forms of discounting correlated with each other across participants but only social discounting correlated with the public-good contribution. Most participants in Experiment 2 predicted that the average contribution would be lower than their own contribution.     

A job with a future? Delay discounting, magnitude effects, and domain independence of utility for career decisions

Decision-makers are often faced with inter-temporal choices in which they are required to choose between options that differ in both value of and delay to outcomes. In this study, assumptions of Discounted Utility Theory (DU) were tested within inter-temporal career choices. According to DU, the same discount rate should be applied regardless of the amount of delay and magnitude of the outcome. In addition, DU predicts that, for economic substitutes, the same discount rate should be applied to both commodities. Contrary to assumptions of DU, but consistent with a hyperbolic discounting model, graduating college students’ job choices reflected non-constant discount rates (i.e., Delay Effect) for future salary and preferred tasks. In addition, these students exhibited lower discount rates for salary and percentage of preferred tasks (i.e., Magnitude Effect). Finally, the two were not found to be economic substitutes as there was a low correlation between discount rate for salary and percentage of preferred tasks (i.e., Domain Independence). Implications for employee recruitment and human resource management are discussed.     

Now for Me,Later for Us? Effects of Group Context on Temporal Discounting

Delayed rewards are less valuable than immediate rewards. This well‐established finding has focused almost entirely on individual outcomes. However, are delayed rewards similarly discounted if they are shared by a group? The current article reports on three experiments exploring the effect of group context on delay discounting. Results indicate that discount rates of individual and group rewards were highly correlated, but that respondents were more willing to wait (decreased discounting) for shared outcomes than for individual outcomes. An explanatory model is proposed suggesting that decreased discount rates in group contexts may be due to the way the effects of both delay and social discounting are combined. That is, in a group context, a person values both a future reward (discounted by delay) and a present reward to another person (discounted by the social distance between them). The results are explained by a combined discount function containing a delay factor and a factor representing the social distance between the decision maker and group members. Practical implications of the fact that shared consequences can increase individual self‐control are also discussed. Copyright © 2011 John Wiley & Sons, Ltd.     

On the functions of the changeover delay

The function of changeover delays in producing matching was examined with pigeons responding on concurrent variable-interval variable-interval schedules. In Experiment 1, no changeover delay was compared to two different types of changeover delay. One type, designated generically as response-response but in the present example as peck-peck, was timed from the first response on the switched-to key; the other, designated generically as pause-response but in the present example as pause-peck, was timed from the last response on the switched-from key. High changeover rates occurred with no changeover delay. Peck-peck and pause-peck changeover delays produced low and intermediate changeover rates, respectively. In Experiment 2, pause-peck and peck-peck changeover delays were compared across a range of relative reinforcement rates. Similar matching relations developed despite differences in the changeover rates and local response patterns as a function of the type of changeover delay. In Experiment 3, both types of changeover delay yielded similar changeover rates when their obtained durations were equal via yoking. The results suggest that changeover delays function to separate responses on one key from reinforcers on the other or to delay reinforcement for changing over. In addition, the distribution of responding during and after the changeover delay may vary considerably without affecting matching.     

Effects of reinforcement rate and delay on the acquisition of lever pressing by rats.

The acquisition of lever pressing by naive rats, in the absence of shaping, was studied as a function of different rates and unsignaled delays of reinforcement. Groups of 3 rats were each exposed to tandem schedules that differed in either the first or the second component. First-component schedules were either continuous reinforcement or random-interval 15, 30, 60 or 120 s; second-component schedules were fixed-time 0, 1, 3, 6, 12, or 24 s. Rate of responding was low under continuous immediate reinforcement and higher under random-interval 15 s. Random interval 30-s and 60-s schedules produced lower rates that were similar to each other. Random-interval 120 s controlled the lowest rate in the immediate-reinforcement condition. Adding a constant 12-s delay to each of the first-component schedule parameters controlled lower response rates that did not vary systematically with reinforcement rate. The continuous and random-interval 60-s schedules of immediate reinforcement controlled higher global and first-component response rates than did the same schedules combined with longer delays, and first-component rates showed some graded effects of delay duration. In addition, the same schedules controlled higher second-component response rates in combination with a 1-s delay than in combination with longer delays. These results were related to those from previous studies on acquisition with delayed reinforcement as well as to those from similar reinforcement procedures used during steady-state responding.     

The Effects of Social Status and Culture on Delay Discounting

People generally tend to discount future outcomes in favor of smaller but immediate gains (i.e., delay discounting). This study examines the hypothesis that culture and social status moderate this tendency, as well as the alternative hypothesis that social status and culture influence delay discounting independently of each other. American and Japanese adults were asked to choose receiving hypothetical monetary rewards either immediately or receiving rewards of different amounts with a delay of 1 year. The results replicated previous findings and supported the alternative hypothesis. Delay discounting was lower when subjective socioeconomic status (i.e., an individual’s perception of her or his social rank) was higher. Also, the Japanese were less likely to discount future rewards than the Americans. However, there was no interaction between social status and culture in influencing the rates of delay discounting.     

TEACHERS' GENERALIZED USE OF DELAY AS A STIMULUS CONTROL PROCEDURE TO INCREASE LANGUAGE USE IN HANDICAPPED CHILDREN

In Experiment 1, classroom teachers were taught to delay their offers of help in naturally occurring situations, and thereby to provide additional opportunities for language use by six moderately retarded language-delayed children. The teachers introduced this delay technique in a multiple-baseline design across the six children. As delays were used, child verbal initiations increased. Follow-up assessment showed that teachers were maintaining greater than baseline levels of the delay technique after 10 weeks. Experiment 2 replicated the findings of Experiment 1, and included a more thorough maintenance assessment, while focusing on teachers' generalization of the delay technique. Teachers were found to generalize their use of delay to 56% of their monitored untaught opportunities. The two experiments show that (a) the delay technique is quick to teach and simple to implement, (b) delays do provide opportunities for children to initiate, (c) teachers can generalize their use of delay to novel self-selected situations, and (d) teachers can maintain their use of delays over time.     

Delay discounting in current and never-before cigarette smokers: similarities and differences across commodity,sign, and magnitude

Research has found that nicotine-dependent individuals delay discount monetary gains at a higher rate than matched controls. Delay discount rates, however, have also been found to vary across within-subject variables such as the magnitude of the outcome (e.g., 10 dollars or 1,000 dollars), whether the outcome constitutes a gain or a loss, and the commodity being evaluated (e.g., money or health). The present study comprehensively investigated the differences in delay discounting between current and never-before cigarette smokers and across these within-subject variables. Both groups exhibited a magnitude, sign, and commodity effect. Current smokers' delay discount rates for monetary outcomes, however, were higher than never-before smokers across all magnitudes and both signs. This trend was also found for delayed health outcomes, but failed to reach significance.     

The role of cultural orientation in bargaining under incomplete information: Differences in causal attributions

This research examines how differences in cultural orientation influence causal attributions and thus the behavioral outcomes in an incomplete information bargaining situation. Using ultimatum bargaining, three experiments demonstrate that acceptance rates differ across Western and East Asian cultures because of the differences in implicit theories of behavior. The results of Experiment 1 shows that East Asians are more sensitive to both external constraints and group influences but only when there is information about the opponent’s situation to discount personality traits. Experiment 2 shows that reasons for an opponent’s behavior mediate the influence of cultural orientation on bargaining outcomes when situational constraints are made salient. Experiment 3 shows that reasons for an opponent’s behavior based on the saliency of a group context mediate the influence of cultural orientation on behavior. The paper concludes by discussing the implications of the findings and suggesting directions for future research.     

Temporal discounting for monetary and close relationship outcomes

The present study extended research on intertemporal choice—in which individuals choose between outcomes that may be received immediately or after a delay—to close relationships. In Experiment 1, New Zealand university students aged 18 to 25 made decisions about hypothetical monetary and relationship outcomes, and in Experiment 2, about relationship outcomes which emphasized companionate or sexual aspects (Ns = 64). Both experiments found effects of delay and magnitude on temporal discounting rates, and domain independence for choices about close relationships, similar to previous studies with monetary outcomes. There were no significant gender differences. Overall, results suggest that people make intertemporal choices about relationships according to a similar process used to make decisions involving other types of outcomes.     

Health and educational outcomes of children who experienced severe neonatal medical complications.

To determine the long-term developmental and educational outcomes of a sample of low birthweight infants with intraventricular hemorrhage (IVH), the authors conducted developmental assessments and interviews 8 years after the initiation of an early intervention project. At the time of the follow-up, 62% of the children were experiencing some developmental or behavior problems, with visual impairments, cerebral palsy, and attention deficits occurring most frequently. Grade of IVH and the number of days spent in the neonatal intensive care unit were the best predictors of later developmental delays. The sample also scored below average on school achievement; approximately 30% of those in school were eligible for special education services. These findings corroborate results from investigations with similar populations whose birth characteristics put them at risk for subsequent developmental delay.     

Unsignaled delay of reinforcement, relative time, and resistance to change

Two experiments with pigeons examined the effects of unsignaled, nonresetting delays of reinforcement on responding maintained by different reinforcement rates. In Experiment 1, 3-s unsignaled delays were introduced into each component of a multiple variable-interval (VI) 15-s VI 90-s VI 540-s schedule. When considered as a proportion of the preceding immediate reinforcement baseline, responding was decreased similarly for the three multiple-schedule components in both the first six and last six sessions of exposure to the delay. In addition, the relation between response rates and reinforcement rates was altered such that both parameters of the single-response version of the matching law (i.e., k and Re) were decreased. Experiment 2 examined the effects of unsignaled delays ranging from 0.5 s to 8.0 s on responding maintained by a multiple VI 20-s VI 120-s schedule of reinforcement. Response rates in both components increased with brief unsignaled delays and decreased with longer delays. As in Experiment 1, response rates as a proportion of baseline were affected similarly for the two components in both the first six and last six sessions of exposure to the delay. Unlike delays imposed between two stimulus events, the effects of delays between responses and reinforcers do not appear to be attenuated when the average time between reinforcers is longer. In addition, the disruptions produced by unsignaled delays appear to be inconsistent with the general finding that responding maintained by higher rates of reinforcement is less resistant to change.     

Responding of pigeons under variable-interval schedules of signaled-delayed reinforcement: effects of delay-signal duration.

Two experiments with pigeons examined the relation of the duration of a signal for delay ("delay signal") to rates of key pecking. The first employed a multiple schedule comprised of two components with equal variable-interval 60-s schedules of 27-s delayed food reinforcement. In one component, a short (0.5-s) delay signal, presented immediately following the key peck that began the delay, was increased in duration across phases; in the second component the delay signal initially was equal to the length of the programmed delay (27 s) and was decreased across phases. Response rates prior to delays were an increasing function of delay-signal duration. As the delay signal was decreased in duration, response rates were generally higher than those obtained under identical delay-signal durations as the signal was increased in duration. In Experiment 2 a single variable-interval 60-s schedule of 27-s delayed reinforcement was used. Delay-signal durations were again increased gradually across phases. As in Experiment 1, response rates increased as the delay-signal duration was increased. Following the phase during which the signal lasted the entire delay, shorter delay-signal-duration conditions were introduced abruptly, rather than gradually as in Experiment 1, to determine whether the gradual shortening of the delay signal accounted for the differences observed in response rates under identical delay-signal conditions in Experiment 1. Response rates obtained during the second exposures to the conditions with shorter signals were higher than those observed under identical conditions as the signal duration was increased, as in Experiment 1. In both experiments, rates and patterns of responding during delays varied greatly across subjects and were not systematically related to delay-signal durations. The effects of the delay signal may be related to the signal's role as a discriminative stimulus for adventitiously reinforced intradelay behavior, or the delay signal may have served as a conditioned reinforcer by virtue of the temporal relation between it and presentation of food.     

Time is on my side: optimism in intertemporal choice

The present research examines the role of optimism on time preferences for both losses and gains. It is argued that optimism has asymmetric effects on time preferences for gains versus losses: one reason why decision makers prefer immediate gains is because they are optimistic that these gains will be followed by additional gains in future. In contrast, decision makers prefer to delay losses because they are optimistic that losses are avoidable in the future. Optimism about outcomes affects time preferences for both gains and losses, such that low optimism reduces the discount rates while increased optimism is associated with higher discount rates. This prediction was supported in two different domains: monetary outcomes (Study 1), and health (Study 2). Implications of these results for both research practice and time preferences in the real world are discussed.     

Delay discounting of different outcomes: Review and theory

Steep delay discounting is characterized by a preference for small immediate outcomes relative to larger delayed outcomes and is predictive of drug abuse, risky sexual behaviors, and other maladaptive behaviors. Nancy M. Petry was a pioneer in delay discounting research who demonstrated that people discount delayed monetary gains less steeply than they discount substances with abuse liability. Subsequent research found steep discounting for not only drugs, but other nonmonetary outcomes such as food, sex, and health. In this systematic review, we evaluate the hypotheses proposed to explain differences in discounting as a function of the type of outcome and explore the trait- and state-like nature of delay discounting. We found overwhelming evidence for the state-like quality of delay discounting: Consistent with Petry and others' work, nonmonetary outcomes are discounted more steeply than monetary outcomes. We propose two hypotheses that together may account for this effect: Decreasing Future Preference and Decreasing Future Worth. We also found clear evidence that delay discounting has trait-like qualities: People who steeply discount monetary outcomes steeply discount nonmonetary outcomes as well. The implication is that changing delay discounting for one outcome could change discounting for other outcomes.     

Effects of signaled versus unsignaled delay of reinforcement on choice

Pigeons chose between 5-s and 15-s delay-of-reinforcement alternatives. The first key peck to satisfy the choice schedule began a delay timer, and food was delivered at the end of the interval. Key pecks during the delay interval were measured, but had no scheduled effect. In Experiment 1, signal conditions and choice schedules were varied across conditions. During unsignaled conditions, no stimulus change signaled the beginning of a delay interval. During differential and nondifferential signal conditions, offset of the choice stimuli and onset of a delay stimulus signaled the beginning of a delay interval. During differential signal conditions, different stimuli were correlated with the 5-s and 15-s delays, whereas the same stimulus appeared during both delay durations during nondifferential signal conditions. Pigeons showed similar, extreme levels of preference for the 5-s delay alternative during unsignaled and differentially signaled conditions. Preference levels were reliably lower with nondifferential signals. Experiment 2 assessed preference with two pairs of unsignaled delays in which the ratio of delays was held constant but the absolute duration was increased fourfold. No effect of absolute duration was found. The results highlight the importance of delayed primary reinforcement effects and challenge models of choice that focus solely on conditioned reinforcement.     

Asymmetric discounting in intertemporal choice: a query-theory account

People are impatient and discount future rewards more when they are asked to delay consumption than when they are offered the chance to accelerate consumption. The three experiments reported here provide a process-level account for this asymmetry, with implications for designing decision environments that promote less impulsivity. In Experiment 1, a thought-listing procedure showed that people decompose discount valuation into two queries. Whether one considers delayed or accelerated receipt of a gift certificate influences the order in which memory is queried to support immediate versus delayed consumption, and the order of queries affects the relative number of patient versus impatient thoughts. Relative frequency and clustering of impatient thoughts predicts discounting and mediates the discounting asymmetry. Experiment 2 implicated query order causally: When participants listed reasons for immediate versus delayed consumption in the order used spontaneously in acceleration and delay decisions, the discounting asymmetry was replicated; reversing the order in which reasons were listed eliminated the asymmetry. The results of Experiment 3, which used an implicit-memory task, support a memory-interference account of the effect of query order.     

Conditioned Reflex and Perception: The Concept of Syncretism

Delay discounting occurs when the subjective value of an outcome decreases because its delivery is delayed. Past research has shown that how steeply participants discount an outcome varies inversely with the value of previously discounted outcomes. In the present study, participants discounted the same hypothetical monetary outcome ($1,000) after their hypothetical annual income was halved (Experiment 1) or doubled (Experiment 2). Rates of discounting decreased and increased, respectively, after these manipulations (although a similar change in discounting was observed for the control and treatment groups in Experiment 2). These results suggest that altering the context in which the discounting task is framed alters the subjective value of the outcome itself, in this case money. This result has implications for understanding contrast effects that are observed in rates of discounting, as well as for researchers and practitioners who are interested in determining methods for altering how individuals discount delayed outcomes.