Effects of reinforcer consumption and magnitude on response rates during noncontingent reinforcement

Results of previous research on the effects of noncontingent reinforcement (NCR) have been inconsistent when magnitude of reinforcement was manipulated. We attempted to clarify the influence of NCR magnitude by including additional controls. In Study 1, we examined the effects of reinforcer consumption time by comparing the same magnitude of NCR when session time was and was not corrected to account for reinforcer consumption. Lower response rates were observed when session time was not corrected, indicating that reinforcer consumption can suppress response rates. In Study 2, we first selected varying reinforcer magnitudes (small, medium, and large) on the basis of corrected response rates observed during a contingent reinforcement condition and then compared the effects of these magnitudes during NCR. One participant exhibited lower response rates when large-magnitude reinforcers were delivered; the other ceased responding altogether even when small-magnitude reinforcers were delivered. We also compared the effects of the same NCR magnitude (medium) during 10-min and 30-min sessions. Lower response rates were observed during 30-min sessions, indicating that the number of reinforcers consumed across a session can have the same effect as the number consumed per reinforcer delivery. These findings indicate that, even when response rate is corrected to account for reinforcer consumption, larger magnitudes of NCR (defined on either a per-delivery or per-session basis) result in lower response rates than do smaller magnitudes.     

Every reinforcer counts: reinforcer magnitude and local preference

Six pigeons were trained on concurrent variable-interval schedules. Sessions consisted of seven components, each lasting 10 reinforcers, with the conditions of reinforcement differing between components. The component sequence was randomly selected without replacement. In Experiment 1, the concurrent-schedule reinforcer ratios in components were all equal to 1.0, but across components reinforcer-magnitude ratios varied from 1:7 through 7:1. Three different overall reinforcer rates were arranged across conditions. In Experiment 2, the reinforcer-rate ratios varied across components from 27:1 to 1:27, and the reinforcer-magnitude ratios for each alternative were changed across conditions from 1:7 to 7:1. The results of Experiment 1 replicated the results for changing reinforcer-rate ratios across components reported by Davison and Baum (2000, 2002): Sensitivity to reinforcer-magnitude ratios increased with increasing numbers of reinforcers in components. Sensitivity to magnitude ratio, however, fell short of sensitivity to reinforcer-rate ratio. The degree of carryover from component to component depended on the reinforcer rate. Larger reinforcers produced larger and longer postreinforcer preference pulses than did smaller reinforcers. Similar results were found in Experiment 2, except that sensitivity to reinforcer magnitude was considerably higher and was greater for magnitudes that differed more from one another. Visit durations following reinforcers measured either as number of responses emitted or time spent responding before a changeover were longer following larger than following smaller reinforcers, and were longer following sequences of same reinforcers than following other sequences. The results add to the growing body of research that informs model building at local levels.     

Zebrafish choice behavior is sensitive to reinforcer rate,immediacy, and magnitude ratios

Behavioral flexibility has, in part, been defined by choice behavior changing as a function of changes in reinforcer payoffs. We examined whether the generalized matching law quantitatively described changes in choice behavior in zebrafish when relative reinforcer rates, delays/immediacy, and magnitudes changed between two alternatives across conditions. Choice was sensitive to each of the three reinforcer properties. Sensitivity estimates to changes in relative reinforcer rates were greater when 2 variable-interval schedules were arranged independently between alternatives (Experiment 1a) than when a single schedule pseudorandomly arranged reinforcers between alternatives (Experiment 1b). Sensitivity estimates for changes in relative reinforcer immediacy (Experiment 2) and magnitude (Experiment 3) were similar but lower than estimates for reinforcer rates. These differences in sensitivity estimates are consistent with studies examining other species, suggesting flexibility in zebrafish choice behavior in the face of changes in payoff as described by the generalized matching law.     

Age‐related differences in delay discounting: Immediate reward,reward magnitude,and social influence

Delay discounting refers to the tendency of individuals to subjectively devalue rewards that are to be received in the future, with high rates of delay discounting being associated with a variety of maladaptive life outcomes (e.g., unhealthy dietary and exercise behaviors). The current study explored the psychological and social processes involved in adult age‐related differences in delay discounting of monetary rewards. Younger adults exhibited higher levels of delay discounting than older adults. This increased level of patience in older adults was found whether smaller‐sooner rewards were to be received immediately or in the future. However, there was an interaction with reward magnitude, whereby younger adults exhibited higher levels of delay discounting for smaller reward magnitudes but not larger reward magnitudes. Social influence on delay discounting was investigated by having participants complete three phases of the delay‐discounting task: an individual precollaboration phase, a collaboration phase in age‐group‐matched dyads, and an individual postcollaboration phase. A convergence effect was observed in that dyad members' postcollaboration choices were significantly more similar compared to their baseline choices during the precollaboration phase. Moreover, levels of convergence were comparable between younger and older adults, suggesting age invariance in social influence on delay discounting. The current results demonstrate a degree of malleability in delay discounting that extends into older adulthood, making interventions targeting the construct a promising avenue for future research.     

A Reinforcer Pathology perspective on relapse

The Reinforcer Pathology theory proposes conditions under which drugs emerge as excessively preferred reinforcers compared to other available reinforcers among drug users. The theory highlights 2 key variables as important determinants of drug preference: (a) excessive preference for immediate rewards (high discounting of future rewards); and (b) excessive valuation for addictive reinforcers (e.g., drugs). Two iterations of the Reinforcer Pathology theory exist with the latest iteration (i.e., Reinforcer Pathology 2.0) specifying that the temporal window of reinforcer integration (measured by delay discounting) is a determinant of reinforcer value. The 2 iterations of the Reinforcer Pathology theory are described. A novel insight and understanding of abstinence and relapse from a Reinforcer Pathology perspective, limitations, and future directions are discussed. The Reinforcer Pathology theory continues the long-standing efforts to scientifically understand and better define novel concepts and methods to further translational research and improve treatment outcomes. Exploring the complementary relation between the Reinforcer Pathology perspective and other current approaches could have a valuable effect.     

Reinforcer pathology II: Reward magnitude,reward delay,and demand for alcohol collectively relate to college students' alcohol related problems

The reinforcer pathologies model of addiction posits that two characteristic patterns of operant behavior characterize addiction. Specifically, individuals suffering from addiction have elevated levels of behavioral economic demand for their substances of abuse and have an elevated tendency to devalue delayed rewards (reflected in high delay discounting rates). Prior research has demonstrated that these behavioral economic markers are significant predictors of many of college students' alcohol-related problems. Delay discounting, however, is a complex behavioral performance likely undergirded by multiple behavioral processes. Emerging analytical approaches have isolated the role of participants' sensitivity to changes in reinforcer magnitude and changes in reinforcer delay. The current study uses these analytic approaches to compare participants' discounting of money versus alcohol, and to build regression models that leverage these new insights to predict a wider range of college students' alcohol related problems. Using these techniques, we were able to 1) demonstrate that individuals differed in their sensitivity to magnitudes of alcohol versus money, but not sensitivity to delays to those commodities and 2) that we could use our behavioral economic measures to predict a range of students' alcohol related problems.     

Concurrent schedules: reinforcer magnitude effects

Five pigeons were trained on pairs of concurrent variable-interval schedules in a switching-key procedure. The arranged overall rate of reinforcement was constant in all conditions, and the reinforcer-magnitude ratios obtained from the two alternatives were varied over five levels. Each condition remained in effect for 65 sessions and the last 50 sessions of data from each condition were analyzed. At a molar level of analysis, preference was described well by a version of the generalized matching law, consistent with previous reports. More local analyses showed that recently obtained reinforcers had small measurable effects on current preference, with the most recently obtained reinforcer having a substantially larger effect. Larger reinforcers resulted in larger and longer preference pulses, and a small preference was maintained for the larger-magnitude alternative even after long inter-reinforcer intervals. These results are consistent with the notion that the variables controlling choice have both short- and long-term effects. Moreover, they suggest that control by reinforcer magnitude is exerted in a manner similar to control by reinforcer frequency. Lower sensitivities when reinforcer magnitude is varied are likely to be due to equal frequencies of different sized preference pulses, whereas higher sensitivities when reinforcer rates are varied might result from changes in the frequencies of different sized preference pulses.     

Delay discounting in current and never-before cigarette smokers: similarities and differences across commodity,sign, and magnitude

Research has found that nicotine-dependent individuals delay discount monetary gains at a higher rate than matched controls. Delay discount rates, however, have also been found to vary across within-subject variables such as the magnitude of the outcome (e.g., 10 dollars or 1,000 dollars), whether the outcome constitutes a gain or a loss, and the commodity being evaluated (e.g., money or health). The present study comprehensively investigated the differences in delay discounting between current and never-before cigarette smokers and across these within-subject variables. Both groups exhibited a magnitude, sign, and commodity effect. Current smokers' delay discount rates for monetary outcomes, however, were higher than never-before smokers across all magnitudes and both signs. This trend was also found for delayed health outcomes, but failed to reach significance.     

Delay discounting of qualitatively different reinforcers in rats

Humans discount larger delayed rewards less steeply than smaller rewards, whereas no such magnitude effect has been observed in rats (and pigeons). It remains possible that rats' discounting is sensitive to differences in the quality of the delayed reinforcer even though it is not sensitive to amount. To evaluate this possibility, Experiment 1 examined discounting of qualitatively different food reinforcers: highly preferred versus nonpreferred food pellets. Similarly, Experiment 2 examined discounting of highly preferred versus nonpreferred liquid reinforcers. In both experiments, an adjusting-amount procedure was used to determine the amount of immediate reinforcer that was judged to be of equal subjective value to the delayed reinforcer. The amount and quality of the delayed reinforcer were varied across conditions. Discounting was well described by a hyperbolic function, but no systematic effects of the quantity or the quality of the delayed reinforcer were observed.     

Preference and resistance to change with constant- and variable-duration terminal links: independence of reinforcement rate and magnitude

Pigeons responded in a three-component multiple concurrent-chains procedure in which the variable-interval reinforcement schedules were the same across components but magnitudes differed across components. The terminal links were arranged either as a variable delay followed by presentation of a reinforcer ("variable duration") or as a fixed period of access to the schedule during which a variable number of reinforcers could be earned ("constant duration"). Relative reinforcement rate was varied parametrically across both types of conditions. After baseline training in each condition, resistance to change of terminal-link responding was assessed by delivering food during the initial links according to a variable-time schedule. Both preference and resistance to change were more sensitive to reinforcement-rate differences in the constant-duration conditions. Sensitivities of preference and resistance to change to relative reinforcement rate did not change depending on relative reinforcement magnitude. Taken together, these results confirm and extend those of prior studies, and suggest that reinforcement rate and magnitude combine additively to determine preference and resistance to change. A single structural relation linking preference and resistance to change describes all the data from this and several related studies.     

Reinforcement magnitude: an evaluation of preference and reinforcer efficacy

Consideration of reinforcer magnitude may be important for maximizing the efficacy of treatment for problem behavior. Nonetheless, relatively little is known about children's preferences for different magnitudes of social reinforcement or the extent to which preference is related to differences in reinforcer efficacy. The purpose of the current study was to evaluate the relations among reinforcer magnitude, preference, and efficacy by drawing on the procedures and results of basic experimentation in this area. Three children who engaged in problem behavior that was maintained by social positive reinforcement (attention, access to tangible items) participated. Results indicated that preference for different magnitudes of social reinforcement may predict reinforcer efficacy and that magnitude effects may be mediated by the schedule requirement.     

Self-control in mentally retarded adolescents: choice as a function of amount and delay of reinforcement.

Three severely mentally retarded adolescents were studied under discrete-trial procedures in which a choice was arranged between edible reinforcers that differed in magnitude and, in some conditions, delay. In the absence of delays the larger reinforcer was consistently chosen. Under conditions in which the smaller reinforcer was not delayed, increasing the delay to delivery of the larger reinforcer decreased the percentage of trials in which that reinforcer was chosen. All subjects directed the majority of choice responses to the smaller reinforcer when the larger reinforcer was sufficiently delayed, although the value at which this occurred differed across subjects. Under conditions in which the larger reinforcer initially was sufficiently delayed to result in preference for the smaller one, progressively increasing in 5-s increments the delay to both reinforcers increased percentage of trials with the larger reinforcer chosen. At sufficiently long delays, 2 of the subjects consistently chose the larger, but more delayed, reinforcer, and the 3rd subject chose that reinforcer on half of the trials. These results are consistent with the findings of prior studies in which adult humans responded to terminate noise and pigeons responded to produce food.     

A Measure of Delay Discounting Within the Academic Domain

It is important to better understand the decision‐making processes involved in student procrastination, in order to develop interventions that reduce this common problem. Students may procrastinate because studying produces delayed reinforcers; however, no task measuring delay discounting of academic outcomes currently exists. In Experiment 1, we developed and piloted a measure of academic discounting modeled on titrating‐amount tasks successfully used in the discounting literature. Participants made hypothetical choices between working for money (the smaller, sooner reinforcer) and working on an assignment that was due at various times (the larger, later reinforcer). Participants showed systematic decreases in the subjective value of the assignment as a function of delay, and the hyperbolic and hyperboloid models described the shape of this decrease in value well. In general, larger delayed rewards are discounted less steeply than smaller delayed rewards (the magnitude effect). In Experiment 2, we observed the magnitude effect in academic discounting: Participants discounted a “not important” assignment more steeply than an “important” assignment. In the hyperboloid model, this change was captured by an increase in the s parameter. Results provide support for the validity of the academic discounting task. Copyright © 2018 John Wiley & Sons, Ltd.     

After-effects of reinforcement magnitude: Dependence upon context

On a fixed-interval schedule with rat subjects the duration of the post-reinforcement pause was found to be an increasing function of the magnitude of the preceding reinforcer. This relationship was observed when two magnitudes were contrasted closely in time, but not when the subjects were trained on each magnitude until the establishment of stable responding. After the behaviour was stable, the effect of the magnitude of reinforcement re-emerged when 50% of the scheduled reinforcers were omitted. Thus, the positive relationship between the magnitude of reinforcement and the duration of the post-reinforcement pause depended on the context of presentation of a given magnitude.     

The effects of constant versus varied reinforcers on preference and resistance to change

Previous research has demonstrated that factors such as reinforcer frequency, amount, and delay have similar effects on resistance to change and preference. In the present study, 4 boys with autism made choices between a constant reinforcer (one that was the same food item every trial) and a varied food reinforcer (one that varied randomly between three possible food items). For all 4 boys, varied reinforcers were preferred over constant reinforcers, and they maintained higher response rates than constant reinforcers. In addition, when a distraction (a video clip) was introduced, responding maintained by varied reinforcers was more resistant to distraction than responding maintained by constant reinforcers. Thus, the present experiment extended the generality of the relation between preference and resistance to change to variation in reinforcer quality.     

Impulsive choice and pre‐exposure to delays: iv. effects of delay‐ and immediacy‐exposure training relative to maturational changes in impulsivity

Impulsive choice describes preference for smaller, sooner rewards over larger, later rewards. Excessive delay discounting (i.e., rapid devaluation of delayed rewards) underlies some impulsive choices, and is observed in many maladaptive behaviors (e.g., substance abuse, gambling). Interventions designed to reduce delay discounting may provide therapeutic gains. One such intervention provides rats with extended training with delayed reinforcers. When compared to a group given extended training with immediate reinforcers, delay‐exposed rats make significantly fewer impulsive choices. To what extent is this difference due to delay‐exposure training shifting preference toward self‐control or immediacy‐exposure training (the putative control group) shifting preference toward impulsivity? The current study compared the effects of delay‐ and immediacy‐exposure training to a no‐training control group and evaluated within‐subject changes in impulsive choice across 51 male Wistar rats. Delay‐exposed rats made significantly fewer impulsive choices than immediacy‐exposed and control rats. Between‐group differences in impulsive choice were not observed in the latter two groups. While delay‐exposed rats showed large, significant pre‐ to posttraining reductions in impulsive choice, immediacy‐exposed and control rats showed small reductions in impulsive choice. These results suggest that extended training with delayed reinforcers reduces impulsive choice, and that extended training with immediate reinforcers does not increase impulsive choice.     

Discrete-trial choice in pigeons: Effects of reinforcer magnitude

The preference of pigeons for large reinforcers which occasionally followed a response versus small reinforcers which invariably followed a response was studied in a discrete-trial situation. Two differently colored keys were associated with the two reinforcement alternatives, and preference was measured as the proportion of choice trials on which the key associated with uncertain reinforcement was pecked. A combination of choice and guidance trials insured that received distributions of reinforcement equalled the scheduled distributions. For five of six subjects, preference for the uncertain reinforcer appeared to be a linear function of the magnitude of the certain reinforcer. In addition, there was greater preference for the response alternative associated with uncertain reinforcement than would be expected on the basis of net reinforcer value.     

Unequal reinforcer magnitudes and relative preference for cooperation in the dyad

College-student subjects, who were paired with a confederate, chose to respond either independently or cooperatively for money reinforcers. The subject's relative preference for cooperation was assessed by a procedure (analogous to the psychophysical method of limits) in which response choice was monitored as reinforcer magnitude for one response mode was systematically varied while the other remained constant. Relative preference for cooperation was assessed when the confederate's payoff for cooperation was greater than the subject's (Experiment I) and when the confederate's payoff for independent responding was less than the subject's (Experiment II). For some subjects, changes in the confederate's reinforcer magnitudes resulted in shifts in relative preference for cooperation, which reduced the earnings differences, even though these preference shifts reduced the subject's absolute earnings. For those subjects for whom within-dyad differences in reinforcer magnitude produced no effect, a changeover button was introduced that allowed the subject to eliminate the payoff difference without reducing her own earnings; some subjects used this changeover button to eliminate earnings differences. Thus, the behavior of subjects varied, in part, as a function of reinforcer magnitudes provided for the confederate.     

Choice between delayed reinforcers in an adjusting-delay schedule: The effects of absolute reinforcer size and deprivation level

Choice between two reinforcers differing in magnitude and delay was investigated in rats using an adjusting-delay discrete-trials schedule in which the two reinforcers were associated with two levers (A and B). The delay to Reinforcer A (the smaller reinforcer) was always 2 sec, whereas the delay to Reinforcer B was varied in accordance with the distribution of choices in successive blocks of trials. In Experiment 1, the mean delay to the large reinforcer during the last 5 of 60 training sessions was greater when the rats were maintained at 80% than when they were maintained at 90% of their free-feeding body weights. In Experiment 2, the delay to the larger reinforcer was greater when the two reinforcers consisted of one and two 45-mg food pellets than when they consisted of three and six pellets. The results are consistent with a model of “self-control” which posits hyperbolic relations between reinforcer value and reinforcer magnitude, and between reinforcer value and delay of reinforcement.     

Smokers discount their drug of abuse in the same way as other consumable rewards

Recent research shows that drug abusers discount delayed monetary rewards more than nonabusers do, and they discount delayed substances of abuse (e.g., drugs) more than delayed money. Furthermore, non-drug-abusers discount food and substances of abuse (e.g., alcohol), more than money. Here, we compare the delay and probability discounting of money with that of a directly consumable reward (chocolate) and with that of a substance of abuse (cigarettes), in a drug-using population (smokers). In line with previous research, we found in two experiments that delay discounting differentiated between smokers and nonsmokers, and between money and a nonabused directly consumable reward (chocolate). In addition, our results show that there appears to be no difference in the extent to which smokers discount their abused substance compared to another directly consumable reward. These findings support the contention that drugs and food are part of the same category of primary reinforcers, whereas money is discounted differently, as a conditioned reinforcer.