Rats place greater value on rewards produced by high effort: An animal analogue of the “effort justification” effect

The effort justification phenomenon, in which greater value is given to rewards that require more effort to obtain, is frequently explained in terms of cognitive dissonance (Festinger & Carlsmith, 1959). Here, we employed a novel combination of operant and lick analysis techniques to show that rats place more value on a sucrose reward when it follows high effort than when the same reward follows low effort. This is the first demonstration of a direct analogue of the effort justification phenomenon in nonhuman animals. As the behavior of rats is normally considered in terms of relatively simple mechanisms, the current results question the need for complex cognitive accounts for the effort justification phenomenon (in rats or humans). As an alternative, we examine the possibility that high effort produces an aversive state that enhances reward value by a process of contrast.     

The Isolation of Motivational, Motoric, and Schedule Effects on Operant Performance: A Modeling Approach

Dissociating motoric and motivational effects of pharmacological manipulations on operant behavior is a substantial challenge. To address this problem, we applied a response‐bout analysis to data from rats trained to lever press for sucrose on variable‐interval (VI) schedules of reinforcement. Motoric, motivational, and schedule factors (effort requirement, deprivation level, and schedule requirements, respectively) were manipulated. Bout analysis found that interresponse times (IRTs) were described by a mixture of two exponential distributions, one characterizing IRTs within response bouts, another characterizing intervals between bouts. Increasing effort requirement lengthened the shortest IRT (the refractory period between responses). Adding a ratio requirement increased the length and density of response bouts. Both manipulations also decreased the bout‐initiation rate. In contrast, food deprivation only increased the bout‐initiation rate. Changes in the distribution of IRTs over time showed that responses during extinction were also emitted in bouts, and that the decrease in response rate was primarily due to progressively longer intervals between bouts. Taken together, these results suggest that changes in the refractory period indicate motoric effects, whereas selective alterations in bout initiation rate indicate incentive‐motivational effects. These findings support the use of response‐bout analyses to identify the influence of pharmacological manipulations on processes underlying operant performance.     

The temporal course of the frustration effect

General activity subsequent to reward (R) and nonreward (N) was monitored at 5-sec intervals with a stabilimeter in the runway goal box. Activity of never-rewarded control Ss was also measured. In Expt 1 it was found that the frustration effect (difference between N- and R-trial activities) disappeared after about 40 sec of goal box confinement. This disappearance of the frustration effect was due to activity increase on R trials rather than activity decrease on N trials as a function of time. Comparison of N-trial activity with control group activity indicated that frustration does not dissipate within 60 sec. Expt 2 investigated activity following reward and nonreward as a function of reward magnitude. Evidence from these experiments suggests that the late R-trial activity increase results from frustration, possibly conditioned to apparatus cues on N trials.     

Temporally spaced responding by pigeons: development and effects of deprivation and extinction

In the first five or six sessions on a DRL 20-sec schedule of reinforcement there developed a stable performance characterized by a relatively constant conditional probability of occurrence (IRTs/op) of interresponse times (IRTs) of durations greater than 5 or 6 sec. Extinction and the level of deprivation changed both the overall rate of responding and the form of the function relating the duration of an IRT to its value of IRTs/op. The value of IRTs/op decreased more rapidly for short than for longer IRTs, resulting in the emergence of a finer discrimination of IRT duration.     

Differential reinforcement of low rates performance by impulsive and reflective children

Third-grade boys classified as either cognitively impulsive or reflective were reinforced for key pressing according to a DRL (differential reinforcement of low rates) 6-sec schedule of reinforcement. Half of each group received instructions about the behavioral requirements for obtaining reinforcements. Prior to DRL training, impulsive Ss showed a low probability of key press responding at long interresponse time (IRT) intervals while reflective Ss exhibited an equal probability of terminating either short or long IRTs. During training and in the absence of instructions, impulsives exhibited a less precise temporal discrimination, characterized by a greater predominance of response bursts (0–2 sec IRTs) following reinforcements, than reflective Ss. While impulsive and reflective Ss displayed similar frequencies of collateral behavior between successively reinforced responses, impulsives engaged in the reinforced response more frequently and tended (p < .08) to obtain fewer reinforcements. Instructions served to enhance the DRL performance.     

I'm worth more than you! Effects of reward interdependence on performance,cohesion, emotion and effort during team competition

ObjectivesTo examine the effects of reward structures on the performance of a motor task. To evaluate the effects of reward interdependence on performance, cohesion, emotion, and effort during intergroup (team) competition.DesignAn experimental design was used to compare the effects of reward interdependence (no, low, high) on performance of a strength and endurance task.MethodParticipants (N = 111) performed a 3-min handgrip task as a member of a team of four under three reward interdependence conditions (no; low, with an even split of prize money; high, with an uneven split of prize money) in head-to-head competitions against another team. Task performance was assessed using the cumulative force production total. Task-related cohesion, enjoyment, anxiety and effort were measured using self-report scales.ResultsPerformance was better with rewards than no reward, and better with high than low reward interdependence. Team cohesion was highest with low reward interdependence. Effort was greater with rewards than no reward. Anxiety and enjoyment did not vary among the reward conditions. Mediation analyses indicated that increased cohesion mediated improvements in performance from no reward to low reward interdependence conditions, and increased effort mediated improvements in performance from no reward to both low and high reward interdependence conditions.ConclusionPerformance of a simple physical task in team competition was facilitated by rewards, with optimal performance associated with unequal rewards (i.e., performance-related pay). The benefits of performing with rewards compared to no rewards were explained by increased cohesion and effort. Social interdependence theory can help explain performance of simple motor tasks during team-based competitions. The findings have implications for the pay structures adopted by sports teams.     

Effort Effect: An Artifact of Effort-Produced Partial Responses?

The Psychological Record - It was previously found (Arm us, 1986, 1988, 1990) that greater response effort requirements led to a greater percentage of short (&lt; 1 s) interresponse times...     

Incentive shift in the rat following training to resist punishment

An incentive shift paradigm was used to test for the similarity of fear and frustration. In Experiment 1, rats trained to resist electric shock punishment showed neither a negative contrast effect nor any performance decrement when reward was shifted from 10 to one pellet. Experiment 2 replicated the basic findings of Experiment 1, but also showed that punishment training did not influence the magnitude of performance shift for animals receiving increases in reward magnitude. Finally, Experiment 3 additionally found that rats sensitized to punishment showed an increase in negative contrast effect. These results support the hypothesized functional similarity between conditioned fear and conditioned frustration with learned persistence or sensitivity to one generalizing to the other as suggested by Amsel's (1972) theory of persistence.     

Briefly delayed reinforcement: An interresponse time analysis

Key-peck responding of pigeons was compared under VI or DRL schedules arranging immediate reinforcement and briefly (.5 sec) delayed reinforcement. Delays were either signaled by a blackout in the chamber, unsignaled, or unsignaled with an additional requirement that responding not occur during the .5 sec interval immediately preceding reinforcement (response delay). Relative to the immediate reinforcement condition, response rates increased during the unsignaled delay, decreased during the signaled delay, and were inconsistent during the response delay condition. An analysis of interresponse times (IRTs) under the different conditions revealed a substantial increase in the frequency of short (0 to .5 sec) IRTs during the unsignaled condition and generally during the response delay conditions compared to that during the immediate reinforcement baseline. Signaled delays decreased the frequency of short (0 to .5 sec) IRTs relative to the immediate reinforcement condition. The results suggest that brief unsignaled delays and, in many instances, response delays increase the frequency of short IRTs by eliminating constraints on responding.     

Differential reinforcement of interresponse times with controlled probability of reinforcement per response

Pigeons were presented food after interresponse times (IRTs) longer or shorter than a fixed percentage of their most recent IRTs. This procedure controlled probability of reinforcement per response while still allowing different classes of IRTs to be reinforced differentially. Support was found for IRT-reinforcement theory in that response rates were determined by the degree and direction of differential reinforcement of IRTs, but were relatively independent of probability of reinforcement per response and of the length of the control system's IRT memory. Stimulus control of these differential response rates was also demonstrated.     

Runway acquisition and extinction as a joint function of magnitude of reward and percentage of rewarded acquisition trials

To determine the joint effects of partial reward and reward magnitude on acquisition and extinction rates, and on acquisition and extinction asymptotes, 215 Wistar albino rats were trained in a Hunter straight runway. The experimental design was a 4 × 4 × 2 factorial combining four reward magnitudes, four reward percentages, and two experimenters. The data revealed that the acquisition rate was an increasing function of both percentage and magnitude of reward and that neither reward magnitude nor percentage of reward significantly affected acquisition asymptote. For extinction, it was found that, for continuous schedules, the larger the reward magnitude the less the resistance to extinction and, for partial schedules, the larger the reward magnitude the greater the resistance to extinction. These results were interpreted within the framework of the sequential effects hypothesis (Capaldi, 1966).     

Schedules of reward shift in the double runway

Eighty food deprived rats received 62 trials in a double runway. On Trials 1-30, reward in the first goal box (GB1) was either always two food pellets or always zero pellets. All subjects received two pellets in the second goal box (GB2). On Trials 31-62 subjects in each preshift group (GB1 reward or GB1 nonreward) were shifted to the opposite GB1 reward level on 0, 25, 50, 75 or 100% of occasions. GB2 reward remained unaltered in all cases. For subjects experiencing reward decrease, second runway (A2) run and goal speeds after nonreward were generally enhanced, both within-group and in comparison with never rewarded controls. No such effect was evident on A2 start speed, nor was there any evidence to suggest that A2 performance after decreased reward was a function of the schedule of decrease. Increased GB1 reward resulted in general within-group impairment of A2 start and run speeds, with no effect on A2 goal performance. However, comparisons of speeds after increased reward with those of always rewarded controls revealed no difference on A2 start or run but indicated impairment of A2 goal performance. With the 50% schedule of reward increase, A2 run speeds after nonreward (the training level) exceeded those of never rewarded controls. Results are discussed with reference to McHose's contrast account of double runway phenomena and Amsel's frustration theory.     

Dopamine D1 receptors in the anterior cingulate cortex regulate effort-based decision making

The anterior cingulate cortex (ACC) has been implicated in encoding whether or not an action is worth performing in view of the expected benefit and the cost of performing the action. Dopamine input to the ACC may be critical for this form of effort-based decision making; however, the role of distinct ACC dopamine receptors is yet unknown. Therefore, we examined in rats the effects of an intra-ACC D1 and D2 receptor blockade on effort-based decision making tested in a T-maze cost-benefit task. In this task, subjects could either choose to climb a barrier to obtain a high reward in one arm or a low reward in the other arm without a barrier. Unlike vehicle-treated rats, rats with intra-ACC infusion of the D1 receptor antagonist SCH23390 exhibited a reduced preference for the high-cost- high-reward response option when having the choice to obtain a low reward with little effort. In contrast, in rats with intra-ACC infusion of the D2 receptor antagonist eticlopride, the preference for the high-cost-high-reward response option was not altered relative to vehicle-treated rats. These data provide the first evidence that D1 receptors in the ACC regulate effort-based decision making.     

Involvement of the rat anterior cingulate cortex in control of instrumental responses guided by reward expectancy

The anterior cingulate cortex (ACC) plays a critical role in stimulus-reinforcement learning and reward-guided selection of actions. Here we conducted a series of experiments to further elucidate the role of the ACC in instrumental behavior involving effort-based decision-making and instrumental learning guided by reward-predictive stimuli. In Experiment 1, rats were trained on a cost-benefit T-maze task in which they could either choose to climb a barrier to obtain a high reward (four pellets) in one arm or a low reward (two pellets) in the other with no barrier present. In line with previous studies, our data reveal that rats with quinolinic acid lesions of the ACC selected the response involving less work and smaller reward. Experiment 2 demonstrates that breaking points of instrumental performance under a progressive ratio schedule were similar in sham-lesioned and ACC-lesioned rats. Thus, lesions of the ACC did not interfere with the effort a rat is willing to expend to obtain a specific reward in this test. In a subsequent task, we examined effort-based decision-making in a lever-press task where rats had the choice between pressing a lever to receive preferred food pellets under a progressive ratio schedule, or free feeding on a less preferred food, i.e. lab chow. Results show that sham- and ACC-lesioned animals had similar breaking points and ingested comparable amounts of less-preferred food. Together, the results of Experiment 1 and 2 suggest that the ACC plays a role in evaluating how much effort to expend for reward; however, the ACC is not necessary in all situations requiring an assessment of costs and benefits. In Experiment 3 we investigated learning and reversal learning of instrumental responses guided by reward predictive stimuli. A reaction time (RT) task demanding conditioned lever release was used in which the upcoming reward magnitude (five vs. one food pellet) was signalled in advance by discriminative visual stimuli. Results revealed that rats with ACC lesions were able to discriminate reward magnitude-predictive stimuli and to adapt instrumental behavior to reversed stimulus-reward magnitude contingencies. Thus, in a simple discrimination task as used here, the ACC appears not to be required to discriminate reward magnitude-predictive stimuli and to use the learned significance of the stimuli to guide instrumental behavior.     

Building blocks of self-control: increased tolerance for delay with bundled rewards

Impulsive choice can be defined as temporary preference for a smaller-sooner reward (SS) over a larger-later reward (LL). Hyperbolic discounting implies that impulsive choices will occur less when organisms choose between a series of SSs versus LLs all at once than when they choose between single SS versus LL pairs. Eight rats were exposed to two conditions of an intertemporal choice paradigm using sucrose solution as reward. In both conditions, the LL was 150 microl delayed by 3 s, while the SS was an immediate reward that ranged from 25-150 microl across sessions. Preference for the LL was greater when the chosen reward was automatically delivered three times in succession (bundled) than when it was chosen singly and delivered after each choice. For each of the 8 rats, the estimated SS amount that produced indifference was higher in the bundled condition than in the single condition. Because bundling in humans may be based on the perception that one's current choice is predictive of future choices, the data presented here may demonstrate an important building block of self-control.     

Response rate correlates with indifference points in a delay‐discounting procedure

To study how effort affects reward value, we replicated Fortes, Vasconcelos and Machado's (2015) study using an adjusting‐delay task. Nine pigeons chose between a standard alternative that gave access to 4 s of food, after a 10 s delay, and an adjusting‐delay alternative that gave access to 12 s of food after a delay that changed dynamically with the pigeons' choices, decreasing when they preferred the standard alternative, and increasing when they preferred the adjusting alternative. The delay value at which preference stabilized defined the indifference point, a measure of reward value. To manipulate effort across phases, we varied the response rate required during the delay of the standard alternative. Results showed that a) the indifference point increased in the higher‐response‐rate phases, suggesting that reward value decreased with effort, and b) in the higher‐response‐rate phases, response rate in the standard alternative was linearly related to the indifference point. We advance several conceptions of how effort may change perceived delay or amount and thereby affect reward value.     

Performance on a fixed-ratio schedule with correlated amount of reward

Four rats were trained to bar press on FR 9 TO 30 sec. They were reinforced with a large or small amount of water according to whether their final IRT was long or short respectively. Four control rats always received the small amount of reinforcement. The control animals produced the high rates of responding typical of fixed-ratio performance. The experimental animals, with one exception, developed superstitious behavior and maintained slow responding throughout the ratio. However, some features of the results pointed to a persistent influence of the factors which favor short IRTs.     

The effects of morphine on the production and discrimination of interresponse times

Recent experiments suggest that the effects of drugs of abuse on the discrimination of the passage of time may differ for experimenter-imposed and subject-produced events. The current experiment examined this suggestion by determining the effects of morphine on the discrimination of interresponse times (IRTs). Pigeons pecked a center key on a random-interval 20-s schedule of matching-to-sample trials. Once the interval had timed out, a choice trial randomly followed either a short (2- to 3-s) or long (6- to 9-s) IRT on the center key. Pecking the side key lit one color produced food after a short IRT, and pecking the side key lit the other color produced food after a long IRT. Two experimental phases differed in the functional role of the different key colors. Under control conditions, the IRT distributions had two modes, one at the lower bound of the short category and a smaller one at the lower bound of the long category. Pigeons accurately categorized the duration of the IRTs: One key color was pecked following short IRTs and the other key color was pecked following long IRTs. Morphine flattened the IRT distribution and reduced the accuracy of categorizing IRTs. Categorization of long IRTs was particularly disrupted. Morphine did not produce overestimation of time as assessed by the production or categorization of IRTs. These results are similar to those obtained previously for the effects of morphine on the discrimination of the duration of experimenter-imposed events.     

Physical and cognitive effort discounting of hypothetical monetary rewards

Effort discounting refers to the decrease in the subjective value of a reward as the effort required to obtain the reward increases. The main aims of this study were to ascertain whether the amount of the reward affects the steepness of the effort discounting process for hypothetical monetary rewards, to identify whether this steepness depends on the type of effort that is required, and to determine whether the steepness of different types of effort covary at the individual level. Two types of effort were studied under hypothetical choice situations: physical effort and cognitive effort. Both physical and cognitive effort discounting were well described by the hyperbolic model. Large rewards were discounted less steeply than small rewards for both types of effort. This finding agrees with the results of prior studies which have found that larger rewards have greater motivational power. In addition, the steepness of physical effort discounting was positively correlated with the steepness of cognitive effort discounting, which suggests that the effort discounting process is a trait‐like characteristic within an individual.     

A frustration effect with adult humans?

In two investigations, the relationship of delays in response opportunity to response latencies was examined. On the basis of a generalized drive approach it was expected that the longer the delays, the greater the frustration, and that consequently the response latencies would be reduced. It was further hypothesized that the frustration effect would be enhanced for Internal locus of control subjects relative to Externals. Although statistically significant relationships between delays in opportunity to respond and response latencies were found, the pattern of the results did not appear to support a generalized drive interpretation of the effects. Consequently, the relationship between partial reward and response latencies that had been previously interpreted as a frustration effect by Libb and Serum (1974) was examined in a third experiment. Again, the results were supportive of specific factors other than a generalized frustration effect.Experiment 3 was conducted by the second author as partial fulfillment of the requirements for a master's degree. We are grateful to Nancy Lindstrom Miller and Gary Ramseyer for their assistance.