Motivation for reward as a function of required effort: Dissociating the ‘liking’ from the ‘wanting’ system in humans

Recent studies investigating the neurobiology of reward motivation in animals have begun to deconstruct reward into separable neural systems involving the ‘liking’ of a reward (hedonic enjoyment of consumption) versus the ‘wanting’ of a reward (incentive salience to obtain reward). To date, however, it is unclear whether these systems are also separable in humans. We examined this question by manipulating the effort (clicking on a moving square) required for participants to obtain a reward (humorous versus non-humorous cartoon). Overall, as the required effort to view a humorous cartoon increased, participants were less likely to choose this reward. Moreover, individual differences in cartoon preference predicted cartoon choice at low levels of required effort, but not at high levels of required effort. These findings suggest that manipulating effort may be a valid method for dissociating the ‘liking’ from ‘wanting’ components of reward motivation in humans.

No pain no gain: The positive impact of punishment on the strategic regulation of accuracy

Previous studies have shown that punishing people through a large penalty for volunteering incorrect information typically leads them to withhold more information (metacognitive response bias), but it does not appear to influence their ability to distinguish between their own correct and incorrect answers (metacognitive accuracy discrimination). The goal of the current study was to demonstrate that punishing people for volunteering incorrect information—versus rewarding volunteering correct information—produces more effective metacognitive accuracy discrimination. All participants completed three different general-knowledge tests: a reward test (high points for correct volunteered answers), a baseline test (equal points/penalties for volunteered correct/incorrect answers) and a punishment test (high penalty for incorrect volunteered answers). Participants were significantly better at distinguishing between their own correct and incorrect answers on the punishment than reward test, which has implications for situations requiring effective accuracy monitoring.     

Experimental setting affects the performance of guppies in a numerical discrimination task

A recent study found that guppies (Poecilia reticulata) can be trained to discriminate 4 versus 5 objects, a numerical discrimination typically achieved only by some mammals and birds. In that study, guppies were required to discriminate between two patches of small objects on the bottom of the tank that they could remove to find a food reward. It is not clear whether this species possesses exceptional numerical accuracy compared with the other ectothermic vertebrates or whether its remarkable performance was due to a specific predisposition to discriminate between differences in the quality of patches while foraging. To disentangle these possibilities, we trained guppies to the same numerical discriminations with a more conventional two-choice discrimination task. Stimuli were sets of dots presented on a computer screen, and the subjects received a food reward upon approaching the set with the larger numerosity. Though the cognitive problem was identical in the two experiments, the change in the experimental setting led to a much poorer performance as most fish failed even the 2 versus 3 discrimination. In four additional experiments, we varied the duration of the decision time, the type of stimuli, the length of training, and whether correction was allowed in order to identify the factors responsible for the difference. None of these parameters succeeded in increasing the performance to the level of the previous study, although the group trained with three-dimensional stimuli learned the easiest numerical task. We suggest that the different results with the two experimental settings might be due to constraints on learning and that guppies might be prepared to accurately estimate patch quality during foraging but not to learn an abstract stimulus–reward association.     

A mechanism for reducing delay discounting by altering temporal attention

Rewards that are not immediately available are discounted compared to rewards that are immediately available. The more a person discounts a delayed reward, the more likely that person is to have a range of behavioral problems, including clinical disorders. This latter observation has motivated the search for interventions that reduce discounting. One surprisingly simple method to reduce discounting is an "explicit-zero" reframing that states default or null outcomes. Reframing a classical discounting choice as "something now but nothing later" versus "nothing now but more later" decreases discount rates. However, it is not clear how this "explicit-zero" framing intervention works. The present studies delineate and test two possible mechanisms to explain the phenomenon. One mechanism proposes that the explicit-zero framing creates the impression of an improving sequence, thereby enhancing the present value of the delayed reward. A second possible mechanism posits an increase in attention allocation to temporally distant reward representations. In four experiments, we distinguish between these two hypothesized mechanisms and conclude that the temporal attention hypothesis is superior for explaining our results. We propose a model of temporal attention whereby framing affects intertemporal preferences by modifying present bias.     

Linear transformations of the payoff matrix and decision criterion learning in perceptual categorization

The effects of payoff-matrix multiplication, payoff-matrix addition, the presence of long-run gains versus long-run losses, category discriminability, and base rate on decision criterion learning were examined in 2 perceptual categorization experiments. Observers were found to be sensitive to the effects of payoff-matrix multiplication (and category discriminability) on the steepness of the objective reward function in line with predictions from the flat-maxima hypothesis and contrary to the predictions from the payoff-variance hypothesis. Decision criterion learning was best in base-rate conditions, was worst when losses were associated with incorrect responding, and was intermediate when no losses were associated with incorrect responding. This performance profile was well captured by the competition between reward and accuracy (COBRA) hypothesis. A hybrid model framework that instantiates both the flat-maxima and COBRA hypotheses was necessary to account for the data from both experiments.     

How humans react to changing rewards during visual foraging

Much is known about the speed and accuracy of search in single-target search tasks, but less attention has been devoted to understanding search in multiple-target foraging tasks. These tasks raise and answer important questions about how individuals decide to terminate searches in cases in which the number of targets in each display is unknown. Even when asked to find every target, individuals quit before exhaustively searching a display. Because a failure to notice targets can have profound effects (e.g., missing a malignant tumor in an X-ray), it is important to develop strategies that could limit such errors. Here, we explored the impact of different reward patterns on these failures. In the Neutral condition, reward for finding a target was constant over time. In the Increasing condition, reward increased for each successive target in a display, penalizing early departure from a display. In the Decreasing condition, reward decreased for each successive target in a display. The experimental results demonstrate that observers will forage for longer (and find more targets) when the value of successive targets increases (and the opposite when value decreases). The data indicate that observers were learning to utilize knowledge of the reward pattern and to forage optimally over the course of the experiment. Simulation results further revealed that human behavior could be modeled with a variant of Charnov’s Marginal Value Theorem (MVT) (Charnov, 1976) that includes roles for reward and learning.     

Cooperation in social dilemmas: free riding may be thwarted by second-order reward rather than by punishment

Cooperation among nonrelatives can be puzzling because cooperation often involves incurring costs to confer benefits on unrelated others. Punishment of noncooperators can sustain otherwise fragile cooperation, but the provision of punishment suffers from a "second-order" free-riding problem because nonpunishers can free ride on the benefits from costly punishment provided by others. One suggested solution to this problem is second-order punishment of nonpunishers; more generally, the threat or promise of higher order sanctions might maintain the lower order sanctions that enforce cooperation in collective action problems. Here the authors report on 3 experiments testing people's willingness to provide second-order sanctions by having participants play a cooperative game with opportunities to punish and reward each other. The authors found that people supported those who rewarded cooperators either by rewarding them or by punishing nonrewarders, but people did not support those who punished noncooperators--they did not reward punishers or punish nonpunishers. Furthermore, people did not approve of punishers more than they did nonpunishers, even when nonpunishers were clearly unwilling to use sanctions to support cooperation. The results suggest that people will much more readily support positive sanctions than they will support negative sanctions.     

Eye blink rate predicts reward decisions in adolescents

The ventral striatum displays hyper‐responsiveness to reward in adolescents relative to other age groups, and animal research on the developmental trajectory of the dopaminergic system suggests that dopamine may underlie adolescent sensitivity to reward. However, practical limitations prevent the direct measurement of dopamine in healthy adolescents. Eye blink rate (EBR) shows promise as a proxy measure of striatal dopamine D2 receptor function. We investigated developmental differences in the relationship between EBR and reward‐seeking behavior on a risky decision‐making task. Increasing EBR was associated with greater reward maximization on the task for adolescent but not adult participants. Furthermore, adolescents demonstrated greater sensitivity to reward value than adults, as evinced by shifts in decision patterns based on increasing potential reward. These findings suggest that previously observed adolescent behavioral and neural hypersensitivity to reward may in fact be due to greater dopamine receptor activity, as represented by the relationship of blink rate and reward‐seeking behavior. They also demonstrate the feasibility and utility of using EBR as a proxy for dopamine in healthy youth in whom direct measurements of dopamine are prohibitively invasive.     

Valence specific response reversal deficits and risk for mania

Response reversal deficits are well documented in bipolar disorder (BD). Although frequently construed as an index of reward processing abnormalities, these response reversal deficits might simply result from more general cognitive inflexibility. Given that both are implicated in BD, our goal was to more carefully test whether reversal learning deficits are specific to reward processing or reflect more general cognitive inflexibility. To more carefully assess deficits, a novel variant of the response reversal task was used to separate responses to in reward versus punishment feedback. In addition, while response reversal deficits are well documented in BD, it is still unknown if these deficits are observable in people at risk for the disorder, whose performance would be unconfounded by medication use and illness course. To assess the presence of premorbid response reversal deficits, we tested students at risk for developing BD, as defined by the well-validated Hypomanic Personality Scale. Undergraduates (n?=?99) were randomly assigned to complete either a reward only or a punishment only version of a response reversal task. Mania risk was related to difficulty reversing responses following reward, but not punishment feedback. Findings suggest that a deficit in response reversal may be an index of reward dysregulation in BD, and that this deficit can be observed even in those at high risk for the development of BD.     

Colour versus quantity as cues in reverse-reward-competent squirrel monkeys (Saimiri sciureus)

To assess the relative salience of colour and quantity cues, squirrel monkeys previously trained to reach for the smaller of two quantities of food in a reverse-reward contingency task received colour discrimination training. After initial failure to discriminate between two colours of dots under a differential reinforcement regime, they learned the task when the S- colour was associated with zero reward. The monkeys then showed good retention on the original reverse-reward task of 1 versus 4 with pairs of dots presented in S+ or S- colours. However, on "mismatch" trials of 1S- versus 4S+ , only 2 of 4 monkeys tested showed a preference--1 monkey chose based on quantity, the other based on colour. Individual differences and the possible roles of overshadowing and blocking are discussed.     

Building blocks of self-control: increased tolerance for delay with bundled rewards

Impulsive choice can be defined as temporary preference for a smaller-sooner reward (SS) over a larger-later reward (LL). Hyperbolic discounting implies that impulsive choices will occur less when organisms choose between a series of SSs versus LLs all at once than when they choose between single SS versus LL pairs. Eight rats were exposed to two conditions of an intertemporal choice paradigm using sucrose solution as reward. In both conditions, the LL was 150 microl delayed by 3 s, while the SS was an immediate reward that ranged from 25-150 microl across sessions. Preference for the LL was greater when the chosen reward was automatically delivered three times in succession (bundled) than when it was chosen singly and delivered after each choice. For each of the 8 rats, the estimated SS amount that produced indifference was higher in the bundled condition than in the single condition. Because bundling in humans may be based on the perception that one's current choice is predictive of future choices, the data presented here may demonstrate an important building block of self-control.     

Choosing schedules of signaled appetitive events over schedules of unsignaled ones

Two experiments were completed allowing albino rats to choose between signaled and unsignaled reward conditions. These experiments examined the effects on preference of (1) response dependent versus response-independent reward and, (2) food pellets versus chocolate milk as the reward. All subjects preferred the signaled condition over the unsignaled condition, whether exposed to response-dependent, or to response-independent delivery of rewards. Preference was controlled most effectively by presenting both the signal itself and the correlated stimulus identifying the signaled condition. The signal presented alone (Extinction 3) controlled preference more effectively than did the stimulus correlated with the signaled condition (Extinction 2). The second experiment showed that the quality of the reinforcer (pellets and chocolate milk) did not affect preference for signaled reward since all subjects preferred the signaled condition at levels comparable to those observed in Experiment 1, with food pellets. These results, along with others, argue against species differences, response-dependency, and reinforcer quality as variables affecting the direction of preference.     

Relationships Among Sleep Disturbance,Reward System Functioning,Anhedonia, and Depressive Symptoms

Recent models propose reward system dysfunction as a key mediator of the relationship between sleep and depression and anhedonia. This study explored interrelationships among sleep disturbance, depressive symptoms, anhedonia, and reward responsiveness. Two-hundred and sixty undergraduate students completed questionnaires and a daily diary paradigm assessing sleep, reward responsiveness, depression, anhedonia, and positive affect over 1 week. Baseline sleep disturbance was associated with depressive symptoms, anhedonia, and reward responsiveness. Daily diary sleep parameters showed differential associations with anticipatory versus consummatory reward responsiveness and positive affect. Poorer sleep quality, shorter sleep duration, and longer awakening after sleep onset predicted blunted anticipatory and consummatory reward responsiveness, while increased sleep onset latency and lower sleep efficiency predicted only decreased consummatory reward responsiveness. All sleep indices, except sleep onset latency, were associated with positive affect. Findings demonstrate unique associations between disparate sleep disturbance and reward responsiveness elements, highlighting new treatment mechanisms for anhedonia and depression.     

The effects of reward on tutor's behaviors in a cross-age tutoring context

Utilizing a cross-age tutoring context, this study examined the effects of reward on the teaching behaviors of the tutor, the tutor's subsequent motivation to continue to teach during a free choice period, and the social interaction between a tutor and a tutee. Third-grade boys and girls (n = 96) who exhibited a positive reinforcement style were asked to teach six addition problems to a first-grade boy or girl (n = 96). The children were randomly assigned to pairs and to one of the three reward conditions. In the performance-contingent reward condition, the tutors were promised a toy if the first-grader learned all of the arithmetic problems. In the noncontingent reward condition, the tutors were promised a toy for teaching the first-grader. In the no reward condition, the tutors taught the first-grader without promise of a toy. The results indicated that the social interaction was rated lower for the children in the performance-contingent group and that the tutors in this group spent less time teaching during the free choice period. However, neither the tutor's teaching style nor the tutee's post-test performance was adversely affected by the reward.     

Unconscious reward cues increase invested effort,but do not change speed–accuracy tradeoffs

While both conscious and unconscious reward cues enhance effort to work on a task, previous research also suggests that conscious rewards may additionally affect speed–accuracy tradeoffs. Based on this idea, two experiments explored whether reward cues that are presented above (supraliminal) or below (subliminal) the threshold of conscious awareness affect such tradeoffs differently. In a speed–accuracy paradigm, participants had to solve an arithmetic problem to attain a supraliminally or subliminally presented high-value or low-value coin. Subliminal high (vs. low) rewards made participants more eager (i.e., faster, but equally accurate). In contrast, supraliminal high (vs. low) rewards caused participants to become more cautious (i.e., slower, but more accurate). However, the effects of supraliminal rewards mimicked those of subliminal rewards when the tendency to make speed–accuracy tradeoffs was reduced. These findings suggest that reward cues initially boost effort regardless of whether or not people are aware of them, but affect speed–accuracy tradeoffs only when the reward information is accessible to consciousness.     

On the self-perception explanation of the overjustification effect: The role of the salience of initial attitude

An experiment was conducted to examine the proposition, derived from self-perception theory, that the “overjustification” effect would be mitigated if initial attitudes toward the task were made salient. Children engaged in a target activity with or without the promise of reward for doing so. Half the children were shown a photograph of themselves freely engaging in the target activity in an earlier session so as to remind them of their intrinsic interest in the task. The other children were not shown such a photograph. Consistent with the predictions. expected reward decreased subsequent intrinsic interest only when initial attitudes were relatively nonsalient. The findings demonstrate the relevance of self-perception processes to the “overjustification” effect.     

The EMG gradient as a reflection of metamotivational state

The significance of motivational statedominance in everyday life and actual motivational state during task performance was investigated in relation to skeletal muscle tension patterns during performance in the continuous preceptual-motor task paradigm. Subjects were recruited to form seriousminded (telic) and playful (paratelic) extreme groups according to their responses to the Telic Dominance Scale (6 subjects in each group). They all performed a continuous perceptual-motor task twice in a neutral context (N1, N2), once with threat of aversive shock at poor performance (T), and once with promise of monetary reward for good performance (R); T and R being balanced around N 2. Measures of tonic (slow) EMG activation were derived from the "passive" forearm, and structured (blind) interview assessed motivational state. It was concluded that motivational state, rather than statedominance, was associated with EMG activation. The telic state was significant to high EMG activation for the N I and T treatments in particular.     

Increasing and decreasing intrinsic interest with contingent rewards: A test of cognitive evaluation theory

The study examined Deci's (Deci, E. L. Intrinsic motivation. New York: Plenum Press, 1975) hypotheses regarding the effects of contingent rewards on intrinsic task interest. Seventy-two male university students worked on a series of puzzles and were given either a high value reward ($1.50) or a low value reward ($.45). The money was given either contingent upon the simple execution of the task (task-contingent), contingent upon the ostensible attainment of a performance criterion level (criterion-contingent), or noncontingent and unexpected (control). Compared to the high payment control subjects, subjects who received the task-contingent high reward rated the task as less interesting, while subjects who received the criterion-contingent high reward rated it as more interesting. Also, subjects expressed less interest in the task after receiving the high task-contingent reward than the low task-contingent reward, but indicated greater interest after receiving the high criterion-contingent reward than the low criterion-contingent reward. It was concluded that substantial support was obtained for Deci's (1975) cognitive evaluation theory.     

奖赏预期对选择性注意的影响——基于客体和空间的注意的区别

为了探究奖赏预期对选择性注意的影响,结合金钱激励延迟任务和双矩形范式,通过两个实验分别考察了奖赏预期的有无以及高低对选择性注意的影响。实验结果表明,在有奖赏预期的情况下,选择性注意更快,奖赏预期的有无对基于空间的注意影响区别不大,但奖赏预期几乎消除了基于客体的注意效应。在低奖赏预期条件下,基于客体和空间的注意均存在;而高奖赏预期条件下,基于空间的注意仍然存在,基于客体的注意几乎被消除了。结果表明奖赏预期可以影响选择性注意,并对基于空间和客体的注意产生不同影响。