Basal ganglia engagement during feedback processing after a substantial delay

The striatum has been shown to play an important role in learning from performance-related feedback that is presented shortly after each response. However, less is known about the neural mechanisms supporting learning from feedback that is substantially delayed from the original response. Since the consequences of one’s actions often do not become known until after a delay, it is important to understand whether delayed feedback can produce neural responses similar to those elicited by immediate-feedback presentation. We investigated this issue by using functional magnetic resonance imaging (fMRI) as participants performed a paired-associate learning task with 180 distinct trials. Feedback indicating response accuracy was presented immediately, after a delay of 25 min, or not at all. Both immediate and delayed feedback led to significant gains in accuracy on a posttest, relative to no feedback. Replicating previous work, we found that the caudate nuclei showed greater activation for positive feedback than for negative feedback when the feedback was presented immediately. In addition, delayed feedback also led to differential caudate activity to positive versus negative feedback. Delayed negative feedback also produced significant activation of the putamen and globus pallidus (the lentiform nucleus), relative to no feedback and delayed positive feedback. This suggests that the caudate nucleus is sensitive to the affective nature of feedback, across different time scales, while the lentiform nucleus may be particularly involved in processing the information carried by negative feedback after a substantial delay.     

Antipsychotic dose modulates behavioral and neural responses to feedback during reinforcement learning in schizophrenia

Schizophrenia is characterized by an abnormal dopamine system, and dopamine blockade is the primary mechanism of antipsychotic treatment. Consistent with the known role of dopamine in reward processing, prior research has demonstrated that patients with schizophrenia exhibit impairments in reward-based learning. However, it remains unknown how treatment with antipsychotic medication impacts the behavioral and neural signatures of reinforcement learning in schizophrenia. The goal of this study was to examine whether antipsychotic medication modulates behavioral and neural responses to prediction error coding during reinforcement learning. Patients with schizophrenia completed a reinforcement learning task while undergoing functional magnetic resonance imaging. The task consisted of two separate conditions in which participants accumulated monetary gain or avoided monetary loss. Behavioral results indicated that antipsychotic medication dose was associated with altered behavioral approaches to learning, such that patients taking higher doses of medication showed increased sensitivity to negative reinforcement. Higher doses of antipsychotic medication were also associated with higher learning rates (LRs), suggesting that medication enhanced sensitivity to trial-by-trial feedback. Neuroimaging data demonstrated that antipsychotic dose was related to differences in neural signatures of feedback prediction error during the loss condition. Specifically, patients taking higher doses of medication showed attenuated prediction error responses in the striatum and the medial prefrontal cortex. These findings indicate that antipsychotic medication treatment may influence motivational processes in patients with schizophrenia.     

Anxiety and expectancy violations: Neural response to false feedback is exaggerated in worriers

This study investigated how individual differences in anxiety modulate the neural response to errors and performance feedback. The design included false feedback on some trials in order to test the hypothesis that anxious people show stronger neural reactions to feedback that is worse than expected. Participants completed a trial-and-error learning task that required learning the correct key to press in response to face images. EEG was recorded during the task, and the response-locked error-related negativity (ERN) and feedback-locked ERN were computed to measure neural responses to error commission and feedback. As expected, errors produced a response-locked ERN and false feedback produced a feedback-locked ERN in the group as a whole. High levels of trait worry predicted a disproportionately larger ERN following false feedback, but did not predict the magnitude of the response-locked ERN following errors. These results imply that worry-prone people react more strongly to violations of expectations, rather than to errors themselves.     

Acetylcholine activity in selective striatal regions supports behavioral flexibility

Daily living often requires individuals to flexibly respond to new circumstances. There is considerable evidence that the striatum is part of a larger neural network that supports flexible adaptations. Cholinergic interneurons are situated to strongly influence striatal output patterns which may enable flexible adaptations. The present experiments investigated whether acetylcholine actions in different striatal regions support behavioral flexibility by measuring acetylcholine efflux during place reversal learning. Acetylcholine efflux selectively increased in the dorsomedial striatum, but not dorsolateral or ventromedial striatum during place reversal learning. In order to modulate the M2-class of autoreceptors, administration of oxotremorine sesquifumurate (100 nM) into the dorsomedial striatum, concomitantly impaired reversal learning and an increase in acetylcholine output. These effects were reversed by the m(2) muscarinic receptor antagonist, AF-DX-116 (20 nM). The effects of oxotremorine sesquifumurate and AF-DX-116 on acetylcholine efflux were selective to behaviorally-induced changes as neither treatment affected acetylcholine output in a resting condition. In contrast to reversal learning, acetylcholine efflux in the dorsomedial striatum did not change during place acquisition. The results reveal an essential role for cholinergic activity and define its locus of control to the dorsomedial striatum in cognitive flexibility.     

Contributions of the striatum to learning, motivation, and performance: an associative account

It has long been recognized that the striatum is composed of distinct functional sub-units that are part of multiple cortico-striatal-thalamic circuits. Contemporary research has focused on the contribution of striatal sub-regions to three main phenomena: learning of associations between stimuli, actions and rewards; selection between competing response alternatives; and motivational modulation of motor behavior. Recent proposals have argued for a functional division of the striatum along these lines, attributing, for example, learning to one region and performance to another. Here, we consider empirical data from human and animal studies, as well as theoretical notions from both the psychological and computational literatures, and conclude that striatal sub-regions instead differ most clearly in terms of the associations being encoded in each region.     

Behavioural context and a distributed system: metabolic mapping studies of the basal ganglia.

Behavioural context is known to affect neural activity in the striatum. Responses of single cells increase to rewarding stimuli, or drop out as a bar press or saccade is learned. Networks that can accomplish a unique response to changing contexts are of particular interest to systems neuroscience and were a part of Hebb's interest in perception and learning. An overall map of the striatum that localizes changes related to this remarkable phenomenon of contextual responses contributes to our understanding of anatomical substrates of neural systems that integrate information, and may lead us to new striatal regions to study synaptic mechanisms of learning.     

测谎中反馈加工的认知神经机制：一项ERP研究

在一个模拟“杀人——侦探”游戏情境中,被试先完成模拟杀人任务,在随后的测谎阶段使用有反馈的ERP测谎范式,以反馈的形式告知被试测谎结果（“+0.5”或“-2”,分别代表“撒谎逃过检测”或“撒谎被发现”）时的脑内时程动态变化。结果发现,在反馈出现后的200～300ms时间段,表示“撒谎被抓住”比“撒谎逃过检测”的反馈信息诱发了一个更正性的ERP成分（P200-300）,在300～400ms时间段,两种反馈诱发的ERP成分（P300）没有差异,而在450～500ms时间段, “撒谎逃过检测”比“撒谎被抓住”的反馈信息诱发了一个更负性的ERP成分（N500）,负性反馈没有诱发FRN。这些结果表明：P200-300、P300和N500都是与加工测谎结果信息有关的ERP成分。P200-300可能对测谎结果的效价（正性或负性）敏感;P300可能反映被试对测谎结果主观意义大小的评价;而N500可能反映测谎结果所导致的大脑皮层兴奋程度。     

Leverpress escape/avoidance training increases neurotrophin levels in rat brain

In addition to their well-known role in neural development, the neurotrophins BDNF and NGF help mediate the plasticity that occurs in the brain to promote learning. Exposure to learning procedures often leads to increases in neurotrophins, while exposure to stress often results in decreases. It is unclear how the neurotrophins would respond to an aversive learning task. Therefore, BDNF and NGF content in the dorsal striatum, hippocampus, and basal forebrain was measured following discrete trial lever-press escape/avoidance conditioning. Conditioning significantly increased levels of both neurotrophins in hippocampus and basal forebrain, relative to home cage controls (HCC). Contrary to expectations, the dorsal striatum did not show any significant changes. However, significant correlations were observed between dorsal striatal neurotrophins and aspects of avoidance performance. This may indicate that the dorsal striatum is involved in the performance aspects of the task. Results are discussed in terms of the role of neurotrophins in the acquisition of new information, and the neural structures involved in different types of memory.     

Leverpress Escape/Avoidance Training Increases Neurotrophin Levels in Rat Brain

In addition to their well-known role in neural development, the neurotrophins BDNF and NGF help mediate the plasticity that occurs in the brain to promote learning. Exposure to learning procedures often leads to increases in neurotrophins, while exposure to stress often results in decreases. It is unclear how the neurotrophins would respond to an aversive learning task. Therefore, BDNF and NGF content in the dorsal striatum, hippocampus, and basal forebrain was measured following discrete trial lever-press escape/avoidance conditioning. Conditioning significantly increased levels of both neurotrophins in hippocampus and basal forebrain, relative to home cage controls (HCC). Contrary to expectations, the dorsal striatum did not show any significant changes. However, significant correlations were observed between dorsal striatal neurotrophins and aspects of avoidance performance. This may indicate that the dorsal striatum is involved in the performance aspects of the task. Results are discussed in terms of the role of neurotrophins in the acquisition of new information, and the neural structures involved in different types of memory.     

Adolescent-specific patterns of behavior and neural activity during social reinforcement learning

Humans are sophisticated social beings. Social cues from others are exceptionally salient, particularly during adolescence. Understanding how adolescents interpret and learn from variable social signals can provide insight into the observed shift in social sensitivity during this period. The present study tested 120 participants between the ages of 8 and 25 years on a social reinforcement learning task where the probability of receiving positive social feedback was parametrically manipulated. Seventy-eight of these participants completed the task during fMRI scanning. Modeling trial-by-trial learning, children and adults showed higher positive learning rates than did adolescents, suggesting that adolescents demonstrated less differentiation in their reaction times for peers who provided more positive feedback. Forming expectations about receiving positive social reinforcement correlated with neural activity within the medial prefrontal cortex and ventral striatum across age. Adolescents, unlike children and adults, showed greater insular activity during positive prediction error learning and increased activity in the supplementary motor cortex and the putamen when receiving positive social feedback regardless of the expected outcome, suggesting that peer approval may motivate adolescents toward action. While different amounts of positive social reinforcement enhanced learning in children and adults, all positive social reinforcement equally motivated adolescents. Together, these findings indicate that sensitivity to peer approval during adolescence goes beyond simple reinforcement theory accounts and suggest possible explanations for how peers may motivate adolescent behavior.     

Better late than never? The effect of feedback delay on ERP indices of reward processing

The feedback negativity (FN), an early neural response that differentiates rewards from losses, appears to be generated in part by reward circuits in the brain. A prominent model of the FN suggests that it reflects learning processes by which environmental feedback shapes behavior. Although there is evidence that human behavior is more strongly influenced by rewards that quickly follow actions, in nonlaboratory settings, optimal behaviors are not always followed by immediate rewards. However, it is not clear how the introduction of a delay between response selection and feedback impacts the FN. Thus, the present study used a simple forced choice gambling task to elicit the FN, in which feedback about rewards and losses was presented after either 1 or 6?s. Results suggest that, at short delays (1?s), participants clearly differentiated losses from rewards, as evidenced in the magnitude of the FN. At long delays (6?s), on the other hand, the difference between losses and rewards was negligible. Results are discussed in terms of eligibility traces and the reinforcement learning model of the FN.     

Weak ventral striatal responses to monetary outcomes predict an unwillingness to resist cigarette smoking

As a group, cigarette smokers exhibit blunted subjective, behavioral, and neurobiological responses to nondrug incentives and rewards, relative to nonsmokers. Findings from recent studies suggest, however, that there are large individual differences in the devaluation of nondrug rewards among smokers. Moreover, this variability appears to have significant clinical implications, since reduced sensitivity to nondrug rewards is associated with poorer smoking cessation outcomes. Currently, little is known about the neurobiological mechanisms that underlie these individual differences in the responsiveness to nondrug rewards. Here, we tested the hypothesis that individual variability in reward devaluation among smokers is linked to the functioning of the striatum. Specifically, functional magnetic resonance imaging was used to examine variability in the neural response to monetary outcomes in nicotine-deprived smokers anticipating an opportunity to smoke—circumstances found to heighten the devaluation of nondrug rewards by smokers in prior work. We also investigated whether individual differences in reward-related brain activity in those expecting to have access to cigarettes were associated with the degree to which the same individuals subsequently were willing to resist smoking in order to earn additional money. Our key finding was that deprived smokers who exhibited the weakest response to rewards (i.e., monetary gains) in the ventral striatum were least willing to refrain from smoking for monetary reinforcement. These results provide evidence that outcome-related signals in the ventral striatum serve as a marker for clinically meaningful individual differences in reward-motivated behavior among nicotine-deprived smokers.     

Interpreting and reacting to feedback in stereotype-relevant performance domains

Feedback on task performance is often phrased in subjective language (e.g., “not bad!”), but how do recipients understand or translate that feedback into a clear, objective, performance metric? We suggest that when feedback is provided in a stereotype-relevant domain, translation is made with reference to stereotyped expectations for one's group. In Study 1, women and men were exposed to negative subjective feedback about their performance on a leadership task; in Study 2, Black and White participants were provided subjective negative feedback, or no feedback, on an academic writing task. Women relative to men, and Black students relative to White students, translated their feedback to indicate objectively worse performance. Furthermore, this translation mediated a drop in the importance placed on the domain among women and Blacks. This research extends the literature on gender- and race-based reactions to feedback by noting the importance of the immediate interpretation of the feedback received.     

时间知觉在老年人跨期选择中的作用

随着人口老龄化速度的加快,老年人的跨期选择问题受到越来越多的关注。时间知觉是影响跨期选择的重要因素之一。然而,目前从时间知觉的角度来探讨老年人跨期选择的研究比较有限。本文在总结国内外相关研究的基础之上,试图分析时间知觉的随龄变化如何影响老年人的跨期选择。具体而言,本文从时间长度知觉、时间成本知觉以及时间知觉相关心理动机三个方面进行探讨,并指出了时间知觉视角的局限性及今后的研究方向。     

The Effect of Goal Difficulty on Escalation of Commitment

Escalation of commitment to a failing course of action is a problem in behavioral decision making that occurs across a wide range of social contexts. In this research, we show that examining escalation of commitment from a goal setting theory perspective provides fresh insights into how goal difficulty influences escalation of commitment. Specifically, through a series of two experiments, we found a curvilinear relationship between goal difficulty and post‐feedback goal commitment, which was mediated by valence and expectancy associated with goal attainment. In turn, it is commitment to goals that leads individuals to continue a previous course of action despite negative feedback. Copyright © 2014 John Wiley & Sons, Ltd.     

Achievement Goals,Feedback, and Task Performance

Prior research indicates that instructions to focus attention on learning versus performance and the nature of feedback provided have distinct implications for subsequent task performance. We first examined how assigned learning and performance goals and feedback valence interact to determine performance change. Individuals with learning goal instructions performed better after negative feedback but worse after positive feedback. In Study 2, we found that implicit theory, an individual difference that is antecedent to general goal orientation, interacted with learning/performance goal instructions to influence performance change after negative feedback. In both studies, goal instructions influenced performance attributions and affective states, but these variables did not mediate the effects of the goal instructions or performance feedback. We discuss the implications of these results for academic and employment settings.     

Manipulating self-efficacy in the exercise environment in women: influences on affective responses.

Self-efficacy was experimentally manipulated in an exercise context, and its effect on affective responses was examined. College women (N = 46) were randomly assigned to a high- or low-efficacy condition, and efficacy expectations were manipulated by means of bogus feedback and graphs depicting contrived normative data. The manipulation successfully influenced affective responses, with participants in the high-efficacy group reporting more positive and less negative affect than did the low-efficacy group. Efficacy was significantly related to feeling-state responses during and after activity but only in the high-efficacy condition. The results suggest that self-efficacy can be manipulated and that these changes are related to the affective experience associated with exercise. Such findings may have important implications for the roles played by self-efficacy and affect in exercise adherence.     

Effects of normative feedback on motor learning are dependent on the frequency of knowledge of results

Studies on normative feedback have shown superior motor learning outcomes for individuals who believe that they are performing better than others through increased self-efficacy. Nevertheless, the effects of normative feedback were never dissociated from the knowledge of results (KR) provided to the learners which potentially interacts with self-efficacy as well. Thus, we investigated whether the effects of normative feedback on motor learning, associated with self-efficacy, would be dependent on the amount of KR provided. Fifty-six participants were randomly assigned to four experimental groups in terms of KR frequency (100% and 33%) and normative feedback (positive and negative). In the acquisition phase, all groups received the average KR of their performance at the end of each block of trials (True feedback) and a fake KR based on their own performance (but said to be from a group of participants who practiced the same task) (False Feedback). The False Feedback indicated better or worse performance of the participant in comparison to the fake group, depending on their experimental group. Retention tests were performed immediately and after 24 h from the acquisition phase. To measure self-efficacy, a questionnaire on participant's efficacy was applied before the first block, after each block of trials and before the retention tests. The results revealed superiority of positive normative feedback and 100% KR frequency, compared to negative normative feedback and 100% KR frequency in the 24h retention test. No difference was found between the groups with a frequency of 33% of KR (positive and negative). All groups increased self-efficacy during practice, but there was no difference between groups at any stage of the study. We conclude that the effects of normative feedback on motor learning are dependent on the KR frequency. However, they were not associated with self-efficacy.     

Candy and the brain: neural response to candy gains and losses

Incentive processing is a critical component of a host of cognitive processes, including attention, motivation, and learning. Neuroimaging studies have clarified the neural systems underlying processing of primary and secondary rewards in adults. However, current reward paradigms have hindered comparisons across these reward types as well as between age groups. To address methodological issues regarding the timing of incentive delivery (during scan vs. postscan) and the age-appropriateness of the incentive type, we utilized fMRI and a modified version of a card-guessing game (CGG), in which candy pieces delivered postscan served as the reinforcer, to investigate neural responses to incentives. Healthy young adults 22–26 years of age won and lost large and small amounts of candy on the basis of their ability to guess the number on a mystery card. BOLD activity was compared following candy gain (large/small), loss (large/small), and neutral feedback. During candy gains, adults recruited regions typically involved in response to monetary and other rewards, such as the caudate, putamen, and orbitofrontal cortex. During losses, they displayed greater deactivation in the hippocampus than in response to neutral and gain feedback. Additionally, individual-difference analyses suggested a negative relationship between reward sensitivity (assessed by the Behavioral Inhibition/Behavioral Activation Scales) and the difference between high- and low-magnitude losses in the caudate and lateral orbitofrontal cortex. Also within the striatum, greater punishment sensitivity was positively related to the difference in activity following high as compared to low gains. Overall, these results show strong overlap with those from previous monetary versions of the CGG and provide a baseline for future work with developmental populations.     

Information content and reward processing in the human striatum during performance of a declarative memory task

Negative feedback can signal poor performance, but it also provides information that can help learners reach the goal of task mastery. The primary aim of this study was to test the hypothesis that the amount of information provided by negative feedback during a paired-associate learning task influences feedback-related processing in the caudate nucleus. To do this, we manipulated the number of response options: With two options, positive and negative feedback provide equal amounts of information, whereas with four options, positive feedback provides more information than does negative feedback. We found that positive and negative feedback activated the caudate similarly when there were two response options. With four options, the caudate’s response to negative feedback was reduced. A secondary goal was to investigate the link between brain-based measures of feedback-related processing and behavioral indices of learning. Analysis of the posttest measures showed that trials with positive feedback were associated with higher posttest confidence ratings. Additionally, when positive feedback was delivered, caudate activity was greater for trials with high than with low posttest confidence. This experiment demonstrated the context sensitivity of feedback processing and provided evidence that feedback processing in the striatum can contribute to the strengthening of the representations available within declarative memory.