Adaptive scaling of reward in episodic memory: a replication study

Reward is thought to enhance episodic memory formation via dopaminergic consolidation. Bunzeck, Dayan, Dolan, and Duzel [(2010). A common mechanism for adaptive scaling of reward and novelty. Human Brain Mapping, 31, 1380–1394] provided functional magnetic resonance imaging (fMRI) and behavioural evidence that reward and episodic memory systems are sensitive to the contextual value of a reward—whether it is relatively higher or lower—as opposed to absolute value or prediction error. We carried out a direct replication of their behavioural study and did not replicate their finding that memory performance associated with reward follows this pattern of adaptive scaling. An effect of reward outcome was in the opposite direction to that in the original study, with lower reward outcomes leading to better memory than higher outcomes. There was a marginal effect of reward context, suggesting that expected value affected memory performance. We discuss the robustness of the reward memory relationship to variations in reward context, and whether other reward-related factors have a more reliable influence on episodic memory.     

Reactivity of the Reward System in Artists During Acceptance and Rejection of Monetary Rewards

Humans possess an invaluable ability of self-expression that extends into visual, literary, musical, and many other fields of creation. More than any other profession, artists are in close contact with this subdomain of creativity. Probably one of the most intriguing aspects of creativity is its negative correlation with the availability of monetary reward. The aim of this study was to investigate the reactivity of the dopaminergic reward system in artists and nonartist controls using the desire-reason-dilemma (DRD) paradigm, which allows separate evaluation of reactivity to the acceptance and rejection of rewards. Using fMRI, blood-oxygen-level-dependent (BOLD) responses were measured in key regions of the reward system, namely the ventral striatum (VS), the ventral tegmental area (VTA), and the anterior ventral prefrontal cortex (AVPFC). In contrast to controls, artists presented significantly weaker VS activation in reward acceptance. Additionally, they showed stronger suppression of the VS by the AVPFC in reward rejection. No other differences in demographic or behavioral data were evidenced. These results support the existence of characteristic neural traits in artists, who display reduced reactions to monetary reward acceptance and increased reactions to monetary reward rejection.     

Computational heterogeneity in the human mesencephalic dopamine system

Recent evidence in animals has indicated that the mesencephalic dopamine system is heterogeneous anatomically, molecularly, and functionally, and it has been suggested that the dopamine system comprises distinct functional systems. Identifying and characterizing these systems in humans will have widespread ramifications for understanding drug addiction and mental health disorders. Model-based studies in humans have suggested an analogous computational heterogeneity, in which dopaminergic targets in striatum encode both experience-based learning signals and counterfactual learning signals that are based on hypothetical information. We used brainstem-tailored fMRI to identify mesencephalic sources of experiential and counterfactual learning signals. Participants completed a decision-making task based on investing in markets. This sequential investment task generated experience-based learning signals, in the form of temporal difference (TD) reward prediction errors, and counterfactual learning signals, in the form of “fictive errors.” Fictive errors are reinforcement learning signals based on hypothetical information about “what could have been.” An additional learning signal was constructed to be relatable to a motivational salience signal. Blood oxygenation level dependent responses in regions of substantia nigra (SN) and ventral tegmental area (VTA), where dopamine neurons are located, coded for TD and fictive errors, and additionally were related to the motivational salience signal. These results are highly consistent with animal electrophysiology and provide direct evidence that human SN and VTA heterogeneously handle important reward-harvesting computations.     

奖赏环路与阿片成瘾:喙内侧被盖核的调节作用

喙内侧被盖核(RMTg)位于腹侧被盖区(VTA)的尾部, 富含抑制性的γ-氨基丁酸(GABA)能神经元.RMTg是中脑边缘多巴胺系统的一个综合调节器.它的GABA能神经元接受外侧缰核(LHb)的输入, 然后投射到VTA多巴胺能神经元, 进而抑制多巴胺的释放.这三个脑区是奖赏环路的重要组成部分, 其中RMTg在阿片类物质激活的奖赏环路中尤为重要.阿片类物质主要通过抑制RMTg GABA能神经元使VTA多巴胺能神经元去抑制, 进而激活奖赏系统.因此, RMTg有望成为治疗药物成瘾(尤其是阿片成瘾)的一个重要靶点.此外, 胆碱类物质作用于RMTg的毒蕈碱受体能够抑制阿片类物质诱导的奖赏效应.未来研究应深入探讨RMTg调控的负性奖赏环路, 这对弱化觅药动机,促进消退和戒断具有重要意义.     

Ventral tegmental area and substantia nigra neural correlates of spatial learning

The ventral tegmental area (VTA) and substantia nigra pars compacta (SNc) may provide modulatory signals that, respectively, influence hippocampal (HPC)- and striatal-dependent memory. Electrophysiological studies investigating neural correlates of learning and memory of dopamine (DA) neurons during classical conditioning tasks have found DA neural activity in VTA and SNc to be tightly coupled with reinforcement expectations. Also, VTA integrity and DA in HPC have been found to regulate the encoding of HPC-dependent memories. Therefore, to determine the nature of the neural code HPC may receive from midbrain DA regions, the present study investigated VTA and SNc neural activity as navigating rats engaged in new spatial learning and experienced changes in expected goal locations. VTA and SNc cells were differentially engaged during training to a series of three novel goal locations. During task acquisition, the peak firing rates of VTA neurons decreased at the time of reward and shifted to time points before reward retrieval, whereas the peak firing rates of SNc neurons remained elevated at the time of reward during training to all three goal locations. Both VTA and SNc egocentric coding was strongest during training to the first goal location, which coincided with the time subjects learned the behavioral rules specific to the task. These data imply that VTA and SNc play complementary yet distinct roles in spatial learning to optimize adaptive behavior.     

Reward prospect rapidly speeds up response inhibition via reactive control

Response inhibition is an important cognitive-control function that allows for already-initiated or habitual behavioral responses to be promptly withheld when needed. A typical paradigm to study this function is the stop-signal task. From this task, the stop-signal response time (SSRT) can be derived, which indexes how rapidly an already-initiated response can be canceled. Typically, SSRTs range around 200 ms, identifying response inhibition as a particularly rapid cognitive-control process. Even so, it has recently been shown that SSRTs can be further accelerated if successful response inhibition is rewarded. Since this earlier study effectively ruled out differential preparatory (proactive) control adjustments, the reward benefits likely relied on boosted reactive control. Yet, given how rapidly such control processes would need to be enhanced, alternative explanations circumventing reactive control are important to consider. We addressed this question with an fMRI study by gauging the overlap of the brain networks associated with reward-related and response-inhibition-related processes in a reward-modulated stop-signal task. In line with the view that reactive control can indeed be boosted swiftly by reward availability, we found that the activity in key brain areas related to response inhibition was enhanced for reward-related stop trials. Furthermore, we observed that this beneficial reward effect was triggered by enhanced connectivity between task-unspecific (reward-related) and task-specific (inhibition-related) areas in the medial prefrontal cortex (mPFC). The present data hence suggest that reward information can be translated very rapidly into behavioral benefits (here, within ~200 ms) through enhanced reactive control, underscoring the immediate responsiveness of such control processes to reward availability in general.     

Neural Correlates of Reward Processing in Adolescents With a History of Inhibited Temperament

ABSTRACT— Functional imaging data were acquired during performance of a reward-contingency task in a unique cohort of adolescents (ages 14–18 years) who were characterized since infancy on measures of temperamental behavioral inhibition. Neural activation was examined in striatal structures (nucleus accumbens, putamen, caudate) with a known role in facilitating response to salient reward-related cues. Adolescents with a history of behavioral inhibition, relative to noninhibited adolescents, showed increased activation in the nucleus accumbens when they believed their selection of an action would affect reward outcome. Neural responses did not differ between the two groups when participants made a prespecified response that they knew would result in reward or when they produced random motor responses that they knew would not be rewarded. These results link inhibited temperament and perturbed neural responses to reward-contingency cues.     

阈下奖励调节认知控制的权衡

本研究采用奖励版的AX-CPT任务, 通过控制奖励线索和反馈的呈现方式, 设置出基线、阈上奖励、阈下奖励三种条件, 考察习得的奖励联结和奖励动机能否以潜意识的方式影响认知任务的表现和认知控制的权衡。结果显示：奖励线索只在阈上呈现时提高AX序列的任务表现; 与基线条件相比, 阈上和阈下奖励条件下认知控制的权衡均偏向主动性控制, 且这种偏向在两者间无显著差异。这表明习得的阈下奖励线索可以调节认知控制的权衡, 使被试像阈上奖励时那样偏向主动性控制。     

Cognitive neuroscience of episodic memory encoding

This paper presents a cognitive neuroscientific perspective on how human episodic memories are formed. Convergent evidence from multiple brain imaging studies using positron emission tomography (PET) and functional magnetic resonance imaging (fMRI) suggests a role for frontal cortex in episodic memory encoding. Activity levels within frontal cortex can predict episodic memory encoding across a wide range of behavioral manipulations known to influence memory performance, such as those present during levels of processing and divided attention manipulations. Activity levels within specific frontal and medial temporal regions can even predict, on an item by item basis, whether an episodic memory is likely to form. Furthermore, separate frontal regions appear to participate in supplying code-specific information, including distinct regions which process semantic attributes of verbal information as well as right-lateralized regions which process nonverbal information. We hypothesize that activity within these multiple frontal regions provides a functional influence (input) to medical temporal regions that bind the information together into a lasting episodic memory trace.     

Behavioral, but not reward, risk modulates activation of prefrontal, parietal, and insular cortices

Risky decisions may involve uncertainty about possible outcomes (i.e., reward risk) or uncertainty about which action should be taken (i.e., behavioral risk). Determining whether different forms of risk have distinct neural correlates is a central goal of neuroeconomic research. In two functional magnetic resonance imaging experiments, subjects viewed shapes that had well-learned response-reward contingencies. Magnitude of a monetary reward was held constant within one experiment, whereas expected value was held constant within the other. Response selection, in the absence of behavioral risk, evoked activation within a broad set of brain regions, as had been found in prior studies. However, behavioral risk additionally modulated activation in prefrontal, parietal, and insular regions, within which no effect of reward risk was observed. Reward delivery, in comparison with omission, evoked increased activity in the ventromedial prefrontal cortex and the nucleus accumbens. We conclude that distinct brain systems are recruited for the resolution of different forms of risk.     

The neural correlates of reward-related trial-and-error learning: an fMRI study with a probabilistic learning task

This fMRI study investigated the neural correlates of reward-related trial-and-error learning in association with changing degrees of stimulus-outcome predictabilities. We found that decreasing predictability was associated with increasing activation in a frontoparietal network. Only maximum predictability was associated with signal decreases across the learning process. The receipt of monetary reward revealed activation in the striatum and associated frontoparietal regions. Present data indicate that during reward-related learning, high uncertainty forces areas relevant for cognitive control to remain activated. In contrast, learning on the basis of predictable stimulus-outcome associations enables the brain to reduce resources in association with the processes of prediction.     

工作记忆与情景记忆中脑区的重合与分离——从脑区定位看两者关系

尽管近期的研究结果认为前额叶在工作记忆和情景记忆任务中均被激活,但是仍然不清楚是否有特定的区域对两种记忆起着不同作用。对于工作记忆和情景记忆在脑区上的重合,研究者们做出了不同的解释。某些研究者认为脑区的重合反映了包含于情景记忆中的工作记忆的作用,另一些研究者则认为这种重合反映的是与工作记忆和情景记忆都有关的反省过程的性质。对此,作者提出了相应的讨论意见。     

Dopaminergic modulation of incentive motivation in adolescence: age-related changes in signaling, individual differences, and implications for the development of self-regulation

Behavioral activation that is associated with incentive-reward motivation increases in adolescence relative to childhood and adulthood. This quadratic developmental pattern is generally supported by behavioral and experimental neuroscience findings. It is suggested that a focus on changes in dopamine neurotransmission is informative in understanding the mechanism for this adolescent increase in reward-related behavioral activation and subsequent decline into adulthood. Evidence is presented to indicate that incentive-reward motivation is modulated by mesoaccumbens dopamine, and that it increases in adolescence before declining into adulthood because of normative developmental changes at the molecular level. Potential mechanisms of variation in functional mesoaccumbens dopamine transmission are discussed with a focus on the interplay between tonic and phasic modes of dopamine transmission in modulating both general incentive-motivational biases and the efficacy of reward learning during exposure to novel reward experiences. Interactions between individual difference factors and these age-related trends are discussed.     

情景记忆成功年老化的神经机制

随着年龄的增长, 大部分老年人的情景记忆会出现衰退, 但也会有一部分老年人的情景记忆表现出成功的年老化, 即记忆成绩较好或随增龄的衰退程度较小。脑保持理论、神经去分化理论、认知储备理论以及神经补偿理论分别从不同角度解释了情景记忆成功年老化的神经机制。基于选择性优化与补偿模型对现有理论进行整合, 发现情景记忆成功年老化可能与个体的认知储备水平直接相关：高认知储备的老年人能够对情景记忆相关的脑区和脑网络进行优化且具备更强的神经补偿能力, 因而其脑功能(比如, 神经表征和神经加工通路的特异性)可能会保持地更好。未来研究需要更多地采用纵向设计来考察各理论之间的关系及其影响因素, 从而更好地解释记忆成功年老化的神经机制并为提升老年人的脑与认知健康提供支持。     

The impact of an unpredictable context and intolerance of uncertainty on the electrocortical response to monetary gains and losses

There is growing evidence that unpredictability and uncertainty can alter reward system functioning. The present study examined the impact of (1) a task-irrelevant unpredictable relative to predictable context and (2) individual differences in intolerance of uncertainty (IU) on the reward-related positivity (RewP), an event-related potential (ERP) response to monetary gains relative to losses. Specifically, 64 participants listened to predictable and unpredictable tone sequences while electroencephalography was recorded during a monetary gambling task. Participants also completed the Intolerance of Uncertainty Scale, which measures both cognitive distress (prospective IU) and behavioral inhibition (inhibitory IU) elicited by uncertainty, in addition to the Depression Anxiety Stress Scale-21 and Penn State Worry Questionnaire. Results indicated that the RewP was reduced during the unpredictable relative to the predictable context. Greater self-reported anxiety elicited by the unpredictable context was associated with a decreased RewP, and a decreased RewP was associated with poorer lose-shift behavioral adjustment. Furthermore, the RewP mediated the relationship between self-reported anxiety elicited by the unpredictable context and lose-shift behavioral adjustment. The IU subscales demonstrated the opposite relationship with the RewP across both contexts—inhibitory IU was associated with an attenuated RewP and prospective IU was associated with an enhanced RewP. In contrast, anxiety, depression, stress, and worry symptomatology were not associated with the RewP. This is the first study to demonstrate that an unpredictable context and individual differences in the degree to which people cannot tolerate uncertainty impact an ERP measure of reward system functioning.     

The posterior parietal paradox: Why do functional magnetic resonance imaging and lesion studies on episodic memory produce conflicting results?

Recent functional magnetic resonance imaging (fMRI) studies addressing healthy subjects point towards posterior parietal cortex (PPC) involvement in episodic memory tasks. This is noteworthy, since neuropsychological studies usually do not connect parietal lesions to episodic memory impairments. Therefore an inventory of the possible factors behind this apparent paradox is warranted. This review compared fMRI studies which demonstrated PPC activity in episodic memory tasks, with findings with studies of patients with PPC lesions. A systematic evaluation of possible explanations for the posterior parietal paradox indicates that PPC activation in fMRI studies does not appear to be attributable to confounding cognitive/psychomotor processes, such as button pressing or stimulus processing. What may be of more importance is the extent to which an episodic memory task loads on three closely related cognitive processes: effort and attention, self‐related activity, and scene and image construction. We discuss to what extent these cognitive processes can account for the paradox between lesion and fMRI results. They are strongly intertwined with the episodic memory and may critically determine in how far the PPC plays a role in a given memory task. Future patient studies might profit from specifically taking these cognitive factors into consideration in the task design.     

Varieties of working memory as seen in biology and in connectionist/control architectures

Biological and computational concepts that underlie the nature working memory are briefly reviewed. The conceptualization of working memory has changed dramatically in the last 30 years. Current biological work has monitored several aspects of memory, including activation decay, sustained activation, long-term connection change, and differential structures for episodic (hippocampal formation) and procedural learning. Current connectionist modeling has identified factors including multiple-region-based processing, control processing as well as data storage, tradeoffs between fast- and slow-connection-change learning effects, and the speeding of acquisition via multiple levels of learning. The need to relate the biological, behavioral, and computational constraints into models of working memory is discussed. Finally, conceptualizations of working memory must acknowledge the need for human learning systems to be robust enough to operate in a dynamic world.     

Neural correlates of successful and unsuccessful verbal memory encoding

Recent neuroimaging studies suggest that episodic memory encoding involves a network of neocortical structures which may act interdependently with medial temporal lobe (mTL) structures to promote the formation of durable memories, and that activation in certain structures is modulated according to task performance. Functional magnetic resonance imaging (fMRI) was used to determine the neural structures recruited during a verbal episodic encoding task and to examine the relationship between activation during encoding and subsequent recognition memory performance across subjects. Our results show performance-correlated activation during encoding both in neocortical and medial temporal structures. Neocortical activations associated with later successful and unsuccessful recognition memory were found to differ not only in magnitude, but also in hemispheric laterality. These performance-related hemispheric effects, which have not been previously reported, may correspond to between-subject differences in encoding strategy.     

Are Episodic Buffer Processes Intact in ADHD? Experimental Evidence and Linkage with Hyperactive Behavior

Working memory deficits are present in a substantial proportion of children with ADHD, and converging evidence links these deficits with ADHD-related behavioral and functional impairments. At the same time, working memory is not a unitary construct, and evidence is lacking regarding the role of several components of this system in ADHD. Preclinical behavioral studies are needed to fractionate the multicomponent working memory system, determine which specific subcomponent(s) are impaired in ADHD, and more importantly link these subcomponent(s) with specific ADHD-related behavioral symptoms/functional impairments. The current study reflects one piece of that puzzle, and focuses on the episodic buffer component of working memory. Across multiple testing days, a well-characterized sample of 86 children ages 8–13 (M=10.52, SD=1.54; 34 girls; 64% Caucasian/Non-Hispanic) with ADHD (n=49) and without ADHD (n=37) completed three counterbalanced working memory tests that were identical in all aspects except the key subcomponent process (phonological, visuospatial, episodic buffer). Gross motor movement during these and control tasks were measured using 4 high-precision actigraphs. There was no evidence of group differences in gender, age, SES, or IQ. Bayesian mixed-model ANOVAs indicated that the ADHD group performed significantly worse on all three working memory tests (d=1.17–1.44) and was significantly more hyperactive than controls (d=0.66–1.05) during the visuospatial and episodic buffer tests. In contrast, the ADHD and Non-ADHD groups were equivalent with regard to effects of episodic buffer demands on performance and hyperactive behavior. The most parsimonious conclusion is that the episodic buffer is likely intact in ADHD, and unrelated to ADHD hyperactivity symptoms.     

The hippocampus, space, and viewpoints in episodic memory

A computational model of how single neurons in and around the rat hippocampus support spatial navigation is reviewed. The extension of this model, to include the retrieval from human long-term memory of spatial scenes and the spatial context of events is discussed. The model explores the link between spatial and mnemonic functions by supposing that retrieval of spatial information from long-term storage requires the imposition of a particular viewpoint. It is consistent with data relating to representational hemispatial neglect and the involvement of the mammillary bodies, anterior thalamus, and hippocampal formation in supporting both episodic recall and the representation of head direction. Some recent behavioural, neuropsychological, and functional neuroimaging experiments are reviewed, in which virtual reality is used to allow controlled study of navigation and memory for events set within a rich large-scale spatial context. These studies provide convergent evidence that the human hippocampus is involved in both tasks, with some lateralization of function (navigation on the right and episodic memory on the left). A further experiment indicates hippocampal involvement in retrieval of spatial information from a shifted viewpoint. I speculate that the hippocampal role in episodic recollection relates to its ability to represent a viewpoint moving within a spatial framework.