Dorsal striatum responses to reward and punishment: Effects of valence and magnitude manipulations

The goal of this research was to further our understanding of how the striatum responds to the delivery of affective feedback. Previously, we had found that the striatum showed a pattern of sustained activation after presentation of a monetary reward, in contrast to a decrease in the hemodynamic response after a punishment. In this study, we tested whether the activity of the striatum could be modulated by parametric variations in the amount of financial reward or punishment. We used an event-related fMRI design in which participants received large or small monetary rewards or punishments after performance in a gambling task. A parametric ordering of conditions was observed in the dorsal striatum according to both magnitude and valence. In addition, an early response to the presentation of feedback was observed and replicated in a second experiment with increased temporal resolution. This study further implicates the dorsal striatum as an integral component of a reward circuitry responsible for the control of motivated behavior, serving to code for such feedback properties as valence and magnitude.     

Interaction of reinforcement conditions and developmental level in a two-choice discrimination task with children

A total of 540 first, fourth, and eighth graders, equally divided by sex, were run on a two-choice discrimination task under one of three reinforcement conditions: reward (R) for correct responses, punishment (P) for incorrect responses, or reward and punishment (RP) for correct and incorrect responses. Half the Ss were tested by a male E; half, by a female E. Across all developmental levels, learning was superior in the P group. Complex interactions involving sex of S and E underline the importance of organismic variables in discrimination learning and in specifying parameters of reinforcer effectiveness.     

Opposing amygdala and ventral striatum connectivity during emotion identification

Lesion and electrophysiological studies in animals provide evidence of opposing functions for subcortical nuclei such as the amygdala and ventral striatum, but the implications of these findings for emotion identification in humans remain poorly described. Here we report a high-resolution fMRI study in a sample of 39 healthy subjects who performed a well-characterized emotion identification task. As expected, the amygdala responded to THREAT (angry or fearful) faces more than NON-THREAT (sad or happy) faces. A functional connectivity analysis of the time series from an anatomically defined amygdala seed revealed a strong anticorrelation between the amygdala and the ventral striatum/ventral pallidum, consistent with an opposing role for these regions in during emotion identification. A second functional connectivity analysis (psychophysiological interaction) investigating relative connectivity on THREAT vs. NON-THREAT trials demonstrated that the amygdala had increased connectivity with the orbitofrontal cortex during THREAT trials, whereas the ventral striatum demonstrated increased connectivity with the posterior hippocampus on NON-THREAT trials. These results indicate that activity in the amygdala and ventral striatum may be inversely related, and that both regions may provide opposing affective bias signals during emotion identification.     

Is all motivation good for learning? Dissociable influences of approach and avoidance motivation in declarative memory

The present study investigated the effects of approach versus avoidance motivation on declarative learning. Human participants navigated a virtual reality version of the Morris water task, a classic spatial memory paradigm, adapted to permit the experimental manipulation of motivation during learning. During this task, participants were instructed to navigate to correct platforms while avoiding incorrect platforms. To manipulate motivational states participants were either rewarded for navigating to correct locations (approach) or punished for navigating to incorrect platforms (avoidance). Participants' skin conductance levels (SCLs) were recorded during navigation to investigate the role of physiological arousal in motivated learning. Behavioral results revealed that, overall, approach motivation enhanced and avoidance motivation impaired memory performance compared to nonmotivated spatial learning. This advantage was evident across several performance indices, including accuracy, learning rate, path length, and proximity to platform locations during probe trials. SCL analysis revealed three key findings. First, within subjects, arousal interacted with approach motivation, such that high arousal on a given trial was associated with performance deficits. In addition, across subjects, high arousal negated or reversed the benefits of approach motivation. Finally, low-performing, highly aroused participants showed SCL responses similar to those of avoidance-motivation participants, suggesting that for these individuals, opportunities for reward may evoke states of learning similar to those typically evoked by threats of punishment. These results provide a novel characterization of how approach and avoidance motivation influence declarative memory and indicate a critical and selective role for arousal in determining how reinforcement influences goal-oriented learning.     

Parallel associative processing in the dorsal striatum: segregation of stimulus-response and cognitive control subregions

Although evidence suggests that the dorsal striatum contributes to multiple learning and memory functions, there nevertheless remains considerable disagreement on the specific associative roles of different neuroanatomical subregions. We review evidence indicating that the dorsolateral striatum (DLS) is a substrate for stimulus–response habit formation – incremental strengthening of simple S–R bonds – via input from sensorimotor neocortex while the dorsomedial striatum (DMS) contributes to behavioral flexibility – the cognitive control of behavior – via prefrontal and limbic circuits engaged in relational and spatial information processing. The parallel circuits through dorsal striatum interact with incentive/affective motivational processing in the ventral striatum and portions of the prefrontal cortex leading to overt responding under specific testing conditions. Converging evidence obtained through a detailed task analysis and neurobehavioral assessment is beginning to illuminate striatal subregional interactions and relations to the rest of the mammalian brain.     

Human anterior and posterior hippocampus respond distinctly to state and trait anxiety

We examined whether anterior and posterior hippocampal subregions in humans show distinct relationships to state and trait anxiety. In rodents, the ventral (but not dorsal) hippocampus is critically involved in contextual anxiety, whereas dorsal hippocampus is affected by chronic stress and genetically bred trait anxiety. These studies suggest that state forms of anxiety may be more associated with anterior (ventral in rodents) hippocampus, whereas trait forms of anxiety maybe more associated with posterior (dorsal in rodents) hippocampus. Participants were placed under alternating blocks of threat of shock and safety conditions while performing a secondary task, and state and trait anxiety measures were obtained. Using subject-specific anatomically defined masks, we found that state anxiety was related to activity in anterior but not posterior hippocampus, whereas trait anxiety showed the opposite pattern. Additionally, a psychophysiological connectivity analysis showed that activity in anterior hippocampus was more strongly related to activity in ventromedial prefrontal cortex under threat than under safety conditions, significantly more so than activity in posterior hippocampus was. Hence, anterior hippocampus shows a distinct moment-to-moment connectivity profile with other neural regions during threat relative to posterior hippocampus. The findings provide several lines of evidence for functional differentiation of anterior and posterior hippocampal involvement across state and trait components of anxiety in humans.     

Evidence for an antagonistic interaction between reward and punishment sensitivity on striatal activity: A verification of the Joint Subsystems Hypothesis

The Reinforcement Sensitivity Theory proposes that the Behavioral Approach System (BAS) comprises dopaminergic brain regions and underpins reward sensitivity causing impulsivity. It has been shown in a supraliminal priming task that highly reward sensitive subjects have a larger reaction time (RT) priming effect and make more commission errors to prime-incongruent targets. We adapted a similar task to event-related fMRI and hypothesized that (1) high reward sensitivity is associated with increased activation in dopaminergic brain regions, the ventral striatum in particular, (2) that BAS related personality traits predict impulsivity and (3) that the BAS effects are larger after adjusting for the interactive influence of trait avoidance, as predicted by the Joint Subsystems Hypothesis. Fourteen healthy females participated in the fMRI experiment and were scored on sensitivity to reward (SR) and trait avoidance, i.e., sensitivity to punishment (SP) and neuroticism (N). SR scores were adjusted by SP and N scores. As hypothesized, adjusted SR scores predicted, more than SR scores alone, activity in the ventral striatum (left caudate nucleus and nucleus accumbens). SR+/ SP− scores predicted increased impulsiveness, i.e., a right side RT priming effect. These results support the Joint Subsystems Hypothesis.     

Reduced spiking in entorhinal cortex during the delay period of a cued spatial response task

Intrinsic persistent spiking mechanisms in medial entorhinal cortex (mEC) neurons may play a role in active maintenance of working memory. However, electrophysiological studies of rat mEC units have primarily focused on spatial modulation. We sought evidence of differential spike rates in the mEC in rats trained on a T-maze, cued spatial delayed response task. Animals begin at the base of the T-maze where a 1-sec white noise and visual light cue are presented on the left or right side of the maze. Rats are rewarded for responding toward the cued direction. In correct trials, we observed decreased spike rates during the delay period, the time interval between cue presentation and reward delivery. Firing-rate histograms show significant decreases during the delay period compared to 5-sec windows from both pre-cue and post-reward periods. We analyzed how running speed and trajectory specificity correlated to spike rate. Twice as many cells were responsive to cue alone compared to running speed. Trajectory specificity did not relate significantly to firing rate. Decreased spike rate may reflect active maintenance in other structures inhibiting mEC. Alternately, the reduction may reflect decreases in background activity during enhanced attention and cholinergic modulation. Lastly, animals often ran through the T-maze choice-point with varying speed. We calculated the spatial posterior probability density from spike rates during these choice-point passes. Slow passes through the choice point were characterized by greater probability of decoding to the reward locations on correct trials compared to quick passes on the maze consistent with similar "look-ahead" properties previously reported in the hippocampus and ventral striatum.     

Analogical reasoning and the differential outcome effect: transitory bridging of the conceptual gap for rhesus monkeys (Macaca mulatta)

Monkeys, unlike chimpanzees and humans, have a marked difficulty acquiring relational matching-to-sample (RMTS) tasks that likely reflect the cognitive foundation upon which analogical reasoning rests. In the present study, rhesus monkeys (Macaca mulatta) completed a categorical (identity and nonidentity) RMTS task with differential reward (pellet ratio) and/or punishment (timeout ratio) outcomes for correct and incorrect choices. Monkeys in either differential reward-only or punishment-only conditions performed at chance levels. However, the RMTS performance of monkeys experiencing both differential reward and punishment conditions was significantly better than chance. Subsequently when all animals experienced nondifferential outcomes tests, their RMTS performance levels were at chance. These results indicate that combining differential reward and punishment contingencies provide an effective, albeit transitory, scaffolding for monkeys to judge analogical relations-between-relations.     

The role of prefrontal cortex in resolving distractor interference

We investigate the hypothesis that those subregions of the prefrontal cortex (PFC) found to support proactive interference resolution may also support delay-spanning distractor interference resolution. Ten subjects performed delayed-recognition tasks requiring working memory for faces or shoes during functional MRI scanning. During the 15-sec delay interval, task-irrelevant distractors were presented. These distractors were either all faces or all shoes and were thus either congruent or incongruent with the domain of items in the working memory task. Delayed-recognition performance was slower and less accurate during congruent than during incongruent trials. Our fMRI analyses revealed significant delay interval activity for face and shoe working memory tasks within both dorsal and ventral PFC. However, only ventral PFC activity was modulated by distractor category, with greater activity for congruent than for incongruent trials. Importantly, this congruency effect was only present for correct trials. In addition to PFC, activity within the fusiform face area was investigated. During face distraction, activity was greater for face relative to shoe working memory. As in ventrolateral PFC, this congruency effect was only present for correct trials. These results suggest that the ventrolateral PFC and fusiform face area may work together to support delay-spanning interference resolution.     

No pain no gain: The positive impact of punishment on the strategic regulation of accuracy

Previous studies have shown that punishing people through a large penalty for volunteering incorrect information typically leads them to withhold more information (metacognitive response bias), but it does not appear to influence their ability to distinguish between their own correct and incorrect answers (metacognitive accuracy discrimination). The goal of the current study was to demonstrate that punishing people for volunteering incorrect information—versus rewarding volunteering correct information—produces more effective metacognitive accuracy discrimination. All participants completed three different general-knowledge tests: a reward test (high points for correct volunteered answers), a baseline test (equal points/penalties for volunteered correct/incorrect answers) and a punishment test (high penalty for incorrect volunteered answers). Participants were significantly better at distinguishing between their own correct and incorrect answers on the punishment than reward test, which has implications for situations requiring effective accuracy monitoring.     

Immediate early gene activation in hippocampus and dorsal striatum: effects of explicit place and response training

Evidence from lesion, electrophysiological, and neuroimaging studies support the hypothesis that the hippocampus and dorsal striatum process afferent inputs in such a way that each structure regulates expression of different behaviors in learning and memory. The present study sought to determine whether rats explicitly trained to perform one of two different learning strategies, spatial or response, would display disparate immediate early gene activation in hippocampus and striatum. c-Fos and Zif268 immunoreactivity (IR) was measured in both hippocampus and striatum 30 or 90 min following criterial performance on a standard plus-maze task (place learners) or a modified T-maze task (response learners). Place and response learning differentially affected c-Fos-IR in striatum but not hippocampus. Specifically, explicit response learning induced greater c-Fos-IR activation in two subregions of the dorsal striatum. This increased c-Fos-IR was dependent upon the number of trials performed prior to reaching behavioral criterion and accuracy of performance during post-testing probe trials. Quantification of Zif268-IR in both hippocampus and striatum failed to distinguish between place and response learners. The changes in c-Fos-IR occurred 30 min, but not 90 min, post-testing. The synthesis of c-Fos early in testing could reflect the recruitment of key structures in learning. Consequently, animals that were able to learn the response task efficiently displayed greater amounts of c-Fos-IR in dorsal striatum.     

奖惩对行为抑制及程序阶段中自主生理反应的影响

采用停止信号任务探讨奖惩条件对行为抑制和程序阶段中自主生理反应的影响。结果显示：(1)奖惩组的反应时显著高于控制组, 抑制失败率则显著低于控制组; (2)奖励组与其它两组相比表现出心率显著增加和皮肤电活动显著降低; (3)各生理指标在准备、工作和奖惩阶段中的奖惩差异性显著(4)奖惩条件下皮肤电活动呈现出一致性程序阶段变化。结果表明：奖惩条件均对行为具有抑制作用, 但在自主生理反应上则有显著差异; 各生理指标在准备、工作和奖惩阶段的情绪效价显著; 皮肤电活动的程序阶段变化显著, 但其奖惩效价较为稳定, 很少接受程序阶段变化的影响。     

Rats depend on habit memory for discrimination learning and retention

We explored the circumstances in which rats engage either declarative memory (and the hippocampus) or habit memory (and the dorsal striatum). Rats with damage to the hippocampus or dorsal striatum were given three different two-choice discrimination tasks (odor, object, and pattern). These tasks differed in the number of trials required for learning (~10, 60, and 220 trials). Dorsal striatum lesions impaired discrimination performance to a greater extent than hippocampal lesions. Strikingly, performance on the task learned most rapidly (the odor discrimination) was severely impaired by dorsal striatum lesions and entirely spared by hippocampal lesions. These findings suggest that discrimination learning in the rat is primarily supported by the dorsal striatum (and habit memory) and that rats engage a habit-based memory system even for a task that takes only a few trials to acquire. Considered together with related studies of humans and nonhuman primates, the findings suggest that different species will approach the same task in different ways.     

Patterns of brain acetylcholine release predict individual differences in preferred learning strategies in rats

Acetylcholine release was measured simultaneously in the hippocampus and dorsal striatum of rats before and during training on a maze that could be learned using either a hippocampus-dependent spatial strategy or a dorsal striatum-dependent turning strategy. A probe trial administered after rats reached a criterion of 9/10 correct responses revealed that about half of the rats used a spatial strategy and half a turning strategy to solve the task. Acetylcholine release in the hippocampus, as well as the ratio of acetylcholine release in the hippocampus vs. the dorsal striatum, measured either before or during training, predicted these individual differences in strategy selection during learning. These findings suggest that differences in release of acetylcholine across brain areas may provide a neurobiological marker of individual differences in selection of the strategies rats use to solve a learning task.     

时间估计任务中的反馈加工和行为调节：来自ERP的证据

本研究采用时间估计任务, 利用高时间分辨率的ERP技术, 试图从时间进程上窥探行为调节的认知神经机制。该时间估计任务提供了指导行为调节的反馈信息有三类：时间估计的准确性、时间估计的不准确方向和不准确程度。行为结果表明, 被试根据前次时间估计的对错、不准确的方向和程度进行相应的调整。脑电成分发现, FRN对于时间估计的对、错敏感, 对时间估计的不准确的方向和程度不敏感, 显示FRN提供了粗略的、需要做出行为调节的早期预警信号; 相反, P300能够区别时间估计上不准确的方向和程度, 结合行为数据, 显示P300反映的是整合不同的信息来更新行为表征, 从而引导被试做出不同方向和不同程度的调节。这些结果表明, 行为调节的认知神经基础可能由两个阶段组成：早期传递需要做出行为调节的预警信号阶段和晚期整合信息来更新行为表征从而指导行为调节的阶段。     

The role of the dorsal and ventral hippocampus in olfactory working memory

Olfactory working memory and pattern separation for odor information was assessed in male rats using a matching-to-sample for odors paradigm. The odor set consisted of a five aliphatic acids with unbranched carbon chains that varied from two- to six-carbons in length. Each trial consisted of a sample phase followed by a choice phase. During the sample phase, rats would receive one of five different odors. Fifteen seconds later during the choice phase one of the previous odors was presented simultaneously side by side with a different odor that was based on the number of aliphatic acids that varied in the carbon chains from two- to six-carbons in length and rats were allowed to choose between the two odors. The rule to be learned in order to receive a food reward was to always choose the odor that occurred during the study phase. Odor separations of 1, 2, 3 or 4 were selected for each choice phase and represented the carbon chain difference between the study phase odor and the test phase odor. Once an animal reached a criterion of 80–90% correct across all temporal separations based on 40 trials, rats received a control, dorsal hippocampal, or ventral hippocampal lesion and were retested on the task. On postoperative trials, only the ventral hippocampal lesion group was impaired relative to both control and dorsal hippocampal groups groups. There were no effects on odor pattern separation. All groups of rats could discriminate between the odors. The data suggest that the ventral hippocampus, but not dorsal hippocampus, supports working memory for odor information.     

Intuitive decision making in complex situations: Somatic markers in an artificial grammar learning task

In this article, we explore the extent to which implicit learning is subtended by somatic markers, as evidenced by skin conductance measures. On each trial, subjects were asked to decide which “word” from a pair of “words” was the “correct” one. Unknown to the subjects, each “word” of a pair was constructed using a different set of rules (Grammar Aand Grammar B). A (monetary) reward was given if the subject chose the “word” from Grammar A. Choosing the Grammar B word resulted in (monetary) punishment. Skin conductance was measured during each of 100 trials. After each set of 10 trials, the subjects were asked how they selected the “correct word.” Task performance increased long before the subjects could even formulate a single relevant rule. In this preconceptual phase of the experiment, skin conductance was larger before incorrect than before correct choices. Thus, it was shown that artificial grammar learning is accompanied by a somatic marker, possibly “warning” the subject of the incorrect decision.     

Effects of decision-phase time constraints on emotion-based learning in the Iowa Gambling Task

Research employing the Iowa Gambling Task (IGT) has frequently shown that learning is impaired in various clinical populations. However, precisely what constitutes "unimpaired" control group learning remains unclear. In order to understand some of the possible factors underlying variability in control group IGT performance, the present study sought to manipulate features of the task to intentionally disrupt learning. Specifically, the present study investigated the effects of time constraints on emotion-based learning during automated administration of the IGT. For two groups of participants, a time-constraint of either 2-s or 4-s was implemented during the critical decision making period, while a control group received no time constraint. We also evaluated participants' subjective experience after every block of 20 trials. Results demonstrated that the 2-s group differed significantly from the control group. Subjective experience measures revealed rapid development of awareness of the advantageous and disadvantageous decks among all three groups. Overall, our findings demonstrate, for the first time, the effects of decision-phase time constraints on emotion-based learning and indicate that the IGT reward/punishment schedules are to some extent cognitively penetrable.     

Rat hippocampus and memory for places of changing significance

Rats were tested once daily on a four-choice delayed match from sample task with a water reward. Each day the correct place changed, and a single exposure to it was provided on information trials. Lesions of the hippocampal formation that involved the fornix, or dorsal hippocampus bilaterally, produced a severe impairment in the performance of previously trained rats. By contrast, lesions of the ventral hippocampus did not preclude reacquisition of the place-memory task. Some otherwise impaired rats with fornical lesions were able to find the water when aided by nonplace cues that consistently signaled reward. Reducing the number of choices from four to two did not aid the impaired rats. Certain lesions of the hippocampal formation in the rat produce a deficit appropriately described as amnesia. The memory deficit is consistent with a role for the hippocampus in processing of place information and shows some parallels to the amnesia seen in persons with temporal lobe lesions.