Learning from delayed feedback: neural responses in temporal credit assignment

When feedback follows a sequence of decisions, relationships between actions and outcomes can be difficult to learn. We used event-related potentials (ERPs) to understand how people overcome this temporal credit assignment problem. Participants performed a sequential decision task that required two decisions on each trial. The first decision led to an intermediate state that was predictive of the trial outcome, and the second decision was followed by positive or negative trial feedback. The feedback-related negativity (fERN), a component thought to reflect reward prediction error, followed negative feedback and negative intermediate states. This suggests that participants evaluated intermediate states in terms of expected future reward, and that these evaluations supported learning of earlier actions within sequences. We examine the predictions of several temporal-difference models to determine whether the behavioral and ERP results reflected a reinforcement-learning process.     

Decision making under risk condition in patients with Parkinson's disease: a behavioural and fMRI study

We aimed to study whether previously described impairment in decision making under risky conditions in patients with Parkinson's disease (PD) is affected by deficits in using information about potential incentives or by processing feedback (in terms of fictitious gains and losses following each decision). Additionally, we studied whether the neural correlates of using explicit information in decision making under risk differ between PD patients and healthy subjects. We investigated ten cognitively intact PD patients and twelve healthy subjects with the Game of Dice Task (GDT) to assess risky decision making, and with an fMRI paradigm to analyse the neural correlates of information integration in the deliberative decision phase. Behaviourally, PD patients showed selective impairment in the GDT but not on the fMRI task that did not include a feedback component. Healthy subjects exhibited lateral prefrontal, anterior cingulate and parietal activations when integrating decision-relevant information. Despite similar behavioural patterns on the fMRI task, patients exhibited reduced parietal activation. Behavioural results suggest that PD patients' deficits in risky decision making are dominated by impaired feedback utilization not compensable by intact cognitive functions. Our fMRI results suggest similarities but also differences in neural correlates when using explicit information for the decision process, potentially indicating different strategy application even if the interfering feedback component is excluded.     

ERPs associated with visual duration discriminations in prefrontal and parietal cortex

An event-related potentials (ERP) study was undertaken to examine the role of prefrontal and parietal association cortices on selective attention and short-term memory functions in a duration discrimination task. Subjects performed better when discriminating the first stimulus relative to the second and not the reverse. Two contingent negative variations (CNV) were obtained for each stimulus duration at prefrontal regions, as well as two P300s at parietal regions. The CNV(S1) component recorded during the first stimulus (S1) appeared to be involved in selective attention at bilateral sites, while the P300(S1) component in the left hemisphere may be implicated in retaining it. The CNV(S2) wave, displayed during the second stimulus (S2), at bilateral sites and the right-sided P300(S2) wave seem to be implicated in working memory. The results indicate that recorded activity at prefrontal and parietal association cortices is tightly linked to task parameters and behavioral performances.     

Toward a taxonomy of attention shifting: Individual differences in fMRI during multiple shift types

Although task switching is often considered one of the fundamental abilities underlying executive functioning and general intelligence, there is little evidence that switching is a unitary construct and little evidence regarding the relationship between brain activity and switching performance. We examined individual differences in multiple types of attention shifting in order to determine whether behavioral performance and fMRI activity are correlated across different types of shifting. The participants (n=39) switched between objects and attributes both when stimuli were perceptually available (external) and when stimuli were stored in memory (internal). We found that there were more switchrelated activations in many regions associated with executive control—including the dorsolateral and medial prefrontal and parietal cortices—when behavioral switch costs were higher (poor performance). Conversely, activation in the ventromedial prefrontal cortex (VMPFC) and the rostral anterior cingulate was consistently correlated with good performance, suggesting a general role for these areas in efficient attention shifting. We discuss these findings in terms of a model of cognitive-emotional interaction in attention shifting, in which reward-related signals in the VMPFC guide efficient selection of tasks in the lateral prefrontal and parietal cortices.     

Decision theory,reinforcement learning,and the brain

Decision making is a core competence for animals and humans acting and surviving in environments they only partially comprehend, gaining rewards and punishments for their troubles. Decision-theoretic concepts permeate experiments and computational models in ethology, psychology, and neuroscience. Here, we review a well-known, coherent Bayesian approach to decision making, showing how it unifies issues in Markovian decision problems, signal detection psychophysics, sequential sampling, and optimal exploration and discuss paradigmatic psychological and neural examples of each problem. We discuss computational issues concerning what subjects know about their task and how ambitious they are in seeking optimal solutions; we address algorithmic topics concerning model-based and model-free methods for making choices; and we highlight key aspects of the neural implementation of decision making.     

Computational and behavioral markers of model-based decision making in childhood

Human decision-making is underpinned by distinct systems that differ in flexibility and associated cognitive cost. A widely accepted dichotomy distinguishes between a cheap but rigid model-free system and a flexible but costly model-based system. Typically, humans use a hybrid of both types of decision-making depending on environmental demands. However, children's use of a model-based system during decision-making has not yet been shown. While prior developmental work has identified simple building blocks of model-based reasoning in young children (1–4 years old), there has been little evidence of this complex cognitive system influencing behavior before adolescence. Here, by using a modified task to make engagement in cognitively costly strategies more rewarding, we show that children aged 5–11-years (N = 85), including the youngest children, displayed multiple indicators of model-based decision making, and that the degree of its use increased throughout childhood. Unlike adults (N = 24), however, children did not display adaptive arbitration between model-free and model-based decision-making. Our results demonstrate that throughout childhood, children can engage in highly sophisticated and costly decision-making strategies. However, the flexible arbitration between decision-making strategies might be a critically late-developing component in human development.     

Frontal brain asymmetry and reward responsiveness: a source-localization study

The influence of approach and avoidance tendencies on affect, reasoning, and behavior has attracted substantial interest from researchers across various areas of psychology. Currently, frontal electroencephalographic (EEG) asymmetry in favor of left prefrontal regions is assumed to reflect the propensity to respond with approach-related tendencies. To test this hypothesis, we recorded resting EEG in 18 subjects, who separately performed a verbal memory task under three incentive conditions (neutral, reward, and punishment). Using a source-localization technique, we found that higher task-independent alpha2 (10.5-12 Hz) activity within left dorsolateral prefrontal and medial orbitofrontal regions was associated with stronger bias to respond to reward-related cues. Left prefrontal resting activity accounted for 54.8% of the variance in reward bias. These findings not only confirm that frontal EEG asymmetry modulates the propensity to engage in appetitively motivated behavior, but also provide anatomical details about the underlying brain systems.     

Reward prospect rapidly speeds up response inhibition via reactive control

Response inhibition is an important cognitive-control function that allows for already-initiated or habitual behavioral responses to be promptly withheld when needed. A typical paradigm to study this function is the stop-signal task. From this task, the stop-signal response time (SSRT) can be derived, which indexes how rapidly an already-initiated response can be canceled. Typically, SSRTs range around 200 ms, identifying response inhibition as a particularly rapid cognitive-control process. Even so, it has recently been shown that SSRTs can be further accelerated if successful response inhibition is rewarded. Since this earlier study effectively ruled out differential preparatory (proactive) control adjustments, the reward benefits likely relied on boosted reactive control. Yet, given how rapidly such control processes would need to be enhanced, alternative explanations circumventing reactive control are important to consider. We addressed this question with an fMRI study by gauging the overlap of the brain networks associated with reward-related and response-inhibition-related processes in a reward-modulated stop-signal task. In line with the view that reactive control can indeed be boosted swiftly by reward availability, we found that the activity in key brain areas related to response inhibition was enhanced for reward-related stop trials. Furthermore, we observed that this beneficial reward effect was triggered by enhanced connectivity between task-unspecific (reward-related) and task-specific (inhibition-related) areas in the medial prefrontal cortex (mPFC). The present data hence suggest that reward information can be translated very rapidly into behavioral benefits (here, within ~200 ms) through enhanced reactive control, underscoring the immediate responsiveness of such control processes to reward availability in general.     

Bi-hemispheric engagement in the retrieval of autobiographical episodes

Functional magnetic resonance imaging (fMRI) was used to study the neural correlates of neutral, stressful, negative and positive autobiographical memories. The brain activity produced by these different kinds of episodic memory did not differ significantly, but a common pattern of activation for different kinds of autobiographical memory was revealed that included (1) largely bilateral portions of the medial and superior temporal lobes, hippocampus and parahippocampus, (2) portions of the ventral, medial, superior and dorsolateral prefrontal cortex, (3) the anterior and posterior cingulate, including the retrosplenial, cortex, (4) the parietal cortex, and (5) portions of the cerebellum. The brain regions that were mainly activated constituted an interactive network of temporal and prefrontal areas associated with structures of the extended limbic system. The main bilateral activations with left-sided preponderance probably reflected reactivation of complex semantic and episodic self-related information representations that included previously experienced contexts. In conclusion, the earlier view of a strict left versus right prefrontal laterality in the retrieval of semantic as opposed to episodic autobiographical memory, may have to be modified by considering contextual variables such as task demands and subject variables. Consequently, autobiographical memory integration should be viewed as based on distributed bi-hemispheric neural networks supporting multi-modal, emotionally coloured components of personal episodes.     

重复信任博弈的决策过程与结果评价

“信任他人或者不信任他人?”是社会互动中经常面临一种行为决择, 直接影响着个体的社会生活。作为一种重要的社会信号机制, 信任是建立良好社会关系的基石, 它能够降低社会交易成本, 易化合作行为。为了探究个体在信任互动情境下大脑活动变化的时间动态特征, 采用事件相关电位技术记录了20名健康被试(充当信任者)完成重复性信任博弈任务(repeated Trust Game, rTG)时的脑电成分。行为结果发现个体选择信任的比例显著高于几率水平。电生理学结果表明, 决策阶段中不信任选择比信任选择诱发了更正的P2成分(150~250 ms), 差异波溯源分析定位于额中回(middle frontal gyrus, BA 46)和额下回(inferior frontal gyrus, BA 46)。反馈阶段中损失反馈比获益反馈诱发了更负的FRN成分(200~300 ms), 而获益反馈比损失反馈诱发了更短的P300潜伏期。本研究为理解人类信任博弈过程的动态加工时程提供了初步的神经电生理学证据。     

多目标追踪的神经机制

多目标追踪任务是研究动态场景中视觉注意加工机制常用的范式。自1998年开始对多目标追踪神经机制的影像学研究以来, 研究者采用ERP和fMRI等技术对多目标注意追踪所涉及的神经电生理活动和脑功能区激活方面开展了大量研究。ERP研究发现, 追踪过程持续的ERP脑电成分如N2pc、CDA的波幅与注意追踪负荷有关, 并且出现在目标与非目标上的探测刺激诱发的脑电成分如N1、P1波幅的差异可反映注意资源的分配, 具体为目标在追踪过程中得到了激活, 而非目标受到了抑制。fMRI研究比较一致地发现了顶叶(包括前顶内沟、后顶内沟、顶上小叶)、背外侧额叶皮层等在注意追踪中的强烈激活。其中顶内沟主要与注意负荷有关, 顶内沟的活动水平直接决定了观察者注意追踪的行为表现。而顶上小叶可能更多的负责注意转移。背外侧额叶皮层可能负责追踪时的感觉运动预测过程。     

Task complexity and habituation effects on frontal P300 topography

The P3(00) event-related potential (ERP) component is usually reported as having a centroparietal maximum. However, the P3 topography is more frontal in early-session trials which may be masked by averaging over the entire session and is evident longer into the test session among elderly subjects. This hyperfrontality is interpreted as a sign of poor prefrontal adaptive functioning. In the present study, P3 amplitude was examined in university students to determine how early a change in amplitude would be evident and the effect of task complexity on the amplitude across electrode sites. ERPs were elicited using a working-memory n-back task where participants pressed a key to target letter presentations in three conditions of increasing complexity. Single-trial ERP waveforms were then averaged in successive sequences of five trials. Results revealed a greater decrease in frontal P3 amplitude compared to the central and parietal P3 after the first block of five target trials until the third block. The results are interpreted as indicating rapid decrease in hyperfrontality with habituation to an easy task. Increases in task complexity (the 2-back paradigm), however, reduced this frontal P3 attenuation. Results support a P3 ERP model of hyperfrontality reflecting short-term adaptive function by the prefrontal cortex.     

Dissociation of the neural systems for working memory maintenance of verbal and nonspatial visual information

Working memory for names and faces was investigated to ascertain whether verbal and nonspatial visual information is maintained in working memory by separate neural systems. The subjects performed a delayed match-to-sample task for famous or unfamous faces and names and a sensorimotor control task. Several occipital, temporal, parietal, and prefrontal areas were activated during all memory delays, in comparison with the control delays. Greater delay activity for unfamous faces than for names was obtained in the right fusiform gyrus, right inferior frontal gyrus (IFG), right IFG/ precentral gyrus, and right medial superior frontal gyrus, whereas greater delay activity for unfamous names than for faces was observed in the precuneus, left insula/postcentral gyrus, and left IFG/ precentral gyrus. There was no significant difference in the prefrontal activity in the comparison between famous faces and names. Greater delay activity for famous names than for faces was obtained in visual association and parietal areas. The results indicate that there is a functional dissociation based on information type within the neural system that is responsible for working memory maintenance of verbal and nonspatial visual information.     

Early maternal availability and prefrontal correlates of reward-related memory

Early emotional experiences affect developing brain systems that subsequently mediate adult learning and memory in rodents. Here we test for similar effects in squirrel monkeys (Saimiri sciureus) four years after disruptions in early maternal availability. These conditions were previously shown to generate differences in emotional behavior, hypothalamic-pituitary-adrenal stress physiology, and right ventral medial prefrontal volumes determined in adulthood by magnetic resonance imaging. This report identifies in the same monkeys variability in reward-related memory on tests with a spatial reversal. Adult monkeys that more often selected locations repeatedly rewarded before each reversal had larger right ventral medial prefrontal volumes, but not hippocampal nor dorsolateral prefrontal volumes on the left or right brain side. Differences in performance were also discerned after each spatial reversal. These findings indicate that maternal availability alters developing ventral medial prefrontal brain regions involved in reward-related memory.     

Self-reference during explicit memory retrieval: an event-related potential analysis

Is there a specific neurocognitive system underlying the subjective sense of having a unitary continuous self across time? If so, it should be possible to isolate functions involved in the sense of self from those supporting mental activities that the self is currently engaged in. We report a study of real-time noninvasive recordings of the brain's electrical activity (event-related potentials, ERPs). We found a common neural signature that is associated with self-referential processing regardless of whether subjects are retrieving general knowledge (noetic awareness) or reexperiencing past episodes (autonoetic awareness). These ERP data are consistent with models of autobiographical memory that postulate a single locus of control over explicit memory for various kinds of self-related information. The temporal properties and scalp distribution of this novel self-reference ERP effect also suggest that it may be a neurophysiological correlate of self-related activation in medial prefrontal and parietal neocortical circuits identified in functional magnetic resonance imaging experiments.     

品牌消费旅程中消费者的认知心理过程——神经营销学视角

品牌消费旅程通常指的是对于品牌消费服务的多维度(包括认知, 情绪, 感觉, 行为和品牌关系)响应。揭示品牌消费旅程的认知心理过程是目前营销学领域之中研究的重点与热点。在梳理当前神经营销领域内有关功能性核磁共振(fMRI)、事件相关电位(ERP)、事件相关震荡(ERO)的相关研究成果后, 将消费者在品牌消费旅程之中的认知心理进程划分为注意吸引、决策形成、消费体验和品牌忠诚四个阶段, 并系统阐述了每个阶段之中消费者心理进程的神经机制与脑区活动, 进而全面立体地揭示消费心理的全貌。未来研究可以进一步探索不同神经指标在具体营销情境下的表征意义, 并结合超扫描技术进一步解析多个被试间的神经耦合情况。     

Motivational influences on cognitive control: Behavior,brain activation,and individual differences

What changes in brain activity are associated with changes in motivational state? The present study addressed this question by having participants perform a cognitive task (AX variant of the Continuous Performance Test; AX-CPT) under three different blocked motivational conditions (reward-incentive, penalty-incentive, and baseline). Behavioral data showed that the incentive conditions modulated task performance, potentially by altering participants’ cognitive control strategy. Neuroimaging data indicated that the reward condition was associated with a sustained increase in a primarily right-lateralized network that included parietal and prefrontal cortex. Additionally, individual differences were observed, such that activation in both reward-related brain regions and frontopolar cortex was linked to the degree of motivation-induced performance enhancement and to motivation-related personality variables. These results suggest that changes in motivational state may modulate performance through sustained activity in cognitive control regions and that the effect of incentives may be affected by the personalities of the participants.     

Learning related modulation of functional retrieval networks in man

The medial temporal lobe has been implicated in studies of episodic memory tasks involving spatio-temporal context and object-location conjunctions. We have previously demonstrated that an increased level of practice in a free-recall task parallels a decrease in the functional activity of several brain regions, including the medial temporal lobe, the prefrontal, the anterior cingulate, the anterior insular, and the posterior parietal cortices, that in concert demonstrate a move from elaborate controlled processing towards a higher degree of automaticity. Here we report data from two experiments that extend these initial observations. We used a similar experimental approach but probed for effects of retrieval paradigms and stimulus material. In the first experiment we investigated practice related changes during recognition of object-location conjunctions and in the second during free-recall of pseudo-words. Learning in a neural network is a dynamic consequence of information processing and network plasticity. The present and previous PET results indicate that practice can induce a learning related functional restructuring of information processing. Different adaptive processes likely subserve the functional re-organisation observed. These may in part be related to different demands for attentional and working memory processing. It appears that the role(s) of the prefrontal cortex and the medial temporal lobe in memory retrieval are complex, perhaps reflecting several different interacting processes or cognitive components. We suggest that an integrative interactive perspective on the role of the prefrontal and medial temporal lobe is necessary for an understanding of the processing significance of these regions in learning and memory. It appears necessary to develop elaborated and explicit computational models for prefrontal and medial temporal functions in order to derive detailed empirical predictions, and in combination with an efficient use and development of functional neuroimaging approaches, to further the understanding of the processing significance of these regions in memory.     

Investigating the modality specificity of response selection using a temporal flanker task

The neurocognitive architecture for response selection is uncertain. Some theorists suggest that it is mediated by an amodal central mechanism, whereas others propose a set of independent control mechanisms. In a functional neuroimaging experiment, we investigated the nature of response selection by examining how its underlying brain mechanisms are affected by stimulus modality. To do this, we used a modified flanker task, in which the target and flanker (distractor) stimuli differed in time rather than space, making it accessible for both visual and auditory stimuli. As in the traditional flanker task, larger reaction times were observed for incongruent than congruent trials (i.e., a congruency effect) for both modalities. Congruency affected brain activation for both modalities in prefrontal cortex, parietal cortex, and the putamen. Modality-dependent activation was found in additional prefrontal and parietal regions for the visual modality and in left inferior prefrontal cortex for the auditory modality. Modality-dependent activity specifically related to response congruency was also found in sensory cortical regions. These data suggest that modality affects the brain regions throughout the cortex mediating response selection even for conceptually identical stimuli and tasks. They are consistent with the hypothesis that (at least partially) independent brain networks mediate response selection and that input modality may be a powerful factor for organizing neural activity to support task performance.     

Cortical correlates of impaired self-regulation in personality disordered patients with traits of psychopathy

Psychopathic personality disordered patients would, by virtue of a failure to self-regulate, be expected to show diminished amplitudes of feedback-related brain potentials. Among a sample of personality disordered patients detained at different levels of security, those who met a Psychopathy Checklist (PCL) criterion of 25 or above were identified (N = 27). Their event-related brain potentials (ERPs), together with those of their nonpsychopathic counterparts (N = 22) and healthy male controls (N = 20), were measured while they performed a visual Go/No Go task, with feedback given for correct and incorrect performance. Psychopathic patients showed a significantly reduced amplitude of an early frontal negative ERP component maximally evoked by negative feedback, and a high rate of errors of commission. Findings are consistent with the idea that psychopathic patients' unsuccessful attempts to self-regulate reflect a cognitive deficit characterised by a failure to attend and respond to a mismatch between expected and obtained outcomes.