Hierarchically organized behavior and its neural foundations: a reinforcement learning perspective

Research on human and animal behavior has long emphasized its hierarchical structure—the divisibility of ongoing behavior into discrete tasks, which are comprised of subtask sequences, which in turn are built of simple actions. The hierarchical structure of behavior has also been of enduring interest within neuroscience, where it has been widely considered to reflect prefrontal cortical functions. In this paper, we reexamine behavioral hierarchy and its neural substrates from the point of view of recent developments in computational reinforcement learning. Specifically, we consider a set of approaches known collectively as hierarchical reinforcement learning, which extend the reinforcement learning paradigm by allowing the learning agent to aggregate actions into reusable subroutines or skills. A close look at the components of hierarchical reinforcement learning suggests how they might map onto neural structures, in particular regions within the dorsolateral and orbital prefrontal cortex. It also suggests specific ways in which hierarchical reinforcement learning might provide a complement to existing psychological models of hierarchically structured behavior. A particularly important question that hierarchical reinforcement learning brings to the fore is that of how learning identifies new action routines that are likely to provide useful building blocks in solving a wide range of future problems. Here and at many other points, hierarchical reinforcement learning offers an appealing framework for investigating the computational and neural underpinnings of hierarchically structured behavior.     

传递性推理的ERP研究

以抽象的内容作为实验材料,测定15名大学生在完成三种不同类型的传递性推理任务和基线任务(记忆判断任务)时的事件相关电位(ERP),探讨传递性推理过程的脑内时程动态变化。研究发现,三种推理任务所诱发的波形基本一致,而推理任务与基线任务之间的ERP波形存在明显的差异。这说明对于传递性推理,被试可能是根据视觉的空间表征对信息进行加工的,结果进一步支持了心理模型理论。     

The biology of visual perspective and depression: A reply to Sutin

A recent meta-analysis by Munafò, Durrant, Lewis, and Flint (2009) [Munafò, M. R., Durrant, C., Lewis, G., & Flint, J. (2009). Gene × environment interactions at the serotonin transporter locus. Biological Psychiatry, 65, 211–219] questioned the meaning of studies searching for endophenotypes associated with the serotonin-transporter-linked promoter region (5-HTTLPR) polymorphism, including our study on visual perspective during autobiographical memory retrieval. However, the association of 3rd person perspective with vulnerability for depression does not rely only on genetics. External consistency is provided by the influence of the 5-HTTLPR polymorphism on brain regions involved in emotion regulation and autobiographical memory retrieval (e.g. the medial prefrontal cortex). Internal consistency is provided by the increased prevalence of 3rd person perspective in both currently and previously depressed patients, as well as in depression-prone individuals. Further studies should examine therapeutic opportunities.     

Infusion of protein synthesis inhibitors in the entorhinal cortex blocks consolidation but not reconsolidation of object recognition memory

Memory consolidation and reconsolidation require the induction of protein synthesis in some areas of the brain. Here, we show that infusion of the protein synthesis inhibitors anisomycin, emetine and cycloheximide in the entorhinal cortex immediately but not 180 min or 360 min after training in an object recognition learning task hinders long-term memory retention without affecting short-term memory or behavioral performance. Inhibition of protein synthesis in the entorhinal cortex after memory reactivation involving either a combination of familiar and novel objects or two familiar objects does not affect retention. Our data suggest that protein synthesis in the entorhinal cortex is necessary early after training for consolidation of object recognition memory. However, inhibition of protein synthesis in this cortical region after memory retrieval does not seem to affect the stability of the recognition trace.     

科学发明问题解决的脑机制再探

选取高生态学效度的科学发明问题作为实验材料, 采用“学习−测试”的两阶段实验范式, 利用静息态功能磁共振成像技术, 基于局部一致性(ReHo)和低频振幅(ALFF)的分析方法, 研究科学发明问题解决的神经机制。在控制被试性别、年龄和常规性问题解决能力后, 结果发现左侧前扣带回(Anterior Cingulate Cortex, ACC)的ReHo值和个体科学发明问题解决的正确率显著正相关, ALFF的结果同样印证了这一发现。结果强调了ACC在科学发明问题解决过程中的重要作用。     

经颅直流电刺激技术在物质依赖治疗中的应用

物质依赖是一种慢性、复发性脑部疾病, 给个体及社会带来极大损害。研究发现, 经颅直流电刺激(transcranial direct current stimulation, tDCS)可以通过调节背外侧前额叶(dorsolateral prefrontal cortex, DLPFC)脑区的兴奋性降低物质依赖者的渴求, 并改善受损的认知功能。tDCS具有副作用小、操作简便、费用低廉等优点, 因此在物质依赖治疗领域应用前景广泛。未来应考虑确定最佳的刺激参数及刺激位置, 同时考虑物质依赖者的种类及个体差异, 与其他疗法结合使用, 并进一步深入探索tDCS治疗物质依赖的神经机制。     

前额叶皮质损伤患者的情绪异常

人类前额叶皮质与情绪之间有着密切的关系,文章在相关的临床和神经成像研究结果基础上,分析了前额叶皮质损伤对情绪行为造成的影响,分别概述了精神分裂症、重性抑郁和焦虑患者的情绪变化、前额叶切除对情绪的影响以及左右前额叶皮质与情绪的关系,重点阐述了前额叶功能异常的神经生理学特点,认为情绪的改变会随着前额叶皮质不同部位的结构缺损或功能变化而有所不同,此有利于对前额叶皮质损伤患者情绪异常的评价与防治。     

Domain-specific and domain-general processes underlying metacognitive judgments

Metacognition and self-awareness are commonly assumed to operate as global capacities. However, there have been few attempts to test this assumption across multiple cognitive domains and metacognitive evaluations. Here, we assessed the covariance between “online” metacognitive processes, as measured by decision confidence judgments in the domains of perception and memory, and error awareness in the domain of attention to action. Previous research investigating metacognition across task domains have not matched stimulus characteristics across tasks raising the possibility that any differences in metacognitive accuracy may be influenced by local task properties. The current experiment measured metacognition in perceptual, memorial and attention tasks that were closely matched for stimulus characteristics. We found that metacognitive accuracy across the three tasks was dissociated suggesting that domain specific networks support an individual’s capacity for accurate metacognition. This finding was independent of objective performance, which was controlled using a staircase procedure. However, response times for metacognitive judgments and error awareness were associated suggesting that shared mechanisms determining how these meta-level evaluations unfold in time may underlie these different types of decision. In addition, the relationship between these laboratory measures of metacognition and reports of everyday functioning from participants and their significant others (informants) was investigated. We found that informant reports, but not self reports, predicted metacognitive accuracy on the perceptual task and participants who underreported cognitive difficulties relative to their informants also showed poorer metacognitive accuracy on the perceptual task. These results are discussed in the context of models of metacognitive regulation and neuropsychological evidence for dissociable metacognitive systems. The potential for the refinement of metacognitive assessment in clinical populations is also discussed.