Toward a unified theory of decision criterion learning in perceptual categorization

Optimal decision criterion placement maximizes expected reward and requires sensitivity to the category base rates (prior probabilities) and payoffs (costs and benefits of incorrect and correct responding). When base rates are unequal, human decision criterion is nearly optimal, but when payoffs are unequal, suboptimal decision criterion placement is observed, even when the optimal decision criterion is identical in both cases. A series of studies are reviewed that examine the generality of this finding, and a unified theory of decision criterion learning is described (Maddox & Dodd, 2001). The theory assumes that two critical mechanisms operate in decision criterion learning. One mechanism involves competition between reward and accuracy maximization: The observer attempts to maximize reward, as instructed, but also places some importance on accuracy maximization. The second mechanism involves a flat-maxima hypothesis that assumes that the observer's estimate of the reward-maximizing decision criterion is determined from the steepness of the objective reward function that relates expected reward to decision criterion placement. Experiments used to develop and test the theory require each observer to complete a large number of trials and to participate in all conditions of the experiment. This provides maximal control over the reinforcement history of the observer and allows a focus on individual behavioral profiles. The theory is applied to decision criterion learning problems that examine category discriminability, payoff matrix multiplication and addition effects, the optimal classifier's independence assumption, and different types of trial-by-trial feedback. In every case the theory provides a good account of the data, and, most important, provides useful insights into the psychological processes involved in decision criterion learning.     

肥胖成因的解释——基于食物奖赏研究的视角

肥胖的形成和发展受生物、心理和社会因素的共同作用,其中食物奖赏对肥胖的产生有重要的作用。食物是一种自然奖赏,它指机体天生对食物的渴望和依赖。食物奖赏包括"wanting"、"liking"以及"learningreinforcement"三个成分,每个成分由相应的神经通路表征。食物奖赏调控机体的摄食行为并以此调控体重变化。目前,关于肥胖与食物奖赏关系的理论模型主要有刺激—敏感化理论、奖赏过度理论以及奖赏不足理论。采用横断面设计、前瞻研究设计和纵向被试内重复测量设计,使用食物图片线索和直接给予美味奶昔的技术方法,人类脑成像研究从不同侧面为以上三个理论模型提供了证据。除此之外,食物奖赏还受基因的调控。目前,研究者关注较多的是多巴胺D2受体基因Taq IA rs1800497的多态性和FTO基因rs9939609的多态性对食物奖赏及体重改变的调控。     

A dual to the von Neumann-Morgenstern theorem

The paper examines Ramsey's proposition that preferences among uncertain prospects may be represented in terms of subjectively expected subjective values. While the von Neumann-Morgenstern approach presupposes probabilities and derives values, the present approach does the reverse: it presupposes values and derives probabilities. Necessary and sufficient conditions are presented for such representations for a fairly wide range of preference structures, including discrete as well as continuous spaces of states-of-nature. An operational procedure is suggested for constructing the subjective values and probabilities.     

Activity under Low Motivational Levels as a Function of the Method of Manipulating the Deprivation Period

Boys and girls in Grades 4, 6, and 8 were presented a 48-word list, in which each item was associated with a 5¢, 3¢, or l¢ reward, in order to test incentive effects on storage by means of a forced choice recognition paradigm assumed to minimize retrieval effects. Results showed (a) higher probabilities of recognition for words associated with higher incentive values, (b) serial position effects, and (c) a suggestive developmental progression in recognition performance. Differential reward influences upon storage and serial position were interpreted in terms of two-process memorial and incentive theories.     

美感对西方绘画无意识加工的影响

采用连续闪烁抑制范式(Continuous Flash Suppression, CFS), 通过比较不同美感的图片的突破抑制时间,考察了美感对西方绘画无意识加工的影响。实验1使用黑白噪音图片, 通过单因素被试内设计考察了高、中、低三种美感等级的彩色西方绘画的突破抑制时间。结果发现, 美感高和美感中等的西方绘画比美感低的西方绘画能更快突破噪音图片的抑制进入意识。实验2考察在彩色噪音图片的抑制下, 美感是否依然影响彩色西方绘画突破抑制的时间。结果发现, 美感不影响西方绘画突破抑制时间, 且突破抑制时间显著长于实验1。这些结果表明美感对西方绘画无意识加工的影响受到双眼竞争的眼间抑制过程的限制, 只有在黑白噪音图抑制的情况下, 美感会影响西方绘画进入意识的速度。与黑白噪音图片相比, 彩色噪音图片可能对颜色信息的抑制更强, 干扰了美感对西方绘画无意识加工的影响。     

Learning from delayed feedback: neural responses in temporal credit assignment

When feedback follows a sequence of decisions, relationships between actions and outcomes can be difficult to learn. We used event-related potentials (ERPs) to understand how people overcome this temporal credit assignment problem. Participants performed a sequential decision task that required two decisions on each trial. The first decision led to an intermediate state that was predictive of the trial outcome, and the second decision was followed by positive or negative trial feedback. The feedback-related negativity (fERN), a component thought to reflect reward prediction error, followed negative feedback and negative intermediate states. This suggests that participants evaluated intermediate states in terms of expected future reward, and that these evaluations supported learning of earlier actions within sequences. We examine the predictions of several temporal-difference models to determine whether the behavioral and ERP results reflected a reinforcement-learning process.     

行为决策中模糊规避研究的回顾与展望

模糊规避是指在相同奖赏的情况下,决策者会偏好有精确概率的事件而不是从主观上判断具有相同模糊概率的事件。自从Ellsberg提出模糊规避的概念以来,模糊规避已在行为决策研究的多个领域得到广泛验证。本文梳理了近五十年来关于模糊规避的研究文献,系统分析了模糊规避的研究范式、心理机制和影响因素,同时提出了未来的研究展望。     

Unconscious reward facilitates motion perceptual learning

Reward signal plays an important role in guiding human learning behaviour. Recent studies have provided evidence that reward signal modulates perceptual learning of basic visual features. Typically, the reward effects on perceptual learning were accompanied with consciously presented reward during the learning process. However, whether an unconsciously presented reward signal that minimizes the contribution of attentional and motivational factors can facilitate perceptual learning remains less well understood. We trained human subjects on a visual motion detection task and subliminally delivered a monetary reward for correct response during the training. The results showed significantly larger learning effect for high reward-associated motion direction than low reward-associated motion direction. Importantly, subjects could neither discriminate the relative values of the subliminal monetary reward nor correctly report the reward-direction contingencies. Our findings suggest that reward signal plays an important modulatory role in perceptual learning even if the magnitude of the reward was not consciously perceived.     

Differential dependence of Pavlovian incentive motivation and instrumental incentive learning processes on dopamine signaling

Here we attempted to clarify the role of dopamine signaling in reward seeking. In Experiment 1, we assessed the effects of the dopamine D(1)/D(2) receptor antagonist flupenthixol (0.5 mg/kg i.p.) on Pavlovian incentive motivation and found that flupenthixol blocked the ability of a conditioned stimulus to enhance both goal approach and instrumental performance (Pavlovian-to-instrumental transfer). In Experiment 2 we assessed the effects of flupenthixol on reward palatability during post-training noncontingent re-exposure to the sucrose reward in either a control 3-h or novel 23-h food-deprived state. Flupenthixol, although effective in blocking the Pavlovian goal approach, was without effect on palatability or the increase in reward palatability induced by the upshift in motivational state. This noncontingent re-exposure provided an opportunity for instrumental incentive learning, the process by which rats encode the value of a reward for use in updating reward-seeking actions. Flupenthixol administered prior to the instrumental incentive learning opportunity did not affect the increase in subsequent off-drug reward-seeking actions induced by that experience. These data suggest that although dopamine signaling is necessary for Pavlovian incentive motivation, it is not necessary for changes in reward experience, or for the instrumental incentive learning process that translates this experience into the incentive value used to drive reward-seeking actions, and provide further evidence that Pavlovian and instrumental incentive learning processes are dissociable.     

Epistemic values and the value of learning

In addition to purely practical values, there are cognitive values which figure in scientific deliberations. One way of introducing cognitive values is to consider the cognitive value that accrues to the act of accepting a hypothesis. Although such values may have a role to play in the matter of theory acceptance, this does not exhaust their significance in scientific decision-making. This paper makes a plea for the consideration of epistemic value??cognitive value that attaches to a state of belief. I defend the notion of cognitive epistemic value against criticisms that have been raised against it. A stability requirement for epistemic value-functions is argued for on the basis of considerations of diachronic coherence. This requirement is sufficient for proving the Value of Learning Theorem, which says that the expected utility of cost-free learning cannot be negative. Under the assumption of stability, the expected cognitive epistemic value of undergoing a learning experience must also be non-negative.     

New Directions for a Classical Paradigm: Human Eyeblink Conditioning

Abstract—The knowledge base on neural substrates an mechanisms involved in classical eyeblink conditioning makes it an ideal paradigm for investigating fundamental issues in learning and memory. New applications for the model system presented here include its use in (a) assessment to evaluate neurocognitive development in infancy, (b) theory building in abnormal psychology to test relationships between obsessive-compulsive behavior and learning rate, (c) evaluation of hypotheses about brain memory systems, and (d) exploration of the role of brain structures such as the cerebellum in learning and timing. Human eyeblink conditioning is a prototype of the utility of a model system that has become well characterized at both the behavioral and the neurobiological levels.     

如何用行为经济学应对不确定性：拓展有效助推的范围

本文提出了决策中不确定性的五种类型及其行为学和心理学的应对机制：用简捷启发式替代加权求和应对信息不确定性, 用直觉应对认知不确定性, 用价值观预测选择偏好应对行为不确定性, 用决策参照点的权重替代概率应对结果不确定性, 用时间换时间以降低延迟折扣应对未来不确定性。新行为经济学应当通过“为什么”的功能性分析, 找到行为助推的心理杠杆。化解不确定性本身就是一种有效的行为助推; 化繁为简是行为助推的关键所在。     

Perceived ownership impacts reward evaluation within medial-frontal cortex

Ownership is a powerful construct. Indeed, in a series of recent studies, perceived ownership has been shown to increase attentional capacity, facilitate a memorial advantage, and elicit positive attitudes. Here, we sought to determine whether self-relevance would bias reward evaluation systems within the brain. To accomplish this, we had participants complete a simple gambling task during which they could “win” or “lose” prizes for themselves or for someone else, while electroencephalographic data were recorded. Our results indicated that the amplitude of the feedback error-related negativity, a component of the event-related brain potential sensitive to reward evaluation, was diminished when participants were not gambling for themselves. Furthermore, our data suggest that the ownership cues that indicated who would win or lose a given gamble either were processed as a potential for an increase in utility (i.e., gain: self-gambles) or were processed in a nonutilitarian manner (other-gambles). Importantly, our results suggest that the medial-frontal reward system is sensitive to perceived ownership, to the extent that it may not process changes in utility when they are not directly relevant to self.     

Social influences on cognitive strategies and cognitive development: The role of communication and instruction

Cognitive strategy training has been shown empirically to be extremely effective in enhancing learning and memory performance in retarded individuals. However, the theoretical status and applied utility of the strategy concept has been subjected to increasing criticism, particularly regarding the lack of definition and specificity in its use, when it is involved in instructional procedures applied to typical learning and information processing tasks. Extensions of research in language and communication provide a social basis for analyzing instruction, in terms of communication theory and pragmatics, and a possible approach to more general studies of the skills and strategies of information processing. The question arises whether rather disparate views of cognitive processing systems can be synthesized, or, by their mutual consideration, at least provide a basis for clearer articulation of research issues. A synthesis of prominent usages of the “strategic processing” construct derived from the social basis of communication theory is advanced.     

Strengthening Connections: Functional Connectivity and Brain Plasticity

The ascendancy of functional neuroimaging has facilitated the addition of network-based approaches to the neuropsychologist’s toolbox for evaluating the sequelae of brain insult. In particular, intrinsic functional connectivity (iFC) mapping of resting state fMRI (R-fMRI) data constitutes an ideal approach to measuring macro-scale networks in the human brain. Beyond the value of iFC mapping for charting how the functional topography of the brain is altered by insult and injury, iFC analyses can provide insights into experience-dependent plasticity at the macro level of large-scale functional networks. Such insights are foundational to the design of training and remediation interventions that will best facilitate recovery of function. In this review, we consider what is currently known about the origin and function of iFC in the brain, and how this knowledge is informative in neuropsychological settings. We then summarize studies that have examined experience-driven plasticity of iFC in healthy control participants, and frame these findings in terms of a schema that may aid in the interpretation of results and the generation of hypotheses for rehabilitative studies. Finally, we outline some caveats to the R-fMRI approach, as well as some current developments that are likely to bolster the utility of the iFC paradigm for neuropsychology.     

Predicting Short‐Term Remembering as Boundedly Optimal Strategy Choice

It is known that, on average, people adapt their choice of memory strategy to the subjective utility of interaction. What is not known is whether an individual's choices are boundedly optimal. Two experiments are reported that test the hypothesis that an individual's decisions about the distribution of remembering between internal and external resources are boundedly optimal where optimality is defined relative to experience, cognitive constraints, and reward. The theory makes predictions that are tested against data, not fitted to it. The experiments use a no‐choice/choice utility learning paradigm where the no‐choice phase is used to elicit a profile of each participant's performance across the strategy space and the choice phase is used to test predicted choices within this space. They show that the majority of individuals select strategies that are boundedly optimal. Further, individual differences in what people choose to do are successfully predicted by the analysis. Two issues are discussed: (a) the performance of the minority of participants who did not find boundedly optimal adaptations, and (b) the possibility that individuals anticipate what, with practice, will become a bounded optimal strategy, rather than what is boundedly optimal during training.     

Instability Of Ex Post Aggregation In The Bolker–Jeffrey Framework And Related Instability Phenomena

Suppose n Bayesian agents need to make a decision as a group. The groupas a whole is also supposed to be a Bayesian agent whose probabilities andutilities are derived or aggregated in reasonable ways from the probabilitiesand utilities of the group members. The aggregation could beex ante, i.e., interms of expected utilities, or it could be ex post, i.e., in terms of utilitiesonly, or in terms of utilities and probabilities separately. This study exploresthe ex post approach. Using the Bolker/Jeffrey framework, we show thatex post aggregation is subject to an instability phenomenon. That is, it mayhappen that the group preference between actions ``flips back and forth' dependingon the level of detail in which the decision problem is described. Structurally verysimilar phenomena also occur elsewhere in social choice theory, in statistics (Simpson'sParadox), and in voting theory (Ostrogorski's Paradox).     

Visual landmarks facilitate rodent spatial navigation in virtual reality environments

Because many different sensory modalities contribute to spatial learning in rodents, it has been difficult to determine whether spatial navigation can be guided solely by visual cues. Rodents moving within physical environments with visual cues engage a variety of nonvisual sensory systems that cannot be easily inhibited without lesioning brain areas. Virtual reality offers a unique approach to ask whether visual landmark cues alone are sufficient to improve performance in a spatial task. We found that mice could learn to navigate between two water reward locations along a virtual bidirectional linear track using a spherical treadmill. Mice exposed to a virtual environment with vivid visual cues rendered on a single monitor increased their performance over a 3-d training regimen. Training significantly increased the percentage of time avatars controlled by the mice spent near reward locations in probe trials without water rewards. Neither improvement during training or spatial learning for reward locations occurred with mice operating a virtual environment without vivid landmarks or with mice deprived of all visual feedback. Mice operating the vivid environment developed stereotyped avatar turning behaviors when alternating between reward zones that were positively correlated with their performance on the probe trial. These results suggest that mice are able to learn to navigate to specific locations using only visual cues presented within a virtual environment rendered on a single computer monitor.     

When is an error not a prediction error? An electrophysiological investigation

A recent theory holds that the anterior cingulate cortex (ACC) uses reinforcement learning signals conveyed by the midbrain dopamine system to facilitate flexible action selection. According to this position, the impact of reward prediction error signals on ACC modulates the amplitude of a component of the event-related brain potential called the error-related negativity (ERN). The theory predicts that ERN amplitude is monotonically related to the expectedness of the event: It is larger for unexpected outcomes than for expected outcomes. However, a recent failure to confirm this prediction has called the theory into question. In the present article, we investigated this discrepancy in three trial-and-error learning experiments. All three experiments provided support for the theory, but the effect sizes were largest when an optimal response strategy could actually be learned. This observation suggests that ACC utilizes dopamine reward prediction error signals for adaptive decision making when the optimal behavior is, in fact, learnable.     

网络游戏成瘾与海洛因成瘾存在相同的神经机制吗?——基于MRI的证据

网游成瘾与海洛因成瘾具有许多相似的临床表现,但其神经机制是否相同还不得而知。综合近5年来的MRI研究发现,两类成瘾存在部分相同区域的脑结构和功能损害,且在成瘾线索诱发下二者的4个成瘾相关环路(认知控制环路、奖赏环路、动机环路和记忆-学习环路)均出现了广泛而增强的脑区激活反应。但海洛因成瘾的脑损害区域偏向更高级的认知控制环路和奖赏环路,损害范围也更广(4个环路的功能连通性均降低),而网游成瘾的脑损害主要发生在相对低级的记忆-学习环路和动机环路,损害范围也较窄(功能连通性降低只发生在认知控制和记忆-学习环路之间)。这些结果说明,两类成瘾行为的神经机制既有相同点,又有差异性。