A mechanistic perspective on canonical neural computation

Although it has been argued that mechanistic explanation is compatible with abstraction (i.e., that there are abstract mechanistic models), there are still doubts about whether mechanism can account for the explanatory power of significant abstract models in computational neuroscience. Chirimuuta has recently claimed that models describing canonical neural computations (CNCs) must be evaluated using a non-mechanistic framework. I defend two claims regarding these models. First, I argue that their prevailing neurocognitive interpretation is mechanistic. Additionally, a criterion recently proposed by Levy and Bechtel to legitimize mechanistic abstract models, and also a criterion proposed by Chirimuuta herself aimed to distinguish between causal and non-causal explanation, can be employed to show why these models are explanatory only under this interpretation (as opposed to a purely mathematical or non-causal interpretation). Second, I argue that mechanism is able to account for the special epistemic achievement implied by CNC models. Canonical neural components contribute to an integrated understanding of different cognitive functions. They make it possible for us to explain these functions by describing different mechanisms constituted by common basic components arranged in different ways.     

Reinforcement learning in the brain

A wealth of research focuses on the decision-making processes that animals and humans employ when selecting actions in the face of reward and punishment. Initially such work stemmed from psychological investigations of conditioned behavior, and explanations of these in terms of computational models. Increasingly, analysis at the computational level has drawn on ideas from reinforcement learning, which provide a normative framework within which decision-making can be analyzed. More recently, the fruits of these extensive lines of research have made contact with investigations into the neural basis of decision making. Converging evidence now links reinforcement learning to specific neural substrates, assigning them precise computational roles. Specifically, electrophysiological recordings in behaving animals and functional imaging of human decision-making have revealed in the brain the existence of a key reinforcement learning signal, the temporal difference reward prediction error. Here, we first introduce the formal reinforcement learning framework. We then review the multiple lines of evidence linking reinforcement learning to the function of dopaminergic neurons in the mammalian midbrain and to more recent data from human imaging experiments. We further extend the discussion to aspects of learning not associated with phasic dopamine signals, such as learning of goal-directed responding that may not be dopamine-dependent, and learning about the vigor (or rate) with which actions should be performed that has been linked to tonic aspects of dopaminergic signaling. We end with a brief discussion of some of the limitations of the reinforcement learning framework, highlighting questions for future research.     

Integrating computation into the mechanistic hierarchy in the cognitive and neural sciences

It is generally accepted that, in the cognitive and neural sciences, there are both computational and mechanistic explanations. We ask how computational explanations can integrate into the mechanistic hierarchy. The problem stems from the fact that implementation and mechanistic relations have different forms. The implementation relation, from the states of an abstract computational system (e.g., an automaton) to the physical, implementing states is a homomorphism mapping relation. The mechanistic relation, however, is that of part/whole; the explaining features in a mechanistic explanation are the components of the explanandum phenomenon and their causal organization. Moreover, each component in one level of mechanism is constituted and explained by components of an underlying level of mechanism. Hence, it seems, computational variables and functions cannot be mechanistically explained by the medium-dependent states and properties that implement them. How then, do the computational and the implementational integrate to create the mechanistic hierarchy? After explicating the general problem (Sect. 2), we further demonstrate it through a concrete example, of reinforcement learning, in the cognitive and neural sciences (Sects. 3 and 4). We then examine two possible solutions (Sect. 5). On one solution, the mechanistic hierarchy embeds at the same levels computational and implementational properties. This picture fits with the view that computational explanations are mechanistic sketches. On the other solution, there are two separate hierarchies, one computational and another implementational, which are related by the implementation relation. This picture fits with the view that computational explanations are functional and autonomous explanations. It is less clear how these solutions fit with the view that computational explanations are full-fledged mechanistic explanations. Finally, we argue that both pictures are consistent with the reinforcement learning example, but that scientific practice does not align with the view that computational models are merely mechanistic sketches (Sect. 6).

     

Motivational deficits in schizophrenia relate to abnormalities in cortical learning rate signals

Individuals from across the psychosis spectrum display impairments in reinforcement learning. In some individuals, these deficits may result from aberrations in reward prediction error (RPE) signaling, conveyed by dopaminergic projections to the ventral striatum (VS). However, there is mounting evidence that VS RPE signals are relatively intact in medicated people with schizophrenia (PSZ). We hypothesized that, in PSZ, reinforcement learning deficits often are not related to RPE signaling per se but rather their impact on learning and behavior (i.e., learning rate modulation), due to dysfunction in anterior cingulate and dorsomedial prefrontal cortex (dmPFC). Twenty-six PSZ and 23 healthy volunteers completed a probabilistic reinforcement learning paradigm with occasional, sudden, shifts in contingencies. Using computational modeling, we found evidence of an impairment in trial-wise learning rate modulation (α) in PSZ before and after a reinforcement contingency shift, expressed most in PSZ with more severe motivational deficits. In a subsample of 22 PSZ and 22 healthy volunteers, we found little evidence for between-group differences in VS RPE and dmPFC learning rate signals, as measured with fMRI. However, a follow-up psychophysiological interaction analysis revealed decreased dmPFC-VS connectivity concurrent with learning rate modulation, most prominently in individuals with the most severe motivational deficits. These findings point to an impairment in learning rate modulation in PSZ, leading to a reduced ability to adjust task behavior in response to unexpected outcomes. At the level of the brain, learning rate modulation deficits may be associated with decreased involvement of the dmPFC within a greater RL network.     

Reinforcement,Rationality, and Intentions: How Robust Is Automatic Reinforcement Learning in Economic Decision Making?

Reinforcement learning is often observed in economic decision making and may lead to detrimental decisions. Because of its automaticity, it is difficult to avoid. In three experimental studies, we investigated whether this process could be controlled by goal intentions and implementation intentions. Participants' decisions were investigated in a probability‐updating task in which the normative rule to maximize expected payoff (Bayes' rule) conflicted with the reinforcement heuristic as a simple decision rule. Some participants were asked to set goal intentions designated to foster the optimization of rational decision making, while other participants were asked to furnish these goal intentions with implementation intentions. Results showed that controlling automatic processes of reinforcement learning is possible by means of goal intentions or implementation intentions that focus decision makers on the analysis of decision feedback. Importantly, such beneficial effects were not achieved by simply instructing participants to analyze the feedback, without defining a goal as the desired end state from a first‐person perspective. Regarding intentions supposed to shut down reinforcement processes by controlling negative affect, effects were more complex and depended on the specified goal‐directed behavior. The goal intention to suppress the disappointment elicited by negative feedback was not effective in controlling reinforcement processes. Furnishing this goal with an implementation intention even backfired and strengthened unwanted reinforcement processes. In contrast, asking participants to keep cool in response to negative decision outcomes through the use of goal intentions or implementation intentions increased decisions in line with Bayes' rule. Copyright © 2017 John Wiley & Sons, Ltd.     

发展的认知神经科学--神经科学与认知发展研究的融合点

本文介绍了发展的神经科学自 2 0世纪 80年代以来在早期突触形成、关键期及丰富环境对脑发育的影响等领域的一些突出成就 ,阐述了从神经科学与认知发展共同研究的主题来考察两个学科的整合的意义 ,并对发展的认知神经科学研究的未来趋势作了展望。     

计算模型在道德认知研究中的应用

道德认知关注道德心理背后的信息加工。近年来, 研究者开始将计算模型应用于道德认知研究, 以探索道德认知如何在大脑中实现。但目前研究者对道德认知进行计算建模的研究处于起步阶段。计算模型(漂移扩散模型、效用模型、强化学习模型和分层高斯过筛器模型)在道德认知行为和生理研究上的运用量化了道德决策、道德判断和道德推理背后的认知过程和神经机制。此外, 这一新进展对理解反社会行为和精神障碍等有所助益。最后, 计算建模有待完善, 未来研究需要关注其潜在的问题。     

Towards learning agents with personality traits: Modeling Openness to Experience

Recent advances in neurosciences and cognitive sciences show us that the human neocortex is not a slave to the experiences from our perception and that the memories stored in hippocampus are goal weighted during the replay of the experiences for the purpose of re-learning from them. Temporal difference reinforcement learning systems that use neural networks as function approximators rely on an experience replay memory structure similar to the hippocampus. We bring forward this similarity and present a novel way of using a goal weighted prioritization of the memory that is biologically inspired. Furthermore, we introduce a novel prioritization criteria called Variety of Experience Index, or VEI, for weighting the selection of the experiences that are stored in the replay memory. Weighting the experiences based on two different extremes of VEI can behaviourally modify the agent’s learning process, generating different types of learning agents that exhibit different personality traits along the dimension of Openness to Experience.     

Hierarchically organized behavior and its neural foundations: a reinforcement learning perspective

Research on human and animal behavior has long emphasized its hierarchical structure—the divisibility of ongoing behavior into discrete tasks, which are comprised of subtask sequences, which in turn are built of simple actions. The hierarchical structure of behavior has also been of enduring interest within neuroscience, where it has been widely considered to reflect prefrontal cortical functions. In this paper, we reexamine behavioral hierarchy and its neural substrates from the point of view of recent developments in computational reinforcement learning. Specifically, we consider a set of approaches known collectively as hierarchical reinforcement learning, which extend the reinforcement learning paradigm by allowing the learning agent to aggregate actions into reusable subroutines or skills. A close look at the components of hierarchical reinforcement learning suggests how they might map onto neural structures, in particular regions within the dorsolateral and orbital prefrontal cortex. It also suggests specific ways in which hierarchical reinforcement learning might provide a complement to existing psychological models of hierarchically structured behavior. A particularly important question that hierarchical reinforcement learning brings to the fore is that of how learning identifies new action routines that are likely to provide useful building blocks in solving a wide range of future problems. Here and at many other points, hierarchical reinforcement learning offers an appealing framework for investigating the computational and neural underpinnings of hierarchically structured behavior.     

A cognitive-perceptual analysis of projective tests used with children.

There are both general and specific problems with projective tests--the production, comprehension, and interpretation of two-dimensional visual representations. At the general level, there is a need to integrate findings from the neuro- and cognitive sciences, cognitive, perceptual, and affective development, and the understanding and interpretation of pictorial material based on the accumulated research base in the arts. At the specific level, much of the research base on projective tests is poor or outdated; evidence for clinical utility is mixed or negative; and the tests possess poor reliability and validity while the putative underlying psychological process of projection" has not been subject to rigorous empirical examination--the term remains vague and elusive. While earlier critiques and reviews have focused on problems in validity and reliability, their has been a lack of attention to the development of children's pictorial abilities as pertain to projective techniques. Although many of the principles delineated here also apply to adolescents and adults, an important challenge for clinicians is to develop and employ better methods in the "projective" assessment of children.     

CONTRIBUTIONS OF COGNITIVE THEORY TO NEW BEHAVIORAL TREATMENTS

Abstract— Although clinical, social, and cognitive psychologists alt use the concept of 'cognition,' they often use a in different ways to refer to different phenomena. We offer a heuristic framework for distinguishing among three general uses of the word cognition, and apply this framework to an evaluation of the experiential avoidance concept presented by Hayes and Gifford (this issue) While acknowledging the promise of such work, we raise concerns about its possible limitations. We recommend that clinical applications of the cognition concept be grounded in the theories and methods of contemporary cognitive and neural sciences In support of our recommendation, we present three examples from experiments from out own research.     

Emotion enhanced retention of cognitive skill learning

Ample evidence suggests that emotional arousal enhances declarative/episodic memory. By contrast, there is little evidence that emotional enhancement of memory (EEM) extends to procedural skill based memory. We examined remote EEM (1.5-month delay) for cognitive skill learning using the weather prediction (WP) probabilistic classification task. Participants viewed interleaved emotionally arousing or neutral pictures during WP acquisition. Arousal retarded initial WP acquisition. While participants in the neutral condition showed substantial forgetting of WP learning across the 1.5-month delay interval, the arousal condition showed no evidence of forgetting across the same time period. Thus, arousal during encoding determined the mnemonic fate of cognitive skill learning. Emotional enhancement of WP retention was independent of verbally stated knowledge of WP learning and EEM for the picture contexts in which learning took place. These results reveal a novel demonstration of EEM for cognitive skill learning, and suggest that emotional arousal may in parallel enhance the neural systems that support procedural learning and its declarative context.     

The cognitive neuroscience of creativity

This article outlines a framework of creativity based on functional neuroanatomy. Recent advances in the field of cognitive neuroscience have identified distinct brain circuits that are involved in specific higher brain functions. To date, these findings have not been applied to research on creativity. It is proposed that there are four basic types of creative insights, each mediated by a distinctive neural circuit. By definition, creative insights occur in consciousness. Given the view that the working memory buffer of the prefrontal cortex holds the content of consciousness, each of the four distinctive neural loops terminates there. When creativity is the result of deliberate control, as opposed to spontaneous generation, the prefrontal cortex also instigates the creative process. Both processing modes, deliberate and spontaneous, can guide neural computation in structures that contribute emotional content and in structures that provide cognitive analysis, yielding the four basic types of creativity. Supportive evidence from psychological, cognitive, and neuroscientific studies is presented and integrated in this article. The new theoretical framework systematizes the interaction between knowledge and creative thinking, and how the nature of this relationship changes as a function of domain and age. Implications for the arts and sciences are briefly discussed.     

Comparison of nonretarded and mentally retarded children on tasks involving direct and rule-governed imitation.

The influence of intellectual level and social reinforcement on imitation learning was examined. Tasks involving direct and rule-governed imitation of a mode were presented to 20 mentally retarded and 20 nonretarded children. The children within each group were randomly assigned to either an affective ("good-fine") or an informative ("correct-right") social reinforcement condition. Reinforcement, administered on a fixed ration (FR4) schedule, was contingent on the child's imitative behavior. A multivariate analysis of variance showed that both the Population X Reinforcement Type interaction and the Reinforcement main effect were significant. Univariate follow-up tests showed that only rule-governed imitation contributed significantly to the multivariate effects. Analysis of simple effects indicated that retarded children performed optimally under affective reinforcement, while the nonretarded children performed highest under informative reinforcement.     

Manipulation is key: on why non-mechanistic explanations in the cognitive sciences also describe relations of manipulation and control

A popular view presents explanations in the cognitive sciences as causal or mechanistic and argues that an important feature of such explanations is that they allow us to manipulate and control the explanandum phenomena. Nonetheless, whether there can be explanations in the cognitive sciences that are neither causal nor mechanistic is still under debate. Another prominent view suggests that both causal and non-causal relations of counterfactual dependence can be explanatory, but this view is open to the criticism that it is not clear how to distinguish explanatory from non-explanatory relations. In this paper, I draw from both views and suggest that, in the cognitive sciences, relations of counterfactual dependence that allow manipulation and control can be explanatory even when they are neither causal nor mechanistic. Furthermore, the ability to allow manipulation can determine whether non-causal counterfactual dependence relations are explanatory. I present a preliminary framework for manipulation relations that includes some non-causal relations and use two examples from the cognitive sciences to show how this framework distinguishes between explanatory and non-explanatory, non-causal relations. The proposed framework suggests that, in the cognitive sciences, causal and non-causal relations have the same criterion for explanatory value, namely, whether or not they allow manipulation and control.     

Analyzing behavior implied by EWA learning: An emphasis on distinguishing reinforcement from belief learning

An important issue in the field of learning is to what extent one can distinguish between behavior resulting from either belief or reinforcement learning. Previous research suggests that it is difficult or even impossible to distinguish belief from reinforcement learning: belief and reinforcement models often fit the empirical data equally well. However, previous research has been confined to specific games in specific settings. In the present study we derive predictions for behavior in games using the EWA learning model (e.g., Camerer & Ho, 1999), a model that includes belief learning and a specific type of reinforcement learning as special cases. We conclude that belief and reinforcement learning can be distinguished, even in 2×2 games. Maximum differentiation in behavior resulting from either belief or reinforcement learning is obtained in games with pure Nash equilibria with negative payoffs and at least one other strategy combination with only positive payoffs. Our results help researchers to identify games in which belief and reinforcement learning can be discerned easily.     

Socioeconomic status and hippocampal volume in children and young adults

An individual's socioeconomic status (SES) is often viewed as a proxy for a host of environmental influences. SES disparities have been linked to variance in brain structures particularly the hippocampus, a neural substrate of learning and memory. However, it is unclear whether the association between SES and hippocampal volume is similar in children and adults. We investigated the relationship between hippocampal volume and SES in a group of children (n = 31, age 8–12 years) and a group of young adults (n = 32, age 18–25 years). SES was assessed with four indicators that loaded on a single factor, therefore a composite SES scores was used in the main analyses. Hippocampal volume was measured using manual demarcation on high resolution structural images. SES was associated with hippocampal volume in the children, but not in adults, suggesting that in childhood, but not adulthood, SES‐related environmental factors influence hippocampal volume. In addition, hippocampal volume, but not SES, was associated with scores on a memory task, suggesting that net effects of postnatal environmental factors, captured by SES, are more distal determinants of memory performance than hippocampal volume. Longitudinal investigation of the association between SES, hippocampal volume and cognitive functioning may further our understanding of the putative neural mechanisms underlying SES‐related environmental effects on cognitive development.     

A neuro-cognitive defense of the unified self

The topic of the self remains one of considerable controversy, and many arguments have been put forth suggesting the intuitive concept of self must be in some way mistaken – in part based on results in the cognitive and neural sciences. In this article I offer the alternative positive proposal that “the self” may indeed refer to a physical/computational system within the brain. To do this, I draw on empirical work regarding the neural basis of consciousness and decision-making, and on philosophical work regarding ecological control, unified group perspectives, and functional/mechanistic explanation. The work I review jointly supports the conclusion that a “core-circuit” of interacting cortical regions – the global workspace network – can be understood as a unified system for consciously perceiving and deciding, and thus fulfills many of the roles intuitively assigned to the self. I conclude that this self-concept need not be mistaken given current empirical knowledge.