Brain mechanisms for predictive control by switching internal models: implications for higher-order cognitive functions

Humans can guide their actions toward the realization of their intentions. Flexible, rapid and precise realization of intentions and goals relies on the brain learning to control its actions on external objects and to predict the consequences of this control. Neural mechanisms that mimic the input–output properties of our own body and other objects can be used to support prediction and control, and such mechanisms are called internal models. We first summarize functional neuroimaging, behavioral and computational studies of the brain mechanisms related to acquisition, modular organization, and the predictive switching of internal models mainly for tool use. These mechanisms support predictive control and flexible switching of intentional actions. We then review recent studies demonstrating that internal models are crucial for the execution of not only immediate actions but also higher-order cognitive functions, including optimization of behaviors toward long-term goals, social interactions based on prediction of others’ actions and mental states, and language processing. These studies suggest that a concept of internal models can consistently explain the neural mechanisms and computational principles needed for fundamental sensorimotor functions as well as higher-order cognitive functions. Electronic supplementary material  The online version of this article (doi:) contains supplementary material, which is available to authorized users.     

The Social Nature of Perception and Action

ABSTRACT— Humans engage in a wide range of social activities. Previous research has focused on the role of higher cognitive functions, such as mentalizing (the ability to infer others' mental states) and language processing, in social exchange. This article reviews recent studies on action perception and joint action suggesting that basic perception–action links are crucial for many social interactions. Mapping perceived actions to one's own action repertoire enables direct understanding of others' actions and supports action identification. Joint action relies on shared action representations and involves modeling of others' performance in relation to one's own. Taking the social nature of perception and action seriously not only contributes to the understanding of dedicated social processes but has the potential to create a new perspective on the individual mind and brain.     

An analysis of fidgeting and associated individual differences

Fidgeting was defined as engaging in manipulations of one's own body parts or other objects, such actions being peripheral or nonessential to central ongoing events or tasks Two studies employed preliminary fidgeting questionnaires of 70 and 106 items, respectively, tapping a wide range of habitual behaviors Item selection and factor analyses led to the development of a final, 40-item, individual difference measure of fidgeting tendency which is balanced for response bias and has an alpha reliability coefficient of 89 Convergent and construct validity were supported by positive correlations between the fidgeting tendency measure and measures of tendencies to engage in extraneous activities (consuming alcoholic drinks, cigarette smoking, eating, daydreaming, restlessness, insomnia) while preoccupied with certain situations and tasks These findings implied that fidgeting is an activity overflow and that it is more probable when the organism's physical activity is constrained by the central or focal act Fidgeting tendency also correlated positively with binge eating and with physical activity In two additional studies, questionnaire fidgeting tendency scores correlated positively with direct observations of fidgeting by subjects and with peer predictions Personality correlates of fidgeting tendency were well defined More fidgety persons were more unpleasant and more arousable (i e, more anxious or hostile) Fidgeting tendency did not correlate significantly with subject sex or with arousal-seeking tendency     

On the role of reduced auditory feedback and kinesic self-stimulation during Stroop Color-Word performance1

Feedback from one's own voice provides important vocal–motor cues for effective cognitive processing. Reduction of such feedback is known to disturb such functioning. Work in our laboratory has shown that kinesic self-stimulation also plays an important role in cognition, and appears to regulate the focusing of attention under conditions of distraction. The present study investigated the effects of both auditory feedback and kinesic self-stimulation in the regulation of cognitive interference during performance of the Stroop Color-Word Task. Twelve subjects were tested on the Stroop task under conditions of normal and occluded hearing. Kinesic self-stimulation and response errors during color-word performance were recorded on video tape. The findings indicated that not only did self-stimulation increase when voice feedback was reduced, but that this increase was associated with a reduction in specific types of color-word performance errors. Individual differences revealed that high kinesic responders made significantly fewer errors in task performance than did low kinesic responders. Results were interpreted as revealing a kinesic feedback mechanism which has adaptive significance in regard to self-editing when auditory feedback is reduced.     

Predictive coding: an account of the mirror neuron system

Is it possible to understand the intentions of other people by simply observing their actions? Many believe that this ability is made possible by the brain’s mirror neuron system through its direct link between action and observation. However, precisely how intentions can be inferred through action observation has provoked much debate. Here we suggest that the function of the mirror system can be understood within a predictive coding framework that appeals to the statistical approach known as empirical Bayes. Within this scheme the most likely cause of an observed action can be inferred by minimizing the prediction error at all levels of the cortical hierarchy that are engaged during action observation. This account identifies a precise role for the mirror system in our ability to infer intentions from actions and provides the outline of the underlying computational mechanisms.     

FREE WILL AND LUCK

I present an account of how agents can know what they are doing when they intentionally execute object-oriented actions. When an agent executes an object-oriented intentional action, she uses perception in such a way that it can fulfil a justificatory role for her knowledge of her own action and it can fulfil this justificatory role without being inferentially linked to the cognitive states that it justifies. I argue for this proposal by meeting two challenges: in an agent's knowledge of her action perception can only play an enabling role (and no justificatory role) for the agent's knowledge and if perception has a justificatory role, then the agent's knowledge must be inferential.     

At one with our actions,but at two with our bodies

Jennifer Hornsby's account of human action frees us from the temptation to think of the person who acts as ‘doing’ the events that are her actions, and thereby removes much of the allure of ‘agent causation’. But her account is spoiled by the claim that physical actions are ‘tryings’ that cause bodily movements. It would be better to think of physical actions and bodily movements as identical; but Hornsby refuses to do this, seemingly because she thinks that to do so would be to endorse the so–called ‘standard causal story’. But Hornsby misses a possibility here, for we can insist on this identity claim without endorsing the standard story if we embrace an account which parallels the disjunctive account in the philosophy of perception. This will leave us with a picture of physical action that saves the insights of Hornsby's account without succumbing to its distortions.     

Predictive coding accelerates word recognition and learning in the early stages of language development

The ability to predict future events in the environment and learn from them is a fundamental component of adaptive behavior across species. Here we propose that inferring predictions facilitates speech processing and word learning in the early stages of language development. Twelve‐ and 24‐month olds’ electrophysiological brain responses to heard syllables are faster and more robust when the preceding word context predicts the ending of a familiar word. For unfamiliar, novel word forms, however, word‐expectancy violation generates a prediction error response, the strength of which significantly correlates with children's vocabulary scores at 12 months. These results suggest that predictive coding may accelerate word recognition and support early learning of novel words, including not only the learning of heard word forms but also their mapping to meanings. Prediction error may mediate learning via attention, since infants’ attention allocation to the entire learning situation in natural environments could account for the link between prediction error and the understanding of word meanings. On the whole, the present results on predictive coding support the view that principles of brain function reported across domains in humans and non‐human animals apply to language and its development in the infant brain. A video abstract of this article can be viewed at: http://hy.fi/unitube/video/e1cbb495-41d8-462e-8660-0864a1abd02c . [Correction added on 27 January 2017, after first online publication: The video abstract link was added.]     

Reward anticipation enhances brain activation during response inhibition

The chance to achieve a reward starts up the required neurobehavioral mechanisms to adapt our thoughts and actions in order to accomplish our objective. However, reward does not equally reinforce everybody but depends on interindividual motivational dispositions. Thus, immediate reward contingencies can modulate the cognitive process required for goal achievement, while individual differences in personality can affect this modulation. We aimed to test the interaction between inhibition-related brain response and motivational processing in a stop signal task by reward anticipation and whether individual differences in sensitivity to reward (SR) modulate such interaction. We analyzed the cognitive–motivational interaction between the brain pattern activation of the regions involved in correct and incorrect response inhibition and the association between such brain activations and SR scores. We also analyzed the behavioral effects of reward on both reaction times for the “go” trials before and after correct and incorrect inhibition in order to test error prediction performance and postinhibition adjustment. Our results show enhanced activation during response inhibition under reward contingencies in frontal, parietal, and subcortical areas. Moreover, activation of the right insula and the left putamen positively correlates with the SR scores. Finally, the possibility of reward outcome affects not only response inhibition performance (e.g., reducing stop signal reaction time), but also error prediction performance and postinhibition adjustment. Therefore, reward contingencies improve behavioral performance and enhance brain activation during response inhibition, and SR is related to brain activation. Our results suggest the conditions and factors that subserve cognitive control strategies in cognitive motivational interactions during response inhibition.     

Transcending the evidentiary boundary: Prediction error minimization,embodied interaction,and explanatory pluralism

In a recent paper, Jakob Hohwy argues that the emerging predictive processing (PP) perspective on cognition requires us to explain cognitive functioning in purely internalistic and neurocentric terms. The purpose of the present paper is to challenge the view that PP entails a wholesale rejection of positions that are interested in the embodied, embedded, extended, or enactive dimensions of cognitive processes. I will argue that Hohwy’s argument from analogy, which forces an evidentiary boundary into the picture, lacks the argumentative resources to make a convincing case for the conceptual necessity to interpret PP in solely internalistic terms. For this reason, I will reconsider the postulation and explanatory role of the evidentiary boundary. I will arrive at an account of prediction error minimization and its foundation on the free energy principle that is fully consistent with approaches to cognition that emphasize the embodied and interactive properties of cognitive processes. This gives rise to the suggestion that explanatory pluralism about the application of PP is to be preferred over Hohwy’s explanatory monism that follows from his internalistic and neurocentric view of predictive cognitive systems.     

Propositional Attitudes and Embodied Skills in the Philosophy of Action

Propositionalism in the philosophy of action is the popular view that intentional actions are bodily movements caused and rationalized by certain ‘internal’ propositional attitude states that constitute the agent's perspective. I attack propositionalism's background claim that the genuinely mental/cognitive dimension of human action resides solely in some range of ‘internal’ agency‐conferring representational states that causally trigger, and thus are always conceptually disentangle‐able from, bodily activity itself. My opposing claim, following Ryle, Wittgenstein, and others, is that mentality and intentionality can be constitutively implicated in bodily actions themselves, as exercises of a distinctive form of embodied practical understanding. I attempt to show this by attending to the fine‐grained contours of various skillful actions.     

The relationship between outcome prediction and cognitive fatigue: A convergence of paradigms

Cognitive fatigue is common after strenuous cognitive effort. A large body of literature has implicated a network of brain areas in fatigue, including the basal ganglia and cortical areas including ventro-medal prefrontal cortex and anterior cingulate cortex (ACC). Furthermore, the ACC has been shown to be involved in processes such as error and conflict monitoring, outcome prediction, and effort processing. Thus, the ACC appears to be one common denominator between clinical work on fatigue and research on outcome prediction and effort. In the present study, we examined whether the same region of the ACC is activated during the processing of errors and fatigue. Cognitive fatigue was induced by having subjects perform a difficult working memory task, during which they rated on-task fatigue. Activation associated with error processing was determined by using error trials on the working memory task. After localizing the region engaged in error processing, we evaluated whether there was a relationship between BOLD activation of that region and on-task fatigue scores. The results showed that as subjects became more fatigued, they responded with longer latencies and increased accuracy for the more difficult task. Moreover, the ACC areas that were activated by error processing were also associated with fatigue. These results suggest that cognitive fatigue may be related to changes in effort and reward. We speculate that as the brain detects these changes, cognitive fatigue is generated as a way for the brain to signal itself that the effort required for the task no longer merits the rewards received for performing it.     

镜像神经元的意义

镜像神经元是一种感觉–运动神经元。它的典型特征是在动作观察和动作执行两个阶段皆被激活。多年来, 由于研究伦理的限制, 研究恒河猴时使用的单细胞电极植入方式无法应用于人类, 因而不能确定人类大脑皮层是否也存在着具有同样功能的神经细胞。但是通过脑成像技术, 神经科学家确定人类大脑皮层存在着具有相同或类似功能的脑区, 称为“镜像神经系统”。文章对镜像神经元及其人类镜像神经系统的意义进行了深入分析, 指出：(1)由于镜像机制把动作知觉和动作执行进行匹配, 观察者仅仅通过他人行为的知觉, 就激活了执行这一动作的神经环路, 产生了一种他人动作的具身模拟, 因而可以直接把握他人的行为意图; (2)镜像神经元所表现出来的那种动作知觉与动作执行的双重激活功能支持了身心一体说, 从方法论上证明了身心二元论的缺陷, 为身心的整体观提供了神经生物学的证据; (3)镜像神经机制把他人的动作与自己的运动系统相匹配, 以自身动作的神经环路对他人的动作做出回应, 促进了人际理解和沟通, 成为社会沟通的“神经桥梁”。     

Both motor prediction and conceptual congruency between preview and action-effect contribute to explicit judgment of agency

The sense of agency is the sense that one is causing an action. The inferential account of the sense of agency proposes that we experience the sense of agency when we infer that one’s own thoughts are the cause of an action. According to this account, the inference occurs when a thought appears in consciousness prior to an action, is consistent with the action, and is not accompanied by conspicuous other causes of the action. Alternatively, a predictive account of the sense of agency proposes that sensory prediction based on efferent (motor) information plays a critical role in generating the sense of agency. The present study investigated whether the sense of agency depended primarily on the conceptual congruence between preview information (i.e., to elicit a thought) and actual sensory feedback as suggested by the inferential account, or whether it depended primarily on the sensory-motor congruence between prediction and actual sensory feedback as suggested by the predictive account. The results indicated that both of these factors did contribute to the sense of agency, although sensory-motor congruence appears to have a more robust impact.     

老年人汉语阅读中预测误差成本的产生机制

阅读的认知加工机制随年龄变老只发生了量变还是也产生了质变, 是毕生发展研究领域的重要问题。要进行有效的语言处理, 读者必须结合已有的经验知识和当前语境对即将出现的信息进行预测加工。因此, 要解决上述问题就要回答老年人阅读中预测加工机制发生了怎样变化, 即预测误差成本是如何产生的。研究拟采用眼动脑电同步记录技术, 在自然阅读过程中实时获取语言处理的行为和神经指标, 聚焦老年人预测误差成本的产生机制, 主要从副中央凹视觉、工作记忆负荷及语言能力三方面解释预测误差成本产生的原因。研究成果对构建汉语阅读的毕生发展模型有重要意义。     

Effects of social context and predictive relevance on action outcome monitoring

Outcome monitoring is crucial for subsequent adjustments in behavior and is associated with a specific electrophysiological response, the feedback-related negativity (FRN). Besides feedback generated by one's own action, the performance of others may also be relevant for oneself, and the observation of outcomes for others' actions elicits an observer FRN (oFRN). To test how these components are influenced by social setting and predictive value of feedback information, we compared event-related potentials, as well as their topographies and neural generators, for performance feedback generated by oneself and others in a cooperative versus competitive context. Our results show that (1) the predictive relevance of outcomes is crucial to elicit an FRN in both players and observers, (2) cooperation increases FRN and P300 amplitudes, especially in individuals with high traits of perspective taking, and (3) contrary to previous findings on gambling outcomes, oFRN components are generated for both cooperating and competing observers, but with smaller amplitudes in the latter. Neural source estimation revealed medial prefrontal activity for both FRN and oFRN, but with additional generators for the oFRN in the dorsolateral and ventral prefrontal cortex, as well as the temporoparietal junction. We conclude that the latter set of brain regions could mediate social influences on action monitoring by representing agency and social relevance of outcomes and are, therefore, recruited in addition to shared prediction error signals generated in medial frontal areas during action outcome observation.     

Learning from delayed feedback: neural responses in temporal credit assignment

When feedback follows a sequence of decisions, relationships between actions and outcomes can be difficult to learn. We used event-related potentials (ERPs) to understand how people overcome this temporal credit assignment problem. Participants performed a sequential decision task that required two decisions on each trial. The first decision led to an intermediate state that was predictive of the trial outcome, and the second decision was followed by positive or negative trial feedback. The feedback-related negativity (fERN), a component thought to reflect reward prediction error, followed negative feedback and negative intermediate states. This suggests that participants evaluated intermediate states in terms of expected future reward, and that these evaluations supported learning of earlier actions within sequences. We examine the predictions of several temporal-difference models to determine whether the behavioral and ERP results reflected a reinforcement-learning process.     

Predictions penetrate perception: Converging insights from brain,behaviour and disorder

It is argued that during ongoing visual perception, the brain is generating top-down predictions to facilitate, guide and constrain the processing of incoming sensory input. Here we demonstrate that these predictions are drawn from a diverse range of cognitive processes, in order to generate the richest and most informative prediction signals. This is consistent with a central role for cognitive penetrability in visual perception. We review behavioural and mechanistic evidence that indicate a wide spectrum of domains—including object recognition, contextual associations, cognitive biases and affective state—that can directly influence visual perception. We combine these insights from the healthy brain with novel observations from neuropsychiatric disorders involving visual hallucinations, which highlight the consequences of imbalance between top-down signals and incoming sensory information. Together, these lines of evidence converge to indicate that predictive penetration, be it cognitive, social or emotional, should be considered a fundamental framework that supports visual perception.     

Embodied inference and spatial cognition

How much about our interactions with--and experience of--our world can be deduced from basic principles? This paper reviews recent attempts to understand the self-organised behaviour of embodied agents, like ourselves, as satisfying basic imperatives for sustained exchanges with the environment. In brief, one simple driving force appears to explain many aspects of perception, action and the perception of action. This driving force is the minimisation of surprise or prediction error, which--in the context of perception--corresponds to Bayes-optimal predictive coding (that suppresses exteroceptive prediction errors) and--in the context of action--reduces to classical motor reflexes (that suppress proprioceptive prediction errors). In what follows, we look at some of the phenomena that emerge from this single principle, such as the perceptual encoding of spatial trajectories that can both generate movement (of self) and recognise the movements (of others). These emergent behaviours rest upon prior beliefs about itinerant (wandering) states of the world--but where do these beliefs come from? In this paper, we focus on the nature of prior beliefs and how they underwrite the active sampling of a spatially extended sensorium. Put simply, to avoid surprising states of the world, it is necessary to minimise uncertainty about those states. When this minimisation is implemented via prior beliefs--about how we sample the world--the resulting behaviour is remarkably reminiscent of searches seen in foraging or visual searches with saccadic eye movements.     

Predictive coding as a model of cognition

Previous work has shown that predictive coding can provide a detailed explanation of a very wide range of low-level perceptual processes. It is also widely believed that predictive coding can account for high-level, cognitive, abilities. This article provides support for this view by showing that predictive coding can simulate phenomena such as categorisation, the influence of abstract knowledge on perception, recall and reasoning about conceptual knowledge, context-dependent behavioural control, and naive physics. The particular implementation of predictive coding used here (PC/BC-DIM) has previously been used to simulate low-level perceptual behaviour and the neural mechanisms that underlie them. This algorithm thus provides a single framework for modelling both perceptual and cognitive brain function.