Extending Dynamical Systems Theory to Model Embodied Cognition

We define a mathematical formalism based on the concept of an ‘‘open dynamical system” and show how it can be used to model embodied cognition. This formalism extends classical dynamical systems theory by distinguishing a ‘‘total system’’ (which models an agent in an environment) and an ‘‘agent system’’ (which models an agent by itself), and it includes tools for analyzing the collections of overlapping paths that occur in an embedded agent's state space. To illustrate the way this formalism can be applied, several neural network models are embedded in a simple model environment. Such phenomena as masking, perceptual ambiguity, and priming are then observed. We also use this formalism to reinterpret examples from the embodiment literature, arguing that it provides for a more thorough analysis of the relevant phenomena.     

Representations in dynamical embodied agents: re-analyzing a minimally cognitive model agent

Understanding the role of 'representations' in cognitive science is a fundamental problem facing the emerging framework of embodied, situated, dynamical cognition. To make progress, I follow the approach proposed by an influential representational skeptic, Randall Beer: building artificial agents capable of minimally cognitive behaviors and assessing whether their internal states can be considered to involve representations. Hence, I operationalize the concept of representing as 'standing in,' and I look for representations in embodied agents involved in simple categorization tasks. In a first experiment, no representation can be found, but the relevance of the task is undermined by the fact that agents with no internal states can reach high performance. A simple modification makes the task more "representationally hungry," and in this case, agents' internal states are found to qualify as representations. I conclude by discussing the benefits of reconciling the embodied-dynamical approach with the notion of representation.     

The effect of psychological distance on perceptual level of construal

Three studies examined the effect of primed psychological distance on level of perceptual construal, using Navon's paradigm of composite letters (global letters that are made of local letters). Relative to a control group, thinking of the more distant future (Study 1), about more distant spatial locations (Study 2), and about more distant social relations (Study 3) facilitated perception of global letters relative to local letters. Proximal times, spatial locations, and social relations had the opposite effect. The results are discussed within the framework of Construal Level Theory of psychological distance ( Liberman & Trope, 2008 ; Trope & Liberman, 2003 ).     

Beyond perceptual symbols: A call for representational pluralism

Recent evidence from cognitive neuroscience suggests that certain cognitive processes employ perceptual representations. Inspired by this evidence, a few researchers have proposed that cognition is inherently perceptual. They have developed an innovative theoretical approach that rests on the notion of perceptual simulation and marshaled several general arguments supporting the centrality of perceptual representations to concepts. In this article, I identify a number of weaknesses in these arguments and defend a multiple semantic code approach that posits both perceptual and non-perceptual representations.     

Modality switching costs emerge in concept creation as well as retrieval

Theories of embodied cognition hold that the conceptual system uses perceptual simulations for the purposes of representation. A strong prediction is that perceptual phenomena should emerge in conceptual processing, and, in support, previous research has shown that switching modalities from one trial to the next incurs a processing cost during conceptual tasks. However, to date, such research has been limited by its reliance on the retrieval of familiar concepts. We therefore examined concept creation by asking participants to interpret modality-specific compound phrases (i.e., conceptual combinations). Results show that modality switching costs emerge during the creation of new conceptual entities: People are slower to simulate a novel concept (e.g., auditory jingling onion) when their attention has already been engaged by a different modality in simulating a familiar concept (e.g., visual shiny penny). Furthermore, these costs cannot be accounted for by linguistic factors alone. Rather, our findings support the embodied view that concept creation, as well as retrieval, requires situated perceptual simulation.     

生成认知与具身机能主义的比较——表征的取舍

在具身认知不同的思潮中，生成认知以其激进的观点反对表征和计算主义，可以称之为是具身认知思潮中的最为激进的一部分。另一方面，由于标准认知中表征和计算的解释力和影响力，具身认知诞生出各种不同程度上的“妥协”性理论，其中，以克拉克为代表的具身的“机能主义”为目前具身认知的主流观点。表征尽管在当前阶段中表现出强大的生命力，但依然无法在所有领域中都得到运用。随着表征的改造与发展，未来的激进具身认知工作者应该将更多的精力投入到具身认知与更多学科的融合研究中。     

Mountain high,valley low: Direction-specific effects of articulation on reaching

Representations underpinning action and language overlap and interact very closely. There are bidirectional interactions between word and action comprehension, semantic processing of language, and response selection. This study extends our understanding of the influence of speech on concurrent motor execution. Participants reached-to-grasp the top or bottom of a vertically oriented bar in response to the location of a word on a computer screen (top/bottom). Words were synonyms for “up” or “down”, and participants were required to articulate the word during movement. We were particularly interested in the influence of articulated word semantics on the transport component of the reach. Using motion capture to analyse action kinematics, we show that irrespective of reach direction, saying “up” synonyms led to greater height of the hand, while saying “down” synonyms was associated with reduced height. This direction-specific influence of articulation on the spatial parameters of the hand supports the idea that linguistic systems are tightly integrated and influence each other.     

How Otto did not extend his mind,but might have: Dynamic systems theory and social-cultural group selection

Proponents of cognitive Situationism argue that the human mind is embodied, embedded in both natural and social-cultural environments and extended creating both extended and distributed cognition. Anti-situationists reject all or some of these claims. I argue that four major objections to extended cognition: (1) the mark of the cognitive, (2) the function-identity fallacy, (3) cognitive bloat, and (4) scientific irrelevance lose much of their sting in the case of distributed cognition, the extension of cognitive agency to a group of cognitive agents, such as a scientific research team. However, I claim that a crucial fifth challenge, that advocates of the extended mind commit the causal-constitution fallacy, has yet to be satisfactorily addressed. I focus on Spyridon Palermos’ use of dynamic systems theory to refute this charge and I argue that his appeal to dynamic systems theory as a way of understanding system-constitution fails. Instead, I suggest a social-cultural group selection hypothesis for understanding system-constitution. But, I leave it for another day to elaborate that hypothesis’ empirical plausibility.     

语言理解中主体职业特征与动作表征的激活

运用句-配范式以及句子理解与身体反应相互作用的实验范式,探讨 “做某事的是某人”与“某人做某事”两类句式理解过程中主体职业特征与动作表征的激活。结果发现,在“做某事的是某人”句子理解过程中,主体的职业特征和动作表征得到了激活,但在“某人做某事”句子理解过程中,主体职业特征表征和动作表征只有其中一种表征处于匹配状态时,另一种表征才能得到明显的激活。这表明,句子理解过程中,主体职业特征表征得到了激活,而主体职业特征表征与动作执行器表征的同时激活则受到对方匹配情况的影响,知觉符号理论得到了进一步的证明。     

Moving Beyond Computational Cognitivism: Understanding Intentionality,Intersubjectivity and Ecology of Mind

The emergence of the Cognitive Sciences, in the middle of the 20th Century, was initially based on an abstract model of the mind: the computer metaphor. The human mind was understood in analogy to the digital computer, as a rule-based, symbol processor. As a consequence, the human being was envisioned as logically-rationally guided, radically disembodied and isolated from culture. Over the last few decades, several disciplines, such as Biology, Mathematics, Philosophy, Psychology and Neuroscience, have begun to address the study of intentionality, intersubjectivity and natural cognition. Searching for a better understanding of these complex issues, a number of approaches have been developed with the promise of capturing the specific qualities of human cognition, radically omitted from a computationalist view of mind. Nevertheless, since these research programs are rather recent, concrete methodological designs and empirical approaches in the form of experimentally testable hypotheses are still scarce. This special issue brings together several perspectives in order to propose alternative research approaches in the topics of Intentionality, Intersubjectivity and Ecology of Mind. We believe it is necessary to discuss and advance towards explicit empirical frames in the form of actual experiments, specific predictions and formal models. The essays presented here constitute an attempt to move in this direction, with the specific aim of reconsidering the study of some forgotten properties of brain and mind.     

Language, perception, and the schematic representation of spatial relations

Schemas are abstract nonverbal representations that parsimoniously depict spatial relations. Despite their ubiquitous use in maps and diagrams, little is known about their neural instantiation. We sought to determine the extent to which schematic representations are neurally distinguished from language on the one hand, and from rich perceptual representations on the other. In patients with either left hemisphere damage or right hemisphere damage, a battery of matching tasks depicting categorical spatial relations was used to probe for the comprehension of basic spatial concepts across distinct representational formats (words, pictures, and schemas). Left hemisphere patients underperformed right hemisphere patients across all tasks. However, focused residual analyses using voxel-based lesion-symptom mapping (VLSM) suggest that (1) left hemisphere deficits in the representation of categorical spatial relations are difficult to distinguish from deficits in naming these relations and (2) the right hemisphere plays a special role in extracting schematic representations from richly textured pictures.     

Complementary interactions between classical and top-down driven inhibitory mechanisms of attention

Selective attention informs decision-making by biasing perceptual processing towards task-relevant stimuli. In experimental and computational literature, this is most often implemented through top-down excitation of selected stimuli. However, physiological and anatomical evidence shows that in certain situations, top-down signals could instead be inhibitory. In this study, we investigated how such an inhibitory mechanism of top-down attention compares with an excitatory one. We did so in a neurorobotics context where the agent was controlled using an established hierarchical architecture. We augmented the architecture with an attentional system that implemented top-down attention biasing as connection gains. We tested four models of top-down attention on the simulated agent performing a foraging task: without top-down biasing, with only excitatory top-down gain, with only inhibitory top-down gain, and with both excitatory and inhibitory top-down gain. We manipulated the reward-distractor ratio that was presented and assessed the agent's performance using accumulated rewards and the latency of the selection. Using these measures, we provide evidence that excitatory and inhibitory mechanisms of attention complement each other.     

The interplay between perceptual organization and categorization in the representation of complex visual patterns by young infants

The relation between perceptual organization and categorization processes in 3- and 4-month-olds was explored. The question was whether an invariant part abstracted during category learning could interfere with Gestalt organizational processes. A 2003 study by Quinn and Schyns had reported that an initial category familiarization experience in which infants were presented with visual patterns consisting of a pacman shape and a complex polygon could interfere with infants' subsequent good continuation-based parsing of a circle from visual patterns consisting of a circle and a complex polygon. However, an alternative noninterference explanation for the results was possible because the pacman had been presented with greater frequency and duration than had the circle. The current study repeated Quinn and Schyns's procedure but provided an equivalent number of familiarization trials and duration of study time for the infants to process the pacman during initial familiarization and the circle during subsequent familiarization. The results replicated the previous findings of Quinn and Schyns. The data are consistent with the interference account and suggest that a cognitive system of adaptable feature creation can take precedence over organizational principles with which a perceptual system comes pre-equipped.     

Reconciling symbolic and dynamic aspects of language: Toward a dynamic psycholinguistics

The present paper examines natural language as a dynamical system. The oft-expressed view of language as “a static system of symbols” is here seen as an element of a larger system that embraces the mutuality of symbols and dynamics. Following along the lines of the theoretical biologist H.H. Pattee, the relation between symbolic and dynamic aspects of language is expressed within a more general framework that deals with the role of information in biological systems. In this framework, symbols are seen as information-bearing entities that emerge under pressures of communicative needs and that serve as concrete constraints on development and communication. In an attempt to identify relevant dynamic aspects of such a system, one has to take into account events that happen on different time scales: evolutionary language change (i.e., a diachronic aspect), processes of communication (language use) and language acquisition. Acknowledging the role of dynamic processes in shaping and sustaining the structures of natural language calls for a change in methodology. In particular, a purely synchronic analysis of a system of symbols as “meaning-containing entities” is not sufficient to obtain answers to certain recurring problems in linguistics and the philosophy of language. A more encompassing research framework may be the one designed specifically for studying informationally based coupled dynamical systems (coordination dynamics) in which processes of self-organization take place over different time scales.     

Dissociation and association of the embodied representation of tool-use verbs and hand verbs: An fMRI study

Embodied semantic theories suppose that representation of word meaning and actual sensory-motor processing are implemented in overlapping systems. According to this view, association and dissociation of different word meaning should correspond to dissociation and association of the described sensory-motor processing. Previous studies demonstrate that although tool-use actions and hand actions have overlapping neural substrates, tool-use actions show greater activations in frontal–parietal–temporal regions that are responsible for motor control and tool knowledge processing. In the present study, we examined the association and the dissociation of the semantic representation of tool-use verbs and hand action verbs. Chinese verbs describing tool-use or hand actions without tools were included, and a passive reading task was employed. All verb conditions showed common activations in areas of left middle frontal gyrus, left inferior frontal gyrus (BA 44/45) and left inferior parietal lobule relative to rest, and all conditions showed significant effects in premotor areas within the mask of hand motion effects. Contrasts between tool-use verbs and hand verbs demonstrated that tool verbs elicited stronger activity in left superior parietal lobule, left middle frontal gyrus and left posterior middle temporal gyrus. Additionally, psychophysiological interaction analyses demonstrated that tool verbs indicated greater connectivity among these regions. These results suggest that the brain regions involved in tool-use action processing also play more important roles in tool-use verb processing and that similar systems may be responsible for word meaning representation and actual sensory-motor processing.     

Dissociating the representation of action- and sound-related concepts in middle temporal cortex

Modality-specific models of conceptual memory propose close links between concepts and the sensory-motor systems. Neuroimaging studies found, in different subject groups, that action-related and sound-related concepts activated different parts of posterior middle temporal gyrus (pMTG), suggesting a modality-specific representation of conceptual features. However, as these different parts of pMTG are close to each other, it is possible that the observed anatomical difference is merely related to interindividual variability. In the present functional magnetic resonance imaging study, we now investigated within the same participant group a possible conceptual feature-specific organization in pMTG. Participants performed lexical decisions on sound-related (e.g., telephone) and action-related (hammer) words. Sound words elicited higher activity in anterior pMTG adjacent to auditory association cortex, but action-related words did so in posterior pMTG close to motion sensitive areas. These results confirm distinct conceptual representations of sound and action in pMTG, just adjacent to the respective modality-specific cortices.     

A dynamic view of cultural influence: A review

Static models of culture's influence have given way to a dynamic view, which identifies not only differences across cultures in people's judgments and decisions, but also the situations and conditions in which these differences do or do not appear. Theory and evidence developed from a cognitive psychological perspective underlie this dynamic approach, including research emerging from the “dynamic constructivist” and “situated cognition” models. In the present review, we focus on findings that confirm the utility of this cognitively oriented approach, and briefly discuss the advantages and complementary nature of the “social collective” and neuroscience approaches to understanding culture.     

Computational explorations of perceptual symbol systems theory

The aim of this paper is twofold. First, we provide a methodological pathway from theories of situated, embodied cognition to simulations with an eye to empirical evidence, and suggest a possible cross-fertilization between cognitive robotics and psychology. Psychological theories, in particular those formulated at an abstract level, include models which are often severely underspecified at the level of mechanisms. This is true in the synchronic, constructive perspective (how can the effects observed in experiments be concretely generated by the model's mechanisms?) and in the diachronic, developmental perspective (how can such mechanisms be learned and developed?). The synthetic method of artificial cognitive systems research, and in particular of cognitive robotics, can complement research in psychology (and neurosciences) by exploring the constructive and developmental aspects of theories. Our second aim is to provide an example of such a methodology by describing simulations aiming at developing a perceptual symbol system (PSS) (Barsalou, 1999). We then describe the two main theoretical constructs of the PSS, perceptual symbols and simulators, illustrate their development in an artificial system, and test the system in prediction, categorization, and abstraction tasks.     

A taste of words: linguistic context and perceptual simulation predict the modality of words

Previous studies have shown that object properties are processed faster when they follow properties from the same perceptual modality than properties from different modalities. These findings suggest that language activates sensorimotor processes, which, according to those studies, can only be explained by a modal account of cognition. The current paper shows how a statistical linguistic approach of word co-occurrences can also reliably predict the category of perceptual modality a word belongs to (auditory, olfactory-gustatory, visual-haptic), even though the statistical linguistic approach is less precise than the modal approach (auditory, gustatory, haptic, olfactory, visual). Moreover, the statistical linguistic approach is compared with the modal embodied approach in an experiment in which participants verify properties that share or shift modalities. Response times suggest that fast responses can best be explained by the linguistic account, whereas slower responses can best be explained by the embodied account. These results provide further evidence for the theory that conceptual processing is both linguistic and embodied, whereby less precise linguistic processes precede precise simulation processes.     

Conflict in the kitchen: Contextual modulation of responsiveness to affordances

Recently, human behavior has been considered the product of continuous interactions between perception, cognition and action in which “affordances” (action possibilities the environment has to offer) play an important role. Converging evidence suggests that multiple action possibilities simultaneously compete for further processing, while external and internal factors (e.g., incoming sensory information, predictions) bias this competition. In the present study we used a stop-task to investigate whether context is able to modulate the strength of the responsiveness to affordances. We therefore placed participants in an actual kitchen and workshop during electroencephalographic recordings. A faster response to context congruent objects demonstrated that the direct surrounding is able to affect responsiveness to affordances. In addition, when responses needed to be withheld, context congruent objects evoked greater response conflict as indicated by an enhanced N2 Event Related Potential (ERP) component.