The Theory of Event Coding (TEC): a framework for perception and action planning

Traditional approaches to human information processing tend to deal with perception and action planning in isolation, so that an adequate account of the perception-action interface is still missing. On the perceptual side, the dominant cognitive view largely underestimates, and thus fails to account for, the impact of action-related processes on both the processing of perceptual information and on perceptual learning. On the action side, most approaches conceive of action planning as a mere continuation of stimulus processing, thus failing to account for the goal-directedness of even the simplest reaction in an experimental task. We propose a new framework for a more adequate theoretical treatment of perception and action planning, in which perceptual contents and action plans are coded in a common representational medium by feature codes with distal reference. Perceived events (perceptions) and to-be-produced events (actions) are equally represented by integrated, task-tuned networks of feature codes--cognitive structures we call event codes. We give an overview of evidence from a wide variety of empirical domains, such as spatial stimulus-response compatibility, sensorimotor synchronization, and ideomotor action, showing that our main assumptions are well supported by the data.     

The planning and control model (PCM) of motorvisual priming: reconciling motorvisual impairment and facilitation effects

Previous research on dual-tasks has shown that, under some circumstances, actions impair the perception of action-consistent stimuli, whereas, under other conditions, actions facilitate the perception of action-consistent stimuli. We propose a new model to reconcile these contrasting findings. The planning and control model (PCM) of motorvisual priming proposes that action planning binds categorical representations of action features so that their availability for perceptual processing is inhibited. Thus, the perception of categorically action-consistent stimuli is impaired during action planning. Movement control processes, on the other hand, integrate multi-sensory spatial information about the movement and, therefore, facilitate perceptual processing of spatially movement-consistent stimuli. We show that the PCM is consistent with a wider range of empirical data than previous models on motorvisual priming. Furthermore, the model yields previously untested empirical predictions. We also discuss how the PCM relates to motorvisual research paradigms other than dual-tasks.     

Is imitation learning the route to humanoid robots?

This review investigates two recent developments in artificial intelligence and neural computation: learning from imitation and the development of humanoid robots. It is postulated that the study of imitation learning offers a promising route to gain new insights into mechanisms of perceptual motor control that could ultimately lead to the creation of autonomous humanoid robots. Imitation learning focuses on three important issues: efficient motor learning, the connection between action and perception, and modular motor control in the form of movement primitives. It is reviewed here how research on representations of, and functional connections between, action and perception have contributed to our understanding of motor acts of other beings. The recent discovery that some areas in the primate brain are active during both movement perception and execution has provided a hypothetical neural basis of imitation. Computational approaches to imitation learning are also described, initially from the perspective of traditional AI and robotics, but also from the perspective of neural network models and statistical-learning research. Parallels and differences between biological and computational approaches to imitation are highlighted and an overview of current projects that actually employ imitation learning for humanoid robots is given.     

Model learning for robot control: a survey

Models are among the most essential tools in robotics, such as kinematics and dynamics models of the robot's own body and controllable external objects. It is widely believed that intelligent mammals also rely on internal models in order to generate their actions. However, while classical robotics relies on manually generated models that are based on human insights into physics, future autonomous, cognitive robots need to be able to automatically generate models that are based on information which is extracted from the data streams accessible to the robot. In this paper, we survey the progress in model learning with a strong focus on robot control on a kinematic as well as dynamical level. Here, a model describes essential information about the behavior of the environment and the influence of an agent on this environment. In the context of model-based learning control, we view the model from three different perspectives. First, we need to study the different possible model learning architectures for robotics. Second, we discuss what kind of problems these architecture and the domain of robotics imply for the applicable learning methods. From this discussion, we deduce future directions of real-time learning algorithms. Third, we show where these scenarios have been used successfully in several case studies.     

Searching for the minimal essential information for skilled perception and action

Summary A common concern for both cognitive/computational and ecological/dynamical models of human motor control is the isolation of the minimal essential information needed to support skilled perception and action. In perception isolating essential features of the optic flow field, which are reliably informative regarding the nature of current events, from nonessential features provides a valuable step towards understanding how the computational complexity of perceptual information processing may be reduced to manageable levels and how relatively direct linkages of low dimensionality may be established between information and control variables. Likewise, in the study of action, discrimination of the movement features that remain immutable (invariant?) across changes in task conditions from the variables that are situationally determined provides a principled insight into the structural framework upon which skilled movement is built. Controversy abounds, however, in the study of perception and action as to whether features isolated as informative and immutable are centrally represented (in the form of a template or program) or are rather directly picked up (in the case of perceptual variables) or are simply an emergent consequence of the underlying dynamics (in the case of action variables). In this paper some examples of putative minimal essential information sources in perception and action are provided, strategies for uncovering such sources are discussed, and attention is directed, with the use of some recent data collected on natural skills, to some systematic expert-novice differences in the utilization of essential information and control variables. Expert-novice differences are highlighted because of the insight they may provide regarding the nature of perceptual-motor skill acquisition.     

Visual illusions affect both movement planning and on-line control: a multiple cue position on bias and goal-directed action

Over the last decade, there has been an interest in the impact of visual illusions on the control of action. Much of this work has been motivated by Milner and Goodale's two visual system model of visual processing. This model is based on a hypothesized dissociation between cognitive judgments and the visual control of action. It holds that action is immune to the visual context that provides the basis for the illusion-induced bias associated with cognitive judgments. Recently, Glover has challenged this position and has suggested that movement planning, but not movement execution is susceptible to visual illusions. Research from our lab is inconsistent with both models of visual-motor processing. With respect to the planning and control model, kinematic evidence shows that the impact of an illusion on manual aiming increases as the limb approaches the target. For the Ebbinghaus illusion, this involved a decrease in the time after peak velocity to accommodate the 'perceived' size of the target. For the Müller-Lyer illusion, the influence of the figure's tails increased from peak velocity to the end of the movement. Although our findings contradict a strong version of the two visual systems hypothesis, we did find dissociations between perception and action in another experiment. In this Müller-Lyer study, perceptual decisions were influenced by misjudgment of extent, while action was influenced by misjudgment of target position. Overall, our findings are consistent with the idea that it is often necessary to use visual context to make adjustments to ongoing movements.     

The basis of shared intentions in human and robot cognition

There is a fundamental difference between robots that are equipped with sensory, motor and cognitive capabilities, vs. simulations or non-embodied cognitive systems. Via their perceptual and motor capabilities, these robotic systems can interact with humans in an increasingly more “natural” way, physically interacting with shared objects in cooperative action settings. Indeed, such cognitive robotic systems provide a unique opportunity to developmental psychologists for implementing their theories and testing their hypotheses on systems that are becoming increasingly “at home” in the sensory--motor and social worlds, where such hypotheses are relevant. The current research is the result of interaction between research in computational neuroscience and robotics on the one hand, and developmental psychology on the other. One of the key findings in the developmental psychology context is that with respect to other primates, humans appear to have a unique ability and motivation to share goals and intentions with others. This ability is expressed in cooperative behavior very early in life, and appears to be the basis for subsequent development of social cognition. Here we attempt to identify a set of core functional elements of cooperative behavior and the corresponding shared intentional representations. We then begin to specify how these capabilities can be implemented in a robotic system, the Cooperator, and tested in human–robot interaction experiments. Based on the results of these experiments we discuss the mutual benefit for both fields of the interaction between robotics and developmental psychology.     

Emergence of self and other in perception and action: an event-control approach

The present paper analyzes the regularities referred to via the concept 'self.' This is important, for cognitive science traditionally models the self as a cognitive mediator between perceptual inputs and behavioral outputs. This leads to the assertion that the self causes action. Recent findings in social psychology indicate this is not the case and, as a consequence, certain cognitive scientists model the self as being epiphenomenal. In contrast, the present paper proposes an alternative approach (i.e., the event-control approach) that is based on recently discovered regularities between perception and action. Specifically, these regularities indicate that perception and action planning utilize common neural resources. This leads to a coupling of perception, planning, and action in which the first two constitute aspects of a single system (i.e., the distal-event system) that is able to pre-specify and detect distal events. This distal-event system is then coupled with action (i.e., effector-control systems) in a constraining, as opposed to 'causal' manner. This model has implications for how we conceptualize the manner in which one infers the intentions of another, anticipates the intentions of another, and possibly even experiences another. In conclusion, it is argued that it may be possible to map the concept 'self' onto the regularities referred to in the event-control model, not in order to reify 'the self' as a causal mechanism, but to demonstrate its status as a useful concept that refers to regularities that are part of the natural order.     

Information processes in movement learning: capacity and structural interference effects

A series of experiments examined motor learning as an information processing activity occurring within a working short-term memory system and where response-produced feedback and knowledge of results (KR) are used to modify the action plan developed from previous attempts at the movement task. Use of interpolated activities in the KR delay interval allowed inferences to be made regarding the capacity and structural characteristics of these information processes. Results indicated no capacity limitations on the learning process but important structural effects were found. The results supported the idea that response-produced feedback is relatively unimportant as a feedback variable early in learning. Rather, the use of KR at a relatively high level of movement planning appears to be the important information processing activity underlying learning. Finally, the results supported the view that these information processes are related to cognitive problem solving activities.     

Information Processes in Movement Learning

A series of experiments examined motor learning as an information processing activity occurring within a working short-term memory system and where response-produced feedback and knowledge of results (KR) are used to modify the action plan developed from previous attempts at the movement task. Use of interpolated activities in the KR delay interval allowed inferences to be made regarding the capacity and structural characteristics of these information processes. Results indicated no capacity limitations on the learning process but important structural effects were found. The results supported the idea that response-produced feedback is relatively unimportant as a feedback variable early in learning. Rather, the use of KR at a relatively high level of movement planning appears to be the important information processing activity underlying learning. Finally, the results supported the view that these information processes are related to cognitive problem solving activities.     

The role of robotic modelling in cognitive science

From the perspective of cognitive robotics, this paper presents a modern interpretation of Newell’s (1973) reasoning and suggestions for why and how cognitive psychologists should develop models of cognitive phenomena. We argue that the shortcomings of current cognitive modelling approaches are due in significant part to a lack of exactly the kind of integration required for the development of embodied autonomous robotics. Moreover we suggest that considerations of embodiment, situatedness, and autonomy, intrinsic to cognitive robotics, provide an appropriate basis for the integration and theoretic cumulation that Newell argued was necessary for psychology to mature. From this perspective we analyse the role of embodiment and modes of situatedness in terms of integration, cognition, emotion, and autonomy. Four complementary perspectives on embodied and situated cognitive science are considered in terms of their potential to contribute to cognitive robotics, cognitive science, and psychological theorizing: minimal cognition and organization, enactive perception and sensorimotor contingency, homeostasis and emotion, and social embedding. In combination these perspectives provide a framework for cognitive robotics, not only wholly compatible with the original aims of cognitive modelling, but as a more appropriate methodology than those currently in common use within psychology.     

Bayes,predictive processing,and the cognitive architecture of motor control

Despite their popularity, relatively scant attention has been paid to the upshot of Bayesian and predictive processing models of cognition for views of overall cognitive architecture. Many of these models are hierarchical; they posit generative models at multiple distinct “levels,” whose job is to predict the consequences of sensory input at lower levels. I articulate one possible position that could be implied by these models, namely, that there is a continuous hierarchy of perception, cognition, and action control comprising levels of generative models. I argue that this view is not entailed by a general Bayesian/predictive processing outlook. Bayesian approaches are compatible with distinct formats of mental representation. Focusing on Bayesian approaches to motor control, I argue that the junctures between different types of mental representation are places where the transitivity of hierarchical prediction may be broken, and I consider the upshot of this conclusion for broader discussions of cognitive architecture.     

Scary warnings and rational precautions: A review of the psychology of fear appeals

Abstract

Research into the effects of fear-arousal on precautionary motivation and action is reviewed. Current models do not adequately distinguish between emotional (i.e., fear arousal) and cognitive (i.e., threat perception) responses to fear appeals and, in general, are not well supported. Evidence suggesting that (i) coping appraisals are more powerful predictors of precautionary action than threat perception and that (ii) fear control processes may interfere with precautionary motivation, recommends cautious and limited use of fear appeals in health promotion. It seems likely that fear arousal is less important in motivating precautionary action than perceptions of action effectiveness and self-efficacy. Moreover, perceived personal relevance may be critical to the emotional and cognitive impact of threat information. Available findings are summarised in the form of a process model that highlights the potential complexity of fear arousal effects. Sequential measurement of fear arousal, other than by self-report, is recommended in studies seeking to clarify these effects.     

An embodied model for sensorimotor grounding and grounding transfer: experiments with epigenetic robots

The grounding of symbols in computational models of linguistic abilities is one of the fundamental properties of psychologically plausible cognitive models. In this article, we present an embodied model for the grounding of language in action based on epigenetic robots. Epigenetic robotics is one of the new cognitive modeling approaches to modeling autonomous mental development. The robot model is based on an integrative vision of language in which linguistic abilities are strictly dependent on and grounded in other behaviors and skills. It uses simulated robots that learn through imitation the names of basic actions. Robots also learn higher order action concepts through the process of grounding transfer. The simulation demonstrates how new, higher order behavioral abilities can be autonomously built on previously grounded basic action categories following linguistic interaction with human users.     

Re-framing the characteristics of concepts and their relation to learning and cognition in artificial agents

In this work, the problems of knowledge acquisition and information processing are explored in relation to the definitions of concepts and conceptual processing, and their implications for artificial agents.The discussion focuses on views of cognition as a dynamic property in which the world is actively represented in grounded mental states which only have meaning in the action context. Reasoning is understood as an emerging property consequence of actions-environment couplings achieved through experience, and concepts as situated and dynamic phenomena enabling behaviours.Re-framing the characteristics of concepts is considered crucial to overcoming settled beliefs and reinterpreting new understandings in artificial systems.The first part presents a review of concepts from cognitive sciences. Support is found for views on grounded and embodied cognition, describing concepts as dynamic, flexible, context-dependent, and distributedly coded.That is argued to contrast with many technical implementations assuming concepts as categories, whilst explains limitations when grounding amodal symbols, or in unifying learning, perception and reasoning.The characteristics of concepts are linked to methods of active inference, self-organization, and deep learning to address challenges posed and to reinterpret emerging techniques.In a second part, an architecture based on deep generative models is presented to illustrate arguments elaborated. It is evaluated in a navigation task, showing that sufficient representations are created regarding situated behaviours with no semantics imposed on data. Moreover, adequate behaviours are achieved through a dynamic integration of perception and action in a single representational domain and process.     

Modality-specific interference between action planning and perceptual processing

This paper reports a study which examined an interaction between action planning and processing of perceptual information in two different sensory modalities. In line with the idea that action planning consists in representing the action’s sensory outcomes, it was assumed that different types of actions should be coupled with different modalities. A visual and auditory oddball paradigm was combined with two types of actions: pointing and knocking (unrelated to the perceptual task). Results showed an interactive effect between the action type and the sensory modality of the oddballs, with impaired detection of auditory oddballs for knocking (congruent) action, as compared to a pointing (incongruent) action. These findings reveal that action planning can interact with modality-specific perceptual processing and that preparing an action presumably binds the respective perceptual features with an action plan, thereby making these features less available for other tasks.     

Competitive Learning: From Interactive Activation to Adaptive Resonance

Functional and mechanistic comparisons are made between several network models of cognitive processing: competitive learning, interactive activation, adaptive resonance, and back propagation. The starting point of this comparison is the article of Rumelhart and Zipser (1985) on feature discovery through competitive learning. All the models which Rumelhart and Zipser (1985) have described were shown in Grossberg (1976b) to exhibit a type of learning which is temporally unstable. Competitive learning mechanisms can be stabilized in response to an arbitrary input environment by being supplemented with mechanisms for learning top-down expectancies, or templates; for matching bottom-up input patterns with the top-down expectancies; and for releasing orienting reactions in a mismatch situation, thereby updating short-term memory and searching for another internal representation. Network architectures which embody all of these mechanisms were called adaptive resonance models by Grossberg (1976c). Self-stabilizing learning models are candidates for use in real-world applications where unpredictable changes can occur in complex input environments. Competitive learning postulates are inconsistent with the postulates of the interactive activation model of McClelland and Rumelhart (1981), and suggest different levels of processing and interaction rules for the analysis of word recognition. Adaptive resonance models use these alternative levels and interaction rules. The self-organizing learning of an adaptive resonance model is compared and contrasted with the teacher-directed learning of a back propagation model. A number of criteria for evaluating real-time network models of cognitive processing are described and applied.     

The link between brain learning, attention, and consciousness

The processes whereby our brains continue to learn about a changing world in a stable fashion throughout life are proposed to lead to conscious experiences. These processes include the learning of top-down expectations, the matching of these expectations against bottom-up data, the focusing of attention upon the expected clusters of information, and the development of resonant states between bottom-up and top-down processes as they reach an attentive consensus between what is expected and what is there in the outside world. It is suggested that all conscious states in the brain are resonant states and that these resonant states trigger learning of sensory and cognitive representations. The models which summarize these concepts are therefore called Adaptive Resonance Theory, or ART, models. Psychophysical and neurobiological data in support of ART are presented from early vision, visual object recognition, auditory streaming, variable-rate speech perception, somatosensory perception, and cognitive-emotional interactions, among others. It is noted that ART mechanisms seem to be operative at all levels of the visual system, and it is proposed how these mechanisms are realized by known laminar circuits of visual cortex. It is predicted that the same circuit realization of ART mechanisms will be found in the laminar circuits of all sensory and cognitive neocortex. Concepts and data are summarized concerning how some visual percepts may be visibly, or modally, perceived, whereas amodal percepts may be consciously recognized even though they are perceptually invisible. It is also suggested that sensory and cognitive processing in the What processing stream of the brain obey top-down matching and learning laws that are often complementary to those used for spatial and motor processing in the brain's Where processing stream. This enables our sensory and cognitive representations to maintain their stability as we learn more about the world, while allowing spatial and motor representations to forget learned maps and gains that are no longer appropriate as our bodies develop and grow from infanthood to adulthood. Procedural memories are proposed to be unconscious because the inhibitory matching process that supports these spatial and motor processes cannot lead to resonance.     

Somatosensory processes subserving perception and action

The functions of the somatosensory system are multiple. We use tactile input to localize and experience the various qualities of touch, and proprioceptive information to determine the position of different parts of the body with respect to each other, which provides fundamental information for action. Further, tactile exploration of the characteristics of external objects can result in conscious perceptual experience and stimulus or object recognition. Neuroanatomical studies suggest parallel processing as well as serial processing within the cerebral somatosensory system that reflect these separate functions, with one processing stream terminating in the posterior parietal cortex (PPC), and the other terminating in the insula. We suggest that, analogously to the organisation of the visual system, somatosensory processing for the guidance of action can be dissociated from the processing that leads to perception and memory. In addition, we find a second division between tactile information processing about external targets in service of object recognition and tactile information processing related to the body itself. We suggest the posterior parietal cortex subserves both perception and action, whereas the insula principally subserves perceptual recognition and learning.