The self in action effects: selective attenuation of self-generated sounds

The immediate experience of self-agency, that is, the experience of generating and controlling our actions, is thought to be a key aspect of selfhood. It has been suggested that this experience is intimately linked to internal motor signals associated with the ongoing actions. These signals should lead to an attenuation of the sensory consequences of one’s own actions and thereby allow classifying them as self-generated. The discovery of shared representations of actions between self and other, however, challenges this idea and suggests similar attenuation of one’s own and other’s sensory action effects.Here, we tested these assumptions by comparing sensory attenuation of self-generated and observed sensory effects. More specifically, we compared the loudness perception of sounds that were either self-generated, generated by another person or a computer. In two experiments, we found a reduced perception of loudness intensity specifically related to self-generation. Furthermore, the perception of sounds generated by another person and a computer did not differ from each other. These findings indicate that one’s own agentive influence upon the outside world has a special perceptual quality which distinguishes it from any sort of external influence, including human and non-human sources. This suggests that a real sense of self-agency is not a socially shared but rather a unique and private experience.     

Baby do–baby see!: How action production influences action perception in infants

With a series of four experiments we show that self-produced actions influence infants’ perception of actions performed by others. After having played with an object, 7–11-month-olds simultaneously watched two videos presenting adults who act on either the same object or a different one. The 9- and 11-month-old preferred to watch the same-object video indicating an influence of action production on action perception at this age. Follow-up studies showed that this influence was restricted to object-related actions. Agentive experience enhanced interest in actions with objects, but not in watching objects or persons per se. These findings indicate that infants are not only interested in acting on objects themselves, but that this experience increases their interest in the actions of other people with the same object.     

Children's imitation of causal action sequences is influenced by statistical and pedagogical evidence

Children are ubiquitous imitators, but how do they decide which actions to imitate? One possibility is that children rationally combine multiple sources of information about which actions are necessary to cause a particular outcome. For instance, children might learn from contingencies between action sequences and outcomes across repeated demonstrations, and they might also use information about the actor’s knowledge state and pedagogical intentions. We define a Bayesian model that predicts children will decide whether to imitate part or all of an action sequence based on both the pattern of statistical evidence and the demonstrator’s pedagogical stance. To test this prediction, we conducted an experiment in which preschool children watched an experimenter repeatedly perform sequences of varying actions followed by an outcome. Children’s imitation of sequences that produced the outcome increased, in some cases resulting in production of shorter sequences of actions that the children had never seen performed in isolation. A second experiment established that children interpret the same statistical evidence differently when it comes from a knowledgeable teacher versus a naïve demonstrator. In particular, in the pedagogical case children are more likely to “overimitate” by reproducing the entire demonstrated sequence. This behavior is consistent with our model’s predictions, and suggests that children attend to both statistical and pedagogical evidence in deciding which actions to imitate, rather than obligately imitating successful action sequences.     

Action influences spatial perception: Neuropsychological evidence

We present neuropsychological evidence indicating that action influences spatial perception. First, we review evidence indicating that actions using a tool can modulate unilateral visual neglect and extinction, where patients are unaware of stimuli presented on one side of space. We show that, at least for some patients, modulation comes about through a combination of visual and motor cueing of attention to the affected side (Experiment 1). Subsequently, we review evidence that action‐relations between stimuli reduce visual extinction; there is less extinction when stimuli fall in the correct colocations for action relative to when they fall in the incorrect relations for action and relative to when stimuli are just associatively related. Finally, we demonstrate that action relations between stimuli can also influence the binding of objects to space, in a patient with Balint's syndrome (Experiment 2). These neuropsychological data indicate that perception–action couplings can be crucial to our conscious representation of space.     

The temporal dynamics of the perceptual consequences of action-effect prediction

An essential aspect of voluntary action control is the ability to predict the perceptual effects of our actions. Although the influence of action-effect prediction on humans’ behavior and perception is unequivocal, it remains unclear when action-effect prediction is generated by the brain. The present study investigates the dynamics of action effect anticipation by tracing the time course of its perceptual consequences. Participants completed an acquisition phase during which specific actions (left and right key-presses) were associated with specific visual effects (upward and downward dots motion). In the test phase they performed a 2 AFC identification task in which they were required to indicate whether the dots moved upward or downward. To isolate any effects of action-effect prediction on perception, participants were presented with congruent and incongruent dot motion in which the association participants learned in the previous acquisition phase was respected and violated, respectively. Crucially, to assess the temporal dynamics of action prediction, congruent and incongruent stimuli were presented at different intervals before or after action execution. We observed higher sensitivity (d′) to motion discrimination in congruent vs. incongruent trials only when stimuli were presented from about 220 ms before the action to 280 ms after the action. The temporal dynamics of our effect suggest that action-effect prediction modulates perception at later stages of motor preparation.     

Infants’ grip strength predicts mu rhythm attenuation during observation of lifting actions with weighted blocks

Research has established that the body is fundamentally involved in perception: bodily experience influences activation of the shared neural system underlying action perception and production during action observation, and bodily characteristics influence perception of the spatial environment. However, whether bodily characteristics influence action perception and its underlying neural system is unknown, particularly in early ontogeny. We measured grip strength in 12‐month‐old infants and investigated relations with mu rhythm attenuation, an electroencephalographic correlate of the neural system underlying action perception, during observation of lifting actions performed with differently weighted blocks. We found that infants with higher grip strength exhibited significant mu attenuation during observation of lifting actions, whereas infants with lower grip strength did not. Moreover, a progressively strong relation between grip strength and mu attenuation during observation of lifts was found with increased block weight. We propose that this relation is attributable to differences in infants’ ability to recognize the effort associated with lifting objects of different weights, as a consequence of their developing strength. Together, our results extend the body's role in perception by demonstrating that bodily characteristics influence action perception by shaping the activation of its underlying neural system.     

Rationales in children’s causal learning from others’ actions

Shown commensurate actions and information by an adult, preschoolers’ causal learning was influenced by the pedagogical context in which these actions occurred. Four-year-olds who were provided with a reason for an experimenter’s action relevant to learning causal structure showed more accurate causal learning than children exposed to the same action and data accompanied by an inappropriate rationale or accompanied by no explanatory information. These results suggest that children’s accurate causal learning is influenced by contextual factors that specify the instructional value of others’ actions.     

The role of motor simulation in action perception: a neuropsychological case study

Research on embodied cognition stresses that bodily and motor processes constrain how we perceive others. Regarding action perception the most prominent hypothesis is that observed actions are matched to the observer’s own motor representations. Previous findings demonstrate that the motor laws that constrain one’s performance also constrain one’s perception of others’ actions. The present neuropsychological case study asked whether neurological impairments affect a person’s performance and action perception in the same way. The results showed that patient DS, who suffers from a frontal brain lesion, not only ignored target size when performing movements but also when asked to judge whether others can perform the same movements. In other words DS showed the same violation of Fitts’s law when performing and observing actions. These results further support the assumption of close perception action links and the assumption that these links recruit predictive mechanisms residing in the motor system.     

Perceptual resonance: action-induced modulation of perception

A direct relationship between perception and action implies bi-directionality, and predicts not only effects of perception on action but also effects of action on perception. Modern theories of social cognition have intensively examined the relation from perception to action and propose that mirroring the observed actions of others underlies action understanding. Here, we suggest that this view is incomplete, as it neglects the perspective of the actor. We will review empirical evidence showing the effects of self-generated action on perceptual judgments. We propose that producing action might prime perception in a way that observers are selectively sensitive to related or similar actions of conspecifics. Therefore, perceptual resonance, not motor resonance, might be decisive for grounding sympathy and empathy and, thus, successful social interactions.     

Spectral information in nonspeech contexts influences children’s categorization of ambiguous speech sounds

For both adults and children, acoustic context plays an important role in speech perception. For adults, both speech and nonspeech acoustic contexts influence perception of subsequent speech items, consistent with the argument that effects of context are due to domain-general auditory processes. However, prior research examining the effects of context on children’s speech perception have focused on speech contexts; nonspeech contexts have not been explored previously. To better understand the developmental progression of children’s use of contexts in speech perception and the mechanisms underlying that development, we created a novel experimental paradigm testing 5-year-old children’s speech perception in several acoustic contexts. The results demonstrated that nonspeech context influences children’s speech perception, consistent with claims that context effects arise from general auditory system properties rather than speech-specific mechanisms. This supports theoretical accounts of language development suggesting that domain-general processes play a role across the lifespan.     

The effect of level of construal on the temporal distance of activity enactment

A series of four studies demonstrates that, across different operationalizations of level of construal, different types of activities, and for both the self and another person as targets, construing activities in high-level terms fosters perception of the more distal future as appropriate for their enactment. Specifically, the studies show that thinking about superordinate “why” (rather than subordinate “how”) aspects of actions, about the implications of actions for one’s personality (rather than the objects that would be used in performing the action), and about abstract (rather than concrete) aspects of actions leads people to expect the actions to be performed in the more distant future. These results suggest that the association of level of construal and time perspective is bi-directional, in that time perspective affects level of construal and, in addition, level of construal affects time perspective.     

Does own experience affect perception of others’ actions in capuchin monkeys (Cebus apella)?

Anticipating another’s actions is an important ability in social animals. Recent research suggests that in human adults and infants one’s own action experience facilitates understanding and anticipation of others’ actions. We investigated the link between first-person experience and perception of another’s action in adult tufted capuchin monkeys (Sapajus apella spp., formerly Cebus apella spp.). In Experiment 1, the monkeys observed a familiar human (actor) trying to open a container using either a familiar or an unfamiliar action. They looked for longer when the actor tried to open the container using a familiar action. In Experiment 2, the actor performed two novel actions on a new container. The monkeys looked equally at the two actions. In Experiment 3, the monkeys were trained to open the container using one of the novel actions in Experiment 2. After training, we repeated the same procedure as in Experiment 2. The monkeys looked for longer when the actor manipulated the container using the action they had practiced than when she used the unfamiliar action. These results show that knowledge derived from one’s own experience impacts perception of another’s action in these New World monkeys.     

Fourteen-month-old infants copy an action style accompanied by social-emotional cues

Human adults exaggerate their actions and facial expressions when interacting with infants. These infant-directed modifications highlight certain aspects of action sequences and attract infants’ attention. This study investigated whether social-emotional aspects of infant-directed modifications, such as smiling, eye contact, and onomatopoeic vocalization, influence infants’ copying of another's action, especially action style, during the process of achieving an outcome. In Study 1, 14-month-old infants (n = 22) saw an experimenter demonstrate goal-directed actions in an exaggerated manner. Either the style or the end state of the actions was accompanied by social-emotional cues from the experimenter. Infants copied the style of the action more often when social-emotional cues accompanied the style than when they accompanied the end state. In Study 2, a different group of 14-month-old infants (n = 22) watched the same exaggerated actions as in Study 1, except that either the style or the end state was accompanied by a physical sound instead of social-emotional cues. The infants copied the end state consistently more often than the style. Taken together, these two studies showed that accompanying social-emotional cues provided by a demonstrator, but not accompanying physical sound, increased infants’ copying of action style. These findings suggest that social-emotional cues facilitate efficient social learning through the adult–infant interaction.     

Action preferences and the anticipation of action outcomes

Skilled performers of time-constrained motor actions acquire information about the action preferences of their opponents in an effort to better anticipate the outcome of that opponent's actions. However, there is reason to doubt that knowledge of an opponent's action preferences would unequivocally influence anticipatory responses in a positive way. It is possible that overt information about an opponent's actions could distract skilled performers from using the advance kinematic information they would usually rely on to anticipate actions, particularly when the opponent performs an ‘unexpected’ action that is not in accordance with his or her previous behaviour. The aim of this study was to examine how the ability to anticipate the outcome of an opponent's actions can be influenced by exposure to the action preferences of that opponent. Two groups of skilled handball goalkeepers anticipated the direction of penalty throws performed by opponents before and after a training intervention that provided situational probability information in the form of action preferences (AP). During the training phase participants in an AP-training group anticipated the action outcomes of two throwers who had a strong preference to throw in one particular direction, whilst participants in a NP-training group viewed players who threw equally to all directions. Exposure to opponents who did have an action preference during the training phase resulted in improved anticipatory performance if the opponent continued to bias their throws towards their preferred direction, but decreased performance if the opponent did not. These findings highlight that skilled observers use information about action preferences to enhance their anticipatory ability, but that doing so can be disadvantageous when the outcomes are no longer consistent with their generated expectations.     

Relations between 18-month-olds’ gaze pattern and target action performance: A deferred imitation study with eye tracking

Deferred imitation studies are used to assess infants’ declarative memory performance. These studies have found that deferred imitation performance improves with age, which is usually attributed to advancing memory capabilities. Imitation studies, however, are also used to assess infants’ action understanding. In this second research program it has been observed that infants around the age of one year imitate selectively, i.e., they imitate certain kinds of target actions and omit others. In contrast to this, two-year-olds usually imitate the model's exact actions. 18-month-olds imitate more exactly than one-year-olds, but more selectively than two-year-olds, a fact which makes this age group especially interesting, since the processes underlying selective vs. exact imitation are largely debated. The question, for example, if selective attention to certain kinds of target actions accounts for preferential imitation of these actions in young infants is still open. Additionally, relations between memory capabilities and selective imitation processes, as well as their role in shaping 18-month-olds’ neither completely selective, nor completely exact imitation have not been thoroughly investigated yet. The present study, therefore, assessed 18-month-olds’ gaze toward two types of actions (functional vs. arbitrary target actions) and the model's face during target action demonstration, as well as infants’ deferred imitation performance. Although infants’ fixation times to functional target actions were not longer than to arbitrary target actions, they imitated the functional target actions more frequently than the arbitrary ones. This suggests that selective imitation does not rely on selective gaze toward functional target actions during the demonstration phase. In addition, a post hoc analysis of interindividual differences suggested that infants’ attention to the model's social-communicative cues might play an important role in exact imitation, meaning the imitation of both functional and arbitrary target actions.     

Social cognition in simple action coordination: A case for direct perception

In this paper we sketch the outlines of an account of the kind of social cognition involved in simple action coordination that is based on direct social perception (DSP) rather than recursive mindreading. While we recognize the viability of a mindreading-based account such as e.g. Michael Tomasello’s, we present an alternative DSP account that (i) explains simple action coordination in a less cognitively demanding manner, (ii) is better able to explain flexibility and strategy-switching in coordination and crucially (iii) allows for formal modeling. This account of action coordination is based on the notion of an agent’s field of affordances. Coordination ensues, we argue, when, given a shared intention, the actions of and/or affordances for one agent shape the field of affordances for another agent. This a form of social perception since in particular perceiving affordances for another person involves seeing that person as an agent. It is a form of social perception since it involves perceiving affordances for another person and registering how another person’s actions influence one’s own perceived field of affordances.     

Understanding social motor coordination

Recently there has been much interest in social coordination of motor movements, or as it is referred to by some researchers, joint action. This paper reviews the cognitive perspective’s common coding/mirror neuron theory of joint action, describes some of its limitations and then presents the behavioral dynamics perspective as an alternative way of understanding social motor coordination. In particular, behavioral dynamics’ ability to explain the temporal coordination of interacting individuals is detailed. Two experiments are then described that demonstrate how dynamical processes of synchronization are apparent in the coordination underlying everyday joint actions such as martial art exercises, hand-clapping games, and conversations. The import of this evidence is that emergent dynamic patterns such as synchronization are the behavioral order that any neural substrate supporting joint action (e.g., mirror systems) would have to sustain.     

Bidirectional semantic interference between action and speech

Research on embodied cognition assumes that language processing involves modal simulations that recruit the same neural systems that are usually used for action execution. If this is true, one should find evidence for bidirectional crosstalk between action and language. Using a direct matching paradigm, this study tested if action-languages interactions are bidirectional (Experiments 1 and 2), and whether the effect of crosstalk between action perception and language production is due to facilitation or interference (Experiment 3). Replicating previous findings, we found evidence for crosstalk when manual actions had to be performed simultaneously to action-word perception (Experiment 1) and also when language had to be produced during simultaneous perception of hand actions (Experiment 2). These findings suggest a clear bidirectional relationship between action and language. The latter crosstalk effect was due to interference between action and language (Experiment 3). By extending previous research of embodied cognition, the present findings provide novel evidence suggesting that bidirectional functional relations between action and language are based on similar conceptual-semantic representations.     

A case of hand waving: Action synchrony and person perception

While previous research has demonstrated that people’s movements can become coordinated during social interaction, little is known about the cognitive consequences of behavioral synchrony. Given intimate links between the systems that regulate perception and action, we hypothesized that the synchronization of movements during a dyadic interaction may prompt increased attention to be directed to an interaction partner, hence facilitate the information that participants glean during a social exchange. Our results supported this prediction. Incidental memories for core aspects of a brief interaction were facilitated following in-phase behavioral synchrony. Specifically, participants demonstrated enhanced memory for an interaction partner’s utterances and facial appearance. These findings underscore the importance of action perception to social cognition.     

Neuro-cognitive mechanisms of decision making in joint action: a human-robot interaction study

In this paper we present a model for action preparation and decision making in cooperative tasks that is inspired by recent experimental findings about the neuro-cognitive mechanisms supporting joint action in humans. It implements the coordination of actions and goals among the partners as a dynamic process that integrates contextual cues, shared task knowledge and predicted outcome of others’ motor behavior. The control architecture is formalized by a system of coupled dynamic neural fields representing a distributed network of local but connected neural populations. Different pools of neurons encode task-relevant information about action means, task goals and context in the form of self-sustained activation patterns. These patterns are triggered by input from connected populations and evolve continuously in time under the influence of recurrent interactions. The dynamic model of joint action is evaluated in a task in which a robot and a human jointly construct a toy object. We show that the highly context sensitive mapping from action observation onto appropriate complementary actions allows coping with dynamically changing joint action situations.