The Social Nature of Perception and Action

ABSTRACT— Humans engage in a wide range of social activities. Previous research has focused on the role of higher cognitive functions, such as mentalizing (the ability to infer others' mental states) and language processing, in social exchange. This article reviews recent studies on action perception and joint action suggesting that basic perception–action links are crucial for many social interactions. Mapping perceived actions to one's own action repertoire enables direct understanding of others' actions and supports action identification. Joint action relies on shared action representations and involves modeling of others' performance in relation to one's own. Taking the social nature of perception and action seriously not only contributes to the understanding of dedicated social processes but has the potential to create a new perspective on the individual mind and brain.     

The felt presence of other minds: Predictive processing,counterfactual predictions,and mentalising in autism

The mental states of other people are components of the external world that modulate the activity of our sensory epithelia. Recent probabilistic frameworks that cast perception as unconscious inference on the external causes of sensory input can thus be expanded to enfold the brain’s representation of others’ mental states. This paper examines this subject in the context of the debate concerning the extent to which we have perceptual awareness of other minds. In particular, we suggest that the notion of perceptual presence helps to refine this debate: are others’ mental states experienced as veridical qualities of the perceptual world around us? This experiential aspect of social cognition may be central to conditions such as autism spectrum disorder, where representations of others’ mental states seem to be selectively compromised. Importantly, recent work ties perceptual presence to the counterfactual predictions of hierarchical generative models that are suggested to perform unconscious inference in the brain. This enables a characterisation of mental state representations in terms of their associated counterfactual predictions, allowing a distinction between spontaneous and explicit forms of mentalising within the framework of predictive processing. This leads to a hypothesis that social cognition in autism spectrum disorder is characterised by a diminished set of counterfactual predictions and the reduced perceptual presence of others’ mental states.     

The proactive brain: using analogies and associations to generate predictions

Rather than passively 'waiting' to be activated by sensations, it is proposed that the human brain is continuously busy generating predictions that approximate the relevant future. Building on previous work, this proposal posits that rudimentary information is extracted rapidly from the input to derive analogies linking that input with representations in memory. The linked stored representations then activate the associations that are relevant in the specific context, which provides focused predictions. These predictions facilitate perception and cognition by pre-sensitizing relevant representations. Predictions regarding complex information, such as those required in social interactions, integrate multiple analogies. This cognitive neuroscience framework can help explain a variety of phenomena, ranging from recognition to first impressions, and from the brain's 'default mode' to a host of mental disorders.     

从动作模仿到社会认知：自我-他人控制的作用

在社会互动中, 人们具有自动模仿他人动作的倾向。尽管这种自动模仿有利于个体理解他人动作的感受, 但有时也会与自身的动作意图产生冲突。因此我们需要将自身动作意图与他人动作进行区分并调控二者之间的冲突。这种能力被称为自我-他人控制(self-other control, SOC)。与动作模仿控制相同, 心理理论、观点采择和共情等更高级的社会认知同样涉及对自我和他人信息的加工。很多证据表明, SOC可能是一种领域普遍的(domain-general)加工机制, 即在动作模仿控制和其他社会认知中, 大脑对自我和他人双方信息的区分和冲突调控共用同一套SOC系统。最近一些研究发现, 相比于抑制自身优势反应的抑制控制(inhibitory control), SOC是社会认知中一个更为关键的影响因素, 抑制控制对社会认知的作用受到SOC的调节。此外, SOC的领域普遍性提示我们, 未来可以通过简单的动作模仿控制训练, 来为社会认知受损个体(如孤独症和述情障碍者)进行康复训练。     

Sociomotor action control

Our actions affect the behavior of other people in predictable ways. In the present article, we describe a theoretical framework for action control in social contexts that we call sociomotor action control. This framework addresses how human agents plan and initiate movements that trigger responses from other people, and we propose that humans represent and control such actions literally in terms of the body movements they consistently evoke from observers. We review evidence for this approach and discuss commonalities and differences to related fields such as joint action, intention understanding, imitation, and interpersonal power. The sociomotor framework highlights a range of open questions pertaining to how representations of other persons’ actions are linked to one’s own motor activity, how specifically they contribute to action initiation, and how they affect the way we perceive the actions of others.     

Primate Culture and Social Learning

The human primate is a deeply cultural species, our cognition being shaped by culture, and cultural transmission amounting to an “epidemic of mental representations” (Sperber, 1996). The architecture of this aspect of human cognition has been shaped by our evolutionary past in ways that we can now begin to discern through comparative studies of other primates. Processes of social learning (learning from others) are important for cognitive science to understand because they are cognitively complex and take many interrelated forms; they shape traditions, cultures and nonsocial aspects of cognition; and in turn they may be shaped by their cultural context. The study of primate social learning and culture has in recent years enjoyed a renaissance, providing a wealth of new findings, key aspects of which are reviewed. The focus is on cognitive issues, including learning about the consequences, sequential structure and hierarchical organization of actions; relating stored knowledge to the assimilation of new social knowledge; feedback guiding the construction of imitations; conceptual grasp of imitation; and the reciprocal relationship between social learning and culture.     

婴儿如何认识他心：“似我”假说述评

Meltzoff的“似我”假说认为,婴儿借助“我他对等”认识解读他人行为和心理状态,通过他人间的互动信息推断和调整自身行为。“似我”认识以动作表征为基础,借助第一人称体验建立双向映射,对“似我”之人进行因果推理,由此构成发展路径引导婴儿认识他心。注视追随与模仿两种初级社会学习机制进一步丰富了婴儿对社交关系的理解,使人类学习的渠道突破了自我体验的局限而变得更加多元。     

What's social about social learning?

Research on social learning in animals has revealed a rich variety of cases where animals--from caddis fly larvae to chimpanzees--acquire biologically important information by observing the actions of others. A great deal is known about the adaptive functions of social learning, but very little about the cognitive mechanisms that make it possible. Even in the case of imitation, a type of social learning studied in both comparative psychology and cognitive science, there has been minimal contact between the two disciplines. Social learning has been isolated from cognitive science by two longstanding assumptions: that it depends on a set of special-purpose modules--cognitive adaptations for social living; and that these learning mechanisms are largely distinct from the processes mediating human social cognition. Recent research challenges these assumptions by showing that social learning covaries with asocial learning; occurs in solitary animals; and exhibits the same features in diverse species, including humans. Drawing on this evidence, I argue that social and asocial learning depend on the same basic learning mechanisms; these are adapted for the detection of predictive relationships in all natural domains; and they are associative mechanisms--processes that encode information for long-term storage by forging excitatory and inhibitory links between event representations. Thus, human and nonhuman social learning are continuous, and social learning is adaptively specialized--it becomes distinctively "social"--only when input mechanisms (perceptual, attentional, and motivational processes) are phylogenetically or ontogenetically tuned to other agents.     

Understanding the flexibility of action–perception coupling

The idea that observing an action triggers an automatic and obligatory activation of an imitative action in the motor system of the observer has recently been questioned by studies examining complementary actions. Instead of a tendency for imitation, cooperative settings may facilitate the execution of dissimilar actions, resulting in a relative disadvantage for imitative actions. The present study aimed at clarifying the contribution of associative learning and interference of task representations to the reversal of congruency effects. To distinguish between the two, an experiment was designed, in which we increased the effects of associative learning and minimized the effects of task interference. Participants completed a series of imitation and complementary action runs, in which they continuously imitated or complemented the actions of a virtual co-actor. Each run was alternated with a test run showing the same actions but including color-cues, and the participants were instructed to respond to color instead of the actor’s posture. Reaction times to test runs showed no reversal of facilitation effects between the imitation and complementary action conditions. This result strongly argues that associative learning cannot adequately account for reversed facilitation effects. Our study provides additional support for action–perception models that allow flexible selection of action–perception coupling and challenges the existing models purely based on stimulus–response associations.

---

‘Goals’ are not an integral component of imitation

Several theories suggest that actions are coded for imitation in terms of mentalistic goals, or inferences about the actor’s intentions, and that these goals solve the correspondence problem by allowing sensory input to be translated into matching motor output. We tested this intention reading hypothesis against general process accounts of imitation using the pen-and-cups task. The task has three components: participants place a pen in one of two cups, using their right or left hand, and one of two grips. Previous research has revealed a colour minimum error pattern; when one of the components is differentially coloured (e.g., one cup is red and the other blue), accuracy is greatest on the coloured dimension. We found the colour minimum error pattern, not only in the standard version of the task, where participants imitate the actions of a human model, but also in three novel variants of the task, in which participants responded on the basis of spatial or arbitrary stimulus–response mappings to ‘geometric’, non-biological stimuli. These stimuli do not afford the attribution of intentions, and therefore our results support generalist theories of imitation by showing that the colour minimum error pattern is due, not to intention reading, but to the operation of task-general processes of perception, attention and motor control.     

Is imitation learning the route to humanoid robots?

This review investigates two recent developments in artificial intelligence and neural computation: learning from imitation and the development of humanoid robots. It is postulated that the study of imitation learning offers a promising route to gain new insights into mechanisms of perceptual motor control that could ultimately lead to the creation of autonomous humanoid robots. Imitation learning focuses on three important issues: efficient motor learning, the connection between action and perception, and modular motor control in the form of movement primitives. It is reviewed here how research on representations of, and functional connections between, action and perception have contributed to our understanding of motor acts of other beings. The recent discovery that some areas in the primate brain are active during both movement perception and execution has provided a hypothetical neural basis of imitation. Computational approaches to imitation learning are also described, initially from the perspective of traditional AI and robotics, but also from the perspective of neural network models and statistical-learning research. Parallels and differences between biological and computational approaches to imitation are highlighted and an overview of current projects that actually employ imitation learning for humanoid robots is given.     

Event structure in perception and conception

Events can be understood in terms of their temporal structure. The authors first draw on several bodies of research to construct an analysis of how people use event structure in perception, understanding, planning, and action. Philosophy provides a grounding for the basic units of events and actions. Perceptual psychology provides an analogy to object perception: Like objects, events belong to categories, and, like objects, events have parts. These relationships generate 2 hierarchical organizations for events: taxonomies and partonomies. Event partonomies have been studied by looking at how people segment activity as it happens. Structured representations of events can relate partonomy to goal relationships and causal structure; such representations have been shown to drive narrative comprehension, memory, and planning. Computational models provide insight into how mental representations might be organized and transformed. These different approaches to event structure converge on an explanation of how multiple sources of information interact in event perception and conception.     

Programmed to learn? The ontogeny of mirror neurons

Mirror neurons are increasingly recognized as a crucial substrate for many developmental processes, including imitation and social learning. Although there has been considerable progress in describing their function and localization in the primate and adult human brain, we still know little about their ontogeny. The idea that mirror neurons result from Hebbian learning while the child observes/hears his/her own actions has received remarkable empirical support in recent years. Here we add a new element to this proposal, by suggesting that the infant's perceptual‐motor system is optimized to provide the brain with the correct input for Hebbian learning, thus facilitating the association between the perception of actions and their corresponding motor programs. We review evidence that infants (1) have a marked visual preference for hands, (2) show cyclic movement patterns with a frequency that could be in the optimal range for enhanced Hebbian learning, and (3) show synchronized theta EEG (also known to favour synaptic Hebbian learning) in mirror cortical areas during self‐observation of grasping. These conditions, taken together, would allow mirror neurons for manual actions to develop quickly and reliably through experiential canalization. Our hypothesis provides a plausible pathway for the emergence of mirror neurons that integrates learning with genetic pre‐programming, suggesting new avenues for research on the link between synaptic processes and behaviour in ontogeny.     

Sounds and scents in (social) action

Although vision seems to predominate in triggering the simulation of the behaviour and mental states of others, the social perception of actions might rely on auditory and olfactory information not only when vision is lacking (e.g. in congenitally blind individuals), but also in daily life (e.g. hearing footsteps along a dark street prompts an appropriate fight-or-fly reaction and smelling the scent of coffee prompts the act of grasping a mug). Here, we review recent evidence showing that non-visual, telereceptor-mediated motor mapping might occur as an autonomous process, as well as within the context of the multimodal perceptions and representations that characterize real-world experiences. Moreover, we discuss the role of auditory and olfactory resonance in anticipating the actions of others and, therefore, in shaping social interactions.     

Weaving the fabric of social interaction: Articulating developmental psychology and cognitive neuroscience in the domain of motor cognition

In this article, we bring together recent findings from developmental science and cognitive neuroscience to argue that perception-action coupling constitutes the fundamental mechanism of motor cognition. A variety of empirical evidence suggests that observed and executed actions are coded in a common cognitive and neural framework, enabling individuals to construct shared representations of self and other actions. We review work to suggest that such shared representations support action anticipation, organization, and imitation. These processes, along with additional computational mechanisms for determining a sense of agency and behavioral regulation, form the fabric of social interaction. In addition, humans possess the capacity to move beyond these basic aspects of action analysis to interpret behavior at a deeper level, an ability that may be outside the scope of the mirror system. Understanding the nature of shared representations from the vantage point of developmental and cognitive science and neuroscience has the potential to inform a range of motor and social processes. This perspective also elucidates intriguing new directions and research questions and generates specific hypotheses regarding the impact of early disorders (e.g., developmental movement disorders) on subsequent action processing.     

Children's Representation and Imitation of Events: How Goal Organization Influences 3‐Year‐Old Children's Memory for Action Sequences

Children's imitation of adults plays a prominent role in human cognitive development. However, few studies have investigated how children represent the complex structure of observed actions which underlies their imitation. We integrate theories of action segmentation, memory, and imitation to investigate whether children's event representation is organized according to veridical serial order or a higher level goal structure. Children were randomly assigned to learn novel event sequences either through interactive hands‐on experience (Study 1) or via storybook (Study 2). Results demonstrate that children's representation of observed actions is organized according to higher level goals, even at the cost of representing the veridical temporal ordering of the sequence. We argue that prioritizing goal structure enhances event memory, and that this mental organization is a key mechanism of social‐cognitive development in real‐world, dynamic environments. It supports cultural learning and imitation in ecologically valid settings when social agents are multitasking and not demonstrating one isolated goal at a time.     

I suffer more from your pain when you act like me: Being imitated enhances affective responses to seeing someone else in pain

Social–psychological research has suggested that being imitated changes the way that we experience others: We like someone who imitates us more, and the interaction with this person runs more smoothly. Whether being imitated also affects basic social reactions, such as empathy for pain, is an open question. Empathy for pain refers to the observation that perceiving another person in pain results in pain-related brain activation in the observer. The aim of the present study was to combine the two lines of research, to investigate whether being imitated can influence empathy for pain. To this end, we developed an experimental approach combining an imitation task with a pain perception task. Subjective reports, as well as physiological responses, indicated that being imitated enhances affective responses to seeing someone else in pain. Furthermore, using rubber hand illusion measures, we provided evidence for the role of shared representations in the sensory and motor domains as a core underlying mechanism. In this way, our study integrated social–psychological research on being imitated with cognitive research on empathy for pain. This has broad implications, since imitation plays a crucial role in our daily social interactions, and our study provides insights into a basic cognitive mechanism that might underlie these social situations.     

Cognitive mechanisms of visuomotor transformation in movement imitation: Examining predictions based on models of apraxia and motor control

When we observe a movement and then reproduce it, how is this visual input transformed into motor output? Studies on stroke patients with apraxia suggest that there may be two distinct routes used for gesture imitation; an indirect route that recruits stored movement memories (motor programs) and a direct route that bypasses them. The present study examined 30 healthy adults ages 18–80 (mean age = 44.0 years, SD = 19.5) to learn how motor programs are recruited or bypassed in movement imitation depending upon task conditions (whether familiar letters or novel shapes are imitated) and perceptual factors (whether shapes or letters are perceived). Subjects were asked to imitate the movements of a model who formed shapes and letters on a sheer mesh screen, and to report whether they perceived the task as a shape or a letter. Movements were recorded using a Vicon motion analysis system, and subsequently analyzed to determine the degree of difference between the demonstrated and produced movements. As predicted, letter perception on the letter tasks resulted in increased temporal error when the demonstrated stroke order conflicted with subjects’ habitual pattern of letter formation. No such interference effects were observed when the letter tasks were perceived as shapes. These findings are discussed in the context of current theories on imitation, and implications for rehabilitation and motor re-learning are presented.     

Visualising mental representations: A primer on noise-based reverse correlation in social psychology

With the introduction of the psychophysical method of reverse correlation, a holy grail of social psychology appears to be within reach – visualising mental representations. Reverse correlation is a data-driven method that yields visual proxies of mental representations, based on judgements of randomly varying stimuli. This review is a primer to an influential reverse correlation approach in which stimuli vary by applying random noise to the pixels of images. Our review suggests that the technique is an invaluable tool in the investigation of social perception (e.g., in the perception of race, gender and personality traits), with ample potential applications. However, it is unclear how these visual proxies are best interpreted. Building on advances in cognitive neuroscience, we suggest that these proxies are visual reflections of the internal representations that determine how social stimuli are perceived. In addition, we provide a tutorial on how to perform reverse correlation experiments using R.