Configural coding of facial expressions: The impact of inversion and photographic negative

In previous research we used the composite paradigm (Young, Hellawell, & Hay, ) to demonstrate that configural cues are important for interpreting facial expressions. However, different configural cues in face perception have been identified, including holistic processing (i.e., perception of facial features as a single gestalt) and second‐order spatial relations (i.e., the spatial relationship between individual features). Previous research has suggested that the composite effect for facial identity operates at the level of holistic encoding. Here we show that the composite effect for facial expression has a similar perceptual basis by using different graphic manipulations (stimulus inversion and photographic negative) in conjunction with the composite paradigm. In relation to Bruce and Young's () functional model of face recognition, a suitable level for the composite effect is a stage of front‐end processing referred to as structural encoding, that is common to both facial identity and facial expression perception.     

On the interplay between familiarity and emotional expression in face perception

Traditional models of face perception (e.g. Bruce and Young 1986) stress independent routes for processing identity and emotional expression. We investigated the interplay between familiarity and emotional expression by systematically varying both factors. In contrast to earlier studies which used binary forced-choice decisions, participants had to judge the familiarity of the presented face and the emotional hedonic valence and emotional arousal of its expressed emotion (angry, happy or neutral), using rating scales. The results demonstrated symmetric, strong interactions between familiarity and expressed emotion. Thus, this study supports more recent models of face perception (Haxby et al. 2000) that were mostly based on brain imaging data. These data together with our behavioural results emphasize the interaction of emotional expression and personal identity and support approaches that propose a relative segregation of these processes, rather than completely independent coding (Calder and Young 2005).     

Face aftereffects suggest interdependent processing of expression and sex and of expression and race

Bruce and Young (1986) proposed that functionally different aspects of faces (e.g., sex, identity, and expression) are processed independently. Although interdependent processing of identity and expression and of identity and sex have been demonstrated previously, evidence for interdependent processing of sex and expression is equivocal. Using a visual adaptation paradigm, we show that expression aftereffects can be simultaneously induced in different directions along anger–fear continua for male and female faces (Experiment 1) and for East Asian and Black African faces (Experiment 2). These findings for sex- and race-contingent expression aftereffects suggest that processing of expression is interdependent with processing of sex and race and are therefore problematic for models of face perception that have emphasized independent processing of functionally different aspects of faces. By contrast, our findings are consistent with models of face processing that propose that invariant physical aspects of faces and changeable social cues can be processed interdependently.     

The role of holistic processing in judgments of facial attractiveness

Previous work has demonstrated that facial identity recognition, expression recognition, gender categorisation, and race categorisation rely on a holistic representation. Here we examine whether a holistic representation is also used for judgments of facial attractiveness. Like past studies, we used the composite paradigm to assess holistic processing (Young et al 1987, Perception 16 747-759). Experiment 1 showed that top halves of upright faces are judged to be more attractive when aligned with an attractive bottom half than when aligned with an unattractive bottom half. To assess whether this effect resulted from holistic processing or more general effects, we examined the impact of the attractive and unattractive bottom halves when upright halves were misaligned and when aligned and misaligned halves were presented upside-down. The bottom halves had no effect in either condition. These results demonstrate that the perceptual processes underlying upright facial-attractiveness judgments represent the face holistically. Our findings with attractiveness judgments and previous demonstrations involving other aspects of face processing suggest that a common holistic representation is used for most types of face processing.     

Symmetrical interaction of sex and expression in face classification tasks

Classification of faces as to their sex or their expression—with sex and expression varying orthogonally—was studied in three experiments. In Experiment 1, expression classification was influenced by sex, with angry male faces being classified faster than angry female faces. Complementarily, sex classification was faster for happy than for angry female faces. In Experiment 2, mutual interaction of sex and expression was also found when the participants were asked to classify top and bottom face segments. In Experiment 3, a face inversion effect was found for both sex and expression classification of whole faces. However, a symmetrical interaction between sex and expression was again found. The results are discussed in terms of configural versus feature processing in the perception of face sex and expression and of their relevance to face perception models that postulate independent processing of different facial features. 2009 The Psychonomic Society, Inc.     

动态面孔表情优势效应的心理机制及神经基础

动态面孔表情优势效应是指较静态面孔表情而言, 个体在识别动态的面孔表情时表现出较好的识别能力。动态面孔表情优势效应的心理机制主要涉及增强的构形加工、补偿角色和面孔模仿能力。此外, 该优势效应的神经网络则由核心的神经网络和扩展的神经网络组成, 前者主要负责早期的知觉编码和刺激的运动加工, 而后者与个体的面孔模仿能力、刺激的动态表征等有关。今后的研究应集中在完善延伸的神经网络、拓展其心理机制; 开展动态面孔表情优势效应的发展性研究; 考察面孔表情的刚性运动特征; 注重在虚拟现实环境下研究动态面孔表情的优势效应。     

The relationships between processing facial identity,emotional expression,facial speech,and gaze direction during development

Four experiments were conducted with 5- to 11-year-olds and adults to investigate whether facial identity, facial speech, emotional expression, and gaze direction are processed independently of or in interaction with one another. In a computer-based, speeded sorting task, participants sorted faces according to facial identity while disregarding facial speech, emotional expression, and gaze direction or, alternatively, according to facial speech, emotional expression, and gaze direction while disregarding facial identity. Reaction times showed that children and adults were able to direct their attention selectively to facial identity despite variations of other kinds of face information, but when sorting according to facial speech and emotional expression, they were unable to ignore facial identity. In contrast, gaze direction could be processed independently of facial identity in all age groups. Apart from shorter reaction times and fewer classification errors, no substantial change in processing facial information was found to be correlated with age. We conclude that adult-like face processing routes are employed from 5 years of age onward.     

面部表情识别与面孔身份识别的独立加工与交互作用机制

面孔识别功能模型认为,面部表情识别与面孔身份识别是两条独立的并行路径。以往诸多研究者都认可并遵循二者分离的原则。但近期研究表明,面部表情识别与面孔身份识别存在交互作用。首先总结和分析已有的面部表情识别的研究成果,评述神经心理学与认知神经科学研究中的论争,然后介绍人脸知觉的分布式神经机制以及相关的实验证据,最后提出面部表情识别与面孔身份识别的多级整合模型,并展望研究前景。     

Probing the visual representation of faces with adaptation: A view from the other side of the mean

Sensory adaptation and visual aftereffects have long given insight into the neural codes underlying basic dimensions of visual perception. Recently discovered perceptual adaptation effects for complex shapes like faces can offer similar insight into high-level visual representations. In the experiments reported here, we demonstrated first that face adaptation transfers across a substantial change in viewpoint and that this transfer occurs via processes unlikely to be specific to faces. Next, we probed the visual codes underlying face recognition using face morphs that varied selectively in reflectance or shape. Adaptation to these morphs affected the perception of "opposite" faces both from the same viewpoint and from a different viewpoint. These results are consistent with high-level face representations that pool local shape and reflectance patterns into configurations that specify facial appearance over a range of three-dimensional viewpoints. These findings have implications for computational models of face recognition and for competing neural theories of face and object recognition.     

Asymmetric interference between sex and emotion in face perception

Previous research with speeded-response interference tasks modeled on the Garner paradigm has demonstrated that task-irrelevant variations in either emotional expression or facial speech do not interfere with identity judgments, but irrelevant variations in identity do interfere with expression and facial speech judgments. Sex, like identity, is a relatively invariant aspect of faces. Drawing on a recent model of face processing according to which invariant and changeable aspects of faces are represented in separate neurological systems, we predicted asymmetric interference between sex and emotion classification. The results of Experiment 1, in which the Garner paradigm was employed, confirmed this prediction: Emotion classifications were influenced by the sex of the faces, but sex classifications remained relatively unaffected by facial expression. A second experiment, in which the difficulty of the tasks was equated, corroborated these findings, indicating that differences in processing speed cannot account for the asymmetric relationship between facial emotion and sex processing. A third experiment revealed the same pattern of asymmetric interference through the use of a variant of the Simon paradigm. To the extent that Garner interference and Simon interference indicate interactions at perceptual and response-selection stages of processing, respectively, a challenge for face processing models is to show how the same asymmetric pattern of interference could occur at these different stages. The implications of these findings for the functional independence of the different components of face processing are discussed.     

The development of emotional face processing during childhood

Our facial expressions give others the opportunity to access our feelings, and constitute an important nonverbal tool for communication. Many recent studies have investigated emotional perception in adults, and our knowledge of neural processes involved in emotions is increasingly precise. Young children also use faces to express their internal states and perceive emotions in others, but little is known about the neurodevelopment of expression recognition. The goal of the current study was to determine the normal development of facial emotion perception. We recorded ERPs in 82 children 4 to 15 years of age during an implicit processing task with emotional faces. Task and stimuli were the same as those used and validated in an adult study; we focused on the components that showed sensitivity to emotions in adults (P1, N170 and frontal slow wave). An effect of the emotion expressed by faces was seen on the P1 in the youngest children. With increasing age this effect disappeared while an emotional sensitivity emerged on N170. Early emotional processing in young children differed from that observed in the adolescents, who approached adults. In contrast, the later frontal slow wave, although showing typical age effects, was more positive for neutral and happy faces across age groups. Thus, despite the precocious utilization of facial emotions, the neural processing involved in the perception of emotional faces develops in a staggered fashion throughout childhood, with the adult pattern appearing only late in adolescence.     

情绪概念加工与情绪面孔知觉的相互影响

通过3个实验探讨情绪概念加工与情绪面孔知觉是否存在相互影响, 以及加工深度对两者关系的影响。实验1采用情绪面孔启动范式, 探讨较深的情绪概念加工层面对情绪概念与情绪面孔的关系的影响。实验2更改概念任务, 探讨较浅的概念加工层面对两者关系的影响。实验3缩短启动面孔呈现时间, 探讨较浅的面孔知觉层面对两者关系的影响。通过研究主要得出以下结论：(1)情绪概念加工与情绪面孔知觉确实存在相互影响; (2)概念加工深度影响两者关系的方向性, 在较深的概念加工层面, 两者的关系是双向的, 在较浅的概念加工层面, 两者的关系是单向的; (3)知觉加工深度也影响两者关系的方向性, 在较深的知觉加工层面, 两者的关系是双向的, 在较浅的知觉加工层面, 没有发现两者的相互影响。本研究为抽象概念的具身表征提供了实证支持。     

Asymmetric dependencies in perceiving identity and emotion: experiments with morphed faces.

We investigated whether an asymmetric relationship between the perception of identity and emotional expressions in faces (Schweinberger & Soukup, 1998) may be related to differences in the relative processing speed of identity and expression information. Stimulus faces were morphed across identity within a given emotional expression, or were morphed across emotion within a given identity. In Experiment 1, consistent classifications of these images were demonstrated across a wide range of morphing, with only a relatively narrow category boundary. At the same time, classification reaction times (RTs) reflected the increased perceptual difficulty of the morphed images. In Experiment 2, we investigated the effects of variations in the irrelevant dimension on judgments of faces with respect to a relevant dimension, using a Garner-type speeded classification task. RTs for expression classifications were strongly influenced by irrelevant identity information. In contrast, RTs for identity classifications were unaffected by irrelevant expression information, and this held even for stimuli in which identity was more difficult and slower to discriminate than expression. This suggests that differences in processing speed cannot account for the asymmetric relationship between identity and emotion perception. Theoretical accounts proposing independence of identity and emotion perception are discussed in the light of these findings.     

Neural computation as a tool to differentiate perceptual from emotional processes: The case of anger superiority effect

Research findings in social and cognitive psychology imply that it is easier to detect angry faces than happy faces in a crowd of neutral faces [Hansen, C. H., & Hansen, R. D. (1988). Finding the face in the crowd – An anger superiority effect. Journal of Personality and Social Psychology, 54(6), 917–924]. This phenomenon has been held to have evolved over phylogenetic development because it was adaptive to quickly and accurately detect a potential threat in the environment. However, across recent studies, a controversy has emerged about the underlying perceptual versus emotional factors responsible for this so-called anger superiority effect [Juth, P., Lundqvist, D., Karlsson, A., & Ohman, A. (2005). Looking for foes and friends: Perceptual and emotional factors when finding a face in the crowd. Emotion, 5(4), 379–395; Purcell, D. G., Stewart, A. L., & Skov, R. B. (1996). It takes a confounded face to pop out of a crowd. Perception, 25(9), 1091–1108]. To tease apart emotional and perceptual processes, we used neural network analyzes of human faces in two different simulations. Results show that a perceptual bias is probably acting against faster and more accurate identification of anger faces compared to happy faces at a purely perceptual level. We suggest that a parsimonious hypothesis related to the simple perceptual properties of the stimuli might explain these behavioral results without reference to evolutionary processes. We discuss the importance of statistical or connectionist analysis for empirical studies that seek to isolate perceptual from emotional factors, but also learned vs. innate factors in the processing of facial expression of emotion.     

Adaptation effects in facial expression recognition

The effect of adaptation on facial expression recognition was investigated by measuring how identification performance of test stimuli falling along a particular expression continuum was affected after adapting to various prototype emotional faces or a control pattern. The results showed that for recognition of fear, happiness, and sadness, inhibition effects were observed on recognition of test expressions following 5 s adaptation to the same emotion, suggesting different neural populations tuned for the encoding of fearful, happy, and sad expressions. Facilitation of recognition of test stimuli differing in emotion to the adapting stimulus was also sometimes observed. The nature of these adaptation effects was investigated by introducing a size transformation or a delay between adapting and test stimuli and was found to survive these changes. The results of a further experiment argued against a criterion effect being the major source by demonstrating the importance of adapting time in generating the effects. Overall, the present study demonstrates the utility of adaptation effects for revealing functional characteristics of facial expression processing.     

Judgment of other people's facial expressions of emotions is influenced by their concurrent affective hand movements

We investigated whether emotional information from facial expression and hand movement quality was integrated when identifying the expression of a compound stimulus showing a static facial expression combined with emotionally expressive dynamic manual actions. The emotions (happiness, neutrality, and anger) expressed by the face and hands were either congruent or incongruent. In Experiment 1, the participants judged whether the stimulus person was happy, neutral, or angry. Judgments were mainly based on the facial expressions, but were affected by manual expressions to some extent. In Experiment 2, the participants were instructed to base their judgment on the facial expression only. An effect of hand movement expressive quality was observed for happy facial expressions. The results conform with the proposal that perception of facial expressions of emotions can be affected by the expressive qualities of hand movements.     

Perceived differences between chimpanzee (Pan troglodytes) and human (Homo sapiens) facial expressions are related to emotional interpretation

Human face perception is a finely tuned, specialized process. When comparing faces between species, therefore, it is essential to consider how people make these observational judgments. Comparing facial expressions may be particularly problematic, given that people tend to consider them categorically as emotional signals, which may affect how accurately specific details are processed. The bared-teeth display (BT), observed in most primates, has been proposed as a homologue of the human smile (J. A. R. A. M. van Hooff, 1972). In this study, judgments of similarity between BT displays of chimpanzees (Pan troglodytes) and human smiles varied in relation to perceived emotional valence. When a chimpanzee BT was interpreted as fearful, observers tended to underestimate the magnitude of the relationship between certain features (the extent of lip corner raise) and human smiles. These judgments may reflect the combined effects of categorical emotional perception, configural face processing, and perceptual organization in mental imagery and may demonstrate the advantages of using standardized observational methods in comparative facial expression research.     

Mutual information, perceptual independence, and holistic face perception

The concept of perceptual independence is ubiquitous in psychology. It addresses the question of whether two (or more) dimensions are perceived independently. Several authors have proposed perceptual independence (or its lack thereof) as a viable measure of holistic face perception (Loftus, Oberg, & Dillon, Psychological Review 111:835–863, 2004; Wenger & Ingvalson, Learning, Memory, and Cognition 28:872–892, 2002). According to this notion, the processing of facial features occurs in an interactive manner. Here, I examine this idea from the perspective of two theories of perceptual independence: the multivariate uncertainty analysis (MUA; Garner & Morton, Definitions, models, and experimental paradigms. Psychological Bulletin 72:233–259, 1969), and the general recognition theory (GRT; Ashby & Townsend, Psychological Review 93:154–179, 1986). The goals of the study were to (1) introduce the MUA, (2) examine various possible relations between MUA and GRT using numerical simulations, and (3) apply the MUA to two consensual markers of holistic face perception^—recognition of facial features (Farah, Wilson, Drain, & Tanaka, Psychological Review 105:482–498, 1998) and the composite face effect (Young, Hellawell, & Hay, Perception 16:747–759, 1987). The results suggest that facial holism is generated by violations of several types of perceptual independence. They highlight the important theoretical role played by converging operations in the study of holistic face perception.     

Asymmetric dependencies in perceiving identity and emotion: Experiments with morphed faces

We investigated whether an asymmetric relationship between the perception of identity and emo-tional expressions in faces (Schweinberger & Soukup, 1998) may be related to differences in the rela-tive processing speed of identity and expression information. Stimulus faces were morphed across identity within a given emotional expression, or were morphed across emotion within a given identity. In Experiment 1, consistent classifications of these images were demonstrated across a wide range of morphing, with only a relatively narrow category boundary. At the same time, classification reaction times (RTs) reflected the increased perceptual difficulty of the morphed images. In Experiment 2, we investigated the effects of variations in the irrelevant dimension on judgments of faces with respect to a relevant dimension, using a Garner-type speeded classification task. RTs for expression classifica-tions were strongly influenced by irrelevant identity information. In contrast, RTs for identity classifi-cations were unaffected by irrelevant expression information, and this held even for stimuli in which identity was more difficult and slower to discriminate than expression. This suggests that differences in processing speed cannot account for the asymmetric relationship between identity and emotion per-ception. Theoretical accounts proposing independence of identity and emotion perception are dis-cussed in the light of these findings.