Evidence and a computational explanation of cultural differences in facial expression recognition

Facial expressions are crucial to human social communication, but the extent to which they are innate and universal versus learned and culture dependent is a subject of debate. Two studies explored the effect of culture and learning on facial expression understanding. In Experiment 1, Japanese and U.S. participants interpreted facial expressions of emotion. Each group was better than the other at classifying facial expressions posed by members of the same culture. In Experiment 2, this reciprocal in-group advantage was reproduced by a neurocomputational model trained in either a Japanese cultural context or an American cultural context. The model demonstrates how each of us, interacting with others in a particular cultural context, learns to recognize a culture-specific facial expression dialect.     

Universals and cultural differences in the judgments of facial expressions of emotion

We present here new evidence of cross-cultural agreement in the judgement of facial expression. Subjects in 10 cultures performed a more complex judgment task than has been used in previous cross-cultural studies. Instead of limiting the subjects to selecting only one emotion term for each expression, this task allowed them to indicate that multiple emotions were evident and the intensity of each emotion. Agreement was very high across cultures about which emotion was the most intense. The 10 cultures also agreed about the second most intense emotion signaled by an expression and about the relative intensity among expressions of the same emotion. However, cultural differences were found in judgments of the absolute level of emotional intensity.     

Nonverbal "accents": cultural differences in facial expressions of emotion

We report evidence for nonverbal "accents," subtle differences in the appearance of facial expressions of emotion across cultures. Participants viewed photographs of Japanese nationals and Japanese Americans in which posers' muscle movements were standardized to eliminate differences in expressions, cultural or otherwise. Participants guessed the nationality of posers displaying emotional expressions at above-chance levels, and with greater accuracy than they judged the nationality of the same posers displaying neutral expressions. These findings indicate that facial expressions of emotion can contain nonverbal accents that identify the expresser's nationality or culture. Cultural differences are intensified during the act of expressing emotion, rather than residing only in facial features or other static elements of appearance. This evidence suggests that extreme positions regarding the universality of emotional expressions are incomplete.     

American-Japanese cultural differences in intensity ratings of facial expressions of emotion

Findings from a recent study by Ekmanet al. (1987) provided evidence for cultural disagreement about the intensity ratings of universal facial expressions of emotion. We conducted a study that examined the basis of these cultural differences. Japanese and American subjects made two separate intensity ratings of Japanese and Caucacian posers portraying anger, disgust, fear, happiness, sadness, and surprise. The Americans had higher mean intensity ratings than the Japanese for all emotions except disgust, regardless of the culture or gender of the poser. Americans gave happy and angry photos the highest intensity ratings, while Japanese gave disgust photos the highest ratings. But there was considerable cross-cultural consistency in the relative differences among photos.This study was conducted as part of a doctoral dissertation by the first author, under the supervision of the second author. David Matsumoto was supported by a minority fellowship from the American Psychological Association, under a Clinical Training Grant from NIMH (MH13833), and by a Regents Fellowship from the University of California, Berkeley. Paul Ekman was supported by a Research Scientist Award from NIMH (MH06092).We gratefully acknowledge the contribution of Wallace V. Friesen to the development of the facial stimuli used in this study, Tsutomu Kudoh for his aid in collecting the Japanese data, and Wallace Friesen and Maureen O'Sullivan for their aid during data analysis.     

Age differences in recognition of emotion in lexical stimuli and facial expressions

Age differences in emotion recognition from lexical stimuli and facial expressions were examined in a cross-sectional sample of adults aged 18 to 85 (N = 357). Emotion-specific response biases differed by age: Older adults were disproportionately more likely to incorrectly label lexical stimuli as happiness, sadness, and surprise and to incorrectly label facial stimuli as disgust and fear. After these biases were controlled, findings suggested that older adults were less accurate at identifying emotions than were young adults, but the pattern differed across emotions and task types. The lexical task showed stronger age differences than the facial task, and for lexical stimuli, age groups differed in accuracy for all emotional states except fear. For facial stimuli, in contrast, age groups differed only in accuracy for anger, disgust, fear, and happiness. Implications for age-related changes in different types of emotional processing are discussed.     

Repeated short presentations of morphed facial expressions change recognition and evaluation of facial expressions

This study investigated whether sensitivity to and evaluation of facial expressions varied with repeated exposure to non-prototypical facial expressions for a short presentation time. A morphed facial expression was presented for 500 ms repeatedly, and participants were required to indicate whether each facial expression was happy or angry. We manipulated the distribution of presentations of the morphed facial expressions for each facial stimulus. Some of the individuals depicted in the facial stimuli expressed anger frequently (i.e., anger-prone individuals), while the others expressed happiness frequently (i.e., happiness-prone individuals). After being exposed to the faces of anger-prone individuals, the participants became less sensitive to those individuals’ angry faces. Further, after being exposed to the faces of happiness-prone individuals, the participants became less sensitive to those individuals’ happy faces. We also found a relative increase in the social desirability of happiness-prone individuals after exposure to the facial stimuli.     

Recognition of posed and genuine facial expressions of emotion in paranoid and nonparanoid schizophrenia

Most previous research reporting emotion-recognition deficits in schizophrenia has used posed facial expressions of emotion and chronic-schizophrenia patients. In contrast, the present research examined the ability of patients with acute paranoid and nonparanoid (disorganized) schizophrenia to recognize genuine as well as posed facial expressions of emotion. Evidence of an emotion-recognition deficit in schizophrenia was replicated, but only when posed facial expressions were used. For genuine expressions of emotion, the paranoid-schizophrenia group was more accurate than controls, nonparanoid-schizophrenia patients, and depressed patients. Future research clearly needs to consider the posed versus genuine nature of the emotional stimuli used and the type of schizophrenia patients examined.     

The cues decoders use in attempting to differentiate emotion-elicited and posed facial expressions

Dynamic facial expressions, either posed or elicited by afectively evocative materials, were objectively scored to determine the movement cues and temporal parameters associated with the two types of expression. Subjects viewed these expressive episodes and rated each of them on a number of scales intended to assess perceived spontaneousness and deliberateness. Subsequent to viewing all stimuli, subjects reported the specfic cues that they felt they had used to discriminate spontaneous from deliberate expressions. The results reveal that (a) subjects were able to accurately report the cues they employed in the rating task and that (b) these cues were not always valid discriminators of posed and spontaneous expressions. Subjects were in fact relatively poor at identifying expressions of the two types and this low discrimination accuracy was found to be a function of the consistent use of these invalid cues. A measure of the level of perceived honest demeanour of the stimulus persons based on their neutral expressions was found to relate to perceivers accuracy in discriminating posed and spontaneous expressions.     

Recognition of emotional facial expressions: the role of facial and contextual information in the accuracy of recognition

Recognition of emotional facial expressions is a central area in the psychology of emotion. This study presents two experiments. The first experiment analyzed recognition accuracy for basic emotions including happiness, anger, fear, sadness, surprise, and disgust. 30 pictures (5 for each emotion) were displayed to 96 participants to assess recognition accuracy. The results showed that recognition accuracy varied significantly across emotions. The second experiment analyzed the effects of contextual information on recognition accuracy. Information congruent and not congruent with a facial expression was displayed before presenting pictures of facial expressions. The results of the second experiment showed that congruent information improved facial expression recognition, whereas incongruent information impaired such recognition.     

False recognition of facial expressions of emotion: causes and implications

This article examines the importance of semantic processes in the recognition of emotional expressions, through a series of three studies on false recognition. The first study found a high frequency of false recognition of prototypical expressions of emotion when participants viewed slides and video clips of nonprototypical fearful and happy expressions. The second study tested whether semantic processes caused false recognition. The authors found that participants made significantly higher error rates when asked to detect expressions that corresponded to semantic labels than when asked to detect visual stimuli. Finally, given that previous research reported that false memories are less prevalent in younger children, the third study tested whether false recognition of prototypical expressions increased with age. The authors found that 67% of eight- to nine-year-old children reported nonpresent prototypical expressions of fear in a fearful context, but only 40% of 6- to 7-year-old children did so. Taken together, these three studies demonstrate the importance of semantic processes in the detection and categorization of prototypical emotional expressions.     

Age-related decrease in recognition of emotional facial and prosodic expressions

The recognition of nonverbal emotional signals and the integration of multimodal emotional information are essential for successful social communication among humans of any age. Whereas prior studies of age dependency in the recognition of emotion often focused on either the prosodic or the facial aspect of nonverbal signals, our purpose was to create a more naturalistic setting by presenting dynamic stimuli under three experimental conditions: auditory, visual, and audiovisual. Eighty-four healthy participants (women = 44, men = 40; age range 20-70 years) were tested for their abilities to recognize emotions either mono- or bimodally on the basis of emotional (happy, alluring, angry, disgusted) and neutral nonverbal stimuli from voice and face. Additionally, we assessed visual and auditory acuity, working memory, verbal intelligence, and emotional intelligence to explore potential explanatory effects of these population parameters on the relationship between age and emotion recognition. Applying unbiased hit rates as performance measure, we analyzed data with linear regression analyses, t tests, and with mediation analyses. We found a linear, age-related decrease in emotion recognition independent of stimulus modality and emotional category. In contrast, the improvement in recognition rates associated with audiovisual integration of bimodal stimuli seems to be maintained over the life span. The reduction in emotion recognition ability at an older age could not be sufficiently explained by age-related decreases in hearing, vision, working memory, and verbal intelligence. These findings suggest alterations in social perception at a level of complexity beyond basic perceptional and cognitive abilities.     

Cerebral asymmetry in autonomic conditioning to facial expressions: Sex differences

In the present experiment, sex differences in hemispheric asymmetry during classical conditioning to emotional stimuli are reported. 125 subjects (62 females and 63 males) were shown a slide of a happy face in the right (or left) visual half field (VHF), and simultaneously a slide of an angry face in the left (or right) VHF. Eight groups were formed by the combination of male and female subjects; left and right VHF positions of the angry/happy faces; and the administration/omission of the shock unconditioned stimulus (UCS). Dependent measures were skin conductance responses recorded from both hands. The results during extinction showed a significant larger SCR magnitude to the shock compared to the no-shock groups only for the female subjects. CS position during conditioning was also important in revealing differential responding to either the happy or angry faces. A right hemisphere effect was found for the angry face CS for both the male and female subjects, however with a greater difference for the females.     

Sex differences in asymmetrically perceiving the intensity of facial expressions.

Emotional facial expressions are often asymmetrical, with the left half of the face typically displaying the stronger affective intensity cues. During facial perception, however, most right-handed individuals are biased toward facial affect cues projecting to their own left visual hemifield. Consequently, mirror-reversed faces are typically rated as more emotionally intense than when presented normally. Mirror-reversal permits the most intense side of the expresser's face to project to the visual hemifield biased for processing facial affect cues. This study replicated the mirror-reversal effect in 21 men and 49 women (aged 18-52 yr.) using a videotaped free viewing presentation but also showed the effect of facial orientation is moderated by the sex of the perceiver. The mirror-reversal effect was significant only for men but not for women, suggesting possible sex differences in cerebral organization of systems for facial perception.     

When familiarity breeds accuracy: cultural exposure and facial emotion recognition

Two studies provide evidence for the role of cultural familiarity in recognizing facial expressions of emotion. For Chinese located in China and the United States, Chinese Americans, and non-Asian Americans, accuracy and speed in judging Chinese and American emotions was greater with greater participant exposure to the group posing the expressions. Likewise, Tibetans residing in China and Africans residing in the United States were faster and more accurate when judging emotions expressed by host versus nonhost society members. These effects extended across generations of Chinese Americans, seemingly independent of ethnic or biological ties. Results suggest that the universal affect system governing emotional expression may be characterized by subtle differences in style across cultures, which become more familiar with greater cultural contact.     

Placing the face in context: cultural differences in the perception of facial emotion

Two studies tested the hypothesis that in judging people's emotions from their facial expressions, Japanese, more than Westerners, incorporate information from the social context. In Study 1, participants viewed cartoons depicting a happy, sad, angry, or neutral person surrounded by other people expressing the same emotion as the central person or a different one. The surrounding people's emotions influenced Japanese but not Westerners' perceptions of the central person. These differences reflect differences in attention, as indicated by eye-tracking data (Study 2): Japanese looked at the surrounding people more than did Westerners. Previous findings on East-West differences in contextual sensitivity generalize to social contexts, suggesting that Westerners see emotions as individual feelings, whereas Japanese see them as inseparable from the feelings of the group.     

Gender differences in judgments of multiple emotions from facial expressions

The authors tested gender differences in emotion judgments by utilizing a new judgment task (Studies 1 and 2) and presenting stimuli at the edge of conscious awareness (Study 2). Women were more accurate than men even under conditions of minimal stimulus information. Women's ratings were more variable across scales, and they rated correct target emotions higher than did men.     

The facial muscle patterning of posed and imagery-induced expressions of emotion by expressive and nonexpressive posers

Much research on emotional facial expression employs posed expressions and expressive subjects. To test the generalizability of this research to more spontaneous expressions of both expressive and nonexpressive posers, subjects engaged in happy, sad, angry, and neutral imagery, and voluntarily posed happy, sad, and angry facial expressions while facial muscle activity (brow, cheek, and mouth regions) and autonomic activity (skin resistance and heart period) were recorded. Subjects were classified as expressive or nonexpressive on the basis of the intensity of their posed expressions. The posed and imagery-induced expressions were similar, but not identical. Brow activity present in the imagery-induced sad expressions was weak or absent in the posed ones. Both nonexpressive and expressive subjects demonstrated similar heart rate acceleration during emotional imagery and demonstrated similar posed and imagery-induced happy expressions, but nonexpressive subjects showed little facial activity during both their posed and imagery-induced sad and angry expressions. The implications of these findings are discussed.     

Internal representations reveal cultural diversity in expectations of facial expressions of emotion

Facial expressions have long been considered the "universal language of emotion." Yet consistent cultural differences in the recognition of facial expressions contradict such notions (e.g., R. E. Jack, C. Blais, C. Scheepers, P. G. Schyns, & R. Caldara, 2009). Rather, culture--as an intricate system of social concepts and beliefs--could generate different expectations (i.e., internal representations) of facial expression signals. To investigate, they used a powerful psychophysical technique (reverse correlation) to estimate the observer-specific internal representations of the 6 basic facial expressions of emotion (i.e., happy, surprise, fear, disgust, anger, and sad) in two culturally distinct groups (i.e., Western Caucasian [WC] and East Asian [EA]). Using complementary statistical image analyses, cultural specificity was directly revealed in these representations. Specifically, whereas WC internal representations predominantly featured the eyebrows and mouth, EA internal representations showed a preference for expressive information in the eye region. Closer inspection of the EA observer preference revealed a surprising feature: changes of gaze direction, shown primarily among the EA group. For the first time, it is revealed directly that culture can finely shape the internal representations of common facial expressions of emotion, challenging notions of a biologically hardwired "universal language of emotion."