Evidence and a computational explanation of cultural differences in facial expression recognition

Facial expressions are crucial to human social communication, but the extent to which they are innate and universal versus learned and culture dependent is a subject of debate. Two studies explored the effect of culture and learning on facial expression understanding. In Experiment 1, Japanese and U.S. participants interpreted facial expressions of emotion. Each group was better than the other at classifying facial expressions posed by members of the same culture. In Experiment 2, this reciprocal in-group advantage was reproduced by a neurocomputational model trained in either a Japanese cultural context or an American cultural context. The model demonstrates how each of us, interacting with others in a particular cultural context, learns to recognize a culture-specific facial expression dialect.     

Universals and cultural differences in the judgments of facial expressions of emotion

We present here new evidence of cross-cultural agreement in the judgement of facial expression. Subjects in 10 cultures performed a more complex judgment task than has been used in previous cross-cultural studies. Instead of limiting the subjects to selecting only one emotion term for each expression, this task allowed them to indicate that multiple emotions were evident and the intensity of each emotion. Agreement was very high across cultures about which emotion was the most intense. The 10 cultures also agreed about the second most intense emotion signaled by an expression and about the relative intensity among expressions of the same emotion. However, cultural differences were found in judgments of the absolute level of emotional intensity.     

Nonverbal "accents": cultural differences in facial expressions of emotion

We report evidence for nonverbal "accents," subtle differences in the appearance of facial expressions of emotion across cultures. Participants viewed photographs of Japanese nationals and Japanese Americans in which posers' muscle movements were standardized to eliminate differences in expressions, cultural or otherwise. Participants guessed the nationality of posers displaying emotional expressions at above-chance levels, and with greater accuracy than they judged the nationality of the same posers displaying neutral expressions. These findings indicate that facial expressions of emotion can contain nonverbal accents that identify the expresser's nationality or culture. Cultural differences are intensified during the act of expressing emotion, rather than residing only in facial features or other static elements of appearance. This evidence suggests that extreme positions regarding the universality of emotional expressions are incomplete.     

Differences in holistic processing do not explain cultural differences in the recognition of facial expression

The aim of this study was to investigate the causes of the own-race advantage in facial expression perception. In Experiment 1, we investigated Western Caucasian and Chinese participants’ perception and categorization of facial expressions of six basic emotions that included two pairs of confusable expressions (fear and surprise; anger and disgust). People were slightly better at identifying facial expressions posed by own-race members (mainly in anger and disgust). In Experiment 2, we asked whether the own-race advantage was due to differences in the holistic processing of facial expressions. Participants viewed composite faces in which the upper part of one expression was combined with the lower part of a different expression. The upper and lower parts of the composite faces were either aligned or misaligned. Both Chinese and Caucasian participants were better at identifying the facial expressions from the misaligned images, showing interference on recognizing the parts of the expressions created by holistic perception of the aligned composite images. However, this interference from holistic processing was equivalent across expressions of own-race and other-race faces in both groups of participants. Whilst the own-race advantage in recognizing facial expressions does seem to reflect the confusability of certain emotions, it cannot be explained by differences in holistic processing.     

American-Japanese cultural differences in intensity ratings of facial expressions of emotion

Findings from a recent study by Ekmanet al. (1987) provided evidence for cultural disagreement about the intensity ratings of universal facial expressions of emotion. We conducted a study that examined the basis of these cultural differences. Japanese and American subjects made two separate intensity ratings of Japanese and Caucacian posers portraying anger, disgust, fear, happiness, sadness, and surprise. The Americans had higher mean intensity ratings than the Japanese for all emotions except disgust, regardless of the culture or gender of the poser. Americans gave happy and angry photos the highest intensity ratings, while Japanese gave disgust photos the highest ratings. But there was considerable cross-cultural consistency in the relative differences among photos.This study was conducted as part of a doctoral dissertation by the first author, under the supervision of the second author. David Matsumoto was supported by a minority fellowship from the American Psychological Association, under a Clinical Training Grant from NIMH (MH13833), and by a Regents Fellowship from the University of California, Berkeley. Paul Ekman was supported by a Research Scientist Award from NIMH (MH06092).We gratefully acknowledge the contribution of Wallace V. Friesen to the development of the facial stimuli used in this study, Tsutomu Kudoh for his aid in collecting the Japanese data, and Wallace Friesen and Maureen O'Sullivan for their aid during data analysis.     

Age differences in recognition of emotion in lexical stimuli and facial expressions

Age differences in emotion recognition from lexical stimuli and facial expressions were examined in a cross-sectional sample of adults aged 18 to 85 (N = 357). Emotion-specific response biases differed by age: Older adults were disproportionately more likely to incorrectly label lexical stimuli as happiness, sadness, and surprise and to incorrectly label facial stimuli as disgust and fear. After these biases were controlled, findings suggested that older adults were less accurate at identifying emotions than were young adults, but the pattern differed across emotions and task types. The lexical task showed stronger age differences than the facial task, and for lexical stimuli, age groups differed in accuracy for all emotional states except fear. For facial stimuli, in contrast, age groups differed only in accuracy for anger, disgust, fear, and happiness. Implications for age-related changes in different types of emotional processing are discussed.     

Disturbance of spontaneous and posed facial expressions in Parkinson's disease

The effects of Parkinson's disease (PD) on spontaneous and posed facial activity and on the control of facial muscles were assessed by comparing 22 PD patients with 22 controls. Facial activity was analysed using the Facial Action Coding System (FACS; Ekman & Friesen, 1978). As predicted, PD patients showed reduced levels of spontaneous and posed facial expression in reaction to unpleasant odours compared to controls. PD patients were less successful than controls in masking or intensifying negative facial expressions. PD patients were also less able than controls to imitate specific facial muscle movements, but did not differ in the ability to pose emotional facial expressions. These results suggest that not only is spontaneous facial activity disturbed in PD, but also to some degree the ability to pose facial expressions, to mask facial expressions with other expressions, and to deliberately move specific muscles in the face.     

Repeated short presentations of morphed facial expressions change recognition and evaluation of facial expressions

This study investigated whether sensitivity to and evaluation of facial expressions varied with repeated exposure to non-prototypical facial expressions for a short presentation time. A morphed facial expression was presented for 500 ms repeatedly, and participants were required to indicate whether each facial expression was happy or angry. We manipulated the distribution of presentations of the morphed facial expressions for each facial stimulus. Some of the individuals depicted in the facial stimuli expressed anger frequently (i.e., anger-prone individuals), while the others expressed happiness frequently (i.e., happiness-prone individuals). After being exposed to the faces of anger-prone individuals, the participants became less sensitive to those individuals’ angry faces. Further, after being exposed to the faces of happiness-prone individuals, the participants became less sensitive to those individuals’ happy faces. We also found a relative increase in the social desirability of happiness-prone individuals after exposure to the facial stimuli.     

Recognition of posed and genuine facial expressions of emotion in paranoid and nonparanoid schizophrenia

Most previous research reporting emotion-recognition deficits in schizophrenia has used posed facial expressions of emotion and chronic-schizophrenia patients. In contrast, the present research examined the ability of patients with acute paranoid and nonparanoid (disorganized) schizophrenia to recognize genuine as well as posed facial expressions of emotion. Evidence of an emotion-recognition deficit in schizophrenia was replicated, but only when posed facial expressions were used. For genuine expressions of emotion, the paranoid-schizophrenia group was more accurate than controls, nonparanoid-schizophrenia patients, and depressed patients. Future research clearly needs to consider the posed versus genuine nature of the emotional stimuli used and the type of schizophrenia patients examined.     

The cues decoders use in attempting to differentiate emotion-elicited and posed facial expressions

Dynamic facial expressions, either posed or elicited by afectively evocative materials, were objectively scored to determine the movement cues and temporal parameters associated with the two types of expression. Subjects viewed these expressive episodes and rated each of them on a number of scales intended to assess perceived spontaneousness and deliberateness. Subsequent to viewing all stimuli, subjects reported the specfic cues that they felt they had used to discriminate spontaneous from deliberate expressions. The results reveal that (a) subjects were able to accurately report the cues they employed in the rating task and that (b) these cues were not always valid discriminators of posed and spontaneous expressions. Subjects were in fact relatively poor at identifying expressions of the two types and this low discrimination accuracy was found to be a function of the consistent use of these invalid cues. A measure of the level of perceived honest demeanour of the stimulus persons based on their neutral expressions was found to relate to perceivers accuracy in discriminating posed and spontaneous expressions.     

Cultural relativism and the recognition of cultural differences

The demand for the recognition of cultural differences is central to a number of debates associated with multiculturalism. Following Charles Taylor's analysis of the relation between modernity and cultural pluralism, it is argued that the demand of cultural relativism, namely, that the equal value of cultures should be recognised, is not justifiable. This however should not serve as an excuse for underestimating the significance of cultural differences or for ethnocentric indifference towards the claim for recognition. The prerequisites for claims towards recognition are further explored by distinguishing between two justifiable claims: on the one hand the claim that the right to differ should be recognised and on the other hand the claim that the inherent value of the difference should be recognised. It is argued that the possibilities of granting recognition are in most cases restricted to the first claim. Although the second claim may also be justified, it is in most cases not possible to meet it. The conclusion is that we here encounter an aporetic ground for both cultural critique and intercultural tolerance; in fact a better ground for tolerance than cultural relativism which easily leads to indifference.     

Recognition of emotional facial expressions: the role of facial and contextual information in the accuracy of recognition

Recognition of emotional facial expressions is a central area in the psychology of emotion. This study presents two experiments. The first experiment analyzed recognition accuracy for basic emotions including happiness, anger, fear, sadness, surprise, and disgust. 30 pictures (5 for each emotion) were displayed to 96 participants to assess recognition accuracy. The results showed that recognition accuracy varied significantly across emotions. The second experiment analyzed the effects of contextual information on recognition accuracy. Information congruent and not congruent with a facial expression was displayed before presenting pictures of facial expressions. The results of the second experiment showed that congruent information improved facial expression recognition, whereas incongruent information impaired such recognition.     

False recognition of facial expressions of emotion: causes and implications

This article examines the importance of semantic processes in the recognition of emotional expressions, through a series of three studies on false recognition. The first study found a high frequency of false recognition of prototypical expressions of emotion when participants viewed slides and video clips of nonprototypical fearful and happy expressions. The second study tested whether semantic processes caused false recognition. The authors found that participants made significantly higher error rates when asked to detect expressions that corresponded to semantic labels than when asked to detect visual stimuli. Finally, given that previous research reported that false memories are less prevalent in younger children, the third study tested whether false recognition of prototypical expressions increased with age. The authors found that 67% of eight- to nine-year-old children reported nonpresent prototypical expressions of fear in a fearful context, but only 40% of 6- to 7-year-old children did so. Taken together, these three studies demonstrate the importance of semantic processes in the detection and categorization of prototypical emotional expressions.     

Sex differences in attention to disgust facial expressions

Research demonstrates that women experience disgust more readily and with more intensity than men. The experience of disgust is associated with increased attention to disgust-related stimuli, but no prior study has examined sex differences in attention to disgust facial expressions. We hypothesised that women, compared to men, would demonstrate increased attention to disgust facial expressions. Participants (n?=?172) completed an eye tracking task to measure visual attention to emotional facial expressions. Results indicated that women spent more time attending to disgust facial expressions compared to men. Unexpectedly, we found that men spent significantly more time attending to neutral faces compared to women. The findings indicate that women’s increased experience of emotional disgust also extends to attention to disgust facial stimuli. These findings may help to explain sex differences in the experience of disgust and in diagnoses of anxiety disorders in which disgust plays an important role.     

Production of spontaneous and posed facial expressions in patients with Huntington's disease: Impaired communication of disgust

Several studies have reported impairment in the recognition of facial expressions of disgust in patients with Huntington's disease (HD) and preclinical carriers of the HD gene. The aim of this study was to establish whether impairment for disgust in HD patients extended to include the ability to express the emotion on their own faces. Eleven patients with HD, and 11 age and education matched healthy controls participated in three tasks concerned with the expression of emotions. One task assessed the spontaneous production of disgust-like facial expressions during the smelling of offensive odorants. A second assessed the production of posed facial expressions during deliberate attempts to communicate emotion. The third task evaluated HD patients’ ability to imitate the specific facial configurations associated with each emotion. Foul odours induced fewer disgust-like facial reactions in HD patients than in controls, and patients’ posed facial expressions of disgust were less accurate than the posed disgust expressions of controls. The effect was selective to disgust; patients had no difficulty posing expressions of other emotions. These impairments were not explained by compromised muscle control: HD patients had no difficulty imitating the facial movements required to display disgust. Viewed together with evidence of difficulty in other aspects of disgust in HD, the findings suggest that a common substrate might participate in both the processing and the expression of this emotion.     

Age-related decrease in recognition of emotional facial and prosodic expressions

The recognition of nonverbal emotional signals and the integration of multimodal emotional information are essential for successful social communication among humans of any age. Whereas prior studies of age dependency in the recognition of emotion often focused on either the prosodic or the facial aspect of nonverbal signals, our purpose was to create a more naturalistic setting by presenting dynamic stimuli under three experimental conditions: auditory, visual, and audiovisual. Eighty-four healthy participants (women = 44, men = 40; age range 20-70 years) were tested for their abilities to recognize emotions either mono- or bimodally on the basis of emotional (happy, alluring, angry, disgusted) and neutral nonverbal stimuli from voice and face. Additionally, we assessed visual and auditory acuity, working memory, verbal intelligence, and emotional intelligence to explore potential explanatory effects of these population parameters on the relationship between age and emotion recognition. Applying unbiased hit rates as performance measure, we analyzed data with linear regression analyses, t tests, and with mediation analyses. We found a linear, age-related decrease in emotion recognition independent of stimulus modality and emotional category. In contrast, the improvement in recognition rates associated with audiovisual integration of bimodal stimuli seems to be maintained over the life span. The reduction in emotion recognition ability at an older age could not be sufficiently explained by age-related decreases in hearing, vision, working memory, and verbal intelligence. These findings suggest alterations in social perception at a level of complexity beyond basic perceptional and cognitive abilities.     

Cerebral asymmetry in autonomic conditioning to facial expressions: Sex differences

In the present experiment, sex differences in hemispheric asymmetry during classical conditioning to emotional stimuli are reported. 125 subjects (62 females and 63 males) were shown a slide of a happy face in the right (or left) visual half field (VHF), and simultaneously a slide of an angry face in the left (or right) VHF. Eight groups were formed by the combination of male and female subjects; left and right VHF positions of the angry/happy faces; and the administration/omission of the shock unconditioned stimulus (UCS). Dependent measures were skin conductance responses recorded from both hands. The results during extinction showed a significant larger SCR magnitude to the shock compared to the no-shock groups only for the female subjects. CS position during conditioning was also important in revealing differential responding to either the happy or angry faces. A right hemisphere effect was found for the angry face CS for both the male and female subjects, however with a greater difference for the females.     

Sex differences in asymmetrically perceiving the intensity of facial expressions.

Emotional facial expressions are often asymmetrical, with the left half of the face typically displaying the stronger affective intensity cues. During facial perception, however, most right-handed individuals are biased toward facial affect cues projecting to their own left visual hemifield. Consequently, mirror-reversed faces are typically rated as more emotionally intense than when presented normally. Mirror-reversal permits the most intense side of the expresser's face to project to the visual hemifield biased for processing facial affect cues. This study replicated the mirror-reversal effect in 21 men and 49 women (aged 18-52 yr.) using a videotaped free viewing presentation but also showed the effect of facial orientation is moderated by the sex of the perceiver. The mirror-reversal effect was significant only for men but not for women, suggesting possible sex differences in cerebral organization of systems for facial perception.