Categorical perception of affective and linguistic facial expressions

Two experiments investigated categorical perception (CP) effects for affective facial expressions and linguistic facial expressions from American Sign Language (ASL) for Deaf native signers and hearing non-signers. Facial expressions were presented in isolation (Experiment 1) or in an ASL verb context (Experiment 2). Participants performed ABX discrimination and identification tasks on morphed affective and linguistic facial expression continua. The continua were created by morphing end-point photo exemplars into 11 images, changing linearly from one expression to another in equal steps. For both affective and linguistic expressions, hearing non-signers exhibited better discrimination across category boundaries than within categories for both experiments, thus replicating previous results with affective expressions and demonstrating CP effects for non-canonical facial expressions. Deaf signers, however, showed significant CP effects only for linguistic facial expressions. Subsequent analyses indicated that order of presentation influenced signers’ response time performance for affective facial expressions: viewing linguistic facial expressions first slowed response time for affective facial expressions. We conclude that CP effects for affective facial expressions can be influenced by language experience.     

Children's understanding of facial expression of emotion: I. Voluntary creation of emotion-faces

22 children (ages 4 to 6 yr.) from a day-care service were asked to "make a face" that would show how they would feel in five situations representing the basic emotions of happiness, sadness, disgust, anger, and fear. Children decoded their own videotaped responses one week later, and they also decoded the same expressions presented by a child whom they did not know. Groups of day-care teachers and university students were employed in decoding the children's facial responses. Recognizability of all emotions by all decoders improved with the age of the child in a linear manner (9% gain per year). Happy and disgusted expressions were the most easily decoded.     

The Matching of Facial Expressions by Deaf and Hearing Children and Their Production and Comprehension of Emotion Labels

This paper reports the results of three tasks comparing the development of the understanding of facial expressions of emotion in deaf and hearing children. Two groups of hearing and deaf children of elementary school age were tested for their ability to match photographs of facial expressions of emotion, and to produce and comprehend emotion labels for the expressions of happiness, sadness, anger, fear, disgust, and surprise. Accuracy data showed comparable levels of performance for deaf and hearing children of the same age. Happiness and sadness were the most accurately matched expressions and the most accurately produced and comprehended labels. Anger was the least accurately matched expression and the most poorly comprehended emotion label. Disgust was the least accurately labeled expression; however, deaf children were more accurate at labeling this expression, and also at labeling fear, than hearing children. Error data revealed that children confused anger with disgust, and fear with surprise. However, the younger groups of deaf and hearing children also showed a tendency to confuse the negative expressions of anger, disgust, and fear with sadness. The results suggest that, despite possible differences in the early socialisation of emotion, deaf and hearing children share a common understanding of the emotions conveyed by distinctive facial expressions.     

The role of motion and intensity in deaf children’s recognition of real human facial expressions of emotion

There is substantial evidence to suggest that deafness is associated with delays in emotion understanding, which has been attributed to delays in language acquisition and opportunities to converse. However, studies addressing the ability to recognise facial expressions of emotion have produced equivocal findings. The two experiments presented here attempt to clarify emotion recognition in deaf children by considering two aspects: the role of motion and the role of intensity in deaf children’s emotion recognition. In Study 1, 26 deaf children were compared to 26 age-matched hearing controls on a computerised facial emotion recognition task involving static and dynamic expressions of 6 emotions. Eighteen of the deaf and 18 age-matched hearing controls additionally took part in Study 2, involving the presentation of the same 6 emotions at varying intensities. Study 1 showed that deaf children’s emotion recognition was better in the dynamic rather than static condition, whereas the hearing children showed no difference in performance between the two conditions. In Study 2, the deaf children performed no differently from the hearing controls, showing improved recognition rates with increasing rates of intensity. With the exception of disgust, no differences in individual emotions were found. These findings highlight the importance of using ecologically valid stimuli to assess emotion recognition.     

Sex differences in the perception of affective facial expressions: Do men really lack emotional sensitivity?

There is evidence that men and women display differences in both cognitive and affective functions. Recent studies have examined the processing of emotions in males and females. However, the findings are inconclusive, possibly the result of methodological differences. The aim of this study was to investigate the perception of emotional facial expressions in men and women. Video clips of neutral faces, gradually morphing into full-blown expressions were used. By doing this, we were able to examine both the accuracy and the sensitivity in labelling emotional facial expressions. Furthermore, all participants completed an anxiety and a depression rating scale. Research participants were 40 female students and 28 male students. Results revealed that men were less accurate, as well as less sensitive in labelling facial expressions. Thus, men show an overall worse performance compared to women on a task measuring the processing of emotional faces. This result is discussed in relation to recent findings.     

听障儿童外部情绪理解研究

通过自编的外部情绪理解问卷,考察了听障儿童表情识别、情绪情景识别和提示物理解等外部情绪理解三个成分的发展状况。结果表明:(1)听力障碍儿童在外部情绪整体理解、表情识别、情绪情景识别和提示物理解四方面的得分都显著低于普通儿童;(2)不同年级听力障碍儿童的外部情绪理解、表情识别、情绪情景识别和提示物理解四方面得分存在显著差异,不同性别听力障碍儿童外部情绪理解差异不显著;年级和性别的交互作用显著;(3)父母听力状况不同的听障儿童,其外部情绪理解水平差异显著。DD儿童的外部情绪整体理解、表情识别和情绪情景识别得分显著高于DH儿童。     

Discriminating signs: perceptual precursors to acquiring a visual-gestural language

We tested hearing 6- and 10-month-olds' ability to discriminate among three American Sign Language (ASL) parameters (location, handshape, and movement) as well as a grammatical marker (facial expression). ASL-na?ve infants were habituated to a signer articulating a two-handed symmetrical sign in neutral space. During test, infants viewed novel two-handed signs that varied in only one parameter or in facial expression. Infants detected changes in the signer's facial expression and in the location of the sign but provided no evidence of detecting the changes in handshape or movement. These findings are consistent with children's production errors in ASL and reveal that infants can distinguish among some parameters of ASL more easily than others.     

The face of bimodal bilingualism: grammatical markers in American Sign Language are produced when bilinguals speak to English monolinguals

Bimodal bilinguals, fluent in a signed and a spoken language, provide unique insight into the nature of syntactic integration and language control. We investigated whether bimodal bilinguals who are conversing with English monolinguals produce American Sign Language (ASL) grammatical facial expressions to accompany parallel syntactic structures in spoken English. In ASL, raised eyebrows mark conditionals, and furrowed eyebrows mark wh-questions; the grammatical brow movement is synchronized with the manual onset of the clause. Bimodal bilinguals produced more ASL-appropriate facial expressions than did nonsigners and synchronized their expressions with the onset of the corresponding English clauses. This result provides evidence for a dual-language architecture in which grammatical information can be integrated up to the level of phonological implementation. Overall, participants produced more raised brows than furrowed brows, which can convey negative affect. Bimodal bilinguals suppressed but did not completely inhibit ASL facial grammar when it conflicted with conventional facial gestures. We conclude that morphosyntactic elements from two languages can be articulated simultaneously and that complete inhibition of the nonselected language is difficult.     

The effects of context on facial affect recognition

In a sample of 325 college students, we examined how context influences judgments of facial expressions of emotion, using a newly developed facial affect recognition task in which emotional faces are superimposed upon emotional and neutral contexts. This research used a larger sample size than previous studies, included more emotions, varied the intensity level of the expressed emotion to avoid potential ceiling effects from very easy recognition, did not explicitly direct attention to the context, and aimed to understand how recognition is influenced by non-facial information, both situationally-relevant and situationally-irrelevant. Both accuracy and RT varied as a function of context. For all facial expressions of emotion other than happiness, accuracy increased when the emotion of the face and context matched, and decreased when they mismatched. For all emotions, participants responded faster when the emotion of the face and image matched and slower when they mismatched. Results suggest that the judgment of the facial expression is itself influenced by the contextual information instead of both being judged independently and then combined. Additionally, the results have implications for developing models of facial affect recognition and indicate that there are factors other than the face that can influence facial affect recognition judgments.     

Dynamic facial expressions are processed holistically,but not more holistically than static facial expressions

There is evidence that facial expressions are perceived holistically and featurally. The composite task is a direct measure of holistic processing (although the absence of a composite effect implies the use of other types of processing). Most composite task studies have used static images, despite the fact that movement is an important aspect of facial expressions and there is some evidence that movement may facilitate recognition. We created static and dynamic composites, in which emotions were reliably identified from each half of the face. The magnitude of the composite effect was similar for static and dynamic expressions identified from the top half (anger, sadness and surprise) but was reduced in dynamic as compared to static expressions identified from the bottom half (fear, disgust and joy). Thus, any advantage in recognising dynamic over static expressions is not likely to stem from enhanced holistic processing, rather motion may emphasise or disambiguate diagnostic featural information.     

Facial expressions of emotions and ethological behavioral categories

The purpose of this study was to measure the capacity of human subjects to match facial expressions of emotions and behavioral categories that represented the motivational states they are supposed to illustrate. 100 university students were shown facial stimuli they had to classify using ethological behavioral categories. The results showed that accuracy of judgment was over-all lower than what was usually found when fundamental emotional categories were used. The data also indicated that the relation between emotional expressions and behavioral tendencies was more complex than expected.     

The impact of rewards on empathic accuracy and emotional mimicry

The notion that motivation influences empathic accuracy has been inferred from aspects of the task, the situation or the relationship between interaction partners or between groups. The present research assessed whether monetary reward influences cognitive and affective empathy. In Study 1, cognitive empathy was assessed for 42 participants who decoded briefly (33 ms) presented expressions of sadness and anger. For half the participants, correctly decoded expressions on male faces were rewarded, for the other half correctly decoded expressions on female faces were rewarded. The results showed that rewards increase empathic accuracy for both emotions equally. In Study 2, facial EMG was measured as well to assess emotional mimicry as an index of affective empathy. Study 2 replicated the findings from Study 1 and found a moderation of affective empathy as indexed through facial mimicry for sadness. Thus, simple monetary rewards affect both cognitive and affective empathy.     

Judgment of other people's facial expressions of emotions is influenced by their concurrent affective hand movements

We investigated whether emotional information from facial expression and hand movement quality was integrated when identifying the expression of a compound stimulus showing a static facial expression combined with emotionally expressive dynamic manual actions. The emotions (happiness, neutrality, and anger) expressed by the face and hands were either congruent or incongruent. In Experiment 1, the participants judged whether the stimulus person was happy, neutral, or angry. Judgments were mainly based on the facial expressions, but were affected by manual expressions to some extent. In Experiment 2, the participants were instructed to base their judgment on the facial expression only. An effect of hand movement expressive quality was observed for happy facial expressions. The results conform with the proposal that perception of facial expressions of emotions can be affected by the expressive qualities of hand movements.     

La socialisation des émotions chez le nourrisson: le rôle des expressions faciales contingentes des mères

Although maternal contingent responses to their infant's facial expressions of emotions is thought to play an important role in the socialization of emotions, available data are still scarce and often inconsistent To further investigate how mothers' contingent facial expressions might influence infant emotional development, we undertook to study mother‐infant dyads in four episodes of face‐to‐face interaction during the first year. Mothers' facial expressions were strongly related to their infant's facial expressions of emotions, most of their contingent responses being produced within one second following infants' facial expressions Specific patterns of responses were also found. The impact of maternal contingent responding on infants' expressive development was also examined.     

Emotion recognition: the role of facial movement and the relative importance of upper and lower areas of the face

In order to investigate the role of facial movement in the recognition of emotions, faces were covered with black makeup and white spots. Video recordings of such faces were played back so that only the white spots were visible. The results demonstrated that moving displays of happiness, sadness, fear, surprise, anger and disgust were recognized more accurately than static displays of the white spots at the apex of the expressions. This indicated that facial motion, in the absence of information about the shape and position of facial features, is informative about these basic emotions. Normally illuminated dynamic displays of these expressions, however, were recognized more accurately than displays of moving spots. The relative effectiveness of upper and lower facial areas for the recognition of these six emotions was also investigated using normally illuminated and spots-only displays. In both instances the results indicated that different facial regions are more informative for different emitions. The movement patterns characterizing the various emotional expressions as well as common confusions between emotions are also discussed.     

Studying the dynamics of emotional expression using synthesized facial muscle movements

Synthetic images of facial expression were used to assess whether judges can correctly recognize emotions exclusively on the basis of configurations of facial muscle movements. A first study showed that static, synthetic images modeled after a series of photographs that are widely used in facial expression research yielded recognition rates and confusion patterns comparable to posed photos. In a second study, animated synthetic images were used to examine whether schematic facial expressions consisting entirely of theoretically postulated facial muscle configurations can be correctly recognized. Recognition rates for the synthetic expressions were far above chance, and the confusion patterns were comparable to those obtained with posed photos. In addition, the effect of static versus dynamic presentation of the expressions was studied. Dynamic presentation increased overall recognition accuracy and reduced confusions between unrelated emotions.     

When deaf signers read English: do written words activate their sign translations?

Deaf bilinguals for whom American Sign Language (ASL) is the first language and English is the second language judged the semantic relatedness of word pairs in English. Critically, a subset of both the semantically related and unrelated word pairs were selected such that the translations of the two English words also had related forms in ASL. Word pairs that were semantically related were judged more quickly when the form of the ASL translation was also similar whereas word pairs that were semantically unrelated were judged more slowly when the form of the ASL translation was similar. A control group of hearing bilinguals without any knowledge of ASL produced an entirely different pattern of results. Taken together, these results constitute the first demonstration that deaf readers activate the ASL translations of written words under conditions in which the translation is neither present perceptually nor required to perform the task.