Interaction effects of perceived gaze direction and dynamic facial expression: Evidence for appraisal theories of emotion

Appraisal theorists suggest that the face expresses cognitive processes involved both in the orienting of attention (primarily gaze direction) and in the evaluation of emotion-eliciting events. Contrary to the assumption of direct emotion recognition by basic emotions theorists, this implies an interaction effect between “perceived gaze direction” and “perceived facial expression” in inferring emotion from the face. These two theoretical perspectives were comparatively tested by requesting participants to decode dynamic synthetic facial expressions of emotion presented with either an averted or a direct gaze. Confirming the interaction predicted by appraisal theories, the perceived specificity and intensity of fear and anger depended on gaze direction (direct gaze for anger and averted gaze for fear).     

The relationships between processing facial identity,emotional expression,facial speech,and gaze direction during development

Four experiments were conducted with 5- to 11-year-olds and adults to investigate whether facial identity, facial speech, emotional expression, and gaze direction are processed independently of or in interaction with one another. In a computer-based, speeded sorting task, participants sorted faces according to facial identity while disregarding facial speech, emotional expression, and gaze direction or, alternatively, according to facial speech, emotional expression, and gaze direction while disregarding facial identity. Reaction times showed that children and adults were able to direct their attention selectively to facial identity despite variations of other kinds of face information, but when sorting according to facial speech and emotional expression, they were unable to ignore facial identity. In contrast, gaze direction could be processed independently of facial identity in all age groups. Apart from shorter reaction times and fewer classification errors, no substantial change in processing facial information was found to be correlated with age. We conclude that adult-like face processing routes are employed from 5 years of age onward.     

Effects of facial expression and gaze direction on approach-avoidance behaviour

Humans must coordinate approach-avoidance behaviours with the social cues that elicit them, such as facial expressions and gaze direction. We hypothesised that when someone is observed looking in a particular direction with a happy expression, the observer would tend to approach that direction, but that when someone is observed looking in a particular direction with a fearful expression, the observer would tend to avoid that direction. Twenty-eight participants viewed stimulus faces with averted gazes and happy or fearful expressions on a computer screen. Participants were asked to grasp (approach) or withdraw from (avoid) a left- or right-hand button depending on the stimulus face's expression. The results were consistent with our hypotheses about avoidance responses, but not with respect to approach responses. Links between social cues and adaptive behaviour are discussed.     

The interaction between gaze direction and facial expressions in newborns

The ability to decode facial expressions is an important component of social interaction and functioning. This ability is even more fundamental early in life, prior to the development of verbal communication. However, it is still unclear whether newborns can detect, discriminate and process facial expressions, and, if so, what the mechanisms underlying this ability are. In this study, we extend the investigation of perceived emotional expression by manipulating gaze direction with different facial expressions. Specifically, newborns were presented with faces displaying neutral, fearful, or happy facial expressions accompanied with direct or averted gaze, and tested in a visual preference paradigm. Four experiments were conducted in which different combinations of expression and gaze were used. However, only in the fourth experiment did newborns show a visual preference for a specific emotional display; they looked significantly longer at a happy face than a neutral one only when both were accompanied with direct gaze. These results provide support for the advantage of happy facial expressions in the development of a face processing system and suggest that this preference reflects experience acquired during the first few days after birth.     

Influence of emotional expression on the processing of gaze direction

Research demonstrates an influence of gaze direction in emotion recognition. Here we examined whether facial affect similarly influences recognition of gaze direction. Across two studies we found that averted relative to direct gaze was processed more quickly and accurately when coupled with fear, and direct relative to averted gaze with anger. Also evident was that slower overall gaze processing was associated with increased interaction effects between emotion and gaze. Examining individual differences, therefore, enabled us to extend previous research examining speed of processing as a moderator of the interaction effect, while holding constant task demands and stimulus features. Unexpectedly, a main effect emerged such that averted relative to direct gaze was found to be processed more quickly and accurately overall. This effect was not moderated by processing speed and is discussed as a potential stimulus-driven effect that may help explain discrepant findings in the literature.

Modulation of reflexive orienting to gaze direction by facial expressions

Facial expression and gaze perception are thought to share brain mechanisms but behavioural interactions, especially from gaze-cueing paradigms, are inconsistent. We conducted a series of gaze-cueing studies using dynamic facial cues to examine orienting across different emotional expression and task conditions, including face inversion. Across experiments, at a short stimulus–onset asynchrony (SOA) we observed both an expression effect (i.e., faster responses when the face was emotional versus neutral) and a cue validity effect (i.e., faster responses when the target was gazed-at), but no interaction between validity and emotion. Results from face inversion suggest that the emotion effect may have been due to both facial expression and stimulus motion. At longer SOAs, validity and emotion interacted such that cueing by emotional faces, fearful faces in particular, was enhanced relative to neutral faces. These results converge with a growing body of evidence that suggests that gaze and expression are initially processed independently and interact at later stages to direct attentional orienting.     

威胁相关面孔表情对眼睛注视方向知觉的影响

以往研究发现眼睛注视方向知觉受面孔表情的影响,愤怒面孔相较于恐惧面孔更倾向被判断为看着观察者。虽然研究者对此提出了不同的解释,但目前尚不清楚愤怒和恐惧表情在注视方向知觉中的这种差异影响到底来自于面孔的结构信息还是物理特征信息。本研究采用注视方向辨别任务,计算直视知觉范围（The Cone of Direct Gaze,CoDG）为因变量,分别以直立,倒置及模糊图片为实验材料,试图通过分离面孔结构信息和物理特征信息,对以上问题进行探讨。结果发现在保留面孔全部信息的情况下（实验1）愤怒面孔的CoDG大于恐惧面孔;在破坏结构信息加工,只保留特征信息加工的情况下（实验2））愤怒和恐惧表情在直视知觉范围上的差异消失了;在削弱物理特征信息加工,保留结构信息加工的情况下（实验3）二者在CoDG上的差异又复现。本研究结果说明不同威胁性面孔表情对眼睛注视知觉的影响主要来自于二者在与情绪意义相关的结构信息加工上的不同,而二者非低级的物理信息上的差异,支持信号共享假说和情绪评价假说对威胁性面孔表情与注视方向整合加工解释的理论基础。     

Perceiver and poser asymmetries in processing facial emotion

This study examined both perceiver and poser asymmetries in processing facial emotion. Posers were left brain-damaged (LBD), right brain-damaged (RBD), and normal control (NC) right-handed males videotaped while expressing happiness and anger. Perceivers rated the facial expressions for asymmetry in original and reversed orientations. Overall, expressions viewed in the reversed orientation were rated as more left-sided than in the original orientation. In the reversed orientation, the more extensive left hemiface of the NCs and LBDs fell in the perceiver's left hemispace. This finding is consistent with previous research demonstrating a left hemispace bias for free-field viewing of emotional faces. Expressions were produced significantly more intensely on the left than the right hemiface by NCs and LBDs; expressions of RBDs were not significantly lateralized.     

Perceptual biases in processing facial identity and emotion

Normal observers demonstrate a bias to process the left sides of faces during perceptual judgments about identity or emotion. This effect suggests a right cerebral hemisphere processing bias. To test the role of the right hemisphere and the involvement of configural processing underlying this effect, young and older control observers and patients with right hemisphere damage completed two chimeric faces tasks (emotion judgment and face identity matching) with both upright and inverted faces. For control observers, the emotion judgment task elicited a strong left-sided perceptual bias that was reduced in young controls and eliminated in older controls by face inversion. Right hemisphere damage reversed the bias, suggesting the right hemisphere was dominant for this task, but that the left hemisphere could be flexibly recruited when right hemisphere mechanisms are not available or dominant. In contrast, face identity judgments were associated most clearly with a vertical bias favouring the uppermost stimuli that was eliminated by face inversion and right hemisphere lesions. The results suggest these tasks involve different neurocognitive mechanisms. The role of the right hemisphere and ventral cortical stream involvement with configural processes in face processing is discussed.     

Sex differences in the lateralized processing of facial emotion

Two reaction time tasks were administered to male and female normal subjects, involving judgments of facial emotion. In the Word-Face task, judgments of similarity or difference of a centrally presented emotion word and an emotional face presented in the left or right visual field were required, and in the Face-Face task, comparisons of a centrally presented and a laterally presented emotional face were required. Results were significant for the matching trials only. Reaction times to negative emotions were faster overall than to positive emotions, and an Emotional Valence by Visual Field interaction was found such that reaction times were faster for negative emotions in the left visual field and for positive emotions in the right visual field. This interaction was significant for the female but not the male subjects, although similar patterns were observed in both sexes. Further, an interaction of Gender, Task, and Emotional Valence was found, such that the two tasks had opposite effects for the two sexes. The Face-Face task appeared to inhibit the performance of the male subjects and facilitate the performance of the female subjects in terms of reaction time. It was suggested that specifying the target emotion by an emotional face elicits a greater emotional response on the part of the subject than specification by a word, and that this emotional elicitation may result in a reactive inhibition in the male subjects and in an elaboration of the emotional response in the female subjects.     

Predictive gaze cues affect face evaluations: The effect of facial emotion

When we see someone change their direction of gaze, we spontaneously follow their eyes because we expect people to look at interesting objects. Bayliss and Tipper (2006) examined the consequences of observing this expectancy being either confirmed or violated by faces producing reliable or unreliable gaze cues. Participants viewed different faces that would consistently look at the target, or consistently look away from the target: The faces that consistently looked towards targets were subsequently chosen as being more trustworthy than the faces that consistently looked away from targets. The current work demonstrates that these gaze contingency effects are only detected when faces create a positive social context by smiling, but not in the negative context when all the faces held angry or neutral expressions. These data suggest that implicit processing of the reward contingencies associated with gaze cues relies on a positive emotional expression to maintain expectations of a favourable outcome of joint attention episodes.     

Consistent left gaze bias in processing different facial cues

While viewing faces, humans often demonstrate a natural gaze bias towards the left visual field, that is, the right side of the viewee’s face is often inspected first and for longer periods. Previous studies have suggested that this gaze asymmetry is a part of the gaze pattern associated with face exploration, but its relation with perceptual processing of facial cues is unclear. In this study we recorded participants’ saccadic eye movements while exploring face images under different task instructions (free viewing, judging familiarity and judging facial expression). We observed a consistent left gaze bias in face viewing irrespective of task demands. The probability of the first fixation and the proportion of overall fixations directed at the left hemiface were indistinguishable across different task instructions or across different facial expressions. It seems that the left gaze bias is an automatic reflection of hemispheric lateralisation in face processing, and is not necessarily correlated with the perceptual processing of a specific type of facial information.     

Evidence for strong dissociation between emotion and facial displays: the case of surprise

Eight experiments examined facial expressions of surprise in adults. Surprise was induced by disconfirming a previously established schema or expectancy. Self-reports and behavioral measures indicated the presence of surprise in most participants, but surprise expressions were observed only in 4%-25%, and most displays consisted of eyebrow raising only; the full, 3-component display was never seen. Experimental variations of surprise intensity, sociality, and duration/complexity of the surprising event did not change these results. Electromyographic measurement failed to detect notably more brow raisings and, in one study, revealed a decrease of frontalis muscle activity in the majority of the participants. Nonetheless, most participants believed that they had shown a strong surprise expression.     

Neural correlates of facial emotion processing in infancy

In the present study we examined the neural correlates of facial emotion processing in the first year of life using ERP measures and cortical source analysis. EEG data were collected cross‐sectionally from 5‐ (N = 49), 7‐ (N = 50), and 12‐month‐old (N = 51) infants while they were viewing images of angry, fearful, and happy faces. The N290 component was found to be larger in amplitude in response to fearful and happy than angry faces in all posterior clusters and showed largest response to fear than the other two emotions only over the right occipital area. The P400 and Nc components were found to be larger in amplitude in response to angry than happy and fearful faces over central and frontal scalp. Cortical source analysis of the N290 component revealed greater cortical activation in the right fusiform face area in response to fearful faces. This effect started to emerge at 5 months and became well established at 7 months, but it disappeared at 12 months. The P400 and Nc components were primarily localized to the PCC/Precuneus where heightened responses to angry faces were observed. The current results suggest the detection of a fearful face in infants’ brain can happen shortly (~200–290 ms) after the stimulus onset, and this process may rely on the face network and develop substantially between 5 to 7 months of age. The current findings also suggest the differential processing of angry faces occurred later in the P400/Nc time window, which recruits the PCC/Precuneus and is associated with the allocation of infants’ attention.     

Effects of early experience on children's recognition of facial displays of emotion

The present research examines visual perception of emotion in both typical and atypical development. To examine the processes by which perceptual mechanisms become attuned to the contingencies of affective signals in the environment, the authors measured the sequential, content-based properties of feature detection in emotion recognition processes. To evaluate the role of experience, they compared typically developing children with physically abused children, who were presumed to have experienced high levels of threat and hostility. As predicted, physically abused children accurately identified facial displays of anger on the basis of less sensory input than did controls, which suggests that physically abused children have facilitated access to representations of anger. The findings are discussed in terms of experiential processes in perceptual learning.     

The effect of facial expression and gaze direction on memory for unfamiliar faces

We report data from an experiment that investigated the influence of gaze direction and facial expression on face memory. Participants were shown a set of unfamiliar faces with either happy or angry facial expressions, which were either gazing straight ahead or had their gaze averted to one side. Memory for faces that were initially shown with angry expressions was found to be poorer when these faces had averted as opposed to direct gaze, whereas memory for individuals shown with happy faces was unaffected by gaze direction. We suggest that memory for another individual's face partly depends on an evaluation of the behavioural intention of that individual.     

The effect of facial expression and gaze direction on memory for unfamiliar faces

We report data from an experiment that investigated the influence of gaze direction and facial expression on face memory. Participants were shown a set of unfamiliar faces with either happy or angry facial expressions, which were either gazing straight ahead or had their gaze averted to one side. Memory for faces that were initially shown with angry expressions was found to be poorer when these faces had averted as opposed to direct gaze, whereas memory for individuals shown with happy faces was unaffected by gaze direction. We suggest that memory for another individual's face partly depends on an evaluation of the behavioural intention of that individual.     

Social anxiety and interpretation biases for facial displays of emotion: emotion detection and ratings of social cost

The current study assessed the processing of facial displays of emotion (Happy, Disgust, and Neutral) of varying emotional intensities in participants with high vs. low social anxiety. Use of facial expressions of varying intensities allowed for strong external validity and a fine-grained analysis of interpretation biases. Sensitivity to perceiving negative evaluation in faces (i.e., emotion detection) was assessed at both long (unlimited) and brief (60 ms) stimulus durations. In addition, ratings of perceived social cost were made indicating what participants judged it would be like to have a social interaction with a person exhibiting the stimulus emotion. Results suggest that high social anxiety participants did not demonstrate biases in their sensitivity to perceiving negative evaluation (i.e. disgust) in facial expressions. However, high social anxiety participants did estimate the perceived cost of interacting with someone showing disgust to be significantly greater than low social anxiety participants, regardless of the intensity of the disgust expression. These results are consistent with a specific type of interpretation bias in which participants with social anxiety have elevated ratings of the social cost of interacting with individuals displaying negative evaluation.     

The combined effect of gaze direction and facial expression on cueing spatial attention

Empirical evidence shows an effect of gaze direction on cueing spatial attention, regardless of the emotional expression shown by a face, whereas a combined effect of gaze direction and facial expression has been observed on individuals' evaluative judgments. In 2 experiments, the authors investigated whether gaze direction and facial expression affect spatial attention depending upon the presence of an evaluative goal. Disgusted, fearful, happy, or neutral faces gazing left or right were followed by positive or negative target words presented either at the spatial location looked at by the face or at the opposite spatial location. Participants responded to target words based on affective valence (i.e., positive/negative) in Experiment 1 and on letter case (lowercase/uppercase) in Experiment 2. Results showed that participants responded much faster to targets presented at the spatial location looked at by disgusted or fearful faces but only in Experiment 1, when an evaluative task was used. The present findings clearly show that negative facial expressions enhance the attentional shifts due to eye-gaze direction, provided that there was an explicit evaluative goal present.     

Emotional attention: effects of emotion and gaze direction on overt orienting of visual attention

In the present study we considered the two factors that have been advocated for playing a role in emotional attention: perception of gaze direction and facial expression of emotions. Participants performed an oculomotor task in which they had to make a saccade towards one of the two lateral targets, depending on the colour of the fixation dot which appeared at the centre of the computer screen. At different time intervals (stimulus onset asynchronies, SOAs: 50,100,150 ms) following the onset of the dot, a picture of a human face (gazing either to the right or to the left) was presented at the centre of the screen. The gaze direction of the face could be congruent or incongruent with respect to the location of the target, and the expression could be neutral or angry. In Experiment 1 the facial expressions were presented randomly in a single block, whereas in Experiment 2 they were shown in separate blocks. Latencies for correct saccades and percentage of errors (saccade direction errors) were considered in the analyses. Results showed that incongruent trials determined a significantly higher percentage of saccade direction errors with respect to congruent trials, thus confirming that gaze direction, even when task-irrelevant, interferes with the accuracy of the observer’s oculomotor behaviour. The angry expression was found to hold attention for a longer time with respect to the neutral one, producing delayed saccade latencies. This was particularly evident at 100 ms SOA and for incongruent trials. Emotional faces may then exert a modulatory effect on overt attention mechanisms.