Developmental changes in face processing skills

Expertise in processing differences among faces in the spacing among facial features (second-order relations) is slower to develop than expertise in processing the shape of individual features or the shape of the external contour. To determine the impact of the slow development of sensitivity to second-order relations on various face-processing skills, we developed five computerized tasks that require matching faces on the basis of identity (with changed facial expression or head orientation), facial expression, gaze direction, and sound being spoken. In Experiment 1, we evaluated the influence of second-order relations on performance on each task by presenting them to adults (N=48) who viewed the faces either upright or inverted. Previous studies have shown that inversion has a larger effect on tasks that require processing the spacing among features than it does on tasks that can be solved by processing the shape of individual features. Adults showed an inversion effect for only one task: matching facial identity when there was a change in head orientation. In Experiment 2, we administered the same tasks to children aged 6, 8, and 10 years (N=72). Compared to adults, 6-year-olds made more errors on every task and 8-year-olds made more errors on three of the five tasks: matching direction of gaze and the two facial identity tasks. Ten-year-olds made more errors than adults on only one task: matching facial identity when there was a change in head orientation (e.g., from frontal to tilted up). Together, the results indicate that the slow development of sensitivity to second-order relations causes children to be especially poor at recognizing the identity of a face when it is seen in a new orientation.     

The relationships between processing facial identity,emotional expression,facial speech,and gaze direction during development

Four experiments were conducted with 5- to 11-year-olds and adults to investigate whether facial identity, facial speech, emotional expression, and gaze direction are processed independently of or in interaction with one another. In a computer-based, speeded sorting task, participants sorted faces according to facial identity while disregarding facial speech, emotional expression, and gaze direction or, alternatively, according to facial speech, emotional expression, and gaze direction while disregarding facial identity. Reaction times showed that children and adults were able to direct their attention selectively to facial identity despite variations of other kinds of face information, but when sorting according to facial speech and emotional expression, they were unable to ignore facial identity. In contrast, gaze direction could be processed independently of facial identity in all age groups. Apart from shorter reaction times and fewer classification errors, no substantial change in processing facial information was found to be correlated with age. We conclude that adult-like face processing routes are employed from 5 years of age onward.     

Facial expressions modulate the gaze orienting effect on sound localization judgement

Recent studies have shown that cueing eye gaze can affect the processing of visual information, and this phenomenon is called the gaze-orienting effect (visual-GOE). Emerging evidence has shown that the cueing eye gaze also affects the processing of auditory information (auditory-GOE). However, it is unclear whether the auditory-GOE is modulated by emotion. We conducted three behavioural experiments to investigate whether cueing eye gaze influenced the orientation judgement to a sound, and whether the effect was modulated by facial expressions. The current study set four facial expressions (angry, fearful, happy, and neutral), manipulated the display type of facial expressions, and changed the sequence of gaze and emotional expressions. Participants were required to judge the sound orientation after facial expressions and gaze cues. The results showed that the orientation judgement of sound was influenced by gaze direction in all three experiments, and the orientation judgement of sound was faster when the face was oriented to the target location (congruent trials) than when the face was oriented away from the target location (incongruent trials). The modulation of emotion on auditory-GOE was observed only when gaze shifted followed by facial expression (Exp3); the auditory-GOE was significantly greater for angry faces than for neutral faces. These findings indicate that auditory-GOE as a social phenomenon exists widely, and the effect was modulated by facial expression. Gaze shift before the presentation of emotion was the key influencing factor for the emotional modulation in an auditory target gaze-orienting task. Our findings suggest that the integration of facial expressions and eye gaze was context-dependent.     

面孔参照或观察者参照：性别二态线索、面孔吸引力与表情对正视面孔偏好的影响

本研究探讨了性别二态线索、面孔吸引力和表情对正视面孔偏好的影响。实验1基于面孔参照,被试普遍偏好正视面孔;相对男性化/厌恶和生气/低吸引力的面孔,被试更偏好女性化/高兴和中性/高吸引力的正视面孔;实验2基于观察者参照,总体并未表现出正视偏好;对低吸引力/中性表情/男性化面孔更偏好斜视面孔。     

The interaction between gaze direction and facial expressions in newborns

The ability to decode facial expressions is an important component of social interaction and functioning. This ability is even more fundamental early in life, prior to the development of verbal communication. However, it is still unclear whether newborns can detect, discriminate and process facial expressions, and, if so, what the mechanisms underlying this ability are. In this study, we extend the investigation of perceived emotional expression by manipulating gaze direction with different facial expressions. Specifically, newborns were presented with faces displaying neutral, fearful, or happy facial expressions accompanied with direct or averted gaze, and tested in a visual preference paradigm. Four experiments were conducted in which different combinations of expression and gaze were used. However, only in the fourth experiment did newborns show a visual preference for a specific emotional display; they looked significantly longer at a happy face than a neutral one only when both were accompanied with direct gaze. These results provide support for the advantage of happy facial expressions in the development of a face processing system and suggest that this preference reflects experience acquired during the first few days after birth.     

Gaze direction and facial expressions exert combined but different effects on attentional resources

Gaze direction and facial expressions are critical components of face processing and have been shown to influence attention deployment. We investigated whether gaze direction (direct vs. averted) combined with a neutral or angry expression modulates the deployment of attentional resources over time. In a Rapid Serial Visual Presentation (RSVP) paradigm participants had to decide the gender of a neutral or an angry target face with direct or averted gaze (T1) and then to judge the orientation of a target picture of a landscape (T2), following the face at different time intervals. Results showed no attentional blink effect (i.e., no deterioration in T2 accuracy) when T1 was an angry face with direct gaze, whereas it was present for angry faces with averted gaze or neutral faces with either averted or direct gaze. These findings are consistent with appraisal theories and are discussed against the background of automatic processing of threat stimuli.     

Gaze direction and facial expressions exert combined but different effects on attentional resources

Gaze direction and facial expressions are critical components of face processing and have been shown to influence attention deployment. We investigated whether gaze direction (direct vs. averted) combined with a neutral or angry expression modulates the deployment of attentional resources over time. In a Rapid Serial Visual Presentation (RSVP) paradigm participants had to decide the gender of a neutral or an angry target face with direct or averted gaze (T1) and then to judge the orientation of a target picture of a landscape (T2), following the face at different time intervals. Results showed no attentional blink effect (i.e., no deterioration in T2 accuracy) when T1 was an angry face with direct gaze, whereas it was present for angry faces with averted gaze or neutral faces with either averted or direct gaze. These findings are consistent with appraisal theories and are discussed against the background of automatic processing of threat stimuli.     

Facial identity and facial expression matching in 5–12‐year‐old children and adults

Facial identity and facial expression matching tasks were completed by 5–12‐year‐old children and adults using stimuli extracted from the same set of normalized faces. Configural and feature processing were examined using speed and accuracy of responding and facial feature selection, respectively. Facial identity matching was slower than face expression matching for all age groups. Large age effects were found on both speed and accuracy of responding and feature use in both identity and expression matching tasks. Eye region preference was found on the facial identity task and mouth region preference on the facial expression task. Use of mouth region information for facial expression matching increased with age, whereas use of eye region information for facial identity matching peaked early. The feature use information suggests that the specific use of primary facial features to arrive at identity and emotion matching judgments matures across middle childhood. Copyright © 2009 John Wiley & Sons, Ltd.     

The neural fate of individual item representations in visual working memory

Previous research has demonstrated an interaction between eye gaze and selected facial emotional expressions, whereby the perception of anger and happiness is impaired when the eyes are horizontally averted within a face, but the perception of fear and sadness is enhanced under the same conditions. The current study reexamined these claims over six experiments. In the first three experiments, the categorization of happy and sad expressions (Experiments 1 and 2) and angry and fearful expressions (Experiment 3) was impaired when eye gaze was averted, in comparison to direct gaze conditions. Experiment 4 replicated these findings in a rating task, which combined all four expressions within the same design. Experiments 5 and 6 then showed that previous findings, that the perception of selected expressions is enhanced under averted gaze, are stimulus and task-bound. The results are discussed in relation to research on facial expression processing and visual attention.     

Dr. Angry and Mr. Smile: when categorization flexibly modifies the perception of faces in rapid visual presentations

Are categorization and visual processing independent, with categorization operating late, on an already perceived input, or are they intertwined, with the act of categorization flexibly changing (i.e. cognitively penetrating) the early perception of the stimulus? We examined this issue in three experiments by applying different categorization tasks (gender, expressive or not, which expression and identity) to identical face stimuli. Stimuli were hybrids: they combined a man or a woman with a particular expression at a coarse spatial scale with a face of the opposite gender with a different expression at the fine spatial scale. Results suggested that the categorization task changes the spatial scales preferentially used and perceived for rapid recognition. A perceptual set effect is shown whereby the scale preference of an important categorization (e.g. identity) transfers to resolve other face categorizations (e.g. expressive or not, which expression). Together, the results suggest that categorization can be closely bound to perception.     

A matching advantage for dynamic human faces

In a series of three experiments, we used a sequential matching task to explore the impact of non-rigid facial motion on the perception of human faces. Dynamic prime images, in the form of short video sequences, facilitated matching responses relative to a single static prime image. This advantage was observed whenever the prime and target showed the same face but an identity match was required across expression (experiment 1) or view (experiment 2). No facilitation was observed for identical dynamic prime sequences when the matching dimension was shifted from identity to expression (experiment 3). We suggest that the observed dynamic advantage, the first reported for non-degraded facial images, arises because the matching task places more emphasis on visual working memory than typical face recognition tasks. More specifically, we believe that representational mechanisms optimised for the processing of motion and/or change-over-time are established and maintained in working memory and that such 'dynamic representations' (Freyd, 1987 Psychological Review 94 427-438) capitalise on the increased information content of the dynamic primes to enhance performance.     

Effects of perceived mutual gaze and gender on face processing and recognition memory

Perceived gaze contact in seen faces may convey important social signals. We examined whether gaze perception affects face processing during two tasks: Online gender judgement, and later incidental recognition memory. Individual faces were presented with eyes directed either straight towards the viewer or away, while these faces were seen in either frontal or three-quarters view. Participants were slower to make gender judgements for faces with direct versus averted eye gaze, but this effect was particularly pronounced for faces with opposite gender to the observer, and seen in three-quarters view. During subsequent surprise recognition-memory testing, recognition was better for faces previously seen with direct than averted gaze, again especially for the opposite gender to the observer. The effect of direct gaze was stronger in both tasks when the head was seen in three-quarters rather than in frontal view, consistent with the greater salience of perceived eye contact for deviated faces. However, in the memory test, face recognition was also relatively enhanced for faces of opposite gender in front views when their gaze was averted rather than direct. Together, these results indicate that perceived eye contact can interact with facial processing during gender judgements and recognition memory, even when gaze direction is task-irrelevant, and particularly for faces of opposite gender to the observer (an influence which controls for stimulus factors when considering observers of both genders). These findings appear consistent with recent neuroimaging evidence that social facial cues can modulate visual processing in cortical regions involved in face processing and memory, presumably via interconnections with brain systems specialized for gaze perception and social monitoring.     

Configural processing in the perception of eye-gaze direction

Traditional accounts of gaze perception emphasise the geometric or configural cues present in the eye; the position of the iris in relation to the corner of the eye, for example. This kind of geometric account has been supported, in part, by findings that gaze judgments are impaired in faces rotated through 180 degrees, a manipulation known to disrupt the processing of relations between facial elements. However, studies involving this manipulation have confounded inversion of the face context with inversion of the eye region. The effects of inversion might therefore have been caused by a disruption of the computation of gaze direction from the eye region itself and/or a disruption of the influence that face context might exert on gaze processing. In the experiment reported here we independently manipulated eye orientation and the orientation of the face context, and measured participants' sensitivity to gaze direction. Performance was severely affected by inversion of the eyes, regardless of the orientation of the face, whereas face inversion had no significant effect on gaze sensitivity. Previous reports of a face-inversion effect on gaze perception can therefore be attributed to inversion of the eye region itself which, we suggest, disrupts some form of configural or relational processing that is normally involved in the computation of eye-gaze direction.     

Consistent left gaze bias in processing different facial cues

While viewing faces, humans often demonstrate a natural gaze bias towards the left visual field, that is, the right side of the viewee’s face is often inspected first and for longer periods. Previous studies have suggested that this gaze asymmetry is a part of the gaze pattern associated with face exploration, but its relation with perceptual processing of facial cues is unclear. In this study we recorded participants’ saccadic eye movements while exploring face images under different task instructions (free viewing, judging familiarity and judging facial expression). We observed a consistent left gaze bias in face viewing irrespective of task demands. The probability of the first fixation and the proportion of overall fixations directed at the left hemiface were indistinguishable across different task instructions or across different facial expressions. It seems that the left gaze bias is an automatic reflection of hemispheric lateralisation in face processing, and is not necessarily correlated with the perceptual processing of a specific type of facial information.     

Processing emotional category congruency between emotional facial expressions and emotional words

Facial expressions are critical for effective social communication, and as such may be processed by the visual system even when it might be advantageous to ignore them. Previous research has shown that categorising emotional words was impaired when faces of a conflicting valence were simultaneously presented. In the present study, we examined whether emotional word categorisation would also be impaired when faces of the same (negative) valence but different emotional category (either angry, sad or fearful) were simultaneously presented. Behavioural results provided evidence for involuntary processing of basic emotional facial expression category, with slower word categorisation when the face and word categories were incongruent (e.g., angry word and sad face) than congruent (e.g., angry word and angry face). Event-related potentials (ERPs) time-locked to the presentation of the word–face pairs also revealed that emotional category congruency effects were evident from approximately 170 ms after stimulus onset.     

Talking heads or talking eyes? Effects of head orientation and sudden onset gaze cues on attention capture

The direction of gaze towards or away from an observer has immediate effects on attentional processing in the observer. Previous research indicates that faces with direct gaze are processed more efficiently than faces with averted gaze. We recently reported additional processing advantages for faces that suddenly adopt direct gaze (abruptly shift from averted to direct gaze) relative to static direct gaze (always in direct gaze), sudden averted gaze (abruptly shift from direct to averted gaze), and static averted gaze (always in averted gaze). Because changes in gaze orientation in previous study co-occurred with changes in head orientation, it was not clear if the effect is contingent on face or eye processing, or whether it requires both the eyes and the face to provide consistent information. The present study delineates the impact of head orientation, sudden onset motion cues, and gaze cues. Participants completed a target-detection task in which head position remained in a static averted or direct orientation while sudden onset motion and eye gaze cues were manipulated within each trial. The results indicate a sudden direct gaze advantage that resulted from the additive role of motion and gaze cues. Interestingly, the orientation of the face towards or away from the observer did not influence the sudden direct gaze effect, suggesting that eye gaze cues, not face orientation cues, are critical for the sudden direct gaze effect.     

Low-level orientation information for social evaluation in face images

Observers make a range of social evaluations based on facial appearance, including judgments of trustworthiness, warmth, competence, and other aspects of personality. What visual information do people use to make these judgments? While links have been made between perceived social characteristics and other high-level properties of facial appearance (e.g., attractiveness, masculinity), there has been comparatively little effort to link social evaluations to low-level visual features, like spatial frequency and orientation sub-bands, known to be critically important for face processing. We explored the extent to which different social evaluations depended critically on horizontal orientation energy vs. vertical orientation energy, as is the case for face identification and emotion recognition. We found that while trustworthiness judgments exhibited this bias for horizontal orientations, competence and dominance did not, suggesting that social evaluations may depend on a multi-channel representation of facial appearance at early stages of visual processing.     

Processing emotional category congruency between emotional facial expressions and emotional words

Facial expressions are critical for effective social communication, and as such may be processed by the visual system even when it might be advantageous to ignore them. Previous research has shown that categorising emotional words was impaired when faces of a conflicting valence were simultaneously presented. In the present study, we examined whether emotional word categorisation would also be impaired when faces of the same (negative) valence but different emotional category (either angry, sad or fearful) were simultaneously presented. Behavioural results provided evidence for involuntary processing of basic emotional facial expression category, with slower word categorisation when the face and word categories were incongruent (e.g., angry word and sad face) than congruent (e.g., angry word and angry face). Event-related potentials (ERPs) time-locked to the presentation of the word-face pairs also revealed that emotional category congruency effects were evident from approximately 170 ms after stimulus onset.     

Discrimination of Facial Identity and Facial Affect by Temporal and Frontal Lobectomy Patients

The purpose of this study was to determine the extent to which lobectomy affects ability to discriminate facial identity or facial expression. Fifteen right temporal, 15 left temporal, 5 right frontal, and 4 left frontal lobectomy patients, pair-matched for age, sex, and education to normal control subjects, participated in this study. Tasks included a Facial Identity Matching Task and a Facial Affect Matching task. The lobectomized patients as a whole were significantly impaired on both tasks (22% decrement in performance). The patients made twice as many errors resulting from perseveration of response-set of the first condition (identity or emotion matching) into the second condition. The site of lobectomy did not influence general performance on any one task or selective performance on any subset of affective categories. It was concluded that all four brain regions play a significant and equal role in face processing, and that circuits more specifically dedicated to visual face processing, which are responsible for hemispheric dominance affects and affect/identity dissociations, are probably located more posteriorly in the brain. Finally, it was concluded that perseveration of acquired habit may, under specific conditions, characterize temporal lobe dysfunction just as much as frontal lobe dysfunction.