Features and their configuration in face recognition

Tanaka and Farah (1993) have proposed a holistic approach to face recognition in which information about the features of a face and their configuration are combined together in the face representation. An implication of the holistic hypothesis is that alterations in facial configuration should interfere with retrieval of features. In four experiments, the effect of configuration on feature recognition was investigated by creating two configurations of a face, one with eyes close together and one with eyes far apart. After subjects studied faces presented in one of the two configurations (eyes-close or eyes-far), they were tested for their recognition of features shown in isolation, in a new face configuration, and in the old face configuration. It was found that subjects recognized features best when presented in the old configuration, next best in the new configuration, and poorest in isolation. Moreover, subjects were not sensitive to configural information in inverted faces (Experiment 2) or nonface stimuli (i.e., houses; Experiments 3 and 4). Importantly, for normal faces, altering the spatial location of the eyes not only impaired subjects’ recognition of the eye features but also impaired their recognition of the nose and mouth features—features whose spatial locations were not directly altered. These findings emphasize the interdependency of featural and configural information in a holistic face representation.     

Perceptibility of schematic face stimuli: Evidence for a perceptual Gestalt

The perceptibility of face, scrambled face, and single-feature stimuli was investigated in three experiments. Stimuli were presented tachistoscopically, followed by a visual noise mask and a forced-choice test of one of three features (eyes, nose, and mouth). In Experiment I, two processing strategies which have been proposed for word perception (involving expectancy and redundancy) were investigated for the stimuli employed here. In Experiments II and III, experimentally induced familiarity was studied for its effect on recognition and perception, and an immediate and delayed perceptual test was employed. Across all three experiments, perception of single-feature and face stimuli were consistently superior to scrambled faces; in Experiment III, differences between single features and faces were eliminated. The effects of perceptual expectancy, internal feature redundancy, familiarity, guessing biases, etc., were shown to be insufficient to account for the superiority of face to scrambled face stimuli. It was argued that the perceptibility of nonredundant features are enhanced when those features are aligned in a well-defined form class. The view that familiarity operates directly on recognitive processes but indirectly on perceptual ones was discussed.     

Effect of study context on item recollection

We examined how visual context information provided during encoding, and unrelated to the target word, affected later recollection for words presented alone using a remember–know paradigm. Experiments 1A and 1B showed that participants had better overall memory—specifically, recollection—for words studied with pictures of intact faces than for words studied with pictures of scrambled or inverted faces. Experiment 2 replicated these results and showed that recollection was higher for words studied with pictures of faces than when no image accompanied the study word. In Experiment 3 participants showed equivalent memory for words studied with unique faces as for those studied with a repeatedly presented face. Results suggest that recollection benefits when visual context information high in meaningful content accompanies study words and that this benefit is not related to the uniqueness of the context. We suggest that participants use elaborative processes to integrate item and meaningful contexts into ensemble information, improving subsequent item recollection.     

Perceptual learning in face processing: Comparison facilitates face recognition

Theoretical analyses of perceptual learning suggest that comparison between similar stimuli aids subsequent discrimination between them. The current studies examined whether the opportunity to compare a target face to other similar faces during a preexposure phase facilitated performance on a matching task. Performance was better when the target face had been presented in alternation with similar comparator faces than if that target had not been exposed before test. Exposure to the target face without comparators, or exposure alternating with dissimilar faces, improved performance relative to the nonexposed control but performance was not as good as when target exposure was given in alternation to similar comparison faces. The effects were not influenced by image changes between exposure and test. It is suggested that exposure to a face in comparison to similar stimuli focuses the central representation of a face on its unique features. In practical terms these results suggest that reliable identification of an individual from their photograph would be improved by viewing that photograph in comparison with photographs of similar people.     

Sad people are more accurate at face recognition than happy people

Mood has varied effects on cognitive performance including the accuracy of face recognition (Lundh & Ost, 1996). Three experiments are presented here that explored face recognition abilities in mood-induced participants. Experiment 1 demonstrated that happy-induced participants are less accurate and have a more conservative response bias than sad-induced participants in a face recognition task. Using a remember/know/guess procedure, Experiment 2 showed that sad-induced participants had more conscious recollections of faces than happy-induced participants. Additionally, sad-induced participants could recognise all faces accurately, whereas, happy- and neutral-induced participants recognised happy faces more accurately than sad faces. In Experiment 3, these effects were not observed when participants intentionally learnt the faces, rather than incidentally learnt the faces. It is suggested that happy-induced participants do not process faces as elaborately as sad-induced participants.     

The effect of inversion on the encoding of normal and “thatcherized” faces

In the “Thatcher illusion” a face, in which the eyes and mouth are inverted relative to the rest of the face, looks grotesque when shown upright but not when inverted. In four experiments we investigated the contribution of local and global processing to this illusion in normal observers. We examined inversion effects (i.e., better performance for upright than for inverted faces) in a task requiring discrimination of whether faces were or were not “thatcherized”. Observers made same/different judgements to isolated face parts (Experiments 1-2) and to whole faces (Experiments 3-4). Face pairs had the same or different identity, allowing for different process- ing strategies using feature-based or configural information, respectively. In Experiment 1, feature-based matching of same-person face parts yielded only a small inversion effect for normal face parts. However, when feature-based matching was prevented by using the face parts of different people on all trials (Experiment 2) an inversion effect occurred for normal but not for thatcherized parts. In Experiments 3 and 4, inversion effects occurred with normal but not with thatcherized whole faces, on both same- and different-person matching tasks. This suggests that a common configural strategy was used with whole (normal) faces. Face context facilitated attention to misoriented parts in same-person but not in different-person matching. The results indicate that (1) face inversion disrupts local configural processing, but not the processing of image features, and (2) thatcherization disrupts local configural processing in upright faces.     

Emotion recognition: The role of featural and configural face information

Several studies investigated the role of featural and configural information when processing facial identity. A lot less is known about their contribution to emotion recognition. In this study, we addressed this issue by inducing either a featural or a configural processing strategy (Experiment 1) and by investigating the attentional strategies in response to emotional expressions (Experiment 2). In Experiment 1, participants identified emotional expressions in faces that were presented in three different versions (intact, blurred, and scrambled) and in two orientations (upright and inverted). Blurred faces contain mainly configural information, and scrambled faces contain mainly featural information. Inversion is known to selectively hinder configural processing. Analyses of the discriminability measure (A′) and response times (RTs) revealed that configural processing plays a more prominent role in expression recognition than featural processing, but their relative contribution varies depending on the emotion. In Experiment 2, we qualified these differences between emotions by investigating the relative importance of specific features by means of eye movements. Participants had to match intact expressions with the emotional cues that preceded the stimulus. The analysis of eye movements confirmed that the recognition of different emotions rely on different types of information. While the mouth is important for the detection of happiness and fear, the eyes are more relevant for anger, fear, and sadness.     

Developmental changes in face processing skills

Expertise in processing differences among faces in the spacing among facial features (second-order relations) is slower to develop than expertise in processing the shape of individual features or the shape of the external contour. To determine the impact of the slow development of sensitivity to second-order relations on various face-processing skills, we developed five computerized tasks that require matching faces on the basis of identity (with changed facial expression or head orientation), facial expression, gaze direction, and sound being spoken. In Experiment 1, we evaluated the influence of second-order relations on performance on each task by presenting them to adults (N=48) who viewed the faces either upright or inverted. Previous studies have shown that inversion has a larger effect on tasks that require processing the spacing among features than it does on tasks that can be solved by processing the shape of individual features. Adults showed an inversion effect for only one task: matching facial identity when there was a change in head orientation. In Experiment 2, we administered the same tasks to children aged 6, 8, and 10 years (N=72). Compared to adults, 6-year-olds made more errors on every task and 8-year-olds made more errors on three of the five tasks: matching direction of gaze and the two facial identity tasks. Ten-year-olds made more errors than adults on only one task: matching facial identity when there was a change in head orientation (e.g., from frontal to tilted up). Together, the results indicate that the slow development of sensitivity to second-order relations causes children to be especially poor at recognizing the identity of a face when it is seen in a new orientation.     

A comparative study of face processing using scrambled faces

It is a widespread assumption that all primate species process faces in the same way because the species are closely related and they engage in similar social interactions. However, this approach ignores potentially interesting and informative differences that may exist between species. This paper describes a comparative study of holistic face processing. Twelve subjects (six chimpanzees Pan troglodytes and six rhesus monkeys Macaca mulatta) were trained to discriminate whole faces (faces with features in their canonical position) and feature-scrambled faces in two separate conditions. We found that both species tended to match the global configuration of features over local features, providing strong evidence of global precedence. In addition, we show that both species were better able to generalize from a learned configuration to an entirely novel configuration when they were first trained to match feature-scrambled faces compared to when they were trained with whole faces. This result implies that the subjects were able to access local information easier when facial features were presented in a scrambled configuration and is consistent with a holistic processing hypothesis. Interestingly, these data also suggest that, while holistic processing in chimpanzees is tuned to own-species faces, monkeys have a more general approach towards all faces. Thus, while these data confirm that both chimpanzees and rhesus monkeys process faces holistically, they also indicate that there are differences between the species that warrant further investigation.     

The face inversion effect—Parts and wholes: Individual features and their configuration

The face inversion effect (FIE) is a reduction in recognition performance for inverted faces (compared to upright faces) that is greater than that typically observed with other stimulus types (e.g., houses). The work of Diamond and Carey, suggests that a special type of configural information, “second-order relational information” is critical in generating this inversion effect. However, Tanaka and Farah concluded that greater reliance on second-order relational information did not directly result in greater sensitivity to inversion, and they suggested that the FIE is not entirely due to a reliance on this type of configural information. A more recent review by McKone and Yovel provides a meta-analysis that makes a similar point. In this paper, we investigated the contributions made by configural and featural information to the FIE. Experiments 1a and1b investigated the link between configural information and the FIE. Remarkably, Experiment 1b showed that disruption of all configural information of the type considered in Diamond and Carey's analysis (both first and second order) was effective in reducing recognition performance, but did not significantly impact on the FIE. Experiments 2 and 3 revealed that face processing is affected by the orientation of individual features and that this plays a major role in producing the FIE. The FIE was only completely eliminated when we disrupted the single feature orientation information in addition to the configural information, by using a new type of transformation similar to Thatcherizing our sets of scrambled faces. We conclude by noting that our results for scrambled faces are consistent with an account that has recognition performance entirely determined by the proportion of upright facial features within a stimulus, and that any ability to make use of the spatial configuration of these features seems to benefit upright and inverted normal faces alike.     

Face processing in cotton-top tamarins (Saguinus oedipus)

Current research on face processing in primates has focused on a few species, mostly macaques and chimpanzees; to date, only one New World monkey, the squirrel monkey, has been tested. We explored face processing, and the inversion effect in particular, in a New World primate species, the cotton-top tamarin (Saguinus oedipus). In phase 1 of our study, we trained subjects to discriminate between two faces and two scrambled faces; we then presented the tamarins with a series of novel probes in order to determine the features underlying classification. Results showed that the tamarins relied on the external contour of the face for discrimination more than the internal features and their configuration. Statistical analyses revealed no differences in accuracy or response times to upright versus inverted stimuli, and thus no inversion effect. In phase 2, we provided subjects with additional training on the face versus scrambled face discrimination task in order to focus their attention on the configuration of the internal features. Accuracy data revealed individual differences in how tamarins classified these stimuli, even though each subject was trained in the same way. In phase 3, we tested for generalization to a new set of face stimuli, as well as for the capacity to show an inversion effect. For one subject who attended to the configuration of internal features, we found significant evidence of generalization, but no evidence for an inversion effect. Electronic Publication     

Attention to internal face features in unfamiliar face matching

Accurate matching of unfamiliar faces is vital in security and forensic applications, yet previous research has suggested that humans often perform poorly when matching unfamiliar faces. Hairstyle and facial hair can strongly influence unfamiliar face matching but are potentially unreliable cues. This study investigated whether increased attention to the more stable internal face features of eyes, nose, and mouth was associated with more accurate face‐matching performance. Forty‐three first‐year psychology students decided whether two simultaneously presented faces were of the same person or not. The faces were displayed for either 2 or 6 seconds, and had either similar or dissimilar hairstyles. The level of attention to internal features was measured by the proportion of fixation time spent on the internal face features and the sensitivity of discrimination to changes in external feature similarity. Increased attention to internal features was associated with increased discrimination in the 2‐second display‐time condition, but no significant relationship was found in the 6‐second condition. Individual differences in eye‐movements were highly stable across the experimental conditions.     

Holistic processing and reliance on global viewing strategies in older adults' face perception

There is increasing evidence that face recognition might be impaired in older adults, but it is unclear whether the impairment is truly perceptual, and face specific. In order to address this question we compared performance in same/different matching tasks with face and non-face objects (watches) among young (mean age 23.7) and older adults (mean age 70.4) using a context congruency paradigm (Meinhardt-Injac, Persike & Meinhardt, 2010, Meinhardt-Injac, Persike and Meinhardt, 2011a). Older adults were less accurate than young adults with both object classes, while face matching was notably impaired. Effects of context congruency and inversion, measured as the hallmarks of holistic processing, were equally strong in both age groups, and were found only for faces, but not for watches. The face specific decline in older adults revealed deficits in handling internal facial features, while young adults matched external and internal features equally well. Comparison with non-face stimuli showed that this decline was face specific, and did not concern processing of object features in general. Taken together, the results indicate no age-related decline in the capabilities to process faces holistically. Rather, strong holistic effects, combined with a loss of precision in handling internal features indicate that older adults rely on global viewing strategies for faces. At the same time, access to the exact properties of inner face details becomes restricted.     

Symmetrical interaction of sex and expression in face classification tasks

Classification of faces as to their sex or their expression—with sex and expression varying orthogonally—was studied in three experiments. In Experiment 1, expression classification was influenced by sex, with angry male faces being classified faster than angry female faces. Complementarily, sex classification was faster for happy than for angry female faces. In Experiment 2, mutual interaction of sex and expression was also found when the participants were asked to classify top and bottom face segments. In Experiment 3, a face inversion effect was found for both sex and expression classification of whole faces. However, a symmetrical interaction between sex and expression was again found. The results are discussed in terms of configural versus feature processing in the perception of face sex and expression and of their relevance to face perception models that postulate independent processing of different facial features. 2009 The Psychonomic Society, Inc.     

Expert face coding: Configural and component coding of own-race and other-race faces

Configural and holistic coding are hallmarks of face perception. Although recent studies have shown that own-race faces are coded more holistically than other-race faces, the evidence for better configural coding of own-race faces is equivocal. We directly measured configural and component coding of own- and other-race male faces in Caucasian and Chinese participants. We manipulated individual features (components) or their spatial relations (configurations) using a novel morphing method to vary difficulty parametrically and tested sensitivity to these changes in a sequential matching task. Both configural and component coding were better for upright own-race than for upright other-race faces. Inversion impaired detection of configural changes more than it did detection of component changes, but also impaired performance more for easier discriminations,independent of type of change. These results challenge explanations of face expertise that rely solely on configural and holistic processing, and also call into question the widespread interpretation of large inversion decrements as diagnostic of configural coding.     

Two routes to face perception: evidence from psychophysics and computational modeling

The aim of this study was to separately analyze the role of featural and configural face representations. Stimuli containing only featural information were created by cutting the faces into their parts and scrambling them. Stimuli only containing configural information were created by blurring the faces. Employing an old-new recognition task, the aim of Experiments 1 and 2 was to investigate whether unfamiliar faces (Exp. 1) or familiar faces (Exp. 2) can be recognized if only featural or configural information is provided. Both scrambled and blurred faces could be recognized above chance level. A further aim of Experiments 1 and 2 was to investigate whether our method of creating configural and featural stimuli is valid. Pre-activation of one form of representation did not facilitate recognition of the other, neither for unfamiliar faces (Exp. 1) nor for familiar faces (Exp. 2). This indicates a high internal validity of our method for creating configural and featural face stimuli. Experiment 3 examined whether features placed in their correct categorical relational position but with distorted metrical distances facilitated recognition of unfamiliar faces. These faces were recognized no better than the scrambled faces in Experiment 1, providing further evidence that facial features are stored independently of configural information. From these results we conclude that both featural and configural information are important to recognize a face and argue for a dual-mode hypothesis of face processing. Using the psychophysical results as motivation, we propose a computational framework that implements featural and configural processing routes using an appearance-based representation based on local features and their spatial relations. In three computational experiments (Experiments 4–6) using the same sets of stimuli, we show how this framework is able to model the psychophysical data.     

The contribution of external and internal features to the matching of unfamiliar faces

The relative efficacy of external and internal features in matching unfamiliar faces was studied in three experiments in which the subjects matched target and test faces differing in terms of external or internal features, or both. In Experiment 1 only full congruency between target and test faces was considered a match; in Experiments 2 and 3 faces sharing the same external and internal features were also considered to be matches. A total of 100 subjects matched 192 pairs of target and test faces in a same-different task. Reaction times and matching errors were recorded for analyses of variance. In all three experiments performance was best when either all features matched or all features mismatched, with mismatches having a slight edge. When matches of external and internal features with the target faces were inconsistent with each other, mismatches of external features led to faster responses in Experiments 1 and 2, and mismatches of internal features led to faster responses in Experiment 3. The results suggest that since faces are configurational stimuli, face matching is influenced by the non-relevant set of features; and that mismatches, especially of external features, influence face matching more than matches do.     

Demonstrating how unfamiliar faces become familiar using a face matching task

Two experiments examine a novel method of assessing face familiarity that does not require explicit identification of presented faces. Earlier research (Clutterbuck & Johnston, 2002; Young, Hay, McWeeny, Flude, & Ellis, 1985) has shown that different views of the same face can be matched more quickly for familiar than for unfamiliar faces. This study examines whether exposure to previously novel faces allows the speed with which they can be matched to be increased, thus allowing a means of assessing how faces become familiar. In Experiment 1, participants viewed two sets of unfamiliar faces presented for either many, short intervals or for few, long intervals. At test, previously familiar (famous) faces were matched more quickly than novel faces or learned faces. In addition, learned faces seen on many, brief occasions were matched more quickly than the novel faces or faces seen on fewer, longer occasions. However, this was only observed when participants performed “different” decision matches. In Experiment 2, the similarity between face pairs was controlled more strictly. Once again, matches were performed on familiar faces more quickly than on unfamiliar or learned items. However, matches made to learned faces were significantly faster than those made to completely novel faces. This was now observed for both same and different match decisions. The use of this matching task as a means of tracking how unfamiliar faces become familiar is discussed.     

The ups and downs of face perception: evidence for holistic encoding of upright and inverted faces

A visual-search task was used to investigate the influence of facial organisation on discrimination of an internal facial feature. Participants searched for a downturned mouth in arrays of one to six faces that differed only in the target feature, with distractor faces containing an upturned mouth. Feature search was tested in four different face contexts: upright unaltered faces, inverted unaltered faces, upright faces in which the internal features were scrambled, or inverted scrambled faces. Normal face organisation facilitated feature search in upright faces, but slowed it in inverted faces. These findings demonstrate an interdependence of features and their configuration in the perceptual analysis of both upright and inverted faces.     

Effects of geometric distortions,Gaussian blur,and contrast negation on recognition of familiar faces

It is assumed that “configural processing” is important for recognizing people whom we know. Studies have shown better discrimination of relational changes between facial features when faces are presented upright than upside down and when presented as photographic negatives. However, studies have also demonstrated tolerance for complex changes to facial configuration of familiar faces when images are geometrically distorted. In two experiments, we explored effects of linear stretching, Gaussian blur, and contrast negation on recognition of familiar or unfamiliar faces (Experiment 1) and familiar or unfamiliar company brand logos (Experiment 2). Only contrast negation severely impaired recognition of familiar faces but not unfamiliar faces or company brand logos. Effects of contrast negation were additional to linear stretching, whereas linear stretching independently had no impact on recognition. These findings highlight an important deficit in the recognition of familiar faces caused by contrast negation.