The ups and downs of face perception: evidence for holistic encoding of upright and inverted faces

A visual-search task was used to investigate the influence of facial organisation on discrimination of an internal facial feature. Participants searched for a downturned mouth in arrays of one to six faces that differed only in the target feature, with distractor faces containing an upturned mouth. Feature search was tested in four different face contexts: upright unaltered faces, inverted unaltered faces, upright faces in which the internal features were scrambled, or inverted scrambled faces. Normal face organisation facilitated feature search in upright faces, but slowed it in inverted faces. These findings demonstrate an interdependence of features and their configuration in the perceptual analysis of both upright and inverted faces.     

Perceptual integrality of componential and configural information in faces

The relative contribution of componential and configural information to face perception is controversial. We addressed this issue in the present study by examining how componential information and configural information interact during face processing, using Garner’s (1974) speeded classification paradigm. When classifying upright faces varying in components (eyes, nose, and mouth) and configural information (intereyes and nose-mouth spacing), observers could not selectively attend to components without being influenced by irrelevant variation in configural information, and vice versa, indicating that componential information and configural information are integral in upright face processing. Performance with inverted faces showed selective attention to components but not to configural information, implying dominance of componential information in processing inverted faces. When faces varied only in components, selective attention to different components was observed in upright and inverted faces, indicating that facial components are perceptually separable. These results provide strong evidence that integrality of componential and configural information, rather than the relative dominance of either, is the hallmark of upright face perception.     

A unique look at face processing: the impact of masked faces on the processing of facial features

This experiment utilized a masked priming paradigm to explore the early processes involved in face recognition. The first experiment investigated implicit processing of the eyes and mouth in an upright face, using prime durations of 33 and 50 ms. The results demonstrate implicit processing of both the eyes and mouth, and support the configural processing theory of face processing. The second experiment used the same method with inverted faces and the third experiment was a combination of Experiments 1 and 2. The fourth experiment utilized misaligned faces as the primes. Based on the pattern of results from these experiments, we suggest that, when a face is inverted, the eyes and mouth are initially processed individually and are not linked until a later stage of processing. An upright face is proposed to be processed by analysis of its configuration, whereas an inverted face is initially processed using first-order relational information, and then converted to an upright representation and transferred to face specific regions for configural analysis.     

Perception of Mooney faces by young infants: the role of local feature visibility, contrast polarity, and motion

We examined the ability of young infants (3- and 4-month-olds) to detect faces in the two-tone images often referred to as Mooney faces. In Experiment 1, this performance was examined in conditions of high and low visibility of local features and with either the presence or absence of the outer head contour. We found that regardless of the presence of the outer head contour, infants preferred upright over inverted two-tone face images only when local features were highly visible (Experiment 1a). We showed that this upright preference disappeared when the contrast polarity of two-tone images was reversed (Experiment 1b), reflecting operation of face-specific mechanisms. In Experiment 2, we investigated whether motion affects infants' perception of faces in Mooney faces. We found that when the faces appeared to be rigidly moving, infants did show an upright preference in conditions of low visibility of local features (Experiment 2a). Again the preference disappeared when the contrast polarity of the image was reversed (Experiment 2b). Together, these results suggest that young infants have the ability to integrate fragmented image features to perceive faces from two-tone face images, especially if they are moving. This suggests that an interaction between motion and form rather than a purely motion-based process (e.g., structure from motion) facilitates infants' perception of faces in ambiguous two-tone images.     

The influence of divided attention on holistic face perception.

There is evidence that upright, but not inverted, faces are encoded holistically. The holistic coding of faces was examined in four experiments by manipulating the attention allocated to target faces. In Experiment 1, participants in a divided attention condition were asked to match two upright flanker faces while encoding a centrally presented upright target face. Although holistic coding was evident in the full attention conditions, dividing attention disrupted holistic coding of target faces. In Experiment 2, we found that while matching upright flanker faces disrupted holistic coding, matching inverted flanker faces did not. Experiment 3 demonstrated that the differential effects of flanker orientation were not due to participants taking longer to match upright, than inverted, flanker faces. In Experiment 4, we found that matching fractured faces had an intermediate effect to that of matching upright and inverted flankers, on the holistic coding of the target faces. The findings emphasize the differences in processing of upright, fractured and inverted faces and suggest that there are limitations in the number of faces that can be holistically coded in a brief time.     

How the Thatcher illusion reveals evolutionary differences in the face processing of primates

Face recognition in humans is a complex cognitive skill that requires sensitivity to unique configurations of eyes, mouth, and other facial features. The Thatcher illusion has been used to demonstrate the importance of orientation when processing configural information within faces. Transforming an upright face so that the eyes and mouth are inverted renders the face grotesque; however, when this “Thatcherized” face is inverted, the effect disappears. Due to the use of primate models in social cognition research, it is important to determine the extent to which specialized cognitive functions like face processing occur across species. To date, the Thatcher illusion has been explored in only a few species with mixed results. Here, we used computerized tasks to examine whether nonhuman primates perceive the Thatcher illusion. Chimpanzees and rhesus monkeys were required to discriminate between Thatcherized and unaltered faces presented upright and inverted. Our results confirm that chimpanzees perceived the Thatcher illusion, but rhesus monkeys did not, suggesting species differences in the importance of configural information in face processing. Three further experiments were conducted to understand why our results differed from previously published accounts of the Thatcher illusion in rhesus monkeys.     

Beyond U-Shaped Development in Infants' Processing of Faces: An Information-Processing Account

The development of the "inversion" effect in face processing was examined in infants 3 to 6 months of age by testing their integration of the internal and external features of upright and inverted faces using a variation of the "switch" visual habituation paradigm. When combined with previous findings showing that 7-month-olds use integrative processing of an upright face, but featural processing of an inverted face (Cohen & Cashon, 2001a), the present findings suggest that from 3 to 7 months, infants' ability to integrate facial features follows an N-shaped developmental pattern for upright faces and an inverted U-shaped pattern for inverted faces. We discuss these results in terms of a set of domain-general information-processing principles.     

When feature information comes first! Early processing of inverted faces

We investigated the early stages of face recognition and the role of featural and holistic face information. We exploited the fact that, on inversion, the alienating disorientation of the eyes and mouth in thatcherised faces is hardly detectable. This effect allows featural and holistic information to be dissociated and was used to test specific face-processing hypotheses. In inverted thatcherised faces, the cardinal features are already correctly oriented, whereas in undistorted faces, the whole Gestalt is coherent but all information is disoriented. Experiment 1 and experiment 3 revealed that, for inverted faces, featural information processing precedes holistic information. Moreover, the processing of contextual information is necessary to process local featural information within a short presentation time (26 ms). Furthermore, for upright faces, holistic information seems to be available faster than for inverted faces (experiment 2). These differences in processing inverted and upright faces presumably cause the differential importance of featural and holistic information for inverted and upright faces.     

The effect of inversion on the encoding of normal and “thatcherized” faces

In the “Thatcher illusion” a face, in which the eyes and mouth are inverted relative to the rest of the face, looks grotesque when shown upright but not when inverted. In four experiments we investigated the contribution of local and global processing to this illusion in normal observers. We examined inversion effects (i.e., better performance for upright than for inverted faces) in a task requiring discrimination of whether faces were or were not “thatcherized”. Observers made same/different judgements to isolated face parts (Experiments 1-2) and to whole faces (Experiments 3-4). Face pairs had the same or different identity, allowing for different process- ing strategies using feature-based or configural information, respectively. In Experiment 1, feature-based matching of same-person face parts yielded only a small inversion effect for normal face parts. However, when feature-based matching was prevented by using the face parts of different people on all trials (Experiment 2) an inversion effect occurred for normal but not for thatcherized parts. In Experiments 3 and 4, inversion effects occurred with normal but not with thatcherized whole faces, on both same- and different-person matching tasks. This suggests that a common configural strategy was used with whole (normal) faces. Face context facilitated attention to misoriented parts in same-person but not in different-person matching. The results indicate that (1) face inversion disrupts local configural processing, but not the processing of image features, and (2) thatcherization disrupts local configural processing in upright faces.     

Face and mouth inversion effects on visual and audiovisual speech perception

Three experiments examined whether image manipulations known to disrupt face perception also disrupt visual speech perception. Research has shown that an upright face with an inverted mouth looks strikingly grotesque whereas an inverted face and an inverted face containing an upright mouth look relatively normal. The current study examined whether a similar sensitivity to upright facial context plays a role in visual speech perception. Visual and audiovisual syllable identification tasks were tested under 4 presentation conditions: upright face-upright mouth, inverted face-inverted mouth, inverted face-upright mouth, and upright face-inverted mouth. Results revealed that for some visual syllables only the upright face-inverted mouth image disrupted identification. These results suggest that upright facial context can play a role in visual speech perception. A follow-up experiment testing isolated mouths supported this conclusion.     

Facial perception of conspecifics: chimpanzees (Pan troglodytes) preferentially attend to proper orientation and open eyes

This paper reports on the use of an eye-tracking technique to examine how chimpanzees look at facial photographs of conspecifics. Six chimpanzees viewed a sequence of pictures presented on a monitor while their eye movements were measured by an eye tracker. The pictures presented conspecific faces with open or closed eyes in an upright or inverted orientation in a frame. The results demonstrated that chimpanzees looked at the eyes, nose, and mouth more frequently than would be expected on the basis of random scanning of faces. More specifically, they looked at the eyes longer than they looked at the nose and mouth when photographs of upright faces with open eyes were presented, suggesting that particular attention to the eyes represents a spontaneous face-scanning strategy shared among monkeys, apes, and humans. In contrast to the results obtained for upright faces with open eyes, the viewing times for the eyes, nose, and mouth of inverted faces with open eyes did not differ from one another. The viewing times for the eyes, nose, and mouth of faces with closed eyes did not differ when faces with closed eyes were presented in either an upright or inverted orientation. These results suggest the possibility that open eyes play an important role in the configural processing of faces and that chimpanzees perceive and process open and closed eyes differently.     

A new look at social attention: Orienting to the eyes is not (entirely) under volitional control

People tend to look at other people's eyes, but whether this bias is automatic or volitional is unclear. To discriminate between these two possibilities, we used a "don't look" (DL) paradigm. Participants looked at a series of upright or inverted faces, and were asked either to freely view the faces or to avoid looking at the eyes, or as a control, the mouth. As previously demonstrated, participants showed a bias to attend to both eyes and mouths during free viewing. In the DL condition, participants told to avoid the eyes of upright faces were unable to fully suppress the tendency to fixate on the faces' eyes, whereas participants told to avoid the mouth of upright faces successfully eliminated their bias to overtly attend to that feature. When faces were inverted, participants were equally able to suppress looks to the eyes and mouth. Together, these results suggest that the tendency to look at the eyes reflects orienting that is both volitional and automatic, and that the engagement of holistic or configural face processing mechanisms during upright face viewing has an influence in guiding gaze automatically to the eyes. (PsycINFO Database Record (c) 2012 APA, all rights reserved).     

Demystifying infant vocal imitation: The roles of mouth looking and speaker’s gaze

Vocal imitation plays a fundamental role in human language acquisition from infancy. Little is known, however, about how infants imitate other's sounds. We focused on three factors: (a) whether infants receive information from upright faces, (b) the infant's observation of the speaker's mouth and (c) the speaker directing their gaze towards the infant. We recorded the eye movements of 6‐month‐olds who participated in experiments watching videos of a speaker producing vowel sounds. We found that an infants’ tendency to vocally imitate such videos increased as a function of (a) seeing upright rather than inverted faces, (b) their increased looking towards the speaker's mouth and (c) whether the speaker directed their gaze towards, rather than away from infants. These latter findings are consistent with theories of motor resonance and natural pedagogy respectively. New light has been shed on the cues and underlying mechanisms linking infant speech perception and production.     

Revisiting the perception of upside-down faces

In two experiments, the effect of orientation on face perception was assessed. Using a scale from 1 ( normal ) to 7 ( bizarre ), participants rated normal, unaltered faces and faces in which changes had been made to spatial-relational properties (eyes and mouth inverted or relative position of the eyes and mouth altered) or to component properties (eyes whitened and teeth blackened). For unaltered and component-distortion faces, bizarreness ratings increased linearly as orientation increased from 0° to 180°. For spatial-distortion faces, a discontinuity in the function relating orientation and bizarreness was in evidence between 90° and 120°. The results provide support for the view that there is a qualitative difference in the processing of upright and inverted faces due to the disproportionate effect of inversion on the encoding of spatial-relational information.     

Attention misplaced: the role of diagnostic features in the face-inversion effect

Inversion disproportionately impairs recognition of face stimuli compared to nonface stimuli arguably due to the holistic manner in which faces are processed. A qualification is put forward in which the first point fixated on is different for upright and inverted faces and this carries some of the face-inversion effect. Three experiments explored this possibility by using fixation crosses to guide attention to the eye or mouth region of the to-be-presented faces in different orientations. Recognition was better when the fixation cross appeared at the eye region than at the mouth region. The face-inversion effect was smaller when the eyes were cued than when the mouth was cued or when there was no cueing. The results suggest that the first facial feature attended to is important for accurate face recognition and this may carry some of the effects of inversion.     

The representations of spacing and part-based information are associated for upright faces but dissociated for objects: Evidence from individual differences

Considerable evidence suggests that qualitatively different processes are involved in the perception of faces and objects. According to a central hypothesis, the extraction of information about the spacing among face parts (e.g., eyes and mouth) is a primary function of face processing mechanisms that is dissociated from the extraction of information about the shape of these parts. Here, we used an individual-differences approach to test whether the shape of face parts and the spacing among them are indeed processed by dissociated mechanisms. To determine whether the pattern of findings that we reveal is unique for upright faces, we also presented similarly manipulated nonface stimuli. Subjects discriminated upright or inverted faces or houses that differed in parts or spacing. Only upright faces yielded a large positive correlation across subjects between performance on the spacing and part discrimination tasks. We found no such correlation for inverted faces or houses. Our findings suggest that face parts and spacing are processed by associated mechanisms, whereas the parts and spacing of nonface objects are processed by distinct mechanisms. These results may be consistent with the idea that faces are special, in that they are processed as nondecomposable wholes.     

Face processing in humans and new world monkeys: the influence of experiential and ecological factors

This study tests whether the face-processing system of humans and a nonhuman primate species share characteristics that would allow for early and quick processing of socially salient stimuli: a sensitivity toward conspecific faces, a sensitivity toward highly practiced face stimuli, and an ability to generalize changes in the face that do not suggest a new identity, such as a face differently oriented. The look rates by adult tamarins and humans toward conspecific and other primate faces were examined to determine if these characteristics are shared. A visual paired comparison (VPC) task presented subjects with either a human face, chimpanzee face, tamarin face, or an object as a sample, and then a pair containing the previous stimulus and a novel stimulus was presented. The stimuli were either presented all in an upright orientation, or all in an inverted orientation. The novel stimulus in the pair was either an orientation change of the same face/object or a new example of the same type of face/object, and the stimuli were shown either in an upright orientation or in an inverted orientation. Preference to novelty scores revealed that humans attended most to novel individual human faces, and this effect decreased significantly if the stimuli were inverted. Tamarins showed preferential looking toward novel orientations of previously seen tamarin faces in the upright orientation, but not in an inverted orientation. Similarly, their preference to look longer at novel tamarin and human faces within the pair was reduced significantly with inverted stimuli. The results confirmed prior findings in humans that novel human faces generate more attention in the upright than in the inverted orientation. The monkeys also attended more to faces of conspecifics, but showed an inversion effect to orientation change in tamarin faces and to identity changes in tamarin and human faces. The results indicate configural processing restricted to particular kinds of primate faces by a New World monkey species, with configural processing influenced by life experience (human faces and tamarin faces) and specialized to process orientation changes specific to conspecific faces.     

Attending to faces: change detection,familiarization, and inversion effects

We tested detection of changes to eye position, eye color (brightness), mouth position, and mouth color in frontal views of faces. Two faces were presented sequentially for 555 ms each, with a blank screen of 120 ms separating the two. Faces were presented either both upright or both inverted. Measures of detection (d') were calculated for several different degrees of change for each of the four dimensions of change. We first compared results to an earlier experiment that used an oddity design, in which subjects indicated which of three simultaneously viewed and otherwise identical faces had been altered on one of these four dimensions. Subjects in both of these experiments were partially cued, in that they knew the four possible types of changes that could occur on a given trial. The change-detection results correlated well with the oddity data. They confirmed that face inversion had little effect upon detection of changes in eye color, a moderate effect upon detection of eye-position or mouth-color changes, and caused a drastic reduction in the detection of mouth-position changes. An experiment in which uncued and fully cued subjects were compared showed that cueing significantly improved detection of feature color changes, but there was little difference between upright and inverted faces. Full cueing eliminated all effects of inversion. Compared to partial cueing, changes in mouth color were poorly detected by uncued subjects. Last, a change in the frequency of the base (unaltered) face in an experiment from 75% to 40% showed that increased short-term familiarity decreased the detection of eye changes and increased the detection of mouth changes, regardless of face orientation and the type of change made (color or position). We conclude that uncued subjects encode the spatial relations of features more than the colors of features, that mouth color in particular is not considered a relevant dimension for encoding, and that familiarization redistributes attention from more to less salient facial regions. Inversion effects are not simply an exaggeration of the salience effects revealed by withdrawing cueing, but represent an interaction of spatial encoding with salience, in that the greatest inversion effects occur for spatial shifts in less salient facial regions, and can be eliminated through the use of focused attention.     

The development of expert face processing: are infants sensitive to normal differences in second-order relational information?

Sensitivity to second-order relational information (i.e., spatial relations among features such as the distance between eyes) is a vital part of achieving expertise with face processing. Prior research is unclear on whether infants are sensitive to second-order differences seen in typical human populations. In the current experiments, we examined whether infants are sensitive to changes in the space between the eyes and between the nose and the mouth that are within the normal range of variability in Caucasian female faces. In Experiment 1, 7-month-olds detected these changes in second-order relational information. Experiment 2 extended this finding to 5-month-olds and also found that infants detect second-order relations in upright faces but not in inverted faces, thereby exhibiting an inversion effect that has been considered to be a hallmark of second-order relational processing during adulthood. These results suggest that infants as young as 5 months are sensitive to second-order relational changes that are within the normal range of human variability. They also indicate that at least rudimentary aspects of face processing expertise are available early in life.     

Facial Attention and Spacetime Fragments

Inverting a face impairs perception of its features and recognition of its identity. Whether faces are special in this regard is a current topic of research and debate. Kanizsa studied the role of facial features and environmental context in perceiving the emotion and identity of upright and inverted faces. He found that observers are biased to interpret faces in a retinal coordinate frame, and that this bias is readily overruled by increased realism of facial features, but not easily overruled by environmental context. An additional factor contributing to a retinal coordinate-frame interpretation may be the ambiguous nature of the face stimuli. Since his facial expressions are interpretable both upright and inverted, they may in both orientations activate an endogenous attentional process for faces. We present visual search and change-blindness experiments that explore how inversion, negation, and facial emotion affect visual attention to static faces. We find that attention to faces is impaired by inversion and negation. We also find that the parts of the face that receive greater attention can be influenced by the emotional expression of the face. We propose to extend these experiments to dynamic faces. To this end, we develop a theory of the visual representation of dynamic faces, in which faces are represented by classes of `spacetime fragments'-moving regions of the face with high informational content. We then present ideas for future experiments which are motivated by the spacetime fragment theory, and which should serve to constrain its further development.