Holistic face processing can be independent of gaze behaviour: Evidence from the composite face illusion

People tend to perceive identical top halves (i.e. above the nose) of two face stimuli as being different when they are aligned with distinct bottom halves. This composite face illusion is generally considered as the most compelling evidence that facial features are integrated into a holistic representation. Here, we recorded eye‐movements during the composite face illusion in a delayed matching task of top halves of faces. Behavioural results showed a strong composite face effect, participants making more mistakes and taking longer time to match two identical top halves of faces when they were aligned (vs. misaligned) with different bottom halves. Importantly, fixation sites and eye‐movements were virtually identical when the top and bottom parts were aligned (composite illusion) or misaligned (no illusion), indicating that holistic face processing can be independent of gaze behaviour. These findings reinforce the view that holistic representations of individual faces can be extracted early on from information at a relatively coarse scale, independently of overt attention.     

Integration of internal and external facial features in 8- to 10-year-old children and adults

Investigation of whole-part and composite effects in 4- to 6-year-old children gave rise to claims that face perception is fully mature within the first decade of life (Crookes & McKone, 2009). However, only internal features were tested, and the role of external features was not addressed, although external features are highly relevant for holistic face perception (Sinha & Poggio, 1996; Axelrod & Yovel, 2010, 2011). In this study, 8- to 10-year-old children and adults performed a same–different matching task with faces and watches. In this task participants attended to either internal or external features. Holistic face perception was tested using a congruency paradigm, in which face and non-face stimuli either agreed or disagreed in both features (congruent contexts) or just in the attended ones (incongruent contexts). In both age groups, pronounced context congruency and inversion effects were found for faces, but not for watches. These findings indicate holistic feature integration for faces. While inversion effects were highly similar in both age groups, context congruency effects were stronger for children. Moreover, children's face matching performance was generally better when attending to external compared to internal features. Adults tended to perform better when attending to internal features. Our results indicate that both adults and 8- to 10-year-old children integrate external and internal facial features into holistic face representations. However, in children's face representations external features are much more relevant. These findings suggest that face perception is holistic but still not adult-like at the end of the first decade of life.     

Why does selective attention to parts fail in face processing?

One hallmark of holistic face processing is an inability to selectively attend to 1 face part while ignoring information in another part. In 3 sequential matching experiments, the authors tested perceptual and decisional accounts of holistic processing by measuring congruency effects between cued and uncued composite face halves shown in spatially aligned or disjointed configurations. The authors found congruency effects when the top and bottom halves of the study face were spatially aligned, misaligned (Experiment 1), or adjacent to one another (Experiment 2). However, at test, congruency effects were reduced by misalignment and abolished for adjacent configurations. This suggests that manipulations at test are more influential than manipulations at study, consistent with a decisional account of holistic processing. When encoding demands for study and test faces were equated (Experiment 3), the authors observed effects of study configuration suggesting that, consistent with a perceptual explanation, encoding does influence the magnitude of holistic processing. Together, these results cannot be accounted for by current perceptual or decisional accounts of holistic processing and suggest the existence of an attention-dependent mechanism that can integrate spatially separated face parts.     

Dissociation between the behavioural and electrophysiological effects of the face and body composite illusions

Several studies have reported similarities between perceptual processes underlying face and body perception, particularly emphasizing the importance of configural processes. Differences between the perception of faces and the perception of bodies were observed by means of a manipulation targeting a specific subtype of configural processing: the composite illusion. The composite face illusion describes the fact that two identical top halves of a face are perceived as being different if they are presented with different bottom parts. This effect disappears, if both halves are laterally shifted. Crucially, the effect of misalignment is not observed for bodies. This study aimed to further explore differences in the time course of face and body perception by using the composite effect. The present results replicated behavioural effects illustrating that misalignment affects the perception of faces but not bodies. Thus, face but not body perception relies on holistic processing. However, differences in the time course of the processing of both stimulus categories emerged at the N170 and P200. The pattern of the behavioural data seemed to be related to the P200. Thus, the present data indicate that holistic processes associated with the effect of misalignment might occur 200 ms after stimulus onset.     

Disrupting perceptual grouping of face parts impairs holistic face processing

Face perception is widely believed to involve integration of facial features into a holistic perceptual unit, but the mechanisms underlying this integration are relatively unknown. We examined whether perceptual grouping cues influence a classic marker of holistic face perception, the “composite-face effect.” Participants made same–different judgments about a cued part of sequentially presented chimeric faces, and holistic processing was indexed as the degree to which the task-irrelevant face halves impacted performance. Grouping was encouraged or discouraged by adjusting the backgrounds behind the face halves: Although the face halves were always aligned, their respective backgrounds could be misaligned and of different colors. Holistic processing of face, but not of nonface, stimuli was significantly reduced when the backgrounds were misaligned and of different colors, cues that discouraged grouping of the face halves into a cohesive unit (Exp. 1). This effect was sensitive to stimulus orientation at short (200 ms) but not at long (2,500 ms) encoding durations, consistent with the previously documented temporal properties of the holistic processing of upright and inverted faces (Exps. 2 and 3). These results suggest that grouping mechanisms, typically involved in the perception of objecthood more generally, might contribute in important ways to the holistic perception of faces.     

The effect of face orientation on holistic processing

Holistic processing, a hallmark of face processing, can be measured by the composite face effect: adults have difficulty recognising the top half of a face when it is aligned with a new bottom half, unless holistic processing is disrupted by misaligning the two halves. Like the recognition of facial identity, holistic processing is impaired when faces are inverted. To obtain a more refined measure of the influence of orientation on holistic face processing, we administered the composite face task to adults at each of seven orientations. In both experiment 1 (in which orientations were randomly intermixed) and experiment 2 (in which orientations were blocked), the composite face effect decreased linearly with rotation. We conclude that holistic processing is tuned to upright faces, diminishes as faces deviate from upright, and becomes insignificant once faces reach a sideways orientation. We propose a hierarchical model of face perception in which linear decreases in holistic processing underlie qualitative shifts in other aspects of face perception.     

The role of holistic processing in judgments of facial attractiveness

Previous work has demonstrated that facial identity recognition, expression recognition, gender categorisation, and race categorisation rely on a holistic representation. Here we examine whether a holistic representation is also used for judgments of facial attractiveness. Like past studies, we used the composite paradigm to assess holistic processing (Young et al 1987, Perception 16 747-759). Experiment 1 showed that top halves of upright faces are judged to be more attractive when aligned with an attractive bottom half than when aligned with an unattractive bottom half. To assess whether this effect resulted from holistic processing or more general effects, we examined the impact of the attractive and unattractive bottom halves when upright halves were misaligned and when aligned and misaligned halves were presented upside-down. The bottom halves had no effect in either condition. These results demonstrate that the perceptual processes underlying upright facial-attractiveness judgments represent the face holistically. Our findings with attractiveness judgments and previous demonstrations involving other aspects of face processing suggest that a common holistic representation is used for most types of face processing.     

Configurational information in face perception

A new facial composites technique is demonstrated, in which photographs of the top and bottom halves of different familiar faces fuse to form unfamiliar faces when aligned with each other. The perception of a novel configuration in such composite stimuli is sufficiently convincing to interfere with identification of the constituent parts (experiment 1), but this effect disappears when stimuli are inverted (experiment 2). Difficulty in identifying the parts of upright composites is found even for stimuli made from parts of unfamiliar faces that have only ever been encountered as face fragments (experiment 3). An equivalent effect is found for composites made from internal and external facial features of well-known people (experiment 4). These findings demonstrate the importance of configurational information in face perception, and that configurations are only properly perceived in upright faces.     

Holistic processing and reliance on global viewing strategies in older adults' face perception

There is increasing evidence that face recognition might be impaired in older adults, but it is unclear whether the impairment is truly perceptual, and face specific. In order to address this question we compared performance in same/different matching tasks with face and non-face objects (watches) among young (mean age 23.7) and older adults (mean age 70.4) using a context congruency paradigm (Meinhardt-Injac, Persike & Meinhardt, 2010, Meinhardt-Injac, Persike and Meinhardt, 2011a). Older adults were less accurate than young adults with both object classes, while face matching was notably impaired. Effects of context congruency and inversion, measured as the hallmarks of holistic processing, were equally strong in both age groups, and were found only for faces, but not for watches. The face specific decline in older adults revealed deficits in handling internal facial features, while young adults matched external and internal features equally well. Comparison with non-face stimuli showed that this decline was face specific, and did not concern processing of object features in general. Taken together, the results indicate no age-related decline in the capabilities to process faces holistically. Rather, strong holistic effects, combined with a loss of precision in handling internal features indicate that older adults rely on global viewing strategies for faces. At the same time, access to the exact properties of inner face details becomes restricted.     

Holistic face processing is mature at 4 years of age: evidence from the composite face effect

Although it is acknowledged that adults integrate features into a representation of the whole face, there is still some disagreement about the onset and developmental course of holistic face processing. We tested adults and children from 4 to 6 years of age with the same paradigm measuring holistic face processing through an adaptation of the composite face effect [Young, A. W., Hellawell, D., & Hay, D. C. (1987). Configurational information in face perception. Perception, 16, 747-759]. In Experiment 1, only 6-year-old children and adults tended to perceive the two identical top parts as different, suggesting that holistic face processing emerged at 6 years of age. However, Experiment 2 suggested that these results could be due to a response bias in children that was cancelled out by always presenting two faces in the same format on each trial. In this condition, all age groups present strong composite face effects, suggesting that holistic face processing is mature as early as after 4 years of experience with faces.     

The context congruency effect is face specific

There is evidence that faces are processed by specialized and independent modules that treat them as global configurations, or wholes (Axelrod and Yovel, 2010, Kanwisher et al., 1997). The holistic nature of face perception has been demonstrated with several experimental paradigms designed to examine whether facial parts interact, or are accessed independently. A recently introduced paradigm (Meinhardt-Injac, Persike, & Meinhardt, 2010) measures the strength of contextual interaction among internal and external facial features in congruent and incongruent target/no-target relationships. For this paradigm it is shown that the context congruency effect is indeed face specific: A strong and asymmetric contextual interaction of the inner and the outer stimulus regions exists for faces, but is absent for watches, which represent a non-facial object class with a comparable inner/outer object structure.     

Electrophysiological correlates of the composite face illusion: Disentangling perceptual and decisional components of holistic face processing in the human brain

When the bottom halves of two faces differ, people’s behavioral judgment of the identical top halves of those faces is impaired: they report that the top halves are different, and/or take more time than usual to provide a response. This behavioral measure is known as the composite face effect (CFE) and has traditionally been taken as evidence that faces are perceived holistically. Recently, however, it has been claimed that this effect is driven almost entirely by decisional, rather than perceptual, factors ( Richler, Gauthier, Wenger, & Palmeri, 2008). To disentangle the contribution of perceptual and decisional brain processes, we aimed to obtain an event-related potential (ERP) measure of the CFE at a stage of face encoding ( Jacques & Rossion, 2009) in the absence of a behavioral CFE effect. Sixteen participants performed a go/no-go task in an oddball paradigm, lifting a finger of their right or left hand when the top half of a face changed identity. This change of identity of the top of the face was associated with an increased ERP signal on occipito-temporal electrode sites at the N170 face-sensitive component (∼160 ms), the later decisional P3b component, and the lateralized readiness potential (LRP) starting at ∼350 ms. The N170 effect was observed equally early when only the unattended bottom part of the face changed, indicating that an identity change was perceived across the whole face in this condition. Importantly, there was no behavioral response bias for the bottom change trials, and no evidence of decisional biases from electrophysiological data (no P3b and LRP deflection in no-go trials). These data show that an early CFE can be measured in ERPs in the absence of any decisional response bias, indicating that the CFE reflects primarily the visual perception of the whole face.     

Holistic face perception: Mind the gap!

The most widely used measurement of holistic face perception, the composite face effect (CFE), is challenged by two apparently contradictory goals: having a defined face part (i.e., the top half), and yet perceiving the face as an integrated unit (i.e., holistically). Here, we investigated the impact of a small gap between top and bottom face halves in the standard composite face paradigm, requiring matching of sequentially presented top face halves. In Experiment 1, the CFE was larger for no-gap than gap stimuli overall, but not for participants who were presented with gap stimuli first, suggesting that the area of the top face half was unknown without a gap. This was confirmed in Experiment 2, in which these two stimulus sets were mixed up: the gap stimuli thus provided information about the area of a top face half and the magnitude of the CFE did not differ between stimulus sets. These observations indicate that the CFE might be artificially inflated in the absence of a stimulus cue that objectively defines a border between the face halves. Finally, in Experiment 3, observers were asked to determine which of two simultaneously presented faces was the composite face. Perceptual judgements for no-gap stimuli approached ceiling; however, with a gap, participants were almost unable to distinguish the composite face from a veridical face. This effect was not only due to low-level segmentation cues at the border of no-gap face halves, because stimulus inversion decreased performance in both conditions. This result indicates that the two halves of different faces may be integrated more naturally with a small gap that eliminates an enhanced contrast border. Collectively, these observations suggest that a small gap between face halves provides an objective definition of the face half to match and is beneficial for valid measurement of the behavioural CFE.     

Does response interference contribute to face composite effects?

Holistic processing of faces can be measured as a failure of selective attention to one face-half under instructions to ignore the other face-half in a naming or same/different matching task. But is interference from the irrelevant half due to response interference rather than to holistic processing? Here, participants learned to name two faces “Fred” and two “Bob.” At test, composites were created from top and bottom halves of different learned faces or of a novel face, and composites were either aligned or misaligned. Naming was slower when the irrelevant half was from a different face as opposed to the same face, regardless of whether it was associated with the same name, a different name, or no name, suggesting holistic processing. Interference was eliminated when composite halves were misaligned. These results suggest that, unlike Stroop effects, composite effects are not due to response interference.     

The composite face effect in six-year-old children: Evidence of adult-like holistic face processing

Holistic processing (i.e., gluing facial features together into a gestalt) is a hallmark of adults’ expert face recognition. Children make more errors than adults on a variety of face processing tasks even during adolescence. To determine whether this slow development can be attributed to immature holistic processing of unfamiliar faces, we tested 6-year-old children (n=24) with a classic measure of adults’ holistic processing, the composite face effect: They made same/different judgements about the top halves of face pairs when each top half was combined with a different bottom half, with which it was aligned (so that holistic processing creates the impression of a different face) or misaligned (a manipulation that disrupts holistic processing). Six-year-olds showed an adult-like composite face effect: Like adults, they made 26% more errors on aligned trials than on misaligned trials. These results suggest that the improvements after age 6 in the recognition of the facial identity are not caused by the onset or increasing strength of holistic face processing.     

The composite face illusion

Few findings in cognitive science have proved as influential as the composite face effect. When the top half of one face is aligned with the bottom half of another, and presented upright, the resulting composite arrangement induces a compelling percept of a novel facial configuration. Findings obtained using composite face procedures have contributed significantly to our understanding of holistic face processing, the detrimental effects of face inversion, the development of face perception, and aberrant face perception in clinical populations. Composite paradigms continue to advance our knowledge of face perception, as exemplified by their recent use for investigating the perceptual mechanisms underlying dynamic face processing. However, the paradigm has been the subject of intense scrutiny, particularly over the last decade, and there is a growing sense that the composite face illusion, whilst easy to illustrate, is deceptively difficult to measure and interpret. In this review, we provide a focussed overview of the existing composite face literature, and identify six priorities for future research. Addressing these gaps in our knowledge will aid the evaluation and refinement of theoretical accounts of the illusion.     

Does temporal integration of face parts reflect holistic processing?

We examined whether temporal integration of face parts reflects holistic processing or response interference. Participants learned to name two faces “Fred” and two “Bob.” At test, the top and bottom halves of different faces formed composites and were presented briefly separated in time. Replicating prior findings (Singer & Sheinberg, Vision Research, 46, 1838–1847, 2006), naming of the target half for aligned composites was slowed when the irrelevant half was from a face with a different name rather than from the original face. However, no interference was observed when the irrelevant half had a name identical to the name of the target half but came from a different learned face, arguing against a true holistic effect. Instead, response interference was obtained when the target half briefly preceded the irrelevant half. Experiment 2 confirmed a double dissociation of holistic processing versus response interference for intact faces versus temporally separated face halves, suggesting that simultaneous presentation of facial information is critical for holistic processing.     

Holistic processing in the composite task depends on face size

Holistic processing is a hallmark of face processing. There is evidence that holistic processing is strongest for faces at identification distance 2–10 metres from the observer. However, this evidence is based on tasks that have been little used in the literature and that are indirect measures of holistic processing. We use the composite task—a well validated and frequently used paradigm—to measure the effect of viewing distance on holistic processing. In line with previous work, we find a congruency x alignment effect that is strongest for faces that are close (2 m equivalent distance) than for faces that are further away (24 m equivalent distance). In contrast, the alignment effect for same trials, used by several authors to measure holistic processing, produced results that are difficult to interpret. We conclude that our results converge with previous findings providing more direct evidence for an effect of size on holistic processing.     

Holistic processing is tuned for in-group faces

Past research has found that mere in-group/out-group categorizations are sufficient to elicit biases in face memory. The current research yields novel evidence that mere social categorization is also sufficient to modulate processes underlying face perception, even for faces for which we have strong perceptual expertise: same-race (SR) faces. Using the composite face paradigm, we find that SR faces categorized as in-group members (i.e., fellow university students) are processed more holistically than are SR faces categorized as out-group members (i.e., students at another university). Hence, holding perceptual expertise with faces constant, categorizing an SR target as an out-group member debilitates the strong holistic processing typically observed for SR faces.