The role of higher level adaptive coding mechanisms in the development of face recognition

Developmental improvements in face identity recognition ability are widely documented, but the source of children’s immaturity in face recognition remains unclear. Differences in the way in which children and adults visually represent faces might underlie immaturities in face recognition. Recent evidence of a face identity aftereffect (FIAE), in which adaptation (exposure) to a particular identity causes a previously neutral face to take on the computationally opposite identity, suggests that adults code faces in an opponent fashion relative to an average face. One previous study showed comparable FIAEs in 8-year-olds and adults but did not demonstrate that adaptation was selective for high-level representations in both groups. Using a developmentally appropriate FIAE task, we investigated whether children show adult-like adaptation for facial identity when adapting and test images differ in size. Both age groups showed an equivalent FIAE, suggesting that qualitative changes in the use of higher level adaptive coding mechanisms do not drive the developmental improvements in face recognition ability, at least from 8 years of age.     

Does response interference contribute to face composite effects?

Holistic processing of faces can be measured as a failure of selective attention to one face-half under instructions to ignore the other face-half in a naming or same/different matching task. But is interference from the irrelevant half due to response interference rather than to holistic processing? Here, participants learned to name two faces “Fred” and two “Bob.” At test, composites were created from top and bottom halves of different learned faces or of a novel face, and composites were either aligned or misaligned. Naming was slower when the irrelevant half was from a different face as opposed to the same face, regardless of whether it was associated with the same name, a different name, or no name, suggesting holistic processing. Interference was eliminated when composite halves were misaligned. These results suggest that, unlike Stroop effects, composite effects are not due to response interference.     

The composite face illusion: A whole window into our understanding of holistic face perception

Two identical top halves of a face are perceived as being different when their bottom halves belong to different faces, showing that the parts of a face cannot be perceived independently from the whole face. When this visual illusion is inserted in a matching task, observers make more mistakes and/or are slower at matching identical top face halves aligned with different bottom halves than when the bottom halves are spatially offset: The composite face effect. This composite face paradigm has been used in more than 60 studies that have provided information about the specificity and nature of perceptual integration between facial parts (“holistic face perception”), the impairment of this process in acquired prosopagnosia, its developmental course, temporal dynamics, and neural basis. Following a review of the main contributions made with the paradigm, I explain its rationale and strengths, and discuss its methodological parameters, making a number of proposals for its optimal use and refinement in order to improve our understanding of holistic face perception. Finally, I explain how this standard composite face paradigm is fundamentally different than the application to facial parts of a congruency/interference paradigm that has a long tradition in experimental psychology since Stroop (1935), and which was originally developed to measure attentional and response interference between different representations rather than perceptual integration. Moreover, a version of this congruency/interference paradigm used extensively over the past years with composite faces lacks a baseline measure and has decisional, attentional, and stimulus confounds, making the findings of these studies impossible to interpret in terms of holistic perception. I conclude by encouraging researchers in this field to concentrate fully on the standard composite face paradigm, gaze contingency, and other behavioural measures that can help us take one of the most important challenges of visual perception research: Understanding the neural mechanisms of holistic face perception.     

Perceptual asymmetries are preserved in memory for highly familiar faces of self and friend

We investigated the effect of familiarity on people's perception of facial likeness by asking participants to choose which of two mirror-symmetric chimeric images (made from the left or right half of a photograph of a face) looked more like an original image. In separate trials the participants made this judgment for their own face and for the face of a close friend; half of them matched to a true image of the original and half matched to a mirror image of the original. In the case of matching to a friend's face presented in the familiar orientation, over 80% of participants chose the left-left composite to be a better likeness to the original, whereas only 62% showed the same left-side bias when matching to a mirror image. The difference is significant, and the result contrasts markedly with a second experiment where participants who were unfamiliar with the faces showed comparable left-side biases when matching to true or mirror reversed images. The result suggests that perceptual asymmetries are retained in our long-term memory for highly familiar faces. While matching to images of self also showed an effect of familiarity, the data in this condition show less evidence of perceptual asymmetry and are discussed in relation to recent research on the representation of one's own face.     

Recognizing disguised faces

Across three experiments, we evaluated the effects of “disguises” on observers’ face identification performance using naturalistic images in which individuals posed with a variety of wigs and eyeglasses. Experiment 1 tested recognition memory performance with and without disguises and showed that any changes in these facial attributes hindered performance, replicating previous findings. More interestingly, Experiment 1 revealed that a change in hairstyle or the removal of eyeglasses had more impact on performance than did the addition of eyeglasses. In Experiment 2, disguised and undisguised faces were presented upright or inverted to examine whether the performance decrements seen in Experiment 1 were attributable to disruption of processing strategies for faces. Despite showing an overall effect of disguise on performance, there was no interaction between inversion and disguise. This suggests that disguises are not directly affecting processing strategies, but instead disguises are likely encoded as part of the overall representation. Similarly, in Experiment 3, the composite face paradigm was used to examine whether the impact of disguises is attributable to disruption of holistic processing for faces. Again, we found an overall effect of disguise on performance, but here we also observed that the presence of disguises interacted with participants’ tendency to form holistic face representations.     

知觉经验对面孔整体加工的影响

基于经验的整体加工理论认为, 随着知觉经验的增多, 人们加工面孔的专家化能力逐渐增强。其中一种表现形式为, 在知觉面孔时更倾向于整体加工。该理论已经得到了一些实验证据的支持, 但未能解释另外一些研究的发现, 即对他族人群没有长期知觉经验的被试在加工他族面孔时也有强烈的整体加工。本研究旨在考察知觉经验对面孔整体加工的影响, 包括两个实验。实验1a通过组合面孔范式测量了中国被试在辨别本族面孔、他族面孔和种族融合面孔时的组合面孔效应(即面孔整体加工强度)。结果发现, 被试在知觉种族融合面孔时没有表现出组合面孔效应, 在知觉本族面孔和他族面孔时都表现出强烈的组合面孔效应。实验1b以灰阶图重复了实验1a。结果发现, 被试在知觉种族融合面孔时仍然没有表现出组合面孔效应, 在知觉本族面孔和他族面孔时表现出强烈的组合面孔效应。实验2通过操纵种族融合面孔的有或无, 测量5 s的知觉经验对融合面孔的组合效应的影响。结果发现, 在经过短时的知觉经验后, 被试在知觉融合面孔时也表现出强烈的组合面孔效应。上述实验结果表明, 中国被试在加工本族面孔、他族面孔和种族融合面孔时所诱发组合面孔效应有显著的、量的差别, 且这种整体加工的差异不会受到面孔肤色信息的影响。在经过短时的知觉经验后, 中国被试在加工种族融合面孔时表现出强烈的整体加工。该研究的发现提示, 不论是短时知觉经验还是长期知觉经验, 知觉经验的增加是面孔整体加工的一个关键要素。     

Remembering and knowing: Two means of access to the personal past

The nature of recollective experience was examined in a recognition memory task. Subjects gave “remember” judgments to recognized items that were accompanied by conscious recollection and “know” judgments to items that were recognized on some other basis. Although a levels-of-processing effect (Experiment 1) and a picture-superiority effect (Experiment 2) were obtained for overall recognition, these effects occurred only for “remember” judgments, and were reversed for “know” judgments. In Experiment 3, targets and lures were either preceded by a masked repetition of their own presentation (thought to increase perceptual fluency) or of an unrelated word. The effect of perceptual fluency was obtained for overall recogrntion and “know” judgments but not for “remember” judgments. The data obtained for confidencejudgments using the same design (Experiment4) indicated that “remember”/”know” judgments are not made solely on the basis of confidence. These data support the two-factor theories of recognition memory by dissociating two forms of recognition, and shed light on the nature of conscious recollection.     

Understanding the multiframe caricature advantage for recognizing facial composites

Eyewitnesses often construct a “composite” face of a person they saw commit a crime, a picture that police use to identify suspects. We described a technique (Frowd, Bruce, Ross, McIntyre, & Hancock, 2007) based on facial caricature to facilitate recognition of these images: Correct naming substantially improves when composites are seen with progressive positive caricature, where distinctive information is enhanced, and then with progressive negative caricature, the opposite. Over the course of four experiments, the underpinnings of this mechanism were explored. Positive-caricature levels were found to be largely responsible for improving naming of composites, with some benefit from negative-caricature levels. Also, different frame-presentation orders (forward, reverse, random, repeated) facilitated equivalent naming benefit relative to static composites. Overall, the data indicate that composites are usually constructed as negative caricatures.     

Perceptual load effects on processing distractor faces indicate face-specific capacity limits

The claim that face perception is mediated by a specialized “face module” that proceeds automatically, independently of attention (e.g., Kanwisher, 2000) can be reconciled with load theory claims that visual perception has limited capacity (e.g., Lavie, 1995) by hypothesizing that face perception has face-specific capacity limits. We tested this hypothesis by comparing the effects of face and nonface perceptual load on distractor face processing. Participants searched a central array of either faces or letter strings for a pop star versus politician's face or name and made speeded classification responses. Perceptual load was varied through the relevant search set size. Response competition effects from a category-congruent or -incongruent peripheral distractor face were eliminated with more than two faces in the face search task, but were unaffected by perceptual load in the name search task. These results support the hypothesis that face perception has face-specific capacity limits and resolve apparent discrepancies in previous research.     

Adaptation and the perception of facial age

We examined how the perceived age of adult faces is affected by adaptation to younger or older adult faces. Observers viewed images of a synthetic male face simulating ageing over a modelled range from 15 to 65 years. Age was varied by changing shape cues or textural cues. Age level was varied in a staircase to find the observer's subjective category boundary between “old” and “young”. These boundaries were strongly biased by adaptation to the young or old face, with significant aftereffects induced by either shape or textural cues. A further experiment demonstrated comparable aftereffects for photorealistic images of average older or younger adult faces, and found that aftereffects showed some selectivity for a change in gender but also strongly transferred across gender. This transfer shows that adaptation can adjust to the attribute of age somewhat independently of other facial attributes. These findings suggest that perceived age, like many other natural facial dimensions, is highly susceptible to adaptation, and that this adaptation can be carried by both the structural and textural changes that normally accompany facial ageing.     

The other-species effect in human perceptions of sexual dimorphism using human and macaque faces

Humans are better able to discriminate among human faces than faces of other species. This difference in perceptual discrimination is known as the “other-species effect”. Models of perception have posited that the ultimate functional significance of the other-species effect is a higher discrimination capability within an organism's most familiar and salient stimulus set while attenuating the ability to discriminate amongst unfamiliar stimuli. Here, human participants made masculinity judgements of human and macaque faces manipulated based on either human or macaque sexual dimorphism. Humans were more accurate at identifying masculine/feminine faces in species-congruent than species-incongruent transforms in both human and macaque faces. We observed an other-species effect whereby accuracy (correctly judging masculinized faces as more masculine) was highest for own-species faces. We also found that both men and women were better at judging the sex-typicality of male faces than female faces, regardless of the species of the face or the species of the manipulation. Our findings demonstrate an other-species effect for the perception of sex-typicality among human raters.     

Electrophysiological correlates of the composite face illusion: Disentangling perceptual and decisional components of holistic face processing in the human brain

When the bottom halves of two faces differ, people’s behavioral judgment of the identical top halves of those faces is impaired: they report that the top halves are different, and/or take more time than usual to provide a response. This behavioral measure is known as the composite face effect (CFE) and has traditionally been taken as evidence that faces are perceived holistically. Recently, however, it has been claimed that this effect is driven almost entirely by decisional, rather than perceptual, factors ( Richler, Gauthier, Wenger, & Palmeri, 2008). To disentangle the contribution of perceptual and decisional brain processes, we aimed to obtain an event-related potential (ERP) measure of the CFE at a stage of face encoding ( Jacques & Rossion, 2009) in the absence of a behavioral CFE effect. Sixteen participants performed a go/no-go task in an oddball paradigm, lifting a finger of their right or left hand when the top half of a face changed identity. This change of identity of the top of the face was associated with an increased ERP signal on occipito-temporal electrode sites at the N170 face-sensitive component (∼160 ms), the later decisional P3b component, and the lateralized readiness potential (LRP) starting at ∼350 ms. The N170 effect was observed equally early when only the unattended bottom part of the face changed, indicating that an identity change was perceived across the whole face in this condition. Importantly, there was no behavioral response bias for the bottom change trials, and no evidence of decisional biases from electrophysiological data (no P3b and LRP deflection in no-go trials). These data show that an early CFE can be measured in ERPs in the absence of any decisional response bias, indicating that the CFE reflects primarily the visual perception of the whole face.     

The influence of attention on holistic face encoding

We examined the influence of attention on the formation of holistic face representations using the composite effect (Perception 16 (1987) 747). In Experiment 1, stimuli composed of a face superimposed on a house were shown during encoding. Subjects delineated either the face or the house, thus manipulating attention away or toward the face. In Experiment 2, an intact face image was presented with letters scrolling from top to bottom. Subjects were asked to either ignore the letters or read them and decipher the words that they formed. Aligned and misaligned composites were shown at testing. Recognition performance was consistently better for misaligned than aligned stimuli, regardless of the allocation of attention during encoding. In Experiment 3, we show that the composite effect can be eliminated by a disruption in holistic processing at the time of encoding. We conclude that holistic encoding is one aspect of face analysis that occurs equally well with or without attention.     

Effect of racial bias on composite construction

We investigated how prior bias about a face's racial characteristics can affect its encoding and resultant facial composite construction. In total, 61 participants (24 Europeans, 18 Indians living in India and 19 Indians living in Europe) saw a racially ambiguous unfamiliar face and were led to believe it was either European or Indian. They created a composite of this face, using EFIT6. Two groups of independent raters (one Indian, the other European) then assessed the apparent race of each composite. A different two groups (one Indian, one European) assessed each composite's degree of resemblance to the target face, to determine whether this was influenced by the constructors' initial categorisation of the target face as “own-race” or “other-race.” Composites appeared significantly more “Asian” or “European” according to the bias induced in their creators, but there was no evidence of any own-race bias in the resemblance ratings for the composites.     

Visual expertise does not predict the composite effect across species: A comparison between spider (Ateles geoffroyi) and rhesus (Macaca mulatta) monkeys

Humans are subject to the composite illusion: two identical top halves of a face are perceived as “different” when they are presented with different bottom halves. This observation suggests that when building a mental representation of a face, the underlying system perceives the whole face, and has difficulty decomposing facial features. We adapted a behavioural task that measures the composite illusion to examine the perception of faces in two nonhuman species. Specifically we had spider (Ateles geoffroyi) and rhesus monkeys (Macaca mulatta) perform a two-forced choice, match-to-sample task where only the top half of sample was relevant to the task. The results of Experiment 1 show that spider monkeys (N = 2) process the faces of familiar species (conspecifics and humans, but not chimpanzees, sheep, or sticks), holistically. The second experiment tested rhesus monkeys (N = 7) with the faces of humans, chimpanzees, gorillas, sheep, and sticks. Contrary to prediction, there was no evidence of a composite effect in the human (or familiar primate) condition. Instead, we present evidence of a composite illusion in the chimpanzee condition (an unfamiliar primate). Together, these experiments show that visual expertise does not predict the composite effect across the primate order.     

Face recognition algorithms and the other‐race effect: computational mechanisms for a developmental contact hypothesis

People recognize faces of their own race more accurately than faces of other races. The “contact” hypothesis suggests that this “other‐race effect” occurs as a result of the greater experience we have with own‐ versus other‐race faces. The computational mechanisms that may underlie different versions of the contact hypothesis were explored in this study. We replicated the other‐race effect with human participants and evaluated four classes of computational face recognition algorithms for the presence of an other‐race effect. Consistent with the predictions of a developmental contact hypothesis, “experience‐based models” demonstrated an other‐race effect only when the representational system was developed through experience that warped the perceptual space in a way that was sensitive to the overall structure of the model's experience with faces of different races. When the model's representation relied on a feature set optimized to encode the information in the learned faces, experience‐based algorithms recognized minority‐race faces more accurately than majority‐race faces. The results suggest a developmental learning process that warps the perceptual space to enhance the encoding of distinctions relevant for own‐race faces. This feature space limits the quality of face representations for other‐race faces.     

Identity From Variation: Representations of Faces Derived From Multiple Instances

Research in face recognition has tended to focus on discriminating between individuals, or “telling people apart.” It has recently become clear that it is also necessary to understand how images of the same person can vary, or “telling people together.” Learning a new face, and tracking its representation as it changes from unfamiliar to familiar, involves an abstraction of the variability in different images of that person's face. Here, we present an application of principal components analysis computed across different photos of the same person. We demonstrate that people vary in systematic ways, and that this variability is idiosyncratic—the dimensions of variability in one face do not generalize well to another. Learning a new face therefore entails learning how that face varies. We present evidence for this proposal and suggest that it provides an explanation for various effects in face recognition. We conclude by making a number of testable predictions derived from this framework.     

Concerning Gibson's ‘on the face of it’: Immediate perception and single-glance face recognition

The paper notes (1) that “thinking Gibsonian” is to be torn between two loves: cognition and reality, (2) that just because of this the cognitive scientist may beneficiently and more thoroughly rediscover that they are only two faces of his one true love: Meaning. This issue is elaborated as “the problem of intentionality” and culminates in the conclusion that Gibson's direct ecological approach to visual perception is not a theory about the constitution of perceptual meaning, which is the domain of the indirect cognitive approach. Therefore the constant Gibsonian attacks on indirect theories might be both misled and misleading. The second half of the paper begins with noting that this may be due to a confounding of the “direct” level of explanation of the perceiver with the “indirect” levels of phenomenologist and visual scientist. Next the scientist's process notions of direct and indirect theories are discussed with respect to single-glance face recognition. Within this second half of the paper about 20 different meanings of “direct” or “immediate” are presented (printed in italics). Most of these are generally acceptable and should, thus, not be invoked in support of entirely different notions, which are very problematic because these seem to eliminate the concept of mind-as-cognitive-process.     

The “parts and wholes” of face recognition: A review of the literature

It has been claimed that faces are recognized as a “whole” rather than by the recognition of individual parts. In a paper published in the Quarterly Journal of Experimental Psychology in 1993, Martha Farah and I attempted to operationalize the holistic claim using the part/whole task. In this task, participants studied a face and then their memory presented in isolation and in the whole face. Consistent with the holistic view, recognition of the part was superior when tested in the whole-face condition compared to when it was tested in isolation. The “whole face” or holistic advantage was not found for faces that were inverted, or scrambled, nor for non-face objects, suggesting that holistic encoding was specific to normal, intact faces. In this paper, we reflect on the part/whole paradigm and how it has contributed to our understanding of what it means to recognize a face as a “whole” stimulus. We describe the value of part/whole task for developing theories of holistic and non-holistic recognition of faces and objects. We discuss the research that has probed the neural substrates of holistic processing in healthy adults and people with prosopagnosia and autism. Finally, we examine how experience shapes holistic face recognition in children and recognition of own- and other-race faces in adults. The goal of this article is to summarize the research on the part/whole task and speculate on how it has informed our understanding of holistic face processing.