Task effects, performance levels, features, configurations, and holistic face processing: A reply to Rossion

A recent article in Acta Psychologica (“Picture-plane inversion leads to qualitative changes of face perception” by Rossion [Rossion, B. (2008). Picture-plane inversion leads to qualitative changes of face perception. Acta Psychologica (Amst), 128(2), 274-289]) criticized several aspects of an earlier paper of ours [Riesenhuber, M., Jarudi, I., Gilad, S., & Sinha, P. (2004). Face processing in humans is compatible with a simple shape-based model of vision. Proceedings of the Royal Society of London B (Supplements), 271, S448-S450]. We here address Rossion’s criticisms and correct some misunderstandings. To frame the discussion, we first review our previously presented computational model of face recognition in cortex [Jiang, X., Rosen, E., Zeffiro, T., Vanmeter, J., Blanz, V., & Riesenhuber, M. (2006). Evaluation of a shape-based model of human face discrimination using FMRI and behavioral techniques. Neuron, 50(1), 159-172] that provides a concrete biologically plausible computational substrate for holistic coding, namely a neural representation learned for upright faces, in the spirit of the original simple-to-complex hierarchical model of vision by Hubel and Wiesel. We show that Rossion’s and others’ data support the model, and that there is actually a convergence of views on the mechanisms underlying face recognition, in particular regarding holistic processing.     

Visual expertise does not predict the composite effect across species: A comparison between spider (Ateles geoffroyi) and rhesus (Macaca mulatta) monkeys

Humans are subject to the composite illusion: two identical top halves of a face are perceived as “different” when they are presented with different bottom halves. This observation suggests that when building a mental representation of a face, the underlying system perceives the whole face, and has difficulty decomposing facial features. We adapted a behavioural task that measures the composite illusion to examine the perception of faces in two nonhuman species. Specifically we had spider (Ateles geoffroyi) and rhesus monkeys (Macaca mulatta) perform a two-forced choice, match-to-sample task where only the top half of sample was relevant to the task. The results of Experiment 1 show that spider monkeys (N = 2) process the faces of familiar species (conspecifics and humans, but not chimpanzees, sheep, or sticks), holistically. The second experiment tested rhesus monkeys (N = 7) with the faces of humans, chimpanzees, gorillas, sheep, and sticks. Contrary to prediction, there was no evidence of a composite effect in the human (or familiar primate) condition. Instead, we present evidence of a composite illusion in the chimpanzee condition (an unfamiliar primate). Together, these experiments show that visual expertise does not predict the composite effect across the primate order.     

The evidence rejects the expertise hypothesis: reply to Gauthier & Bukach

In Robbins, R. & McKone, E. (2006). No face-like processing for object-of-expertise in three behavioural tasks. Cognition this issue, we showed face-like holistic/configural processing does not occur for objects-of-expertise on standard paradigms including inversion, part-whole, part-in-configurally-transformed-whole, and the standard composite task. In this reply to the discussion by Gauthier, I., & Bukach, C. (2006). Should we reject the expertise hypothesis? Cognition, this issue, we focus on several issues: the fact that they do not dispute our review of previous data; the strength of null effects obtained from multiple studies and paradigms; the evidence for domain-specificity of the neural substrate supporting face processing; the difference between the expertise hypothesis (as a theory about the origin of face processing) and studies of how experience affects object processing in general; and the problems with G&B's proposed alteration to the standard composite paradigm. We argue that overwhelming evidence suggests the expertise hypothesis should be put to rest so that researchers can focus on what the origin of "special" processing for faces actually is, and investigate the many interesting changes to object recognition that do occur with experience.     

Engaging the motor system with masked orthographic primes: A kinematic analysis

We report two experiments in which participants categorized target words (e.g., BLOOD or CUCUMBER) according to their canonical colour of red or green by pointing to a red square on the left or a green square on the right. Unbeknownst to the participants, the target words were preceded by the prime words “red” or “green”. We found that the curvature of participants’ pointing trajectories was greater following incongruent primes (green–BLOOD) than it was following congruent primes, indicating that individuals initiated a response on the basis of the prime and then corrected that response mid-flight. This finding establishes that the processing of masked orthographic stimuli extends down to include the formulation of an overt manual response.     

Impaired holistic and analytic face processing in congenital prosopagnosia: Evidence from the eye-contingent mask/window paradigm

There is abundant evidence that face recognition, in comparison to the recognition of other objects, is based on holistic processing rather than analytic processing. One line of research that provides evidence for this hypothesis is based on the study of people who experience pronounced difficulties in visually identifying conspecifics on the basis of their face. Earlier, we developed a behavioural paradigm to directly test analytic vs. holistic face processing. In comparison to a to be remembered reference face stimulus, one of two test stimuli was either presented in full view, with an eye-contingently moving window (only showing the fixated face feature, and therefore only affording analytic processing), or with an eye-contingently moving mask or scotoma (masking the fixated face feature, but still allowing holistic processing). In the present study we use this paradigm (that we used earlier in acquired prosopagnosia) to study face perception in congenital prosopagnosia (people having difficulties recognizing faces from birth on, without demonstrable brain damage). We observe both holistic and analytic face processing deficits in people with congenital prosopagnosia. Implications for a better understanding, both of congenital prosopagnosia and of normal face perception, are discussed.     

A unique look at face processing: the impact of masked faces on the processing of facial features

This experiment utilized a masked priming paradigm to explore the early processes involved in face recognition. The first experiment investigated implicit processing of the eyes and mouth in an upright face, using prime durations of 33 and 50 ms. The results demonstrate implicit processing of both the eyes and mouth, and support the configural processing theory of face processing. The second experiment used the same method with inverted faces and the third experiment was a combination of Experiments 1 and 2. The fourth experiment utilized misaligned faces as the primes. Based on the pattern of results from these experiments, we suggest that, when a face is inverted, the eyes and mouth are initially processed individually and are not linked until a later stage of processing. An upright face is proposed to be processed by analysis of its configuration, whereas an inverted face is initially processed using first-order relational information, and then converted to an upright representation and transferred to face specific regions for configural analysis.     

Tolerance for distorted faces: Challenges to a configural processing account of familiar face recognition

Face recognition is widely held to rely on ‘configural processing’, an analysis of spatial relations between facial features. We present three experiments in which viewers were shown distorted faces, and asked to resize these to their correct shape. Based on configural theories appealing to metric distances between features, we reason that this should be an easier task for familiar than unfamiliar faces (whose subtle arrangements of features are unknown). In fact, participants were inaccurate at this task, making between 8% and 13% errors across experiments. Importantly, we observed no advantage for familiar faces: in one experiment participants were more accurate with unfamiliars, and in two experiments there was no difference. These findings were not due to general task difficulty – participants were able to resize blocks of colour to target shapes (squares) more accurately. We also found an advantage of familiarity for resizing other stimuli (brand logos). If configural processing does underlie face recognition, these results place constraints on the definition of ‘configural’. Alternatively, familiar face recognition might rely on more complex criteria – based on tolerance to within-person variation rather than highly specific measurement.     

Inferring social attributes from different face regions: Evidence for holistic processing

Two experiments investigated the role that different face regions play in a variety of social judgements that are commonly made from facial appearance (sex, age, distinctiveness, attractiveness, approachability, trustworthiness, and intelligence). These judgements lie along a continuum from those with a clear physical basis and high consequent accuracy (sex, age) to judgements that can achieve a degree of consensus between observers despite having little known validity (intelligence, trustworthiness). Results from Experiment 1 indicated that the face's internal features (eyes, nose, and mouth) provide information that is more useful for social inferences than the external features (hair, face shape, ears, and chin), especially when judging traits such as approachability and trustworthiness. Experiment 2 investigated how judgement agreement was affected when the upper head, eye, nose, or mouth regions were presented in isolation or when these regions were obscured. A different pattern of results emerged for different characteristics, indicating that different types of facial information are used in the various judgements. Moreover, the informativeness of a particular region/feature depends on whether it is presented alone or in the context of the whole face. These findings provide evidence for the importance of holistic processing in making social attributions from facial appearance.     

Identity specific adaptation with composite faces

A composite face, made from the top half of a celebrity face and the bottom half of an unfamiliar face, appears to be a single, “new” face (e.g., Young, Hellawell, & Hay, 1987). Composite faces were used within the face identity aftereffect (FIAE) paradigm, in which prolonged exposure to a face reduces sensitivity to it (adaptation). Adaptation occurred both with an intact face and with composites containing its upper half, but only when composites were explicitly recognized during the adaptation phase. Unrecognized composites produced no adaptation. These findings imply that the FIAE is a relatively high-level perceptual effect, given that identical stimuli either did or did not produce adaptation depending on whether or not they were recognized. They also suggest a perceptual locus for the “composite face effect”.     

Culture,Emotion, and Well-being: Good Feelings in Japan and the United States

We tested the hypothesis that “good feelings”—the central element of subjective well-being—are associated with interdependence and interpersonal engagement of the self in Japan, but with independence and interpersonal disengagement of the self in the United States. Japanese and American college students (total N = 913) reported how frequently they experienced various emotional states in daily life. In support of the hypothesis, the reported frequency of general positive emotions (e.g. calm, elated) was most closely associated with the reported frequency of interpersonally engaged positive emotions (e.g. friendly feelings) in Japan, but with the reported frequency of interpersonally disengaged positive emotions (e.g. pride) in the United States. Further, for Americans the reported frequency of experience was considerably higher for positive emotions than for negative emotions, but for Japanese it was higher for engaged emotions than for disengaged emotions. Implications for cultural constructions of emotion in general and subjective well-being in particular are discussed.     

Face gender recognition in developmental prosopagnosia: Evidence for holistic processing and use of configural information

Face identification deficits in developmental prosopagnosics (DPs) have been thought to be due to general difficulties with processing configural face information and integrating configural and parts information into a coherent whole (holistic processing). Gender recognition provides a further opportunity to more fully examine this issue as this ability may be intact in DPs and it has been shown to depend on processing configural information and holistic processing in neurotypical individuals. In the present study we first determined that, indeed, gender discrimination performance was similar in DPs and controls. Second, we found that inversion and scrambling (which we propose measures holistic processing and sensitivity to configural information, respectively) produced comparable deficits in DPs and controls, suggesting that both groups use holistic processing and configural information to recognize gender. This indicates that holistic processing and using configural face information are not general impairments in DP and may be more specific to face identity.     

The electrophysiological correlates of integrated face and body-part perception

Previous studies have suggested that the human visual system processes faces and bodies holistically—that is, the different body parts are integrated into a unified representation. However, the time course of this integrative process is less known. In the present study, we investigated this issue by recording event-related potentials evoked by a face and two hands presented simultaneously and in different configurations. When the hands were rotated to obtain a biologically implausible configuration, a reduction of the P2 amplitude was observed relative to the condition in which the face and hands were retained in their veridical configuration and were supplemented with visual cues to highlight further the overall body posture. Our results show that the P2 component is sensitive to manipulations affecting the configuration of face and hand stimuli and suggest that the P2 reflects the operation of perceptual mechanisms responsible for the integrated processing of visually presented body parts.     

Two routes to face perception: evidence from psychophysics and computational modeling

The aim of this study was to separately analyze the role of featural and configural face representations. Stimuli containing only featural information were created by cutting the faces into their parts and scrambling them. Stimuli only containing configural information were created by blurring the faces. Employing an old-new recognition task, the aim of Experiments 1 and 2 was to investigate whether unfamiliar faces (Exp. 1) or familiar faces (Exp. 2) can be recognized if only featural or configural information is provided. Both scrambled and blurred faces could be recognized above chance level. A further aim of Experiments 1 and 2 was to investigate whether our method of creating configural and featural stimuli is valid. Pre-activation of one form of representation did not facilitate recognition of the other, neither for unfamiliar faces (Exp. 1) nor for familiar faces (Exp. 2). This indicates a high internal validity of our method for creating configural and featural face stimuli. Experiment 3 examined whether features placed in their correct categorical relational position but with distorted metrical distances facilitated recognition of unfamiliar faces. These faces were recognized no better than the scrambled faces in Experiment 1, providing further evidence that facial features are stored independently of configural information. From these results we conclude that both featural and configural information are important to recognize a face and argue for a dual-mode hypothesis of face processing. Using the psychophysical results as motivation, we propose a computational framework that implements featural and configural processing routes using an appearance-based representation based on local features and their spatial relations. In three computational experiments (Experiments 4–6) using the same sets of stimuli, we show how this framework is able to model the psychophysical data.     

Why 8-year-olds cannot tell the difference between Steve Martin and Paul Newman: factors contributing to the slow development of sensitivity to the spacing of facial features

Children are nearly as sensitive as adults to some cues to facial identity (e.g., differences in the shape of internal features and the external contour), but children are much less sensitive to small differences in the spacing of facial features. To identify factors that contribute to this pattern, we compared 8-year-olds' sensitivity to spacing cues with that of adults under a variety of conditions. In the first two experiments, participants made same/different judgments about faces differing only in the spacing of facial features, with the variations being kept within natural limits. To measure the effect of attention, we reduced the salience of featural information by blurring faces and occluding features (Experiment 1). To measure the role of encoding speed and memory limitations, we presented pairs of faces simultaneously and for an unlimited time (Experiment 2). To determine whether participants' sensitivity would increase when spacing distortions were so extreme as to make the faces grotesque, we manipulated the spacing of features beyond normal limits and asked participants to rate each face on a "bizarreness" scale (Experiment 3). The results from the three experiments indicate that low salience, poor encoding efficiency, and limited memory can partially account for 8-year-olds' poor performance on face processing tasks that require sensitivity to the spacing of features, a kind of configural processing that underlies adults' expertise. However, even when the task is modified to compensate for these problems, children remain less sensitive than adults to the spacing of features.     

Telling faces together: Learning new faces through exposure to multiple instances

We are usually able to recognize novel instances of familiar faces with little difficulty, yet recognition of unfamiliar faces can be dramatically impaired by natural within-person variability in appearance. In a card-sorting task for facial identity, different photos of the same unfamiliar face are often seen as different people. Here we report two card-sorting experiments in which we manipulate whether participants know the number of identities present. Without constraints, participants sort faces into many identities. However, when told the number of identities present, they are highly accurate. This minimal contextual information appears to support viewers in “telling faces together”. In Experiment 2 we show that exposure to within-person variability in the sorting task improves performance in a subsequent face-matching task. This appears to offer a fast route to learning generalizable representations of new faces.     

A holistic account of the own-race effect in face recognition: evidence from a cross-cultural study

A robust finding in the cross-cultural research is that people's memories for faces of their own race are superior to their memories for other-race faces. However, the mechanisms underlying the own-race effect have not been well defined. In this study, a holistic explanation was examined in which Caucasian and Asian participants were asked to recognize features of Caucasian and Asian faces presented in isolation and in the whole face. The main finding was that Caucasian participants recognized own-race faces more holistically than Asian faces whereas Asian participants demonstrated holistic recognition for both own-race and other-race faces. The differences in holistic recognition between Caucasian and Asian participants mirrored differences in their relative experience with own-race and other-race faces. These results suggest that the own-race effect may arise from the holistic recognition of faces from a highly familiar racial group.     

Race categorization modulates holistic face encoding

Recent studies have shown that same-race (SR) faces are processed more holistically than other-race (OR) faces, a difference that may underlie the greater difficulty at recognizing OR than SR faces (the "other-race effect"). This article provides original evidence suggesting that the holistic processing of faces may be sensitive to the observers' racial categorization of the face. In Experiment 1, Caucasian participants performed a face-composite task with Caucasian faces, Asian faces, and racially ambiguous morphed face stimuli. Identical morphed face stimuli were processed more holistically when categorized as SR than as OR faces. Experiment 2 further suggests that this finding was not underlain by strategic or training effects. Overall, these results support the view that one's categorization of a face as belonging to the same or another race plays a critical role in the holistic processing of this face.     

The own-species face bias: A review of developmental and comparative data

Face recognition is characterized in part by biases resulting in enhanced perception and memory for individuals within frequently encountered groups and impaired recognition for individuals within uncommonly encountered groups. These biases are found across multiple categories and levels, including species, race, age, and gender (Scherf & Scott, 2012). At the highest level of categorization, human adults and nonhuman primates exhibit improved recognition (Dufour, Pascalis, & Petit, 2006; Pascalis & Bachevalier, 1998) and discrimination abilities (Pascalis, de Haan, & Nelson, 2002) for individuals within their own species, reflecting what has been called an “own-species” or “species-specific” bias in face processing. The own-species bias is a model system for delineating the role of experience and the developmental trajectory of face-processing biases, for further understanding the malleability of face biases in adults, and for examining and comparing face-processing abilities across species. Here, we will review findings from studies investigating the perception of other-species faces during development, into adulthood and across species.     

Hemisphere-dependent holistic processing of familiar faces

In two behavioral experiments involving lateralized stimulus presentation, we tested whether one of the most commonly used measures of holistic face processing—the composite face effect—would be more pronounced for stimuli presented to the right as compared to the left hemisphere. In experiment 1, we investigated the composite face effect in a verbal identification task, similar to its original report (Young, Hellawell, & Hay, 1987). Aligning top and bottom halves of composite face stimuli led to performance decreases irrespective of hemifield, indicating holistic processing of comparable magnitude for inputs provided separately to either hemisphere. However, when matching of the same top parts was required in experiment 2, an alignment-dependent performance decrease was found for stimuli presented in the left, but not right visual field. These observations suggest that the right hemisphere dominates in early stages of holistic processing, as indexed by the composite face effect, but that later processes such as face identification and naming are based on unified representations that are independent of input lateralization. Moreover, the composite face effect may not rely on the exact same representation(s) when measured in matching and identification tasks.