Face inversion disrupts the perception of vertical relations between features in the right human occipito‐temporal cortex

The impact of inversion on the extraction of relational and featural face information was investigated in two fMRI experiments. Unlike previous studies, the contribution of horizontal and vertical spatial relations were considered separately since they have been shown to be differentially vulnerable to face inversion (Goffaux & Rossion, 2007). Hence, inversion largely affects the perception of vertical relations (e.g. eye or mouth height) while the processing of features (e.g. eye shape and surface) and of horizontal relations (e.g. inter‐ocular distance) is affected to a far lesser extent. Participants viewed pairs of faces that differed either at the level of one local feature (i.e. the eyes) or of the spatial relations of this feature with adjacent features. Changes of spatial relations were divided into two conditions, depending on the vertical or horizontal axis of the modifications. These stimulus conditions were presented in separate blocks in the first (block) experiment while they were presented in a random order in the second event‐related (ER) experiment. Face‐preferring voxels located in the right‐lateralized middle fusiform gyrus (rMFG) largely decreased their activity with inversion. Inversion‐related decreases were more moderate in left‐lateralized middle fusiform gyrus (lMFG). ER experiment revealed that inversion affected rMFG and lMFG activity in distinct stimulus conditions. Whereas inversion affected lMFG processing only in featural condition, inversion selectively affected the processing of vertical relations in rMFG. Correlation analyses further indicated that the inversion effect (IE) observed in rMFG and right inferior occipital gyrus (rIOG) reliably predicted the large behavioural IE observed for the processing of vertical relations. In contrast, lMFG IE correlated with the weak behavioural IE observed for the processing of horizontal relations. Our findings suggest that face configuration is mostly encoded in rMFG, whereas more local aspects of face information, such as features and horizontal spatial relations drive lMFG processing. These findings corroborate the view that the vulnerability of face perception to inversion stems mainly from the disrupted processing of vertical face relations in the right‐lateralized network of face‐preferring regions (rMFG, rIOG).     

Face inversion disproportionately impairs the perception of vertical but not horizontal relations between features

Upside-down inversion disrupts the processing of spatial relations between the features of a face, while largely preserving local feature analysis. However, recent studies on face inversion failed to observe a clear dissociation between relational and featural processing. To resolve these discrepancies and clarify how inversion affects face perception, the authors monitored inversion effects separately for vertical and horizontal distances between features. Inversion dramatically declined performance in the vertical-relational condition, but it impaired featural and horizontal-relational performance only moderately. Identical observations were made whether upright and inverted trials were blocked or randomly interleaved. The largest performance decrement was found for vertical relations even when faces were rotated by 90 degrees. Evidence that inversion dramatically disrupts the ability to extract vertical but not horizontal feature relations supports the view that inversion qualitatively changes face perception by rendering some of the processes activated by upright faces largely ineffective.     

The horizontal and vertical relations in upright faces are transmitted by different spatial frequency ranges

Faces convey distinct types of information: features and their spatial relations, which are differentially vulnerable to inversion. While inversion largely disrupts the processing of vertical spatial relations (e.g. eyes’ height), its effect is moderate for horizontal relations (e.g. interocular distance) and local feature properties. The SF ranges optimally transmitting horizontal and vertical face relations were here investigated to further address their functional role in face perception. Participants matched upright and inverted pairs of faces that differed at the level of local featural properties, horizontal relations in vertical relations. Irrespective of SF, the inversion effect was larger for vertical than horizontal and featural cues. Most interestingly, SF differentially influenced the processing of vertical, horizontal and featural cues in upright faces. Vertical relations were optimally processed in intermediate SF, which are known to carry useful information for face individuation. In contrast, horizontal relations were best conveyed by high SF, which are involved in the processing of local face properties. These findings not only confirm that horizontal and vertical relations play distinct functional roles in face perception, but they also further suggest a unique role of vertical relations in face individuation.     

The shape of facial features and the spacing among them generate similar inversion effects: A reply to

It is often argued that picture-plane face inversion impairs discrimination of the spacing among face features to a greater extent than the identity of the facial features. However, several recent studies have reported similar inversion effects for both types of face manipulations. In a recent review, Rossion (2008) claimed that similar inversion effects for spacing and features are due to methodological and conceptual shortcomings and that data still support the idea that inversion impairs the discrimination of features less than that of the spacing among them. Here I will claim that when facial features differ primarily in shape, the effect of inversion on features is not smaller than the one on spacing. It is when color/contrast information is added to facial features that the inversion effect on features decreases. This obvious observation accounts for the discrepancy in the literature and suggests that the large inversion effect that was found for features that differ in shape is not a methodological artifact. These findings together with other data that are discussed are consistent with the idea that the shape of facial features and the spacing among them are integrated rather than dissociated in the holistic representation of faces.     

Decay of Stimulus Spatial Code in Horizontal and Vertical Simon Tasks

Evidence on the processes underlying the horizontal and vertical Simon effect is still controversial. The present study uses experimental manipulations to selectively delay the stages of response execution, response selection, and stimulus identification in three experiments. A reduction is observed for both horizontal and vertical Simon effects when response execution is delayed by a go-signal presented 400–600 ms post-stimulus onset or when a spatial precue is presented 200–400 ms before the stimulus. When the overlap between stimulus spatial code formation and response selection is prevented by decreasing stimulus discriminability, the horizontal Simon effect decays, but the vertical Simon effect does not change. Activation theories, which propose a decay of the automatically activated response ipsilateral to the stimulus, mainly apply to the horizontal Simon effect. In contrast, translation theories, which propose that the effect occurs when stimulus features are translated into a response code, are more suitable to account for the vertical Simon effect.     

The effects of face inversion on the perception of long-range and local spatial relations in eye and mouth configuration

A recent study hypothesized a configurational anisotropy in the face inversion effect, with vertical relations more difficult to process. However, another difference in the stimuli of that report was that the vertical but not horizontal shifts lacked local spatial references. Difficulty processing long-range spatial relations might also be predicted from a relevance-interaction explanation, which proposes that in inverted faces, spatial relations are processed efficiently only within high-relevance local regions. The authors performed 2 experiments to distinguish between these hypotheses. Experiment 1 showed that the inversion effect for vertical shifts of the eyes alone was more similar to that for horizontal eye shifts than for vertical shifts of the eyes and eyebrows. In Experiment 2, focused attention reduced the inversion effect for vertical mouth position more than that for vertical shifts of the eyes and brows. The authors concluded that face inversion impairs the perception of both local spatial relations in low-relevance regions and long-range spatial relations extending across multiple facial regions, consistent with a loss of efficient whole-face processing of the spatial relations between features.     

Chinese characters elicit face-like N170 inversion effects

Recognition of both faces and Chinese characters is commonly believed to rely on configural information. While faces typically exhibit behavioral and N170 inversion effects that differ from non-face stimuli (Rossion, Joyce, Cottrell, & Tarr, 2003), the current study examined whether a similar reliance on configural processing may result in similar inversion effects for faces and Chinese characters. Participants were engaged in an orientation judgment task (Experiment 1) and a one-back identity matching task (Experiment 2). Across two experiments, the N170 was delayed and enhanced in magnitude for upside-down faces and compound Chinese characters, compared to upright stimuli. The inversion effects for these two stimulus categories were bilateral for latency and right-lateralized for amplitudes. For simple Chinese characters, only the latency inversion effects were significant. Moreover, the size of the right-hemisphere inversion effects in N170 amplitude was larger for faces than Chinese characters. These findings show the N170 inversion effects from non-face stimuli closely parallel effects seen with faces. Face-like N170 inversion effects elicited by Chinese compound characters were attributed to the difficulty of part-whole integration as well as the disrupted regularity in relational information due to inversion. Hemispheric difference in Chinese character processing is also discussed.     

Chinese characters elicit face-like N170 inversion effects

Recognition of both faces and Chinese characters is commonly believed to rely on configural information. While faces typically exhibit behavioral and N170 inversion effects that differ from non-face stimuli (Rossion, Joyce, Cottrell, & Tarr, 2003), the current study examined whether a similar reliance on configural processing may result in similar inversion effects for faces and Chinese characters. Participants were engaged in an orientation judgment task (Experiment 1) and a one-back identity matching task (Experiment 2). Across two experiments, the N170 was delayed and enhanced in magnitude for upside-down faces and compound Chinese characters, compared to upright stimuli. The inversion effects for these two stimulus categories were bilateral for latency and right-lateralized for amplitudes. For simple Chinese characters, only the latency inversion effects were significant. Moreover, the size of the right-hemisphere inversion effects in N170 amplitude was larger for faces than Chinese characters. These findings show the N170 inversion effects from non-face stimuli closely parallel effects seen with faces. Face-like N170 inversion effects elicited by Chinese compound characters were attributed to the difficulty of part-whole integration as well as the disrupted regularity in relational information due to inversion. Hemispheric difference in Chinese character processing is also discussed.     

Dynamic aspects of stimulus-response correspondence: evidence for two mechanisms involved in the Simon effect

It has been recently proposed that the time course of the Simon effect may vary across tasks, which might reflect different types of stimulus-response (S-R) transmissions (E. Wascher, U. Schatz, T. Kuder, & R. Verleger, 2001). The authors tested this notion in 4 experiments by comparing Simon effects evoked by horizontal and vertical S-R arrangements. The temporal properties of the effect, as well as lateralized readiness potential-difference waves, indicated a fast and transient influence of the horizontal, but a slow and sustained influence of the vertical spatial stimulus feature on performance. Additional evidence for this temporal dissociation was obtained in experiments that induced a shortening or lengthening of the mean response time. Thus, the data strongly indicate that there are 2 temporally dissociable mechanisms involved in generating the Simon effect for horizontal and vertical S-R relations.     

Older adults' configural processing of faces: role of second-order information

Problems with face recognition are frequent in older adults. However, the mechanisms involved have only been partially discovered. In particular, it is unknown to what extent these problems may be related to changes in configural face processing. Here, we investigated the face inversion effect (FIE) together with the ability to detect modifications in the vertical or horizontal second-order relations between facial features. We used a same/different unfamiliar face discrimination task with 33 young and 33 older adults. The results showed dissociations in the performances of older versus younger adults. There was a lack of inversion effect during the recognition of original faces by older adults. However, for modified faces, older adults showed a pattern of performance similar to that of young participants, with preserved FIE for vertically modified faces and no detectable FIE for horizontally modified faces. Most importantly, the detection of vertical modifications was preserved in older relative to young adults whereas the detection of horizontal modifications was markedly diminished. We conclude that age has dissociable effects on configural face-encoding processes, with a relative preservation of vertical compared to horizontal second-order relations processing. These results help to understand some divergent results in the literature and may explain the spared familiar face identification abilities in the daily lives of older adults.     

Emotion recognition (sometimes) depends on horizontal orientations

Face recognition depends critically on horizontal orientations (Goffaux & Dakin, Frontiers in Psychology, 1(143), 1–14, 2010): Face images that lack horizontal features are harder to recognize than those that have this information preserved. We asked whether facial emotional recognition also exhibits this dependency by asking observers to categorize orientation-filtered happy and sad expressions. Furthermore, we aimed to dissociate image-based orientation energy from object-based orientation by rotating images 90 deg in the picture plane. In our first experiment, we showed that the perception of emotional expression does depend on horizontal orientations, and that object-based orientation constrained performance more than image-based orientation did. In Experiment 2, we showed that mouth openness (i.e., open vs. closed mouths) also influenced the emotion-dependent reliance on horizontal information. Finally, we describe a simple computational analysis that demonstrates that the impact of mouth openness was not predicted by variation in the distribution of orientation energy across horizontal and vertical orientation bands. Overall, our results suggest that emotion recognition largely does depend on horizontal information defined relative to the face, but that this bias is modulated by multiple factors that introduce variation in appearance across and within distinct emotions.     

Different mechanisms for role relations versus verb–action congruence effects: Evidence from ERPs in picture–sentence verification

Extant accounts of visually situated language processing do make general predictions about visual context effects on incremental sentence comprehension; these, however, are not sufficiently detailed to accommodate potentially different visual context effects (such as a scene–sentence mismatch based on actions versus thematic role relations, e.g., (Altmann & Kamide, 2007; Knoeferle & Crocker, 2007; Taylor & Zwaan, 2008; Zwaan & Radvansky, 1998)). To provide additional data for theory testing and development, we collected event-related brain potentials (ERPs) as participants read a subject–verb–object sentence (500 ms SOA in Experiment 1 and 300 ms SOA in Experiment 2), and post-sentence verification times indicating whether or not the verb and/or the thematic role relations matched a preceding picture (depicting two participants engaged in an action). Though incrementally processed, these two types of mismatch yielded different ERP effects. Role–relation mismatch effects emerged at the subject noun as anterior negativities to the mismatching noun, preceding action mismatch effects manifest as centro-parietal N400s greater to the mismatching verb, regardless of SOAs. These two types of mismatch manipulations also yielded different effects post-verbally, correlated differently with a participant's mean accuracy, verbal working memory and visual-spatial scores, and differed in their interactions with SOA. Taken together these results clearly implicate more than a single mismatch mechanism for extant accounts of picture–sentence processing to accommodate.     

The inversion effect reveals species differences in face processing

Face recognition is a complex skill that requires the integration of facial features across the whole face, e.g., holistic processing. It is unclear whether, and to what extent, other species process faces in a manner that is similar to humans. Previous studies on the inversion effect, a marker of holistic processing, in nonhuman primates have revealed mixed results in part because many studies have failed to include alternative image categories necessary to understand whether the effects are truly face-specific. The present study re-examined the inversion effect in rhesus monkeys and chimpanzees using comparable testing methods and a variety of high quality stimuli including faces and nonfaces. The data support an inversion effect in chimpanzees only for conspecifics' faces (expert category), suggesting face-specific holistic processing similar to humans. Rhesus monkeys showed inversion effects for conspecifics, but also for heterospecifics' faces (chimpanzees), and nonfaces images (houses), supporting important species differences in this simple test of holistic face processing.     

Integration of internal and external facial features in 8- to 10-year-old children and adults

Investigation of whole-part and composite effects in 4- to 6-year-old children gave rise to claims that face perception is fully mature within the first decade of life (Crookes & McKone, 2009). However, only internal features were tested, and the role of external features was not addressed, although external features are highly relevant for holistic face perception (Sinha & Poggio, 1996; Axelrod & Yovel, 2010, 2011). In this study, 8- to 10-year-old children and adults performed a same–different matching task with faces and watches. In this task participants attended to either internal or external features. Holistic face perception was tested using a congruency paradigm, in which face and non-face stimuli either agreed or disagreed in both features (congruent contexts) or just in the attended ones (incongruent contexts). In both age groups, pronounced context congruency and inversion effects were found for faces, but not for watches. These findings indicate holistic feature integration for faces. While inversion effects were highly similar in both age groups, context congruency effects were stronger for children. Moreover, children's face matching performance was generally better when attending to external compared to internal features. Adults tended to perform better when attending to internal features. Our results indicate that both adults and 8- to 10-year-old children integrate external and internal facial features into holistic face representations. However, in children's face representations external features are much more relevant. These findings suggest that face perception is holistic but still not adult-like at the end of the first decade of life.     

Configural coding of facial expressions: The impact of inversion and photographic negative

In previous research we used the composite paradigm (Young, Hellawell, & Hay, ) to demonstrate that configural cues are important for interpreting facial expressions. However, different configural cues in face perception have been identified, including holistic processing (i.e., perception of facial features as a single gestalt) and second‐order spatial relations (i.e., the spatial relationship between individual features). Previous research has suggested that the composite effect for facial identity operates at the level of holistic encoding. Here we show that the composite effect for facial expression has a similar perceptual basis by using different graphic manipulations (stimulus inversion and photographic negative) in conjunction with the composite paradigm. In relation to Bruce and Young's () functional model of face recognition, a suitable level for the composite effect is a stage of front‐end processing referred to as structural encoding, that is common to both facial identity and facial expression perception.     

Aberrant first fixations when looking at inverted faces in various poses: The result of the centre‐of‐gravity effect?

Face recognition is essential in everyday human life, and all faces are encountered in different poses. However, when a face is inverted, difficulties arise for recognition and eye movements may (Barton, Radcliffe, Cherkasova, Edleman, & Intriligator, 2006) or may not be disrupted (Williams & Henderson, 2007). The present study explored the effects of orientation and pose on recognition and eye movements during a standard old/new recognition task in order to resolve whether inversion disrupts eye movements. Eye‐tracking data looked at the first fixations, the number of fixations, and the duration of fixations over a face. A standard inversion effect was observed, but the three‐quarter view advantage was not observed. Eye‐movement data revealed that the eyes were the most sampled feature (in terms of first fixation, number of fixations, and duration of fixation) for all upright faces, however, other features were sampled first for inverted faces. These results are consistent with Barton et al.'s (2006) but not Williams and Henderson's (2007) findings: possible explanations for this are discussed with the caveat that the same images were used from learning to test.     

The time course of processing external and internal features of unfamiliar faces

The time course of processing internal and external facial features was studied in a sequential face matching task, where first a target face was presented, followed by a test face. The exposure duration of the test face was varied systematically (90, 120, 150 ms, and self-paced). In three tasks, participants were instructed to match either the whole face, only external features, or only internal features of the target and test face. Taken together, the results in all the three tasks provide evidence for very fast matching processes. For upright faces, maximal performance was achieved at 90 ms exposure duration and longer exposure durations (120, 150 ms, self-paced) did not improve accuracy. For inverted whole faces, reduced exposure duration resulted in an increase of matching errors, suggesting that below 150 ms of exposure duration, inverted faces cannot be matched reliably. When matching selected facial features only, no such inversion effect was found. Our data challenges previous claims that external features are matched faster than internal: no difference of time course was found between external and internal features. However, external features were matched more accurately.     

Distinguishing non-spatial from spatial biases in visual selection: neuropsychological evidence

Recently, it has been shown that patients with visual extinction can show enhanced awareness of contralesional stimuli that match the contents of working memory [Soto, D., & Humphreys, G.W. (2006). Proceedings of the National Academy of Sciences, USA, 103, 4789–4792]. Here we investigate whether these effects extend to cases where the working memory cue is verbal rather than visual (Study 1), and we show that non-spatial cues can overcome problems in spatial disengagement in patients, and this can affect the first eye movement made in search (Study 2). We discuss the implications of the data for understanding the relations between the cueing of attention from visual and conceptual information held in working memory and the cueing of spatial attention, and for understanding the relations between non-spatial and spatial biases in selection.     

Task effects, performance levels, features, configurations, and holistic face processing: A reply to Rossion

A recent article in Acta Psychologica (“Picture-plane inversion leads to qualitative changes of face perception” by Rossion [Rossion, B. (2008). Picture-plane inversion leads to qualitative changes of face perception. Acta Psychologica (Amst), 128(2), 274-289]) criticized several aspects of an earlier paper of ours [Riesenhuber, M., Jarudi, I., Gilad, S., & Sinha, P. (2004). Face processing in humans is compatible with a simple shape-based model of vision. Proceedings of the Royal Society of London B (Supplements), 271, S448-S450]. We here address Rossion’s criticisms and correct some misunderstandings. To frame the discussion, we first review our previously presented computational model of face recognition in cortex [Jiang, X., Rosen, E., Zeffiro, T., Vanmeter, J., Blanz, V., & Riesenhuber, M. (2006). Evaluation of a shape-based model of human face discrimination using FMRI and behavioral techniques. Neuron, 50(1), 159-172] that provides a concrete biologically plausible computational substrate for holistic coding, namely a neural representation learned for upright faces, in the spirit of the original simple-to-complex hierarchical model of vision by Hubel and Wiesel. We show that Rossion’s and others’ data support the model, and that there is actually a convergence of views on the mechanisms underlying face recognition, in particular regarding holistic processing.     

The effect of feature displacement on the perception of well-known faces

The effect of feature displacement within two well-known faces (Terry Wogan and Cyril Smith) was examined. Image processing equipment was used to produce stimuli in which the features of an original facial image were displaced to form a number of modified images. This technique was first reported by Haig, in a recognition study in which the effect of feature displacement within unfamiliar faces was investigated. In the present experiment a perceptual judgement task was carried out in which subjects were presented with a number of modified faces and asked to judge how dissimilar these were with respect to an original image. A multi-dimensional scaling analysis of the comparative judgements of the subjects revealed a two-dimensional solution involving displacement of the eyes and mouth. A clear division between up/down and inward/outward displacement within these features (particularly the eyes) was observed. A similar pattern of results was found for both well-known faces. This result indicates that subjects were responding to changes in the facial configuration produced by the different types of feature displacement (horizontal or vertical), as opposed to movement of the features per se. Finally, the results also indicate that the displacement of inner features (mouth, eyes, nose) was more noticeable than displacement of the outer facial features (eg hairline).