Automatic spatial coding of perceived gaze direction is revealed by the Simon effect

In a typical Simon task, the (irrelevant) spatial position of the stimulus interferes with the processing of the salient characteristic (e.g., color). We used the Simon effect to investigate the automatic processing of gaze cues. We show that a simple drawing of schematic eyes automatically generates a spatially defined code of gaze direction. Although completely irrelevant to the task, direction of gaze influenced reaction times in a spatially selective two-choice discrimination based on eye color. Moreover, in one experiment employing an orthogonal manipulation of stimulus position and gaze direction, we found that coding of gaze direction is independent of stimulus spatial coding. Our finding of a “gazedirection Simon effect” is congruent with the hypothesis that gaze direction is coded by a specialized mechanism.     

Affective evaluations of objects are influenced by observed gaze direction and emotional expression

Gaze direction signals another person's focus of interest. Facial expressions convey information about their mental state. Appropriate responses to these signals should reflect their combined influence, yet current evidence suggests that gaze-cueing effects for objects near an observed face are not modulated by its emotional expression. Here, we extend the investigation of perceived gaze direction and emotional expression by considering their combined influence on affective judgments. While traditional response-time measures revealed equal gaze-cueing effects for happy and disgust faces, affective evaluations critically depended on the combined product of gaze and emotion. Target objects looked at with a happy expression were liked more than objects looked at with a disgust expression. Objects not looked at were rated equally for both expressions. Our results demonstrate that facial expression does modulate the way that observers utilize gaze cues: Objects attended by others are evaluated according to the valence of their facial expression.     

Spatial attentional distribution is modulated by head direction during eccentric gaze

Visual perception during eccentric gaze can be facilitated when a visual stimulus appears in front of the head direction. This study investigated the relative effects of gaze location and head direction on visual perception in central and peripheral vision. Participants identified the orientation of a T-shaped figure presented in the centre of a monitor and simultaneously localised a dot appearing in the periphery, while head direction relative to gaze location was to the left, right or centre. Effects of head direction were found only when the dot appeared far from the gaze fixation point, such that dot detection was superior when it appeared to the left (right) of fixation in the left (right) head direction. Experimental results indicated this was not due to a small shift of gaze location. Thus this study suggests that head direction influences visual perception particularly in peripheral vision where visual acuity decreases.     

Dissociating Simon and affordance compatibility effects: Silhouettes and photographs

When a graspable object’s handle is oriented to the same side as the response hand, responses are quicker and more accurate than when it is oriented to the opposite side. This effect has been attributed to the affordance of the object’s handle (Tucker & Ellis, 1998). Recent findings suggest this effect results instead from an abstract spatial response code (i.e., Simon effect; Cho & Proctor, 2010). However, the stimuli used in these previous studies differ in the amount of object and environmental depth information they contain, which may be critical to conveying an affordance. This information could explain these disparate findings as well as dissociate Simon and affordance compatibility effects. Four experiments demonstrate that the Simon effect results from the absence of this information, as in a silhouette, and the affordance effect results from its presence, as in a photograph. A fifth experiment confirmed that modifying information associated with the affordance, rather than the modification itself, produced the effects observed in the previous experiments. These findings support the following: (a) the internal details of an object and environmental depth can dissociate Simon and affordance compatibility effects, (b) this information is necessary to convey the object’s graspable affordance, and (c) the outer shape of the object is not sufficient to elicit an affordance effect. These findings are discussed in relation to the theory of embodied cognition.     

Spatial interference between gaze direction and gaze location: A study on the eye contact effect

Perceived gaze in faces is an important social cue that influences spatial orienting of attention. In three experiments, we examined whether the social relevance of gaze direction modulated spatial interference in response selection, using three different stimuli: faces, isolated eyes, and symbolic eyes (Experiments 1, 2, and 3, respectively). Each experiment employed a variant of the spatial Stroop paradigm in which face location and gaze direction were put into conflict. Results showed a reverse congruency effect between face location to the right or left of fixation and gaze direction only for stimuli with a social meaning to participants (Experiments 1 and 2). The opposite was observed for the nonsocial stimuli used in Experiment 3. Results are explained as facilitation in response to eye contact.     

Spatial Simon effects with nonspatial responses

Recent studies have shown that spatial Simon effects can be modulated by short-term associations that are set up as a result of task instructions. I examined whether spatial Simon effects can also be produced by short-term associations even when the responses are unrelated to spatial position. Participants were to say “cale” or “cole” on the basis of the direction of arrows (i.e., left or right), the meaning of words (i.e.,left orright), and the color of squares presented left or right of the screen center. Responses to squares were faster when the correct response was associated with the same position as the irrelevant position of the square (e.g., say “cale” to a square on the left when “cale” was assigned to the wordleft and the left arrow). This new type of stimulus-response compatibility effect provides the first evidence for short-term associations that involve mode-independent representations.     

Spatial interference between gaze direction and gaze location: a study on the eye contact effect

Perceived gaze in faces is an important social cue that influences spatial orienting of attention. In three experiments, we examined whether the social relevance of gaze direction modulated spatial interference in response selection, using three different stimuli: faces, isolated eyes, and symbolic eyes (Experiments 1, 2, and 3, respectively). Each experiment employed a variant of the spatial Stroop paradigm in which face location and gaze direction were put into conflict. Results showed a reverse congruency effect between face location to the right or left of fixation and gaze direction only for stimuli with a social meaning to participants (Experiments 1 and 2). The opposite was observed for the nonsocial stimuli used in Experiment 3. Results are explained as facilitation in response to eye contact.     

Abrupt onsets and gaze direction cues trigger independent reflexive attentional effects

The present study investigated whether the direction of visual signals is influenced independently by two automatic visual orienting phenomena: orienting to a gazed-at location and inhibition of return (IOR) to the location of an abrupt onset. A schematic face served as both a nonpredictive gaze direction cue and an abrupt onset cue. Results indicated that target detection was facilitated at the gazed-at location, and that it was inhibited at the abrupt onset location. Importantly, these two different reflexive attention effects were triggered by the same event and exhibited overlapping time courses. Moreover, the IOR effect did not vary as a function of the facilitatory effect of gaze. These findings strongly suggest that reflexive attention to gaze direction and reflexive inhibition to an abrupt onset are independent processes, and that gaze direction does not produce IOR at the gazed-at location.     

Modulation of reflexive orienting to gaze direction by facial expressions

Facial expression and gaze perception are thought to share brain mechanisms but behavioural interactions, especially from gaze-cueing paradigms, are inconsistent. We conducted a series of gaze-cueing studies using dynamic facial cues to examine orienting across different emotional expression and task conditions, including face inversion. Across experiments, at a short stimulus–onset asynchrony (SOA) we observed both an expression effect (i.e., faster responses when the face was emotional versus neutral) and a cue validity effect (i.e., faster responses when the target was gazed-at), but no interaction between validity and emotion. Results from face inversion suggest that the emotion effect may have been due to both facial expression and stimulus motion. At longer SOAs, validity and emotion interacted such that cueing by emotional faces, fearful faces in particular, was enhanced relative to neutral faces. These results converge with a growing body of evidence that suggests that gaze and expression are initially processed independently and interact at later stages to direct attentional orienting.     

Spatial S-R compatibility effects in an intentional imitation task

The active intermodal mapping hypothesis suggests that intentional imitation is mediated by a highly efficient, special-purpose mechanism of actor-centered movement encoding. In the present study, using methods from stimulus-response (S-R) compatibility research, we found no evidence to support this hypothesis. In two experiments, the performance of adult participants instructed to imitate actorcentered spatial properties of head, arm, and leg movements was affected by task-irrelevant, egocentric spatial cues. In Experiment 1, participants imitated using the same side of their bodies as did the model, and performance was less accurate when egocentric stimulus location was response incompatible than when it was response compatible. This effect was reversed in Experiment 2 when participants imitated using the opposite side of their bodies. These findings, in line with general process theories of imitation, imply that intentional imitation is mediated by the same processes that mediate responding to inanimate stimuli on the basis of arbitrary S-R mappings.     

Emotional attention: effects of emotion and gaze direction on overt orienting of visual attention

In the present study we considered the two factors that have been advocated for playing a role in emotional attention: perception of gaze direction and facial expression of emotions. Participants performed an oculomotor task in which they had to make a saccade towards one of the two lateral targets, depending on the colour of the fixation dot which appeared at the centre of the computer screen. At different time intervals (stimulus onset asynchronies, SOAs: 50,100,150 ms) following the onset of the dot, a picture of a human face (gazing either to the right or to the left) was presented at the centre of the screen. The gaze direction of the face could be congruent or incongruent with respect to the location of the target, and the expression could be neutral or angry. In Experiment 1 the facial expressions were presented randomly in a single block, whereas in Experiment 2 they were shown in separate blocks. Latencies for correct saccades and percentage of errors (saccade direction errors) were considered in the analyses. Results showed that incongruent trials determined a significantly higher percentage of saccade direction errors with respect to congruent trials, thus confirming that gaze direction, even when task-irrelevant, interferes with the accuracy of the observer’s oculomotor behaviour. The angry expression was found to hold attention for a longer time with respect to the neutral one, producing delayed saccade latencies. This was particularly evident at 100 ms SOA and for incongruent trials. Emotional faces may then exert a modulatory effect on overt attention mechanisms.     

Eye remember you: the effects of gaze direction on face recognition in children and adults

Children and adults were tested on a forced‐choice face recognition task in which the direction of eye gaze was manipulated over the course of the initial presentation and subsequent test phase of the experiment. To establish the effects of gaze direction on the encoding process, participants were presented with to‐be‐studied faces displaying either direct or deviated gaze (i.e. encoding manipulation). At test, all the faces depicted persons with their eyes closed. To investigate the effects of gaze direction on the efficiency of the retrieval process, a second condition (i.e. retrieval manipulation) was run in which target faces were presented initially with eyes closed and tested with either direct or deviated gaze. The results revealed the encoding advantages enjoyed by faces with direct gaze was present for both children and adults. Faces with direct gaze were also recognized better than faces with deviated gaze at retrieval, although this effect was most pronounced for adults. Finally, the advantage for direct gaze over deviated gaze at encoding was greater than the advantage for direct gaze over deviated gaze at retrieval. We consider the theoretical implications of these findings.     

Detection of emotional faces is modulated by the direction of eye gaze

Emotionally expressive faces have shown enhanced detectability over neutral faces, but little is known about the effect of eye gaze on detecting the presence of emotional faces. Emotional expressions and gaze direction are both cues to the intentions of another person, and gaze direction has been shown to affect recognition accuracy and perceived intensity of emotional faces. The current study showed that fearful faces were detected more frequently with an averted gaze than with a direct gaze in an attentional blink task, whereas angry and happy faces were detected more frequently with a direct gaze than with an averted gaze. The results are in line with the shared signal hypothesis and appraisal theory and suggest that selection for awareness was based on a rapid evaluation of the intentions of another person as conveyed by their facial expression and gaze direction.     

Increased spatial salience in the social Simon task: A response-coding account of spatial compatibility effects

A spatial compatibility effect (SCE) is typically observed in forced two-choice tasks in which a spatially defined response (e.g., pressing a left vs. a right key) has to be executed to a nonspatial feature of a stimulus (e.g., discriminating red from green) that is additionally connoted by a spatial feature (e.g., the stimulus points to the left or the right). Responses are faster and more accurate when the response side and the spatial stimulus feature are compatible than when they are incompatible. Previous research has demonstrated that SCEs are diminished when stimuli from only one response category are responded to in individual go/no-go tasks, whereas SCEs reemerge when two participants work jointly on two complementary, individual go/no-go tasks in a joint go/no-go task setting. This social Simon effect has been considered evidence for shared task representations. We show that SCEs emerge in individual go/no-go tasks when the spatial dimension is made more salient, whereas SCEs are eliminated in joint go/no-go tasks when the spatial dimension is made less salient. These findings are consistent with an account of social Simon effects in terms of spatial response coding, whereas they are inconsistent with an account of shared task representations. The relevance of social factors for spatial response coding is discussed.     

Vertical gaze direction and the resting posture of the eyes

With horizontal gaze, the resting posture of binocular vergence typically corresponds to a distance of about 1 m. The effect of vertical direction of gaze on this basic resting posture was investigated. The dark vergence of twenty-four subjects was measured while they fixated a dim monocular light-point at vertical directions ranging from -45 degree (lowered) to +30 degrees (elevated). In one condition, gaze was varied by changes in eye position with the head held upright; in a second condition, gaze was varied by changes in head inclination with the eyes held in constant (horizontal) position with respect to the head. In both conditions, dark vergence shifted in the convergent (nearer) direction with lowered gaze and in the divergent (farther) direction with elevated gaze. The effect of varied eye inclination was larger, more variable across subjects, and more stable over time than that of varied head inclination. These findings indicate that multiple mechanisms contribute to gaze-related variations of the resting posture of the eyes. They may help to explain the variations of space perception and visual fatigue that are observed with different gaze inclinations.     

Integrating gaze direction and expression in preferences for attractive faces

Few studies have investigated how physical and social facial cues are integrated in the formation of face preferences. Here we show that expression differentially qualifies the strength of attractiveness preferences for faces with direct and averted gaze. For judgments of faces with direct gaze, attractiveness preferences were stronger for smiling faces than for faces with neutral expressions. By contrast, for judgments of faces with averted gaze, attractiveness preferences were stronger for faces with neutral expressions than for smiling faces. Because expressions can differ in meaning depending on whether they are directed toward or away from oneself, it is only by integrating gaze direction, facial expression, and physical attractiveness that one can unambiguously identify the most attractive individuals who are likely to reciprocate one's own social interest.     

The interaction between gaze direction and facial expressions in newborns

The ability to decode facial expressions is an important component of social interaction and functioning. This ability is even more fundamental early in life, prior to the development of verbal communication. However, it is still unclear whether newborns can detect, discriminate and process facial expressions, and, if so, what the mechanisms underlying this ability are. In this study, we extend the investigation of perceived emotional expression by manipulating gaze direction with different facial expressions. Specifically, newborns were presented with faces displaying neutral, fearful, or happy facial expressions accompanied with direct or averted gaze, and tested in a visual preference paradigm. Four experiments were conducted in which different combinations of expression and gaze were used. However, only in the fourth experiment did newborns show a visual preference for a specific emotional display; they looked significantly longer at a happy face than a neutral one only when both were accompanied with direct gaze. These results provide support for the advantage of happy facial expressions in the development of a face processing system and suggest that this preference reflects experience acquired during the first few days after birth.