Visual saliency and semantic incongruency influence eye movements when inspecting pictures

Models of low-level saliency predict that when we first look at a photograph our first few eye movements should be made towards visually conspicuous objects. Two experiments investigated this prediction by recording eye fixations while viewers inspected pictures of room interiors that contained objects with known saliency characteristics. Highly salient objects did attract fixations earlier than less conspicuous objects, but only in a task requiring general encoding of the whole picture. When participants were required to detect the presence of a small target, then the visual saliency of nontarget objects did not influence fixations. These results support modifications of the model that take the cognitive override of saliency into account by allowing task demands to reduce the saliency weights of task-irrelevant objects.

The pictures sometimes contained incongruent objects that were taken from other rooms. These objects were used to test the hypothesis that previous reports of the early fixation of congruent objects have not been consistent because the effect depends upon the visual conspicuity of the incongruent object. There was an effect of incongruency in both experiments, with earlier fixation of objects that violated the gist of the scene, but the effect was only apparent for inconspicuous objects, which argues against the hypothesis.     

Visual imagery differences and eye movements in the recall of pictures

Groups of vivid and poor visualizers were given a picture memory task. and horizontal and vertical components of the electro-oculogram were recorded. This allowed a detailed investigation of each S’s eye movements in the perception. imagery, and recall phases of the task. The vivid visualizers gave a higher accuracy of recall Eye movement rate was lower in visual imagery than it was in perception, especially in the goup of vivid visualizers. There was some evidence of scanning activity prior to recall, but only if positional cues were provided or if recall was incorrect. No scanning occurred prior to accurate recall unprompted by a positional cue. These results provide no support to the theories of image construction proposed by Hebb (1949, 1968) and Neisser (1967). As suggested by Singer (1966), an absence of eye movement may be a necessary condition for vivid visual imagery.     

Eye movements to pictures reveal transient semantic activation during spoken word recognition

Two experiments explore the activation of semantic information during spoken word recognition. Experiment 1 shows that as the name of an object unfolds (e.g., lock), eye movements are drawn to pictorial representations of both the named object and semantically related objects (e.g., key). Experiment 2 shows that objects semantically related to an uttered word's onset competitors become active enough to draw visual attention (e.g., if the uttered word is logs, participants fixate on key because of partial activation of lock), despite that the onset competitor itself is not present in the visual display. Together, these experiments provide detailed information about the activation of semantic information associated with a spoken word and its phonological competitors and demonstrate that transient semantic activation is sufficient to impact visual attention.     

Visual short-term memory during smooth pursuit eye movements

Visual short-term memory (VSTM) was probed while observers performed smooth pursuit eye movements. Smooth pursuit keeps a moving object stabilized in the fovea. VSTM capacity for position was reduced during smooth pursuit compared with a condition with eye fixation. There was no difference between a condition in which the items were approximately stabilized on the retina because they moved with the pursuit target and a condition in which the items moved across the retina because they were stationary in space. The reduction of capacity for position was eliminated when miniature items were presented on the pursuit target. Similarly, VSTM capacity for color did not differ between smooth pursuit and fixation. The results suggest that visuospatial attention is tied to the target during smooth pursuit, which impairs VSTM for the position of peripheral objects. Sensory memory during smooth pursuit was only slightly impaired.     

Quantifying the contribution of low-level saliency to human eye movements in dynamic scenes

We investigated the contribution of low-level saliency to human eye movements in complex dynamic scenes. Eye movements were recorded while naive observers viewed a heterogeneous collection of 50 video clips (46,489 frames; 4-6 subjects per clip), yielding 11,916 saccades of amplitude ≥2°. A model of bottom-up visual attention computed instantaneous saliency at the instant each saccade started and at its future endpoint location. Median model-predicted saliency was 45% the maximum saliency, a significant factor 2.03 greater than expected by chance. Motion and temporal change were stronger predictors of human saccades than colour, intensity, or orientation features, with the best predictor being the sum of all features. There was no significant correlation between model-predicted saliency and duration of fixation. A majority of saccades were directed to a minority of locations reliably marked as salient by the model, suggesting that bottom-up saliency may provide a set of candidate saccade target locations, with the final choice of which location of fixate more strongly determined top-down.     

Distinct processing for pictures of animals and objects: evidence from eye movements

Many studies have suggested that emotional stimuli orient and engage attention. There is also evidence that animate stimuli, such as those from humans and animals, cause attentional bias. However, categorical and emotional factors are usually mixed, and it is unclear to what extent human context influences attentional allocation. To address this issue, we tracked participants' eye movements while they viewed pictures with animals and inanimate images (i.e., category) as focal objects. These pictures had either negative or neutral emotional valence, and either human body parts or nonhuman parts were near the focal objects (i.e., context). The picture's valence, arousal, position, size, and most of the low-level visual features were matched across categories. The results showed that nonhuman animals were more likely to be attended to and to be attended to for longer times than inanimate objects. The same pattern held for the human contexts (vs. nonhuman contexts). The effects of emotional valence, category, and context interacted. Specifically, in images with a negative valence, focal animals and objects with human context had comparable numbers of gaze fixations and gaze duration. These results highlighted the attentional bias to animate parts of a picture and clarified that the effects of category, valence, and picture context interacted to influence attentional allocation.     

The influence of eye movements and illumination on auditory localization

A critical analysis of a recent paper by Shelton and Searle (1980) on the visual facilitation of auditory localization is presented. The author claims that Shelton and Searle fail to make the relevant distinction between Warren’s (1970) frame of reference hypothesis and Jones’s (1975) spatial memory hypothesis. Shelton and Searle’s claim that they have demonstrated the existence of two distinct forms of visual facilitation is questioned. Data from an experiment in which auditory localization was tested under two levels of illumination (light and dark) and with two kinds of eye movement instructions (fixed and movement) are presented. The results show that facilitation occurs only when eye movements take place in a lighted environment. This is interpreted as supporting Warren’s frame of reference hypothesis.     

Visual search,complex backgrounds,mental counters,and eye movements

For 30 days, three Ss searched for 1, 5, or 10 targets (N), any one of which had to occur once (as in past research). twice, or four times (FREQ). The N=1 and N=5 items were subsets of N=10 items. When FREQ=1. Ss searched longer for 5 targets than for 1 or 10 targets. This nonintuitive result was due to (a) Ss searching for all 10 targets even at N=5, and (b) the reported “pop-out effect” of the 5 target items as well as the other 5 items in the N=10 set. When FREQ > 1, Ss could search for only 2-3 targets at once, because they could not keep mental counters on more items. The presence of a complex background merely added a constant to search time. The main change in eye movements, with 30 days practice, was to reduce greatly the number of eye fixations, thus suggesting an increase in the useful fidld of view and/or a change in search strategy. A speculative model, based upon two comparison processes, was suggested to explain the results.     

The influence of eye movements and eye position on perception of the visual vertical

Summary Four experiments are reported which involved the adjustment of two points of light in an otherwise dark room to the visual vertical. In the first two experiments different point separations (corresponding to visual angles of about 2° and 23°) were employed in an attempt to control the degree of scanning eye movements. Binocular adjustments to the visual vertical during body tilt were influenced by the point separation (Exp. 1), but monocular adjustments with the head upright were not (Exp. 2). Using the larger point separation and fixating the top point the visual vertical using the right eye was counterclockwise of that with fixation of the bottom point when the head was upright (Exp. 3), but this difference was not found for judgments during tilt (Exp. 4). The results were discussed in terms of the rotational changes in eye position accompanying scanning eye movements and ocular elevation and depression.

---

Zusammenfassung Es wird über vier Experimente berichtet, in denen es um die Einstellungen zweier Lichtpunkte in eine vertikale Richtung geht. In den ersten zwei Experimenten sollte der Einfluß der Fixierbewegungen der Augen geprüft werden. Der Winkelabstand der Lichtpunkte betrug 2° oder 23°. Bei seitwärts geneigtem Körper und binokularer Betrachtung wurde ein Unterschied gefunden (Exp. 1). Bei aufrecht gehaltenem Kopf und monokularer Betrachtung dagegen zeigte sich kein Unterschied (Exp. 2). Wurde in aufrechter Körperlage und bei einem Punktabstand von 23° der obere Punkt mit dem rechten Auge fixiert, so verschob sich die scheinbare Vertikale im Uhrzeigersinn; sie verschob sich noch weiter im Uhrzeigersinn, wenn der untere Punkt fixiert wurde (Exp. 3). Bei geneigtem Körper wurde dagegen kein Unterschied festgestellt (Exp. 4). Die Ergebnisse werden im Zusammenhang mit der Augendrehung diskutiert, die mit dem scanning sowie mit der Auf- und Abbewegung der Augen beim Fixieren verbunden ist.

---

---

This research was carried out while I was a recipient of a Forschungsstipendium from the Alexander von Humboldt-Stiftung, appreciation for which is gratefully acknowledged. I also wish to thank the subjects who participated in the experiments.     

Models of high-dimensional semantic space predict language-mediated eye movements in the visual world

In the visual world paradigm, participants are more likely to fixate a visual referent that has some semantic relationship with a heard word, than they are to fixate an unrelated referent [Cooper, R. M. (1974). The control of eye fixation by the meaning of spoken language. A new methodology for the real-time investigation of speech perception, memory, and language processing. Cognitive Psychology, 6, 813-839]. Here, this method is used to examine the psychological validity of models of high-dimensional semantic space. The data strongly suggest that these corpus-based measures of word semantics predict fixation behavior in the visual world and provide further evidence that language-mediated eye movements to objects in the concurrent visual environment are driven by semantic similarity rather than all-or-none categorical knowledge. The data suggest that the visual world paradigm can, together with other methodologies, converge on the evidence that may help adjudicate between different theoretical accounts of the psychological semantics.     

Visual span in expert chess players: evidence from eye movements

The reported research extends classic findings that after briefly viewing structured, but not random, chess positions, chess masters reproduce these positions much more accurately than less-skilled players. Using a combination of the gaze-contingent window paradigm and the change blindness flicker paradigm, we documented dramatically larger visual spans for experts while processing structured, but not random, chess positions. In addition, in a check-detection task, a minimized 3 × 3 chessboard containing a King and potentially checking pieces was displayed. In this task, experts made fewer fixations per trial than less-skilled players, and had a greater proportion of fixations between individual pieces, rather than on pieces. Our results provide strong evidence for a perceptual encoding advantage for experts attributable to chess experience, rather than to a general perceptual or memory superiority.     

Effects of context on eye movements when reading about possible and impossible events

Plausibility violations resulting in impossible scenarios lead to earlier and longer lasting eye movement disruption than violations resulting in highly unlikely scenarios (K. Rayner, T. Warren, B. J. Juhasz, & S. P. Liversedge, 2004; T. Warren & K. McConnell, 2007). This could reflect either differences in the timing of availability of different kinds of information (e.g., selectional restrictions, world knowledge, and context) or differences in their relative power to guide semantic interpretation. The authors investigated eye movements to possible and impossible events in real-world and fantasy contexts to determine when contextual information influences detection of impossibility cued by a semantic mismatch between a verb and an argument. Gaze durations on a target word were longer to impossible events independent of context. However, a measure of the time elapsed from first fixating the target word to moving past it showed disruption only in the real-world context. These results suggest that contextual information did not eliminate initial disruption but moderated it quickly thereafter.     

The influence of text legibility on eye movements during reading

Subjects read passages of text on a video monitor as their eye movements were recorded. In Experiment 1, the passages were presented either in a format (ClearType) designed to display smoother, clearer characters on LCD monitors by eliminating pixilation or in standard format (non‐ClearType). The passages were also presented in three different fonts (Times New Roman, Harrington and Script MT bold) which differed in how easy the letters were to encode. While there were no comprehension differences due to ClearType or font, ClearType led to faster reading, fewer fixations, and shorter fixation durations than non‐ClearType. There were also font differences, with Times New Roman leading to faster reading than the other two fonts. In Experiment 2, we replicated the results that ClearType lead to faster reading than non‐ClearType when subjects read single sentences with high or low frequency target words. Across both experiments, word frequency interacted with presentation format. Copyright © 2009 John Wiley & Sons, Ltd.     

Event-related potentials and the semantic matching of pictures

Event-related potentials (ERPs) were recorded from one midline and three pairs of lateral electrodes while subjects determined whether pairs of sequentially presented pictures were semantically associated. The ERPs evoked by the second picture of each pair differed as a consequence of whether it was associated with its predecessor, such that ERPs to nonassociated pictures were more negative-going than those to associated items. These differences resulted from the modulation of two ERP components, one frontally distributed and centered on an N300 deflection, the other distributed more widely over the scalp and encompassing an N450 deflection. The modulation of N450 is interpreted as further evidence that the "N400" ERP component is sensitive to semantic relationships between nonverbal stimuli. The earlier N300 effects, which do not appear to occur when ERPs are evoked by semantically primed and unprimed words, could suggest that the semantic processing of pictorial stimuli involves neural systems different from those associated with the semantic processing of words.     

Probing the influence of unconscious fear-conditioned visual stimuli on eye movements

Efficient threat detection from the environment is critical for survival. Accordingly, fear-conditioned stimuli receive prioritized processing and capture overt and covert attention. However, it is unknown whether eye movements are influenced by unconscious fear-conditioned stimuli. We performed a classical fear-conditioning procedure and subsequently recorded participants’ eye movements while they were exposed to fear-conditioned stimuli that were rendered invisible using interocular suppression. Chance-level performance in a forced-choice-task demonstrated unawareness of the stimuli. Differential skin conductance responses and a change in participants’ fearfulness ratings of the stimuli indicated the effectiveness of conditioning. However, eye movements were not biased towards the fear-conditioned stimulus. Preliminary evidence suggests a relation between the strength of conditioning and the saccadic bias to the fear-conditioned stimulus. Our findings provide no strong evidence for a saccadic bias towards unconscious fear-conditioned stimuli but tentative evidence suggests that such an effect may depend on the strength of the conditioned response.     

The influence of parafoveal typographical errors on eye movements in reading

Three experiments are reported, examining the effects of a typographical error in parafoveal vision on aspects of foveal inspection time and saccade targeting. All the experiments involved reading for comprehension. A contingent presentation procedure ensured that typographical errors were restored to their correct form before they were viewed in foveal vision: They were never available for foveal processing. In Experiment 1, the error was formed by replacing the first letter of the target word with a second occurrence of its second letter, producing an illegal nonword. This manipulation had no significant effect on foveal inspection time, but lowered the probability that a short word (“de” or “du”) prior to the target would be skipped. In Experiment 2 the familiarity of the target's initial letters was maintained constant across conditions. This manipulation removed the target 1 skipping effect, suggesting that the outcome of Experiment 1 was due to orthographic rather than lexical illegality, but revealed shorter foveal inspection times as a function of the presence of the error. Experiment 3 manipulated lexical and sublexical properties of the parafoveal typing error. Properties of the parafoveal error again influenced prior foveal inspection times. The pattern of results suggested that the determining properties were sublexical rather than lexical. The results as a whole are incompatible with a view of information processing in reading in which foveal processing remains immune from concurrent parafoveal influences.     

The influence of semantic context on initial eye landing sites in words

To determine the role of ongoing processing on eye guidance in reading, two studies examined the effects of semantic context on the eyes' initial landing position in words of different levels of processing difficulty. Results from both studies clearly indicate a shift of the initial fixation location towards the end of the words for words that can be predicted from a prior semantic context. However, shifts occur only in high-frequency words and with prior fixations close to the beginning of the target word. These results suggest that ongoing perceptual and linguistic processes can affect the decision of where to send the eyes next in reading. They are explained in terms of the easiness of processing associated with the target words when located in parafoveal vision. It is concluded that two critical factors might help observing effects of linguistic variables on initial landing sites, namely, the frequency of the target word and the position where the eyes are launched from as regards to the beginning of the target word. Results also provide evidence for an early locus of semantic context effects in reading.