Direct control of fixation times in scene viewing: Evidence from analysis of the distribution of first fixation duration

Participants' eye movements were monitored in two scene viewing experiments that manipulated the task-relevance of scene stimuli and their availability for extrafoveal processing. In both experiments, participants viewed arrays containing eight scenes drawn from two categories. The arrays of scenes were either viewed freely (Free Viewing) or in a gaze-contingent viewing mode where extrafoveal preview of the scenes was restricted (No Preview). In Experiment 1a, participants memorized the scenes from one category that was designated as relevant, and in Experiment 1b, participants chose their preferred scene from within the relevant category. We examined first fixations on scenes from the relevant category compared to the irrelevant category (Experiments 1a and 1b), and those on the chosen scene compared to other scenes not chosen within the relevant category (Experiment 1b). A survival analysis was used to estimate the first discernible influence of the task-relevance on the distribution of first-fixation durations. In the free viewing condition in Experiment 1a, the influence of task relevance occurred as early as 81 ms from the start of fixation. In contrast, the corresponding value in the no preview condition was 254 ms, demonstrating the crucial role of extrafoveal processing in enabling direct control of fixation durations in scene viewing. First fixation durations were also influenced by whether or not the scene was eventually chosen (Experiment 1b), but this effect occurred later and affected fewer fixations than the effect of scene category, indicating that the time course of scene processing is an important variable mediating direct control of fixation durations.     

The effect of the first glimpse at a scene on eye movements during search

Previewing scenes briefly makes finding target objects more efficient when viewing is through a gaze-contingent window (windowed viewing). In contrast, showing a preview of a randomly arranged search display does not benefit search efficiency when viewing during search is of the full display. Here, we tested whether a scene preview is beneficial when the scene is fully visible during search. Scene previews, when presented, were 250 ms in duration. During search, the scene was either fully visible or windowed. A preview always provided an advantage, in terms of decreasing the time to initially fixate and respond to targets and in terms of the total number of fixations. In windowed visibility, a preview reduced the distance of fixations from the target position until at least the fourth fixation. In full visibility, previewing reduced the distance of the second fixation but not of later fixations. The gist information derived from the initial glimpse of a scene allowed for placement of the first one or two fixations at information-rich locations, but when nonfoveal information was available, subsequent eye movements were only guided by online information.     

Eye movement control during scene viewing: Immediate degradation and enhancement effects of spatial frequency filtering

What controls how long the eyes remain fixated during scene perception? We investigated whether fixation durations are under the immediate control of the quality of the current scene image. Subjects freely viewed photographs of scenes in preparation for a later memory test while their eye movements were recorded. Using the saccade-contingent display change method, scenes were degraded (Experiment 1) or enhanced (Experiment 2) via blurring (low-pass filtering) during predefined saccades. Results showed that fixation durations immediately after a display change were influenced by the degree of blur, with a monotonic relationship between degree of blur and fixation duration. The results also demonstrated that fixation durations can be both increased and decreased by changes in the degree of blur. The results suggest that fixation durations in scene viewing are influenced by the ease of processing of the image currently in view. The results are consistent with models of saccade generation in scenes in which moment-to-moment difficulty in visual and cognitive processing modulates fixation durations.     

On the control of visual fixation durations in free viewing of complex images

The mechanisms for the substantial variation in the durations of visual fixations in scene perception are not yet well understood. During free viewing of paintings, gaze-contingent irrelevant distractors (Exp. 1) and non-gaze-related time-locked display changes (Exp. 2) were presented. We demonstrated that any visual change-its onset and offset-prolongs the ongoing fixation (i.e., delays the following saccade), strongly suggesting that fixation durations are under the direct control of the stimulus information. The strongest influence of distraction was observed for fixations preceded by saccades within the parafoveal range (<5° of visual angle). We assume that these fixations contribute to the focal in contrast to the ambient mode of attention (Pannasch & Velichkovsky, Visual Cognition, 17, 1109-1131, 2009; Velichkovsky, Memory, 10, 405-419, 2002). Recent findings about two distinct "subpopulations of fixations," one under the direct and another under the indirect control of stimulation (e.g., Henderson & Smith, Visual Cognition, 17, 1055-1082, 2009), are reconsidered in view of these results.     

Effects of viewing time,fixations, and viewing strategies on visual memory for briefly presented natural objects

We investigated the impact of viewing time and fixations on visual memory for briefly presented natural objects. Participants saw a display of eight natural objects arranged in a circle and used a partial report procedure to assign one object to the position it previously occupied during stimulus presentation. At the longest viewing time of 7,000 ms or 10 fixations, memory performance was significantly higher than at the shorter times. This increase was accompanied by a primacy effect, suggesting a contribution of another memory component—for example, visual long-term memory (VLTM). We found a very limited beneficial effect of fixations on objects; fixated objects were only remembered better at the shortest viewing times. Our results revealed an intriguing difference between the use of a blocked versus an interleaved experimental design. When trial length was predictable, in the blocked design, target fixation durations increased with longer viewing times. When trial length was unpredictable, fixation durations stayed the same for all viewing lengths. Memory performance was not affected by this design manipulation, thus also supporting the idea that the number and duration of fixations are not closely coupled to memory performance.     

Investigating spatial vision and dynamic attentional selection using a gaze-contingent multiresolutional display

This study examined spatial vision and attentional selection using a gaze-contingent multiresolutional display, with a dynamic, gaze-centered, high-resolution window and lower resolution periphery. Visual search times and eye movements from 15 participants in a 3 x 3 design (Window Radius x Peripheral Resolution) suggest that contrast sensitivity as a function of retinal eccentricity affects attentional selection and visual processing. Smaller windows led to longer search times and shorter saccades; lower peripheral resolution also shortened saccades (all ps < .05) as a result of avoiding fixating degraded areas. Fixation durations, although longer for smaller windows (p < .05), were unaffected by whether the next saccade went within or outside the window. These results are explained through (a) competition among potential saccade targets where above-threshold filtering reduces an object's relative salience and (b) generally disrupted visual processing.     

Eye movements during scene viewing: Evidence for mixed control of fixation durations

Recent behavioral and computational research on eye movement control during scene viewing has focused on where the eyes move. However, fixations also differ in their durations, and when the eyes move may be another important indicator of perceptual and cognitive activity. Here we used a scene onset delay paradigm to investigate the degree to which individual fixation durations are under direct moment-to-moment control of the viewer’s current visual scene. During saccades just prior to critical fixations, the scene was removed from view so that when the eyes landed, no scene was present. Following a manipulated delay period, the scene was restored to view. We found that one population of fixations was under the direct control of the current scene, increasing in duration as delay increased. A second population of fixations was relatively constant across delay. The pattern of data did not change whether delay duration was random or blocked, suggesting that the effects were not under the strategic control of the viewer. The results support a mixed control model in which the durations of some fixations proceed regardless of scene presence, whereas others are under the direct moment-to-moment control of ongoing scene analysis.     

Not fixating at the line of text comes at a cost

Previous research on eye guidance in reading has investigated systematic tendencies with respect to horizontal fixation locations on letters within words and the relationship between fixation location in a word and the duration of the fixation. The present study investigates where readers place their eyes vertically on the line of text and how vertical fixation location is related to fixation duration. Analyses were based on a large corpus of eye movement recordings from single-sentence reading. The vertical preferred viewing location was found to be within the vertical extent of the font, but fixations beyond the vertical boundaries of the text also frequently occurred. Analyzing fixation duration as a function of vertical fixation location revealed a vertical optimal viewing position (vOVP) effect: Fixations were shortest when placed optimally on the line of text, and fixation duration gradually increased for fixations that fell above or below the line of text. The vOVP effect can be explained by the limits of visual resolution along the vertical meridian. It is concluded that vertical and horizontal landing positions in single-sentence reading are associated with differences in fixation durations in opposite ways.     

Perception onset time during fixations in free viewing

In this study, we investigated when visual perception begins in fixations. During picture viewing, the picture was degraded at the beginning of selected saccades and changed back to the original after varying intervals. Participants manually responded whenever they detected changes. The change-backs were undetected when they occurred <6 msec after the end of the saccade, marked by the peak of the overshoot in dual Purkinje image eyetracker data, and detection reached asymptote 32 msec after that marker. Eye velocity at the change-backtime also affected detection likelihood. Apparently, perception begins around the time at which the eyes stop rotating at the end of a saccade, giving a psychological justification for measuring fixation durations from then. This also specifies the deadline for gaze-contingent display changes to occur without detectable image motion. Investigators using the dual PurMnje image eyetracker should consider the peak of the overshoot as the fixation onset time and measure intrafixational presentation times from then.     

Brief foveal masking during scene perception

In the present experiment, participants were exploring line drawings of scenes in the context of an object-decision task, while eye-contingent display changes manipulated the appearance of the foveal part of the image. Foveal information was replaced by an ovoid noise mask for 83 ms, after a preset delay of 15, 35, 60, or 85 ms following the onset of fixations. In control conditions, a red ellipse appeared for 83 ms, centered around the fixation position, after the same delays as in the noise-mask conditions. It was found that scene exploration was hampered especially when foveal masking occurred early during fixations, replicating earlier findings. Furthermore, fixation durations were shown to increase linearly as the mask delay decreased, which validates the fixation duration as a measure of perceptual processing speed.     

Fixation durations in scene viewing: Modeling the effects of local image features,oculomotor parameters,and task

Scene perception requires the orchestration of image- and task-related processes with oculomotor constraints. The present study was designed to investigate how these factors influence how long the eyes remain fixated on a given location. Linear mixed models (LMMs) were used to test whether local image statistics (including luminance, luminance contrast, edge density, visual clutter, and the number of homogeneous segments), calculated for 1° circular regions around fixation locations, modulate fixation durations, and how these effects depend on task-related control. Fixation durations and locations were recorded from 72 participants, each viewing 135 scenes under three different viewing instructions (memorization, preference judgment, and search). Along with the image-related predictors, the LMMs simultaneously considered a number of oculomotor and spatiotemporal covariates, including the amplitudes of the previous and next saccades, and viewing time. As a key finding, the local image features around the current fixation predicted this fixation’s duration. For instance, greater luminance was associated with shorter fixation durations. Such immediacy effects were found for all three viewing tasks. Moreover, in the memorization and preference tasks, some evidence for successor effects emerged, such that some image characteristics of the upcoming location influenced how long the eyes stayed at the current location. In contrast, in the search task, scene processing was not distributed across fixation durations within the visual span. The LMM-based framework of analysis, applied to the control of fixation durations in scenes, suggests important constraints for models of scene perception and search, and for visual attention in general.     

Recognition advantage of happy faces in extrafoveal vision: Featural and affective processing

Happy, surprised, disgusted, angry, sad, fearful, and neutral facial expressions were presented extrafoveally (2.5° away from fixation) for 150 ms, followed by a probe word for recognition (Experiment 1) or a probe scene for affective valence evaluation (Experiment 2). Eye movements were recorded and gaze-contingent masking prevented foveal viewing of the faces. Results showed that (a) happy expressions were recognized faster than others in the absence of fixations on the faces, (b) the same pattern emerged when the faces were presented upright or upside-down, (c) happy prime faces facilitated the affective evaluation of emotionally congruent probe scenes, and (d) such priming effects occurred at 750 but not at 250 ms prime–probe stimulus–onset asynchrony. This reveals an advantage in the recognition of happy faces outside of overt visual attention, and suggests that this recognition advantage relies initially on featural processing and involves processing of positive affect at a later stage.     

Object–scene inconsistencies do not capture gaze: evidence from the flash-preview moving-window paradigm

In the present study, we investigated the influence of object-scene relationships on eye movement control during scene viewing. We specifically tested whether an object that is inconsistent with its scene context is able to capture gaze from the visual periphery. In four experiments, we presented rendered images of naturalistic scenes and compared baseline consistent objects with semantically, syntactically, or both semantically and syntactically inconsistent objects within those scenes. To disentangle the effects of extrafoveal and foveal object-scene processing on eye movement control, we used the flash-preview moving-window paradigm: A short scene preview was followed by an object search or free viewing of the scene, during which visual input was available only via a small gaze-contingent window. This method maximized extrafoveal processing during the preview but limited scene analysis to near-foveal regions during later stages of scene viewing. Across all experiments, there was no indication of an attraction of gaze toward object-scene inconsistencies. Rather than capturing gaze, the semantic inconsistency of an object weakened contextual guidance, resulting in impeded search performance and inefficient eye movement control. We conclude that inconsistent objects do not capture gaze from an initial glimpse of a scene.     

Describing scenes hardly seen

Knowledge about scene categories, the so-called gist, can be extracted very rapidly, while recognition and naming of individual scene objects is a more effortful process. We investigate this phenomenon by presenting action scenes involving two actors for durations varying between 100 and 300 ms. Incoherence was created by mirroring individual scene actors. Upon masked presentation participants had to report content, actors and objects and to indicate whether the scene was meaningful or not. Scene coherence was judged correctly at all presentation durations. Actors were correctly identified in about one-third of the cases even with presentation durations of 100 ms, and identification rate increased up to 80% with longer durations. Identification depended on scene coherence, on the position of agents in the scene, and on the position of actors relative to the fixation cross. These interdependencies of scene and object perception indicate that the visual system seems to be very sensitive to meaningful interactions of living entities. A series of fixations is not necessary to identify actors of a scene.     

True three-dimensional displays that allow viewers to dynamically shift accommodation, bringing objects displayed at different viewing distances into and out of focus.

Under natural viewing conditions, viewers do not just passively perceive. Instead, they dynamically scan the visual scene by shifting their eye fixation and focus between objects at different viewing distances. In doing so, the oculomotor processes of accommodation (eye focus) and vergence (angle between lines of sight of two eyes) must be shifted synchronously to place new objects in sharp focus in the center of each retina. Accordingly, nature has reflexively linked accommodation and vergence, such that a change in one process automatically drives a matching change in the other. Conventional stereoscopic displays force viewers to try to decouple these processes, because while they must dynamically vary vergence angle to view objects at different stereoscopic distances, they must keep accommodation at a fixed distance--or else the entire display will slip out of focus. This decoupling generates eye fatigue and compromises image quality when viewing such displays. In an effort to solve this accommodation/vergence mismatch problem, we have built various prototype displays that can vary the focus of objects at different distances in a displayed scene to match vergence and stereoscopic retinal disparity demands and better simulate natural viewing conditions. By adjusting the focus of individual objects in a scene to match their stereoscopic retinal disparity, the cues to ocular accommodation and vergence are brought into agreement. As in natural vision, the viewer brings different objects into focus by shifting accommodation. As the mismatch between accommodation and vergence is decreased, natural viewing conditions are better simulated and eye fatigue should decrease.     

Spatial frequency filtering and the direct control of fixation durations during scene viewing

The present study employed a saccade-contingent change paradigm to investigate the effect of spatial frequency filtering on fixation durations during scene viewing. Subjects viewed grayscale scenes while encoding them for a later memory test. During randomly chosen saccades, the scene was replaced with an alternate version that remained throughout the critical fixation that followed. In Experiment 1, during the critical fixation, the scene could be changed to high-pass and low-pass spatial frequency filtered versions. Under both conditions, fixation durations increased, and the low-pass condition produced a greater effect than the high-pass condition. In subsequent experiments, we manipulated the familiarity of scene information during the critical fixation by flipping the filtered scenes upside down or horizontally. Under these conditions, we observed lengthening of fixation durations but no difference between the high-pass and low-pass conditions, suggesting that the filtering effect is related to the mismatch between information extracted within the critical fixation and the ongoing scene representation in memory. We also conducted control experiments that tested the effect of changes to scene orientation (Experiment 2a) and the addition of color to a grayscale scene (Experiment 2b). Fixation distribution analysis suggested two effects on the distribution fixation durations: a fast-acting effect that was sensitive to all transsaccadic changes tested and a later effect in the tail of the distribution that was likely tied to the processing of scene information. These findings are discussed in the context of theories of oculomotor control during scene viewing.     

Eye Movements and Visual Encoding During Scene Perception

ABSTRACT— The amount of time viewers could process a scene during eye fixations was varied by a mask that appeared at a certain point in each eye fixation. The scene did not reappear until the viewer made an eye movement. The main finding in the studies was that in order to normally process a scene, viewers needed to see the scene for at least 150 ms during each eye fixation. This result is surprising because viewers can extract the gist of a scene from a brief 40- to 100-ms exposure. It also stands in marked contrast to reading, as readers need only to view the words in the text for 50 to 60 ms to read normally. Thus, although the same neural mechanisms control eye movements in scene perception and reading, the cognitive processes associated with each task drive processing in different ways.     

Reading with a filtered fovea: The influence of visual quality at the point of fixation during reading

Reading relies critically on processing text in foveal vision during brief fixational pauses, and high-quality visual input from foveal text is fundamental to theories of reading. However, the quality of visual input from foveal text that is actually functional for reading and the effects of this input on reading performance are unclear. To investigate these issues, a moving, gaze-contingent foveal filtering technique was developed to display areas of text within foveal vision that provided only coarse, medium, or fine scale visual input during each fixational pause during reading. Normal reading times were unaffected when foveal text up to three characters wide at the point of fixation provided any one visual input (coarse, medium, or fine). Wider areas of coarse visual input lengthened reading times, but reading still occurred, and normal reading times were completely unaffected when only medium or fine visual input extended across the entire fovea. Further analyses revealed that each visual input had no effect on the number of fixations made when normal text was read, that adjusting fixation durations helped preserve reading efficiency for different visual inputs, and that each visual input had virtually no effect on normal saccades. These findings indicate that, despite the resolving power of foveal vision and the emphasis placed on high-quality foveal visual input by theories of reading, normal reading functions with similar success using a range of restricted visual inputs from foveal text, even at the point of fixation. Some implications of these findings for theories of reading are discussed.     

场景情境和目标模板在视觉搜索中的交互作用

以真实场景图像中的物体搜索为实验任务, 操纵场景情境和目标模板, 采用眼动技术将搜索过程分为起始阶段、扫描阶段和确认阶段, 考察场景情境和目标模板对视觉搜索过程的影响机制。结果发现, 场景情境和目标模板的作用方式及时间点不同, 二者交互影响搜索的正确率和反应时, 仅场景情境影响起始阶段的时间, 随后二者交互影响扫描阶段和确认阶段的时间及主要眼动指标。作者由此提出了场景情境和目标模板在视觉搜索中的交互作用模型。     

Stable individual differences across images in human saccadic eye movements.

Individual differences in eye movements during picture viewing were examined across image format, content, and foveal quality in 3 experiments. Experiment 1 demonstrated that an individual's fixation durations were strongly related across 3 types of scene formats and that saccade amplitudes followed the same pattern. In Experiment 2, a similar relationship was observed for fixation durations across faces and scenes, although the amplitude relationship did not hold as strongly. In Experiment 3, the duration and amplitude relationships were observed when foveal information was degraded and even removed. Eye movement characteristics differ across individuals, but there is a great deal of consistency within individuals when viewing different types of images.