Prioritization of new objects in real-world scenes: evidence from eye movements

The authors examined the prioritization of abruptly appearing objects in real-world scenes by measuring the eyes' propensity to be directed to the new object. New objects were fixated more often than chance whether they appeared during fixations (transient onsets) or saccades (nontransient onsets). However, onsets that appeared during fixations were fixated sooner and more often than those coincident with saccades. Prioritization of onsets during saccades, but not fixations, were affected by manipulations of memory: Reducing scene viewing time prior to the onset eliminated prioritization, whereas prior study of the scenes increased prioritization. Transient objects draw attention quickly and do not depend on memory, but without a transient signal, new objects are prioritized over several saccades as memory is used to explicitly identify the change. These effects were not modulated by observers' expectations concerning the appearance of new objects, suggesting the prioritization of a transient is automatic and that memory-guided prioritization is implicit.     

The Role of Fixation Position in Detecting Scene Changes Across Saccades

Target objects presented within color images of naturalistic scenes were deleted or rotated during a saccade to or from the target object or to a control region of the scene. Despite instructions to memorize the details of the scenes and to monitor for object changes, viewers frequently failed to notice the changes. However, the failure to detect change was mediated by three other important factors: First, accuracy generally increased as the distance between the changing region and the fixation immediately before or after the change decreased. Second, changes were sometimes initially missed, but subsequently noticed when the changed region was later refixated. Third, when an object disappeared from a scene, detection of that disappearance was greatly improved when the deletion occurred during the saccade toward that object. These results suggest that fixation position and saccade direction play an important role in determining whether changes will be detected. It appears that more information can be retained across views than has been suggested by previous studies.     

Eye movements and visual memory: detecting changes to saccade targets in scenes

Saccade-contingent change detection provides a powerful tool for investigating scene representation and scene memory. In the present study, critical objects presented within color images of naturalistic scenes were changed during a saccade toward or away from the target. During the saccade,the critical object was changed to another object type, to a visually different token of the same object type, or was deleted from the scene. There were three main results. First, the deletion of a saccade target was special: Detection performance for saccade target deletions was very good, and this level of performance did not decline with the amplitude of the saccade. In contrast, detection of type and token changes at the saccade target, and of all changes including deletions at a location that had just been fixated but was not the saccade target, decreased as the amplitude of the saccade increased. Second, detection performance for type and token changes, both when the changing object was the target of the saccade and when the object had just been fixated but was not the saccade target, was well above chance. Third, mean gaze durations were reliably elevated for those trials in which the change was not overtly detected. The results suggest that the presence of the saccade target plays a special role in trassaccadic integration, and together with other recent findings, suggest more generally that a relatively rich scene representation is retained across saccades and stored in visual memory.     

Local and global contextual constraints on the identification of objects in scenes.

Objects likely to appear in a given real-world scene are frequently found to be easier to recognize. Two different sources of contextual information have been proposed as the basis for this effect: global scene background and individual companion objects. The present paper examines the relative importance of these two elements in explaining the context-sensitivity of object identification in full scenes. Specific sequences of object fixations were elicited during free scene exploration, while fixation times on designated target objects were recorded as a measure of ease of target identification. Episodic consistency between the target, the global scene background, and the object fixated just prior to the target (the prime), were manipulated orthogonally. Target fixation times were examined for effects of prime and background. Analyses show effects of both factors, which are modulated by the chronology and spatial extent of scene exploration. The results are discussed in terms of their implications for a model of visual object recognition in the context of real-world scenes.     

Constructing visual representations of natural scenes: the roles of short- and long-term visual memory

A "follow-the-dot" method was used to investigate the visual memory systems supporting accumulation of object information in natural scenes. Participants fixated a series of objects in each scene, following a dot cue from object to object. Memory for the visual form of a target object was then tested. Object memory was consistently superior for the two most recently fixated objects, a recency advantage indicating a visual short-term memory component to scene representation. In addition, objects examined earlier were remembered at rates well above chance, with no evidence of further forgetting when 10 objects intervened between target examination and test and only modest forgetting with 402 intervening objects. This robust prerecency performance indicates a visual long-term memory component to scene representation.     

Effects of foveal priming and extrafoveal preview on object identification

The results of three different experiments suggested that the relation between an object in the fovea on fixation n and an object subsequently brought into the fovea on fixation n + 1 affects the time to identify the second object. In Experiment 1 we extended previous work by demonstrating that a previously seen related priming object speeded the time to name a target object even when a saccade intervened between the two objects. In Experiment 2 we replicated this result and further showed that the benefit on naming time was due to facilitation from the related object rather than inhibition from the unrelated object. In addition, naming of the target object was much slower in both experiments when there was not a peripheral preview of the target object on fixation n. However, because the effect of the foveal priming object was greater when the target was not present than when it was present, priming did not appear to make extraction of the extrafoveal information more efficient. In Experiment 3, fixation times were recorded while subjects looked at four objects in order to identify them. Fixation time on an object was shorter when a related object was fixated immediately before it, even though the four objects did not form a scene. The size of the facilitation was roughly comparable to that in several analogous experiments where scenes were used. The results suggest that the effects of a predictive scene context on object identification may be explainable in terms of an object-to-object or "intralevel" priming mechanism.     

Optimal and preferred eye landing positions in objects and scenes

Viewing position effects are commonly observed in reading, but they have only rarely been investigated in object perception or in the realistic context of a natural scene. In two experiments, we explored where people fixate within photorealistic objects and the effects of this landing position on recognition and subsequent eye movements. The results demonstrate an optimal viewing position—objects are processed more quickly when fixation is in the centre of the object. Viewers also prefer to saccade to the centre of objects within a natural scene, even when making a large saccade. A central landing position is associated with an increased likelihood of making a refixation, a result that differs from previous reports and suggests that multiple fixations within objects, within scenes, occur for a range of reasons. These results suggest that eye movements within scenes are systematic and are made with reference to an early parsing of the scene into constituent objects.     

Eye movement control during scene viewing: Immediate degradation and enhancement effects of spatial frequency filtering

What controls how long the eyes remain fixated during scene perception? We investigated whether fixation durations are under the immediate control of the quality of the current scene image. Subjects freely viewed photographs of scenes in preparation for a later memory test while their eye movements were recorded. Using the saccade-contingent display change method, scenes were degraded (Experiment 1) or enhanced (Experiment 2) via blurring (low-pass filtering) during predefined saccades. Results showed that fixation durations immediately after a display change were influenced by the degree of blur, with a monotonic relationship between degree of blur and fixation duration. The results also demonstrated that fixation durations can be both increased and decreased by changes in the degree of blur. The results suggest that fixation durations in scene viewing are influenced by the ease of processing of the image currently in view. The results are consistent with models of saccade generation in scenes in which moment-to-moment difficulty in visual and cognitive processing modulates fixation durations.     

Stuck on semantics: Processing of irrelevant object-scene inconsistencies modulates ongoing gaze behavior

People have an amazing ability to identify objects and scenes with only a glimpse. How automatic is this scene and object identification? Are scene and object semantics—let alone their semantic congruity—processed to a degree that modulates ongoing gaze behavior even if they are irrelevant to the task at hand? Objects that do not fit the semantics of the scene (e.g., a toothbrush in an office) are typically fixated longer and more often than objects that are congruent with the scene context. In this study, we overlaid a letter T onto photographs of indoor scenes and instructed participants to search for it. Some of these background images contained scene-incongruent objects. Despite their lack of relevance to the search, we found that participants spent more time in total looking at semantically incongruent compared to congruent objects in the same position of the scene. Subsequent tests of explicit and implicit memory showed that participants did not remember many of the inconsistent objects and no more of the consistent objects. We argue that when we view natural environments, scene and object relationships are processed obligatorily, such that irrelevant semantic mismatches between scene and object identity can modulate ongoing eye-movement behavior.     

Eye movements and picture processing during recognition

Eye movements were monitored during a recognition memory test of previously studied pictures of full-color scenes. The test scenes were identical to the originals, had an object deleted from them, or included a new object substituted for an original object. In contrast to a prior report (Parker, 1978), we found no evidence that object deletions or substitutions could be recognized on the basis of information acquired from the visual periphery. Deletions were difficult to recognize peripherally, and the eyes were not attracted to them. Overall, the amplitude of the average saccade to the critical object during the memory test was less than 4.5 degrees of visual angle in all conditions and averaged 4.1 degrees across conditions. We conclude that information about object presence and identity in a scene is limited to a relatively small region around the current fixation point.     

Perceptual effects of scene context on object identification

Summary In a number of studies the context provided by a real-world scene has been claimed to have a mandatory, perceptual effect on the identification of individual objects in such a scene. This claim has provided a basis for challenging widely accepted data-driven models of visual perception in order to advocate alternative models with an outspoken top-down character. The present paper offers a review of the evidence to demonstrate that the observed scene-context effects may be the product of post-perceptual and task-dependent guessing strategies. A new research paradigm providing an on-line measure of genuine perceptual effects of context on object identification is proposed. First-fixation durations for objects incidentally fixated during the free exploration of real-world scenes are shown to increase when the objects are improbable in the scene or violate certain aspects of their typical spatial appearance in it. These effects of contextual violations are shown to emerge only at later stages of scene exploration, contrary to the notion of schema-driven scene perception effective from the very first scene fixation. In addition, evidence is reported in support of the existence of a facilitatory component in scene-context effects. This is taken to indicate that the context directly affects the ease of perceptual object processing and does not merely serve as a framework for checking the plausibility of the output of perceptual processes. Finally, our findings are situated against other contrasting results. Some future research questions are highlighted.     

Identification of objects in scenes: the role of scene background in object naming.

Research with brief presentations of scenes has indicated that scene context facilitates object identification. In the present experiments we used a paradigm in which an object in a scene is "wiggled"--drawing both attention and an eye fixation to itself--and then named. Thus the effect of scene context on object identification can be examined in a situation in which the target object is fixated and hence is fully visible. Experiment 1 indicated that a scene background that was episodically consistent with a target object facilitated the speed of naming. In Experiments 2 and 3, we investigated the time course of scene background information acquisition using display changes contingent on eye movements to the target object. The results from Experiment 2 were inconclusive; however, Experiment 3 demonstrated that scene background information present only on either the first or second fixation on a scene significantly affected naming time. Thus background information appears to be both extracted and able to affect object identification continuously during scene viewing.     

Processing of extrafoveal objects during multiple-object naming

In 3 experiments, the authors investigated the extent to which objects that are about to be named are processed prior to fixation. Participants named pairs or triplets of objects. One of the objects, initially seen extrafoveally (the interloper), was replaced by a different object (the target) during the saccade toward it. The interloper-target pairs were identical or unrelated objects or visually and conceptually unrelated objects with homophonous names (e.g., animal- baseball bat). The mean latencies and gaze durations for the targets were shorter in the identity and homophone conditions than in the unrelated condition. This was true when participants viewed a fixation mark until the interloper appeared and when they fixated on another object and prepared to name it while viewing the interloper. These results imply that objects that are about to be named may undergo far-reaching processing, including access to their names, prior to fixation.     

靶刺激变换与情景信息属性对真实情景再认的影响

采用眼动研究范式,通过两个实验考察靶刺激变换与情景信息属性对情景再认的影响。实验一结果显示,靶刺激变换对情景再认、靶刺激所属兴趣区的凝视时间均有显著影响,这表明在情景再认过程中,观察者会有意识地搜索靶刺激,靶刺激具有诊断效应;实验二应用了知觉信息与语义信息重合和分离两种情景材料。结果显示,观察者对语义信息的首次注视次数显著多于知觉信息;观察者对知觉信息和语义信息分离条件下语义信息的首次注视时间显著长于重合条件下。这一结果提示,在情景识别过程中,语义信息具有注意优先性,但其优先性会受到知觉信息启动的干扰。     

TAM: Explaining off-object fixations and central fixation tendencies as effects of population averaging during search

Understanding how patterns are selected for both recognition and action, in the form of an eye movement, is essential to understanding the mechanisms of visual search. It is argued that selecting a pattern for fixation is time consuming-requiring the pruning of a population of possible saccade vectors to isolate the specific movement to the potential target. To support this position, two experiments are reported showing evidence for off-object fixations, where fixations land between objects rather than directly on objects, and central fixations, where initial saccades land near the center of scenes. Both behaviors were modeled successfully using TAM (Target Acquisition Model; Zelinsky, 2008). TAM interprets these behaviors as expressions of population averaging occurring at different times during saccade target selection. A large population early during search results in the averaging of the entire scene and a central fixation; a smaller population later during search results in averaging between groups of objects and off-object fixations.     

Does oculomotor inhibition of return influence fixation probability during scene search?

Oculomotor inhibition of return (IOR) is believed to facilitate scene scanning by decreasing the probability that gaze will return to a previously fixated location. This "foraging" hypothesis was tested during scene search and in response to sudden-onset probes at the immediately previous (one-back) fixation location. The latencies of saccades landing within 1o of the previous fixation location were elevated, consistent with oculomotor IOR. However, there was no decrease in the likelihood that the previous location would be fixated relative to distance-matched controls or an a priori baseline. Saccades exhibit an overall forward bias, but this is due to a general bias to move in the same direction and for the same distance as the last saccade (saccadic momentum) rather than to a spatially specific tendency to avoid previously fixated locations. We find no evidence that oculomotor IOR has a significant impact on return probability during scene search.     

To see and remember: visually specific information is retained in memory from previously attended objects in natural scenes

What is the nature of the representation formed during the viewing of natural scenes? We tested two competing hypotheses regarding the accumulation of visual information during scene viewing. The first holds that coherent visual representations disintegrate as soon as attention is withdrawn from an object and thus that the visual representation of a scene is exceedingly impoverished. The second holds that visual representations do not necessarily decay upon the withdrawal of attention, but instead can be accumulated in memory from previously attended regions. Target objects in line drawings of natural scenes were changed during a saccadic eye movement away from those objects. Three findings support the second hypothesis. First, changes to the visual form of target objects (token substitution) were successfully detected, as indicated by both explicit and implicit measures, even though the target object was not attended when the change occurred. Second, these detections were often delayed until well after the change. Third, changes to semantically inconsistent target objects were detected better than changes to semantically consistent objects.     

Detection and Identification of Change in Naturalistic Scenes

Research using change detection paradigms has demonstrated that only limited scene information remains available for conscious report following initial inspection of a scene. Previous researchers have found higher change identification rates for deletions of parts of objects in line drawings of scenes than additions. Other researchers, however, have found an asymmetry in the opposite direction for addition/deletion of whole objects in line drawings of scenes. Experiment 1 investigated subjects' accuracy in detecting and identifying changes made to successive views of high quality photographs of naturalistic scenes that involved the addition and deletion of objects, colour changes to objects, and changes to the spatial location of objects. Identification accuracy for deletions from scenes was highest, with lower identification rates for object additions and colour changes, and the lowest rates for identification of location changes. Data further suggested that change identification rates for the presence/absence of objects were a function of the number of identical items present in the scene. Experiment 2 examined this possibility further, and also investigated whether the higher identification rates for deletions found in Experiment 1 were found for changes involving whole objects or parts of objects. Results showed higher identification rates for deletions, but only where a unique object was deleted from a scene. The presence of an identical object in the scene abolished this deletion identification advantage. Results further showed that the deletion/addition asymmetry occurs both when the objects are parts of a larger object and when they are entire objects in the scene.     

Perceptual conditions necessary to induce change blindness

Change blindness is a failure to detect a change in an scene when the change occurs along with some visual disturbances. Disturbances are thought to play a delocalizing role that affects the saliency of the “target” transient signal coming from the change location, which would otherwise capture attention and render the change visible. For instance, it is hypothesized that the appearance of new objects in the “mudsplashes” paradigm generates transient signals that compete with the target object's transient signal for attracting attention. Thus, experiments using the mudsplashes paradigm do not rule out a possible role of object changes in capturing attention. Here, by reversing image contrast polarity, we develop a new paradigm to produce change blindness when a real global transient signal is the only visual event occurring, with no edges added or deleted except in the target object. The results show that transient signals, per se, are able to prevent change detection. However, abrupt transients are not necessary if object change occurs in the zero-contrast phase of a smoothly fading and reappearing image, leaving attention as the only common factor affecting all cases of change blindness.     

Searching in the dark: Cognitive relevance drives attention in real-world scenes

We investigated whether the deployment of attention in scenes is better explained by visual salience or by cognitive relevance. In two experiments, participants searched for target objects in scene photographs. The objects appeared in semantically appropriate locations but were not visually salient within their scenes. Search was fast and efficient, with participants much more likely to look to the targets than to the salient regions. This difference was apparent from the first fixation and held regardless of whether participants were familiar with the visual form of the search targets. In the majority of trials, salient regions were not fixated. The critical effects were observed for all 24 participants across the two experiments. We outline a cognitive relevance framework to account for the control of attention and fixation in scenes.