Object–scene inconsistencies do not capture gaze: evidence from the flash-preview moving-window paradigm

In the present study, we investigated the influence of object-scene relationships on eye movement control during scene viewing. We specifically tested whether an object that is inconsistent with its scene context is able to capture gaze from the visual periphery. In four experiments, we presented rendered images of naturalistic scenes and compared baseline consistent objects with semantically, syntactically, or both semantically and syntactically inconsistent objects within those scenes. To disentangle the effects of extrafoveal and foveal object-scene processing on eye movement control, we used the flash-preview moving-window paradigm: A short scene preview was followed by an object search or free viewing of the scene, during which visual input was available only via a small gaze-contingent window. This method maximized extrafoveal processing during the preview but limited scene analysis to near-foveal regions during later stages of scene viewing. Across all experiments, there was no indication of an attraction of gaze toward object-scene inconsistencies. Rather than capturing gaze, the semantic inconsistency of an object weakened contextual guidance, resulting in impeded search performance and inefficient eye movement control. We conclude that inconsistent objects do not capture gaze from an initial glimpse of a scene.     

Selective attention during scene perception: Evidence from negative priming

In two experiments, we examined the role of semantic scene content in guiding attention during scene viewing. In each experiment, performance on a lexical decision task was measured following the brief presentation of a scene. The lexical decision stimulus named an object that was either present or not present in the scene. The results of Experiment 1 revealed no priming from inconsistent objects (whose identities conflicted with the scene in which they appeared), but negative priming from consistent objects. The results of Experiment 2 indicated that negative priming from consistent objects occurs only when inconsistent objects are present in the scenes. Together, the results suggest that observers are likely to attend to inconsistent objects, and that representations of consistent objects are suppressed in the presence of an inconsistent object. Furthermore, the data suggest that inconsistent objects draw attention because they are relatively difficult to identify in an inappropriate context.     

Scene consistency in object and background perception

Does knowledge about which objects and settings tend to co-occur affect how people interpret an image? The effects of consistency on perception were investigated using manipulated photographs containing a foreground object that was either semantically consistent or inconsistent with its setting. In four experiments, participants reported the foreground object, the setting, or both after seeing each picture for 80 ms followed by a mask. In Experiment 1, objects were identified more accurately in a consistent than an inconsistent setting. In Experiment 2, backgrounds were identified more accurately when they contained a consistent rather than an inconsistent foreground object. In Experiment 3, objects were presented without backgrounds and backgrounds without objects; comparison with the other experiments indicated that objects were identified better in isolation than when presented with a background, but there was no difference in accuracy for backgrounds whether they appeared with a foreground object or not. Finally, in Experiment 4, consistency effects remained when both objects and backgrounds were reported. Semantic consistency information is available when a scene is glimpsed briefly and affects both object and background perception. Objects and their settings are processed interactively and not in isolation.     

Influence of semantic consistency and perceptual features on visual attention during scene viewing in toddlers

Conceptual representations of everyday scenes are built in interaction with visual environment and these representations guide our visual attention. Perceptual features and object-scene semantic consistency have been found to attract our attention during scene exploration. The present study examined how visual attention in 24-month-old toddlers is attracted by semantic violations and how perceptual features (i. e. saliency, centre distance, clutter and object size) and linguistic properties (i. e. object label frequency and label length) affect gaze distribution. We compared eye movements of 24-month-old toddlers and adults while exploring everyday scenes which either contained an inconsistent (e.g., soap on a breakfast table) or consistent (e.g., soap in a bathroom) object. Perceptual features such as saliency, centre distance and clutter of the scene affected looking times in the toddler group during the whole viewing time whereas looking times in adults were affected only by centre distance during the early viewing time. Adults looked longer to inconsistent than consistent objects either if the objects had a high or a low saliency. In contrast, toddlers presented semantic consistency effect only when objects were highly salient. Additionally, toddlers with lower vocabulary skills looked longer to inconsistent objects while toddlers with higher vocabulary skills look equally long to both consistent and inconsistent objects. Our results indicate that 24-month-old children use scene context to guide visual attention when exploring the visual environment. However, perceptual features have a stronger influence in eye movement guidance in toddlers than in adults. Our results also indicate that language skills influence cognitive but not perceptual guidance of eye movements during scene perception in toddlers.     

To see and remember: visually specific information is retained in memory from previously attended objects in natural scenes

What is the nature of the representation formed during the viewing of natural scenes? We tested two competing hypotheses regarding the accumulation of visual information during scene viewing. The first holds that coherent visual representations disintegrate as soon as attention is withdrawn from an object and thus that the visual representation of a scene is exceedingly impoverished. The second holds that visual representations do not necessarily decay upon the withdrawal of attention, but instead can be accumulated in memory from previously attended regions. Target objects in line drawings of natural scenes were changed during a saccadic eye movement away from those objects. Three findings support the second hypothesis. First, changes to the visual form of target objects (token substitution) were successfully detected, as indicated by both explicit and implicit measures, even though the target object was not attended when the change occurred. Second, these detections were often delayed until well after the change. Third, changes to semantically inconsistent target objects were detected better than changes to semantically consistent objects.     

Scene context influences without scene gist: Eye movements guided by spatial associations in visual search

Although the use of semantic information about the world seems ubiquitous in every task we perform, it is not clear whether we rely on a scene’s semantic information to guide attention when searching for something in a specific scene context (e.g., keys in one’s living room). To address this question, we compared contribution of a scene’s semantic information (i.e., scene gist) versus learned spatial associations between objects and context. Using the flash-preview–moving-window paradigm Castelhano and Henderson (Journal of Experimental Psychology: Human Perception and Performance 33:753–763, 2007), participants searched for target objects that were placed in either consistent or inconsistent locations and were semantically consistent or inconsistent with the scene gist. The results showed that learned spatial associations were used to guide search even in inconsistent contexts, providing evidence that scene context can affect search performance without consistent scene gist information. We discuss the results in terms of hierarchical organization of top-down influences of scene context.     

靶刺激变换与情景信息属性对真实情景再认的影响

采用眼动研究范式,通过两个实验考察靶刺激变换与情景信息属性对情景再认的影响。实验一结果显示,靶刺激变换对情景再认、靶刺激所属兴趣区的凝视时间均有显著影响,这表明在情景再认过程中,观察者会有意识地搜索靶刺激,靶刺激具有诊断效应;实验二应用了知觉信息与语义信息重合和分离两种情景材料。结果显示,观察者对语义信息的首次注视次数显著多于知觉信息;观察者对知觉信息和语义信息分离条件下语义信息的首次注视时间显著长于重合条件下。这一结果提示,在情景识别过程中,语义信息具有注意优先性,但其优先性会受到知觉信息启动的干扰。     

Effect of background information on object identification

Previous research using briefly presented displays has indicated that objects in a coherent scene are easier to identify than are objects in incoherent backgrounds. Of interest is whether the identification of the target object depends on the identification of the scene or the identification of other diagnostic objects in the scene. Experiment 1 indicated objects are more difficult to identify when located in an "episodically" inconsistent background even when the same diagnostic objects are present in both inconsistent and consistent backgrounds. Experiment 2 demonstrated that the degree to which noncued (cohort) objects are consistent with the target object has no effect on the object identification task. Experiment 3 showed consistent episodic background information facilitated object identification and inconsistent episodic background information did not interfere relative to "nonsense" backgrounds roughly equated on visual characteristics. Implications for models of scene perception are discussed.     

Testing a conceptual locus

Changes to objects that are inconsistent with the scene in which they appear are detected more accurately than changes to consistent objects. In three experiments, we tested whether this inconsistent object advantage derives from the differential retention of conceptual codes generated from a brief view of a real-world scene in accordance with a conceptual short-term memory (CSTM) hypothesis. A scene was presented for 250 msec, followed by a brief mask and a test scene in which a target object was either changed or not changed. In Experiment 1, changes that altered conceptual content (object deletion) were contrasted with visual changes (left-right orientation changes). In Experiment 2, the duration of the mask was manipulated to vary the amount of time available for conceptual consolidation of the initial scene. In Experiment 3, the type of mask was manipulated: Either a meaningless pattern mask or a meaningful, and thus conceptually disruptive, scene was shown. The inconsistent object advantage was obtained in each experiment, yet in none was it modulated in the direction predicted by the CSTM hypothesis. Instead, the inconsistent object advantage is likely to be caused by contextual influence on memory for visual object representations.     

The role of context on boundary extension

Boundary extension (BE) is a memory error in which observers remember more of a scene than they actually viewed. This error reflects one’s prediction that a scene naturally continues and is driven by scene schema and contextual knowledge. In two separate experiments we investigated the necessity of context and scene schema in BE. In Experiment 1, observers viewed scenes that either contained semantically consistent or inconsistent objects as well as objects on white backgrounds. In both types of scenes and in the no-background condition there was a BE effect; critically, semantic inconsistency in scenes reduced the magnitude of BE. In Experiment 2 when we used abstract shapes instead of meaningful objects, there was no BE effect. We suggest that although scene schema is necessary to elicit BE, contextual consistency is not required.     

The influence of color on the perception of scene gist

In 3 experiments the authors used a new contextual bias paradigm to explore how quickly information is extracted from a scene to activate gist, whether color contributes to this activation, and how color contributes, if it does. Participants were shown a brief presentation of a scene followed by the name of a target object. The target object could be consistent or inconsistent with scene gist but was never actually present in the scene. Scene gist activation was operationalized as the degree to which participants respond "yes" to consistent versus inconsistent objects, reflecting a response bias produced by scene gist. Experiment 1 demonstrated that scene gist is activated after a 42-ms exposure and that the strength of the activation increases with longer presentation durations. Experiments 2 and 3 explored the contribution of color to the activation of scene gist. The results revealed that color has an influence across a wide variety of scenes and is directly associated with scene gist.     

Look what I have felt: unidentified haptic line drawings are identified after sketching

This study examined the influences of semantic characteristics of objects in real-world scenes on allocation of attention as reflected in eye movement measures. Stimuli consisted of full-color photographic scenes, and within each scene, the semantic salience of two target objects was manipulated while the objects' perceptual salience was kept constant. One of the target objects was either inconsistent or consistent with the scene category. In addition, the second target object was either smoking-related or neutral. Two groups of college students, namely current cigarette smokers (N=18) and non-smokers (N=19), viewed each scene for 10s while their eye movements were recorded. While both groups showed preferential allocation of attention to inconsistent objects, smokers also selectively attended to smoking-related objects. Theoretical implications of the results are discussed.     

Semantic consistency versus perceptual salience in visual scenes: Findings from change detection

In a one-shot change detection task, we investigated the relationship between semantic properties (high consistency, i.e., diagnosticity, versus inconsistency with regard to gist) and perceptual properties (high versus low salience) of objects in guiding attention in visual scenes and in constructing scene representations. To produce the change an object was added or deleted in either the right or left half of coloured drawings of daily-life events. Diagnostic object deletions were more accurately detected than inconsistent ones, indicating rapid inclusion into early scene representation for the most predictable objects. Detection was faster and more accurate for high salience than for low salience changes. An advantage was found for diagnostic object changes in the high salience condition, although it was limited to additions when considering response speed. For inconsistent objects of high salience, deletions were detected faster than additions. These findings may indicate that objects are primarily selected on a perceptual basis with subsequent and supplementary effect of semantic consistency, in the sense of facilitation due to object diagnosticity or lengthening of processing time due to inconsistency.     

Stuck on semantics: Processing of irrelevant object-scene inconsistencies modulates ongoing gaze behavior

People have an amazing ability to identify objects and scenes with only a glimpse. How automatic is this scene and object identification? Are scene and object semantics—let alone their semantic congruity—processed to a degree that modulates ongoing gaze behavior even if they are irrelevant to the task at hand? Objects that do not fit the semantics of the scene (e.g., a toothbrush in an office) are typically fixated longer and more often than objects that are congruent with the scene context. In this study, we overlaid a letter T onto photographs of indoor scenes and instructed participants to search for it. Some of these background images contained scene-incongruent objects. Despite their lack of relevance to the search, we found that participants spent more time in total looking at semantically incongruent compared to congruent objects in the same position of the scene. Subsequent tests of explicit and implicit memory showed that participants did not remember many of the inconsistent objects and no more of the consistent objects. We argue that when we view natural environments, scene and object relationships are processed obligatorily, such that irrelevant semantic mismatches between scene and object identity can modulate ongoing eye-movement behavior.     

Eye movements to pictures reveal transient semantic activation during spoken word recognition

Two experiments explore the activation of semantic information during spoken word recognition. Experiment 1 shows that as the name of an object unfolds (e.g., lock), eye movements are drawn to pictorial representations of both the named object and semantically related objects (e.g., key). Experiment 2 shows that objects semantically related to an uttered word's onset competitors become active enough to draw visual attention (e.g., if the uttered word is logs, participants fixate on key because of partial activation of lock), despite that the onset competitor itself is not present in the visual display. Together, these experiments provide detailed information about the activation of semantic information associated with a spoken word and its phonological competitors and demonstrate that transient semantic activation is sufficient to impact visual attention.     

Is naming faces different from naming objects? Semantic interference in a face- and object-naming task

A current debate regarding face and object naming concerns whether they are equally vulnerable to semantic interference. Although some studies have shown similar patterns of interference, others have revealed different effects for faces and objects. In Experiment 1, we compared face naming to object naming when exemplars were presented in a semantically homogeneous context (grouped by their category) or in a semantically heterogeneous context (mixed) across four cycles. The data revealed significant slowing for both face and object naming in the homogeneous context. This semantic interference was explained as being due to lexical competition from the conceptual activation of category members. When focusing on the first cycle, a facilitation effect for objects but not for faces appeared. This result permits us to explain the previously observed discrepancies between face and object naming. Experiment 2 was identical to Experiment 1, with the exception that half of the stimuli were presented as face/object names for reading. Semantic interference was present for both face and object naming, suggesting that faces and objects behave similarly during naming. Interestingly, during reading, semantic interference was observed for face names but not for object names. This pattern is consistent with previous assumptions proposing the activation of a person identity during face name reading.     

Nontarget objects can influence perceptual processes during object recognition

Previous experiments have shown that objects are recognized more readily in a semantically consistent visual context. However, the benefit from context could be explained by response bias, and may not reflect the influence of context on the perceptual processes of recognition. We conducted a six-alternative forced-choice experiment to measure semantic and perceptual errors. A target object appeared briefly, surrounded by four context objects. The target was more accurately identified when the context consisted of objects semantically related to the target. The large number of semantic errors, which increased when the context presentation preceded the target, showed that response bias did account for a proportion of the context effect. Nevertheless, significant facilitation was still present after a bias correction. Recognition of an object can be affected by context not only when it is embedded in a coherent naturalistic scene, but also when it is simply near other related objects. Materials associated with this article may be accessed at www.psychonomic.org/archive.     

Consistency effects between objects in scenes

How does context influence the perception of objects in scenes? Objects appear in a given setting with surrounding objects. Do objects in scenes exert contextual influences on each other? Do these influences interact with background consistency? In three experiments, we investigated the role of object-to-object context on object and scene perception. Objects (Experiments 1 and 3) and backgrounds (Experiment 2) were reported more accurately when the objects and their settings were consistent than when they were inconsistent, regardless of the number of foreground objects. In Experiment 3, related objects (from the same setting) were reported more accurately than were unrelated objects (from different settings), independently of consistency with the background. Consistent with an interactive model of scene processing, both object-to-object context and object-background context affect object perception.     

汉语词汇产生中语义、语音层次之间的交互作用

采用3种方法考察汉语词汇产生中语义激活（词条选择）和音位编码这两个阶段之间的关系。实验一、二分别采用图片命名法和干扰字命名法,发现目标图片（“牛”）的语义相关词（“羊”）的语音信息得到了激活,说明汉语词汇产生中存在多重语音激活,音位编码在词条选择完成之前既已开始。实验三采用语义范畴判断法,考察语音信息的激活是否会向上反馈到词条和语义层次,发现目标图片（“羊”）的同音字（“阳”）不会促进对图片的语义判断。研究表明,多重语音激活与语音激活向词条和语义层次反馈是两个独立的,可以分割的问题。虽然实验一和二的结果符合相互作用理论,实验三的结果符合模块化观点,但就总体而言,本研究更加支持词汇产生中的模块化观点