Incidental visual memory for objects in scenes

Three experiments were conducted to investigate the existence of incidentally acquired, long-term, detailed visual memory for objects embedded in previously viewed scenes. Participants performed intentional memorization and incidental visual search learning tasks while viewing photographs of real-world scenes. A visual memory test for previously viewed objects from these scenes then followed. Participants were not aware that they would be tested on the scenes following incidental learning in the visual search task. In two types of memory tests for visually specific object information (token discrimination and mirror-image discrimination), performance following both the memorization and visual search conditions was reliably above chance. These results indicate that recent demonstrations of good visual memory during scene viewing are not due to intentional scene memorization. Instead, long-term visual representations are incidentally generated as a natural product of scene perception.     

Object-position binding in visual memory for natural scenes and object arrays

Nine experiments examined the means by which visual memory for individual objects is structured into a larger representation of a scene. Participants viewed images of natural scenes or object arrays in a change detection task requiring memory for the visual form of a single target object. In the test image, 2 properties of the stimulus were independently manipulated: the position of the target object and the spatial properties of the larger scene or array context. Memory performance was higher when the target object position remained the same from study to test. This same-position advantage was reduced or eliminated following contextual changes that disrupted the relative spatial relationships among contextual objects (context deletion, scrambling, and binding change) but was preserved following contextual change that did not disrupt relative spatial relationships (translation). Thus, episodic scene representations are formed through the binding of objects to scene locations, and object position is defined relative to a larger spatial representation coding the relative locations of contextual objects.     

Object–scene inconsistencies do not capture gaze: evidence from the flash-preview moving-window paradigm

In the present study, we investigated the influence of object-scene relationships on eye movement control during scene viewing. We specifically tested whether an object that is inconsistent with its scene context is able to capture gaze from the visual periphery. In four experiments, we presented rendered images of naturalistic scenes and compared baseline consistent objects with semantically, syntactically, or both semantically and syntactically inconsistent objects within those scenes. To disentangle the effects of extrafoveal and foveal object-scene processing on eye movement control, we used the flash-preview moving-window paradigm: A short scene preview was followed by an object search or free viewing of the scene, during which visual input was available only via a small gaze-contingent window. This method maximized extrafoveal processing during the preview but limited scene analysis to near-foveal regions during later stages of scene viewing. Across all experiments, there was no indication of an attraction of gaze toward object-scene inconsistencies. Rather than capturing gaze, the semantic inconsistency of an object weakened contextual guidance, resulting in impeded search performance and inefficient eye movement control. We conclude that inconsistent objects do not capture gaze from an initial glimpse of a scene.     

Influence of semantic consistency and perceptual features on visual attention during scene viewing in toddlers

Conceptual representations of everyday scenes are built in interaction with visual environment and these representations guide our visual attention. Perceptual features and object-scene semantic consistency have been found to attract our attention during scene exploration. The present study examined how visual attention in 24-month-old toddlers is attracted by semantic violations and how perceptual features (i. e. saliency, centre distance, clutter and object size) and linguistic properties (i. e. object label frequency and label length) affect gaze distribution. We compared eye movements of 24-month-old toddlers and adults while exploring everyday scenes which either contained an inconsistent (e.g., soap on a breakfast table) or consistent (e.g., soap in a bathroom) object. Perceptual features such as saliency, centre distance and clutter of the scene affected looking times in the toddler group during the whole viewing time whereas looking times in adults were affected only by centre distance during the early viewing time. Adults looked longer to inconsistent than consistent objects either if the objects had a high or a low saliency. In contrast, toddlers presented semantic consistency effect only when objects were highly salient. Additionally, toddlers with lower vocabulary skills looked longer to inconsistent objects while toddlers with higher vocabulary skills look equally long to both consistent and inconsistent objects. Our results indicate that 24-month-old children use scene context to guide visual attention when exploring the visual environment. However, perceptual features have a stronger influence in eye movement guidance in toddlers than in adults. Our results also indicate that language skills influence cognitive but not perceptual guidance of eye movements during scene perception in toddlers.     

Semantically incongruent objects attract eye gaze when viewing scenes for change

Past research has shown that change detection performance is often more efficient for target objects that are semantically incongruent with a surrounding scene context than for target objects that are semantically congruent with the scene context. One account of these findings is that attention is attracted to objects for which the identity of the object conflicts with the meaning of the scene, perhaps as a violation of expectancies created by earlier recruitment of scene gist information. An alternative account of the performance benefit for incongruent objects is that attention is more apt to linger on incongruent objects, as perhaps identifying these objects is more difficult due to conflicting information from the scene context. In the current experiment, we present natural scenes in a change detection task while monitoring eye movements. We find that eye gaze is attracted to these objects relatively early during scene processing.     

The effect of spatial and nonspatial contextual information on visual object memory

Recent research has found visual object memory can be stored as part of a larger scene representation rather than independently of scene context. The present study examined how spatial and nonspatial contextual information modulate visual object memory. Two experiments tested participants’ visual memory by using a change detection task in which a target object's orientation was either the same as it appeared during initial viewing or changed. In addition, we examined the effect of spatial and nonspatial contextual manipulations on change detection performance. The results revealed that visual object representations can be maintained reliably after viewing arrays of objects. Moreover, change detection performance was significantly higher when either spatial or nonspatial contextual information remained the same in the test image. We concluded that while processing complex visual stimuli such as object arrays, visual object memory can be stored as part of a comprehensive scene representation, and both spatial and nonspatial contextual changes modulate visual memory retrieval and comparison.     

The role of attention in the contextual enhancement of visual memory for natural scenes

The role of attention in memory for objects in natural scenes was investigated using a visual memory. In Experiment 1, participants were asked to memorize six cued (to be attended) objects in a natural scene, and were subsequently tested on one of the cued objects. Four types of test scene images were created by jumbling different sections in scene's background: Attended sections changed, unattended sections changed, both sections changed, and both unchanged. In Experiment 2, the procedure was the same as that of the jumble condition except that scenes to be memorized were also jumbled. Results showed that jumbling of attended sections reduced memory performance, whereas jumbling unattended sections did not, irrespective of the regularity of scene to be memorized. This finding suggests that attention plays an important role in mental construction of a natural scene representation, and leads to enhanced visual memory.     

The relationship between online visual representation of a scene and long-term scene memory

In 3 experiments the author investigated the relationship between the online visual representation of natural scenes and long-term visual memory. In a change detection task, a target object either changed or remained the same from an initial image of a natural scene to a test image. Two types of changes were possible: rotation in depth, or replacement by another object from the same basic-level category. Change detection during online scene viewing was compared with change detection after delay of 1 trial (Experiments 2A and 2B) until the end of the study session (Experiment 1) or 24 hr (Experiment 3). There was little or no decline in change detection performance from online viewing to a delay of 1 trial or delay until the end of the session, and change detection remained well above chance after 24 hr. These results demonstrate that long-term memory for visual detail in a scene is robust.     

Failures of retrieval and comparison constrain change detection in natural scenes

In a change detection paradigm, a target object in a natural scene either rotated in depth, was replaced by another object token, or remained the same. Change detection performance was reliably higher when a target postcue allowed participants to restrict retrieval and comparison processes to the target object (Experiment 1). Change detection performance remained excellent when the target object was not attended at change (Experiment 2) and when a concurrent verbal working memory load minimized the possibility of verbal encoding (Experiment 3). Together, these data demonstrate that visual representations accumulate in memory from attended objects as the eyes and attention are oriented within a scene and that change blindness derives, at least in part, from retrieval and comparison failure.     

Object detection in natural scenes: Independent effects of spatial and category-based attention

Humans are remarkably efficient in detecting highly familiar object categories in natural scenes, with evidence suggesting that such object detection can be performed in the (near) absence of attention. Here we systematically explored the influences of both spatial attention and category-based attention on the accuracy of object detection in natural scenes. Manipulating both types of attention additionally allowed for addressing how these factors interact: whether the requirement for spatial attention depends on the extent to which observers are prepared to detect a specific object category—that is, on category-based attention. The results showed that the detection of targets from one category (animals or vehicles) was better than the detection of targets from two categories (animals and vehicles), demonstrating the beneficial effect of category-based attention. This effect did not depend on the semantic congruency of the target object and the background scene, indicating that observers attended to visual features diagnostic of the foreground target objects from the cued category. Importantly, in three experiments the detection of objects in scenes presented in the periphery was significantly impaired when observers simultaneously performed an attentionally demanding task at fixation, showing that spatial attention affects natural scene perception. In all experiments, the effects of category-based attention and spatial attention on object detection performance were additive rather than interactive. Finally, neither spatial nor category-based attention influenced metacognitive ability for object detection performance. These findings demonstrate that efficient object detection in natural scenes is independently facilitated by spatial and category-based attention.     

Constructing visual representations of natural scenes: the roles of short- and long-term visual memory

A "follow-the-dot" method was used to investigate the visual memory systems supporting accumulation of object information in natural scenes. Participants fixated a series of objects in each scene, following a dot cue from object to object. Memory for the visual form of a target object was then tested. Object memory was consistently superior for the two most recently fixated objects, a recency advantage indicating a visual short-term memory component to scene representation. In addition, objects examined earlier were remembered at rates well above chance, with no evidence of further forgetting when 10 objects intervened between target examination and test and only modest forgetting with 402 intervening objects. This robust prerecency performance indicates a visual long-term memory component to scene representation.     

Semantic consistency versus perceptual salience in visual scenes: Findings from change detection

In a one-shot change detection task, we investigated the relationship between semantic properties (high consistency, i.e., diagnosticity, versus inconsistency with regard to gist) and perceptual properties (high versus low salience) of objects in guiding attention in visual scenes and in constructing scene representations. To produce the change an object was added or deleted in either the right or left half of coloured drawings of daily-life events. Diagnostic object deletions were more accurately detected than inconsistent ones, indicating rapid inclusion into early scene representation for the most predictable objects. Detection was faster and more accurate for high salience than for low salience changes. An advantage was found for diagnostic object changes in the high salience condition, although it was limited to additions when considering response speed. For inconsistent objects of high salience, deletions were detected faster than additions. These findings may indicate that objects are primarily selected on a perceptual basis with subsequent and supplementary effect of semantic consistency, in the sense of facilitation due to object diagnosticity or lengthening of processing time due to inconsistency.     

Failure to detect changes to people during a real-world interaction

Recent research on change detection has documented surprising failures to detect visual changes occurring between views of a scene, suggesting the possibility that visual representations contain few details. Although these studies convincingly demonstrate change blindness for objects in still images and motion pictures, they may not adequately assess the capacity to represent objects in the real world. Here we examine and reject the possibility that change blindness in previous studies resulted from passive viewing of 2-D displays. In one experiment, an experimenter initiated a conversation with a pedestrian, and during the interaction, he was surreptitiously replaced by a different experimenter. Only half of the pedestrians detected the change. Furthermore, successful detection depended on social group membership; pedestrians from the same social group as the experimenters detected the change but those from a different social group did not. A second experiment further examined the importance of this effect of social group. Provided that the meaning of the scene is unchanged, changes to attended objects can escape detection even when they occur during a natural, real-world interaction. The discussion provides a set of guidelines and suggestions for future research on change blindness.     

How does information from low and high spatial frequencies interact during scene categorization?

Current models of visual perception suggest that, during scene categorization, low spatial frequencies (LSF) are rapidly processed and activate plausible interpretations of visual input. This coarse analysis would be used to guide subsequent processing of high spatial frequencies (HSF). The present study aimed to further examine how information from LSF and HSF interact and influence each other during scene categorization. In a first experimental session, participants had to categorize LSF and HSF filtered scenes belonging to two different semantic categories (artificial vs. natural). In a second experimental session, we used hybrid scenes as stimuli made by combining LSF and HSF from two different scenes which were semantically similar or dissimilar. Half of the participants categorized LSF scenes in hybrids, and the other half categorized HSF scenes in hybrids. Stimuli were presented for 30 or 100?ms. Session 1 results showed better performance for LSF than HSF scene categorization. Session 2 scene categorization was faster when participants attended and categorized LSF than HSF scene in hybrids. The semantic interference of a semantically dissimilar HSF scene on LSF scene categorization was greater than the semantic interference of a semantically dissimilar LSF scene on HSF scene categorization, irrespective of exposure duration. These results suggest a LSF advantage for scene categorization, and highlight the prominent role of HSF information when there is uncertainty about the visual stimulus, in order to disentangle between alternative interpretations.     

Gaze cues influence the allocation of attention in natural scene viewing

In two experiments we examined whether the allocation of attention in natural scene viewing is influenced by the gaze cues (head and eye direction) of an individual appearing in the scene. Each experiment employed a variant of the flicker paradigm in which alternating versions of a scene and a modified version of that scene were separated by a brief blank field. In Experiment 1, participants were able to detect the change made to the scene sooner when an individual appearing in the scene was gazing at the changing object than when the individual was absent, gazing straight ahead, or gazing at a nonchanging object. In addition, participants' ability to detect change deteriorated linearly as the changing object was located progressively further from the line of regard of the gazer. Experiment 2 replicated this change detection advantage of gaze-cued objects in a modified procedure using more critical scenes, a forced-choice change/no-change decision, and accuracy as the dependent variable. These findings establish that in the perception of static natural scenes and in a change detection task, attention is preferentially allocated to objects that are the target of another's social attention.     

Gaze cues influence the allocation of attention in natural scene viewing

In two experiments we examined whether the allocation of attention in natural scene viewing is influenced by the gaze cues (head and eye direction) of an individual appearing in the scene. Each experiment employed a variant of the flicker paradigm in which alternating versions of a scene and a modified version of that scene were separated by a brief blank field. In Experiment 1, participants were able to detect the change made to the scene sooner when an individual appearing in the scene was gazing at the changing object than when the individual was absent, gazing straight ahead, or gazing at a nonchanging object. In addition, participants' ability to detect change deteriorated linearly as the changing object was located progressively further from the line of regard of the gazer. Experiment 2 replicated this change detection advantage of gaze-cued objects in a modified procedure using more critical scenes, a forced-choice change/no-change decision, and accuracy as the dependent variable. These findings establish that in the perception of static natural scenes and in a change detection task, attention is preferentially allocated to objects that are the target of another's social attention.     

Selective attention during scene perception: Evidence from negative priming

In two experiments, we examined the role of semantic scene content in guiding attention during scene viewing. In each experiment, performance on a lexical decision task was measured following the brief presentation of a scene. The lexical decision stimulus named an object that was either present or not present in the scene. The results of Experiment 1 revealed no priming from inconsistent objects (whose identities conflicted with the scene in which they appeared), but negative priming from consistent objects. The results of Experiment 2 indicated that negative priming from consistent objects occurs only when inconsistent objects are present in the scenes. Together, the results suggest that observers are likely to attend to inconsistent objects, and that representations of consistent objects are suppressed in the presence of an inconsistent object. Furthermore, the data suggest that inconsistent objects draw attention because they are relatively difficult to identify in an inappropriate context.     

Look what I have felt: unidentified haptic line drawings are identified after sketching

This study examined the influences of semantic characteristics of objects in real-world scenes on allocation of attention as reflected in eye movement measures. Stimuli consisted of full-color photographic scenes, and within each scene, the semantic salience of two target objects was manipulated while the objects' perceptual salience was kept constant. One of the target objects was either inconsistent or consistent with the scene category. In addition, the second target object was either smoking-related or neutral. Two groups of college students, namely current cigarette smokers (N=18) and non-smokers (N=19), viewed each scene for 10s while their eye movements were recorded. While both groups showed preferential allocation of attention to inconsistent objects, smokers also selectively attended to smoking-related objects. Theoretical implications of the results are discussed.     

Scene and position specificity in visual memory for objects

This study investigated whether and how visual representations of individual objects are bound in memory to scene context. Participants viewed a series of naturalistic scenes, and memory for the visual form of a target object in each scene was examined in a 2-alternative forced-choice test, with the distractor object either a different object token or the target object rotated in depth. In Experiments 1 and 2, object memory performance was more accurate when the test object alternatives were displayed within the original scene than when they were displayed in isolation, demonstrating object-to-scene binding. Experiment 3 tested the hypothesis that episodic scene representations are formed through the binding of object representations to scene locations. Consistent with this hypothesis, memory performance was more accurate when the test alternatives were displayed within the scene at the same position originally occupied by the target than when they were displayed at a different position.     

Eye movements and visual memory: detecting changes to saccade targets in scenes

Saccade-contingent change detection provides a powerful tool for investigating scene representation and scene memory. In the present study, critical objects presented within color images of naturalistic scenes were changed during a saccade toward or away from the target. During the saccade,the critical object was changed to another object type, to a visually different token of the same object type, or was deleted from the scene. There were three main results. First, the deletion of a saccade target was special: Detection performance for saccade target deletions was very good, and this level of performance did not decline with the amplitude of the saccade. In contrast, detection of type and token changes at the saccade target, and of all changes including deletions at a location that had just been fixated but was not the saccade target, decreased as the amplitude of the saccade increased. Second, detection performance for type and token changes, both when the changing object was the target of the saccade and when the object had just been fixated but was not the saccade target, was well above chance. Third, mean gaze durations were reliably elevated for those trials in which the change was not overtly detected. The results suggest that the presence of the saccade target plays a special role in trassaccadic integration, and together with other recent findings, suggest more generally that a relatively rich scene representation is retained across saccades and stored in visual memory.