Semantic guidance of attention within natural scenes

When viewing real-world scenes composed of a myriad of objects, detecting changes can be quite difficult, especially when transients are masked. In general, changes are noticed more quickly and accurately if they occur at the currently (or a recently) attended location. Here, we examine the effects of explicit and implicit semantic cues on the guidance of attention in a change detection task. Participants first attempted to read aloud a briefly presented prime word, then looked for a difference between two alternating versions of a real-world scene. Helpful primes named the object that changed, while misdirecting primes named another (unchanging) object in the picture. Robust effects were found for both explicit and implicit priming conditions, with helpful primes yielding faster change detection times than misdirecting or neutral primes. This demonstrates that observers are able to use higher order semantic information as a cue to guide attention within a natural scene, even when the semantic information is presented outside of explicit awareness.     

Active and passive scene recognition across views.

Recent evidence suggests that scene recognition across views is impaired when an array of objects rotates relative to a stationary observer, but not when the observer moves relative to a stationary display [Simons, D.J., Wang, R.F., 1998. Perceiving real-world viewpoint changes. Psychological Science 9, 315-320]. The experiments in this report examine whether the relatively poorer performance by stationary observers across view changes results from a lack of perceptual information for the rotation or from the lack of active control of the perspective change, both of which are present for viewpoint changes. Three experiments compared performance when observers passively experienced the view change and when they actively caused the change. Even with visual information and active control over the display rotation, change detection performance was still worse for orientation changes than for viewpoint changes. These findings suggest that observers can update a viewer-centered representation of a scene when they move to a different viewing position, but such updating does not occur during display rotations even with visual and motor information for the magnitude of the change. This experimental approach, using arrays of real objects rather than computer displays of isolated individual objects, can shed light on mechanisms that allow accurate recognition despite changes in the observer's position and orientation.     

Discriminating direction of motion trajectories from angular speed and background information

The effects of a background scene on the perception of the trajectory of an approaching object and its relation to changes in angular speed and angular size were examined in five experiments. Observers judged the direction (upward or downward) of two sequentially presented motion trajectories simulating a sphere traveling toward the observer at a constant 3-D speed from a fixed distance. In Experiments 1–4, we examined the effects of changes in angular speed and the presence of a scene background, with changes in angular size based either on the trajectories being discriminated or on an intermediate trajectory. In Experiment 5, we examined the effects of changes in angular speed and scene background, with angular size either constant or consistent with an intermediate 3-D trajectory. Overall, we found that (1) observers were able to judge the direction of object motion trajectories from angular speed changes; (2) observers were more accurate with a 3-D scene background, as compared with a uniform background, suggesting that scene information is important for recovering object motion trajectories; and (3) observers were more accurate in judging motion trajectories based on angular speed when the angular size function was consistent with motion in depth than when the angular size was constant.     

Perceived depth of 3-D objects in 3-D scenes

Effects of information specifying the position of an object in a 3-D scene were investigated in two experiments with twelve observers. To separate the effects of the change in scene position from the changes in the projection that occur with increased distance from the observer, the same projections were produced by simulating (a) a constant object at different scene positions and (b) different objects at the same scene position. The simulated scene consisted of a ground plane, a ceiling plane, and a cylinder on a pole attached to both planes. Motion-parallax scenes were studied in one experiment; texture-gradient scenes were studied in the other. Observers adjusted a line to match the perceived internal depth of the cylinder. Judged depth for objects matched in simulated size decreased as simulated distance from the observer increased. Judged depth decreased at a faster rate for the same projections shown at a constant scene position. Adding object-centered depth information (object rotation) increased judged depth for the motion-parallax displays. These results demonstrate that the judged internal depth of an object is reduced by the change in projection that occurs with increased distance, but this effect is diminished if information for change in scene position accompanies the change in projection.     

How Are Different Properties of a Scene Encoded in Visual Memory?

Recent studies in scene perception suggest that much of what observers believe they see is not retained in visual memory. Depending on the roles they play in organizing the perception of a scene, different visual properties may require different amounts of attention to be incorporated into a mental representation of the scene. The goal of this study was to compare how three visual properties of scenes, colour, object position, and object presence, are encoded in visual memory. We used a variation on the change detection “flicker” task and measured the time to detect scene changes when: (1) a cue was provided regarding the type of change; and, (2) no cue was provided. We hypothesized that cueing would enhance the processing of visual properties that require more attention to be encoded into scene representations, whereas cueing would not have an effect for properties that are readily or automatically encoded in visual memory. In Experiment 1, we found that there was a cueing advantage for colour changes, but not for position or presence changes. In Experiment 2, we found the same cueing effect regardless of whether the colour change altered the configuration of the scene or not. These results are consistent with the idea that properties that typically help determine the configuration of the scene, for example, position and presence, are better encoded in scene representations than are surface properties such as colour.     

Attentional coding of categorical relations in scene perception: evidence from the flicker paradigm

The purpose of the present investigation was to determine whether the positions of objects in a scene are coded relative to one another categorically (i.e., above, below, or side of; Experiment 1) and to determine whether spatial position in scene perception is coded preattentively or only under focused attention (Experiment 2). In Experiment 1, participants viewed alternating versions of a scene in which one of the objects in the scene changed its categorical relationship to the closest object in the scene, changed only its metric relationship to the closest object in a scene, or appeared and disappeared. Participants were faster at detecting changes that disrupted categorical relations than at detecting changes that disrupted only metric relations. In Experiment 2, this categorical advantage still occurred even when participants were cued to the location of the change. These results suggest that categorical spatial relations are being coded in scene perception and that attention is required in order to encode spatial relations.     

The role of attentional breadth in perceptual change detection

Previous research has shown that changes to scenes are often surprisingly hard to detect. The research reported here investigated the relationship between individual differences in attention and change detection. We did this by assessing participantś breadth of attention in a functional field of view task (FFOV) and relating this measure to the speed with which individuals detected changes in scenes. We also examined how the salience, meaningfulness, and eccentricity of the scene changes affected perceptual change performance. In order to broaden the range of individual differences in attentional breadth, both young and old adults participated in the study. A strong negative relationship was obtained between attentional breadth and the latency with which perceptual changes were detected; observers with broader attentional windows detected changes faster. Salience and eccentricity had large effects on change detection, but meaning aided the performance of young adults only and only when changes also had low salience.     

Perceiving Real-World Viewpoint Changes

Retinal images vary as observers move through the environment, but observers seem to have little difficulty recognizing objects and scenes across changes in view. Although real-world view changes can be produced both by object rotations (orientation changes) and by observer movements (viewpoint changes), research on recognition across views has relied exclusively on display rotations. However, research on spatial reasoning suggests a possible dissociation between orientation and viewpoint. Here we demonstrate that scene recognition in the real world depends on more than the retinal projection of the visible array; viewpoint changes have little effect on detection of layout changes, but equivalent orientation changes disrupt performance significantly. Findings from our three experiments suggest that scene recognition across view changes relies on a mechanism that updates a viewer-centered representation during observer movements, a mechanism not available for orientation changes. These results link findings from spatial tasks to work on object and scene recognition and highlight the importance of considering the mechanisms underlying recognition in real environments.     

Failure to detect changes to attended objects in motion pictures

Our intuition that we richly represent the visual details of our environment is illusory. When viewing a scene, we seem to use detailed representations of object properties and interobject relations to achieve a sense of continuity across views. Yet, several recent studies show that human observers fail to detect changes to objects and object properties when localized retinal information signaling a change is masked or eliminated (e.g., by eye movements). However, these studies changed arbitrarily chosen objects which may have been outside the focus of attention. We draw on previous research showing the importance of spatiotemporal information for tracking objects by creating short motion pictures in which objects in both arbitrary locations and the very center of attention were changed. Adult observers failed to notice changes in both cases, even when the sole actor in a scene transformed into another person across an instantaneous change in camera angle (or “cut”).     

Efficient summary statistical representation when change localization fails

People are sensitive to the summary statistics of the visual world (e.g., average orientation/speed/facial expression). We readily derive this information from complex scenes, often without explicit awareness. Given the fundamental and ubiquitous nature of summary statistical representation, we tested whether this kind of information is subject to the attentional constraints imposed by change blindness. We show that information regarding the summary statistics of a scene is available despite limited conscious access. In a novel experiment, we found that while observers can suffer from change blindness (i.e., not localize where change occurred between two views of the same scene), observers could nevertheless accurately report changes in the summary statistics (or “gist”) about the very same scene. In the experiment, observers saw two successively presented sets of 16 faces that varied in expression. Four of the faces in the first set changed from one emotional extreme (e.g., happy) to another (e.g., sad) in the second set. Observers performed poorly when asked to locate any of the faces that changed (change blindness). However, when asked about the ensemble (which set was happier, on average), observer performance remained high. Observers were sensitive to the average expression even when they failed to localize any specific object change. That is, even when observers could not locate the very faces driving the change in average expression between the two sets, they nonetheless derived a precise ensemble representation. Thus, the visual system may be optimized to process summary statistics in an efficient manner, allowing it to operate despite minimal conscious access to the information presented.     

Visual orientation in a mirror world tilted 90 degrees

Previously, we showed that many supine observers in a furnished room tilted 90 degrees perceive themselves and the room as upright. We called this the "levitation illusion" because the arms feel weightless when held out from the body. We now report that a familiar scene viewed by supine observers through a mirror at 45 degrees appears vertical when, optically, it is horizontal and above the head. However, the body feels pitched upright only partially. This visual-righting effect, like the levitation illusion, is due to the polarity axis of the scene being accepted as vertical even in the presence of conflicting information from the gravity sense organs. In experiment 1 we tested the potency of objects containing either intrinsic polarity (due to familiar tops and bottoms) or extrinsic polarity (due to support relationships) to generate a visual-righting illusion. To almost all observers, a blank surface seen in the mirror appeared like a ceiling. A scene containing an object with intrinsic polarity, such as a chair or person, seen in the mirror appeared vertical to almost all observers. A scene containing a pair of objects with only extrinsic polarity, such as a ball on a shelf (but not a ball under a shelf), also appeared vertical to most observers. In experiment 2 we found that a polarised scene was more likely to produce a visual-righting illusion when it was arranged as a view through a window rather than as a picture inside a room.     

Detection and Identification of Change in Naturalistic Scenes

Research using change detection paradigms has demonstrated that only limited scene information remains available for conscious report following initial inspection of a scene. Previous researchers have found higher change identification rates for deletions of parts of objects in line drawings of scenes than additions. Other researchers, however, have found an asymmetry in the opposite direction for addition/deletion of whole objects in line drawings of scenes. Experiment 1 investigated subjects' accuracy in detecting and identifying changes made to successive views of high quality photographs of naturalistic scenes that involved the addition and deletion of objects, colour changes to objects, and changes to the spatial location of objects. Identification accuracy for deletions from scenes was highest, with lower identification rates for object additions and colour changes, and the lowest rates for identification of location changes. Data further suggested that change identification rates for the presence/absence of objects were a function of the number of identical items present in the scene. Experiment 2 examined this possibility further, and also investigated whether the higher identification rates for deletions found in Experiment 1 were found for changes involving whole objects or parts of objects. Results showed higher identification rates for deletions, but only where a unique object was deleted from a scene. The presence of an identical object in the scene abolished this deletion identification advantage. Results further showed that the deletion/addition asymmetry occurs both when the objects are parts of a larger object and when they are entire objects in the scene.     

Selective attention during scene perception: Evidence from negative priming

In two experiments, we examined the role of semantic scene content in guiding attention during scene viewing. In each experiment, performance on a lexical decision task was measured following the brief presentation of a scene. The lexical decision stimulus named an object that was either present or not present in the scene. The results of Experiment 1 revealed no priming from inconsistent objects (whose identities conflicted with the scene in which they appeared), but negative priming from consistent objects. The results of Experiment 2 indicated that negative priming from consistent objects occurs only when inconsistent objects are present in the scenes. Together, the results suggest that observers are likely to attend to inconsistent objects, and that representations of consistent objects are suppressed in the presence of an inconsistent object. Furthermore, the data suggest that inconsistent objects draw attention because they are relatively difficult to identify in an inappropriate context.     

Iconic memory for natural scenes: Evidence using a modified change-detection procedure

ABSTRACT

Change blindness for the contents of natural scenes suggests that only items that are attended while the scene is still visible are stored, leading some to characterize our visual experiences as sparse. Experiments on iconic memory for arrays of discrete symbols or objects, however, indicate observers have access to more visual information for at least several hundred milliseconds at offset of a display. In the experiment presented here, we demonstrate an iconic memory for complex natural or real-world scenes. Using a modified change detection task in which to-be changed objects are cued at offset of the scene, we show that more information from a natural scene is briefly stored than change blindness predicts and more than is contained in visual short-term memory. In our experiment, a cue appearing 0, 300, or 1000?msec after offset of the pre-change scene or at onset of the second scene presentation (a Post Cue) directed attention to the location of a possible change. Compared to a no-cue condition, subjects were significantly better at detecting changes and identifying what changed in the cue condition, with the cue having a diminishing effect as a function of time and no effect when its onset coincided with that of the second scene presentation. The results suggest that an iconic memory of a natural scene exists for at least 1000?msec after scene offset, from which subjects can access the identity of items in the pre-change scene. This implies that change blindness underestimates the amount of information available to the visual system from a brief glance of a natural scene.     

Oscillatory motion induces change blindness

Change blindness is the relative inability of normally sighted observers to detect large changes in scenes when the low-level signals associated with those changes are either masked or of extremely low magnitude. Change detection can be inhibited by saccadic eye movements, artificial saccades or blinks, and 'mud splashes'. We now show that change detection is also inhibited by whole image motion in the form of sinusoidal oscillations. The degree of disruption depends upon the frequency of oscillation, which at 3 Hz is equivalent to that produced by artificial blinks. Image motion causes the retinal image to be blurred and this is known to affect object recognition. However, our results are inconsistent with good change detection followed by a delay due to poor recognition of the changing object. Oscillatory motion can induce eye movements that potentially mask or inhibit the low-level signals related to changes in the scene, but we show that eye movements promote rather than inhibit change detection when the image is moving.     

Minimalist surface-colour matching

Some theories of surface-colour perception assume that observers estimate the illuminant on a scene so that its effects can be discounted. A critical test of this interpretation of colour constancy is whether surface-colour matching is worse when the number of surfaces in a scene is so small that any illuminant estimate is unreliable. In the experiment reported here, observers made asymmetric colour matches between pairs of simultaneously presented Mondrian-like patterns under different daylights. The patterns had either 49 surfaces or a minimal 2 surfaces. No significant effect of number was found, suggesting that illuminant estimates are unnecessary for surface-colour matching.     

The effects of distributed and focused attention on rapid scene categorization

It has been argued that distributed attention facilitates the rapid extraction of summary statistics that underpins rapid scene categorization. We directly examined this hypothesis by investigating whether distributed or focused attention is more compatible with the extraction of both summary statistics (Experiment 1) and semantic scene information (Experiments 2–4). Experiment 1 replicated Chong and Treisman’s (2005a) result that mean circle size judgments are more compatible with a distributed attention task than a focused attention task. Experiment 2 investigated whether this finding extends to simple scene categorization by replacing the averaging task with an animal detection task. Consistent with Experiment 1, the ability to detect the presence of an animal was more compatible with a distributed attention task than a focused attention task. Experiments 3 and 4 addressed whether distributed attention influences scene categorization tasks. When observers were asked to classify scenes based on their basic level (e.g., beach or forest; Experiment 3), there was no statistically significant difference between focused and distributed attention task conditions; however, superordinate level categorization (e.g., natural or manmade; Experiment 4) was faster when combined with a task requiring distributed attention compared to a task requiring focused attention.     

The role of context on boundary extension

Boundary extension (BE) is a memory error in which observers remember more of a scene than they actually viewed. This error reflects one’s prediction that a scene naturally continues and is driven by scene schema and contextual knowledge. In two separate experiments we investigated the necessity of context and scene schema in BE. In Experiment 1, observers viewed scenes that either contained semantically consistent or inconsistent objects as well as objects on white backgrounds. In both types of scenes and in the no-background condition there was a BE effect; critically, semantic inconsistency in scenes reduced the magnitude of BE. In Experiment 2 when we used abstract shapes instead of meaningful objects, there was no BE effect. We suggest that although scene schema is necessary to elicit BE, contextual consistency is not required.     

Immediate correction and adaptation based on viewing a prismatically displaced scene

It was shown that when observers view a scene of a room through displacing prisms there is an immediate correction of the prismatic distortion. Objects appear to lie in a direction closer to their true direction than to that produced by the refraction of the prisms. It was also shown that a brief period of exposure to the prismatically viewed scene, without movement or sight of the body, results in substantial adaptation to the displacement.     

Immediate correction and adaptation based on viewing a prismatically displaced scene

It was shown that when observers view a scene of a room through displacing prisms there is an immediate correction of the prismatic distortion. Objects appear to lie in a direction closer to their true direction than to that produced by the refraction of the prisms. It was also shown that a brief period of exposure to the prismatically viewed scene, without movement or sight of the body, results in substantial adaptation to the displacement.