Young infants' eye movements over "natural" scenes and "experimental" scenes

Eye movements of 30 4-month-olds were tracked as infants viewed animals and vehicles in “natural” scenes and, for comparison, in homogeneous “experimental” scenes. Infants showed equivalent looking time preferences for natural and experimental scenes overall, but fixated natural scenes and objects in natural scenes more than experimental scenes and objects in experimental scenes and shifted fixations between objects and contexts more in natural than in experimental scenes. The findings show how infants treat objects and contexts in natural scenes and suggest that they treat more commonly used experimental scenes differently.     

Seeking the boundary of boundary extension

Boundary extension (BE) is a remarkably consistent visual memory error in which participants remember seeing a more wide-angle image of a scene than was actually viewed (Intraub & Richardson, Journal of Experimental Psychology: Learning, Memory, and Cognition 15:179–187, 1989). Multiple stimulus factors are thought to contribute to the occurrence of BE, including object recognition, conceptual knowledge of scenes, and amodal perception at the view boundaries (Intraub, Wiley Interdisciplinary Reviews: Cognitive Science 3:117–127, 2012). In the present study, we used abstract scenes instead of images of the real world, in order to remove expectations based on semantic associations with objects and the schematic context of the view. Close-angle and wide-angle scenes were created using irregular geometric shapes rated by independent observers as lacking any easily recognizable structure. The abstract objects were tested on either a random-dot or a blank background in order to assess the influence of implied continuation of the image beyond its boundaries. The random-dot background conditions had background occlusion cues either present or absent at the image border, in order to test their influence on BE in the absence of high-level information about the scenes. The results indicate that high-level information about objects and schematic context is unnecessary for BE to occur, and that occlusion information at the image boundary also has little influence on BE. Contrary to previous studies, we also found clear BE for all conditions, despite using scenes depicting undetailed objects on a blank white background. The results highlighted the ubiquitous nature of BE and the adaptability of scene perception processes.     

Searching in the dark: Cognitive relevance drives attention in real-world scenes

We investigated whether the deployment of attention in scenes is better explained by visual salience or by cognitive relevance. In two experiments, participants searched for target objects in scene photographs. The objects appeared in semantically appropriate locations but were not visually salient within their scenes. Search was fast and efficient, with participants much more likely to look to the targets than to the salient regions. This difference was apparent from the first fixation and held regardless of whether participants were familiar with the visual form of the search targets. In the majority of trials, salient regions were not fixated. The critical effects were observed for all 24 participants across the two experiments. We outline a cognitive relevance framework to account for the control of attention and fixation in scenes.     

PRIMING SPATIAL LAYOUT OF SCENES

Abstract— Observers responded to full-color images of scenes by indicating which of two critical objects was closer in the pictorial space. These target images were preceded by prime images of the same scene primes critical objects, or by control primes or different-scene primes. Reaction times were faster following same-scene primes than following the various control and different-scene primes. Same-scene facilitation was obtained with color primes, line-drawing primes, and primes with shifted views. The effect occurred with natural scenes having gist and simple artificial scenes having little or no gist. The results indicate that prime-induced representations influence the perception of spatial layout in pictures.     

Detection and Identification of Change in Naturalistic Scenes

Research using change detection paradigms has demonstrated that only limited scene information remains available for conscious report following initial inspection of a scene. Previous researchers have found higher change identification rates for deletions of parts of objects in line drawings of scenes than additions. Other researchers, however, have found an asymmetry in the opposite direction for addition/deletion of whole objects in line drawings of scenes. Experiment 1 investigated subjects' accuracy in detecting and identifying changes made to successive views of high quality photographs of naturalistic scenes that involved the addition and deletion of objects, colour changes to objects, and changes to the spatial location of objects. Identification accuracy for deletions from scenes was highest, with lower identification rates for object additions and colour changes, and the lowest rates for identification of location changes. Data further suggested that change identification rates for the presence/absence of objects were a function of the number of identical items present in the scene. Experiment 2 examined this possibility further, and also investigated whether the higher identification rates for deletions found in Experiment 1 were found for changes involving whole objects or parts of objects. Results showed higher identification rates for deletions, but only where a unique object was deleted from a scene. The presence of an identical object in the scene abolished this deletion identification advantage. Results further showed that the deletion/addition asymmetry occurs both when the objects are parts of a larger object and when they are entire objects in the scene.     

Revisiting scene primes for object locations

Sanocki and Epstein (1997) provided evidence that an immediate prior experience of a scene, as a prime, can induce representations of its spatial layout, facilitating the subsequent spatial processing of objects in the target scene. In their experiments, observers responded to target scenes by indicating which of two critical objects was closer in the pictorial space. Reaction times to target scenes that were preceded by same-scene primes without the critical objects were faster than reaction times to target scenes that were preceded by different scene or control primes (geometrical figures). By manipulating the nature of the prime and the interval between prime and target, and by cueing the position of the critical objects, we obtain evidence that the facilitating effect of the same-scene primes can also be explained by the sudden appearance of the critical objects in the target scene. In same-scene conditions, the critical objects cause a local onset, whereas in different-scene and control conditions the entire target scene causes a global onset. As a result, the local onset in the same-scene condition produces a shift of attention towards the critical objects, resulting in faster processing of the critical objects.     

I see what you're saying: the integration of complex speech and scenes during language comprehension

The effect of language-driven eye movements in a visual scene with concurrent speech was examined using complex linguistic stimuli and complex scenes. The processing demands were manipulated using speech rate and the temporal distance between mentioned objects. This experiment differs from previous research by using complex photographic scenes, three-sentence utterances and mentioning four target objects. The main finding was that objects that are more slowly mentioned, more evenly placed and isolated in the speech stream are more likely to be fixated after having been mentioned and are fixated faster. Surprisingly, even objects mentioned in the most demanding conditions still show an effect of language-driven eye-movements. This supports research using concurrent speech and visual scenes, and shows that the behavior of matching visual and linguistic information is likely to generalize to language situations of high information load.     

Visual search in scenes involves selective and nonselective pathways

How does one find objects in scenes? For decades, visual search models have been built on experiments in which observers search for targets, presented among distractor items, isolated and randomly arranged on blank backgrounds. Are these models relevant to search in continuous scenes? This article argues that the mechanisms that govern artificial, laboratory search tasks do play a role in visual search in scenes. However, scene-based information is used to guide search in ways that had no place in earlier models. Search in scenes might be best explained by a dual-path model: a 'selective' path in which candidate objects must be individually selected for recognition and a 'nonselective' path in which information can be extracted from global and/or statistical information.     

Look what I have felt: unidentified haptic line drawings are identified after sketching

This study examined the influences of semantic characteristics of objects in real-world scenes on allocation of attention as reflected in eye movement measures. Stimuli consisted of full-color photographic scenes, and within each scene, the semantic salience of two target objects was manipulated while the objects' perceptual salience was kept constant. One of the target objects was either inconsistent or consistent with the scene category. In addition, the second target object was either smoking-related or neutral. Two groups of college students, namely current cigarette smokers (N=18) and non-smokers (N=19), viewed each scene for 10s while their eye movements were recorded. While both groups showed preferential allocation of attention to inconsistent objects, smokers also selectively attended to smoking-related objects. Theoretical implications of the results are discussed.     

Semantic interference from objects both in and out of a scene context

Prior studies have found that, despite the intentions of the participants, objects automatically activate their semantic representations; however, this research examined only objects presented in isolation without a background context. The present set of experiments examined the automaticity issue for objects presented in isolation as well as in scenes. In Experiments 1 and 2, words were categorized more slowly when they were embedded inside incongruent objects (e.g., the word chair in a picture of a duck) than inside neutral nonobjects, suggesting that the meanings of the objects were activated despite participants' intentions. A new interference task was introduced in Experiment 3. When the same objects and words from the first 2 experiments were inserted into scenes in which those objects were probable or improbable, interference occurred from probable pictured objects but not from improbable pictured objects. Implications for theories of automaticity and models of object identification are discussed.     

The role of context on boundary extension

Boundary extension (BE) is a memory error in which observers remember more of a scene than they actually viewed. This error reflects one’s prediction that a scene naturally continues and is driven by scene schema and contextual knowledge. In two separate experiments we investigated the necessity of context and scene schema in BE. In Experiment 1, observers viewed scenes that either contained semantically consistent or inconsistent objects as well as objects on white backgrounds. In both types of scenes and in the no-background condition there was a BE effect; critically, semantic inconsistency in scenes reduced the magnitude of BE. In Experiment 2 when we used abstract shapes instead of meaningful objects, there was no BE effect. We suggest that although scene schema is necessary to elicit BE, contextual consistency is not required.     

Integration without awareness: expanding the limits of unconscious processing

Human conscious awareness is commonly seen as the climax of evolution. However, what function-if any-it serves in human behavior is still debated. One of the leading suggestions is that the cardinal function of conscious awareness is to integrate numerous inputs-including the multitude of features and objects in a complex scene-across different levels of analysis into a unified, coherent, and meaningful perceptual experience. Here we demonstrate, however, that integration of objects with their background scenes can be achieved without awareness of either. We used a binocular rivalry technique known as continuous flash suppression to induce perceptual suppression in a group of human observers. Complex scenes that included incongruent objects escaped perceptual suppression faster than normal scenes did. We conclude that visual awareness is not needed for object-background integration or for processing the likelihood of an object to appear within a given semantic context, but may be needed for dealing with novel situations.     

Does visual expertise improve visual recognition memory?

In general, humans have impressive recognition memory for previously viewed pictures. Many people spend years becoming experts in highly specialized image sets. For example, cytologists are experts at searching micrographs filled with potentially cancerous cells and radiologists are expert at searching mammograms for indications of cancer. Do these experts develop robust visual long-term memory for their domain of expertise? If so, is this expertise specific to the trained image class, or do such experts possess generally superior visual memory? We tested recognition memory of cytologists, radiologists, and controls with no medical experience for three visual stimulus classes: isolated objects, scenes, and mammograms or micrographs. Experts were better than control observers at recognizing images from their domain, but their memory for those images was not particularly good (D’ ~ 1.0) and was much worse than memory for objects or scenes (D’ > 2.0). Furthermore, experts were not better at recognizing scenes or isolated objects than control observers.     

Incongruent object/context relationships in visual scenes: Where are they processed in the brain?

Rapid object visual categorization in briefly flashed natural scenes is influenced by the surrounding context. The neural correlates underlying reduced categorization performance in response to incongruent object/context associations remain unclear and were investigated in the present study using fMRI. Participants were instructed to categorize objects in briefly presented scenes (exposure duration = 100 ms). Half of the scenes consisted of objects pasted in an expected (congruent) context, whereas for the other half, objects were embedded in incongruent contexts. Object categorization was more accurate and faster in congruent relative to incongruent scenes. Moreover, we found that the two types of scenes elicited different patterns of cerebral activation. In particular, the processing of incongruent scenes induced increased activations in the parahippocampal cortex, as well as in the right frontal cortex. This higher activity may indicate additional neural processing of the novel (non experienced) contextual associations that were inherent to the incongruent scenes. Moreover, our results suggest that the locus of object categorization impairment due to contextual incongruence is in the right anterior parahippocampal cortex. Indeed in this region activity was correlated with the reaction time increase observed with incongruent scenes. Representations for associations between objects and their usual context of appearance might be encoded in the right anterior parahippocampal cortex.     

How does the purpose of inspection influence the potency of visual salience in scene perception?

Salience-map models have been taken to suggest that the locations of eye fixations are determined by the extent of the low-level discontinuities in an image. While such models have found some support, an increasing emphasis on the task viewers are performing implies that these models must combine with cognitive demands to describe how the eyes are guided efficiently. An experiment is reported in which eye movements to objects in photographs were examined while viewers performed a memory-encoding task or one of two search tasks. The objects depicted in the scenes had known salience ranks according to a popular model. Participants fixated higher-salience objects sooner and more often than lower-salience objects, but only when memorising scenes. This difference shows that salience-map models provide useful predictions even in complex scenes and late in viewing. However, salience had no effects when searching for a target defined by category or exemplar. The results suggest that salience maps are not used to guide the eyes in these tasks, that cognitive override by task demands can be total, and that modelling top-down search is important but may not be easily accomplished within a salience-map framework.     

Semantic memory for contextual regularities within and across scene categories: Evidence from eye movements

When encountering familiar scenes, observers can use item-specific memory to facilitate the guidance of attention to objects appearing in known locations or configurations. Here, we investigated how memory for relational contingencies that emerge across different scenes can be exploited to guide attention. Participants searched for letter targets embedded in pictures of bedrooms. In a between-subjects manipulation, targets were either always on a bed pillow or randomly positioned. When targets were systematically located within scenes, search for targets became more efficient. Importantly, this learning transferred to bedrooms without pillows, ruling out learning that is based on perceptual contingencies. Learning also transferred to living room scenes, but it did not transfer to kitchen scenes, even though both scene types contained pillows. These results suggest that statistical regularities abstracted across a range of stimuli are governed by semantic expectations regarding the presence of target-predicting local landmarks. Moreover, explicit awareness of these contingencies led to a central tendency bias in recall memory for precise target positions that is similar to the spatial category effects observed in landmark memory. These results broaden the scope of conditions under which contextual cuing operates and demonstrate how semantic memory plays a causal and independent role in the learning of associations between objects in real-world scenes.     

Functions of parahippocampal place area and retrosplenial cortex in real-world scene analysis: An fMRI study

We used functional MRI to investigate several hypotheses concerning the functions of posterior parahippocampal cortex and retrosplenial cortex, two regions that preferentially activate to images of real-world scenes compared to images of other meaningful visual stimuli such as objects and faces. We compared activation resulting from photographs of rooms, city streets, cityscapes, and landscapes against activation to a control condition of objects. Activation in posterior parahippocampal cortex, including parahippocampal place area, was greater for all scene types than objects, and greater for scenes that clearly convey information about local three-dimensional (3-D) structure (city streets and rooms) than scenes that do not (cityscapes and landscapes). Similar differences were observed in retrosplenial cortex, though activation was also greater for city streets than rooms. These results suggest that activation in both cortical areas is primarily related to analysis or representation of local 3-D space. The results are not consistent with hypotheses that these areas reflect panoramic spatial volume, an artificial versus natural category distinction, an indoor versus outdoor distinction, or the number of explicit objects depicted in a scene image.     

Statistical regularities guide the spatial scale of attention

Visual scenes contain information on both a local scale (e.g., a tree) and a global scale (e.g., a forest). The question of whether the visual system prioritizes local or global elements has been debated for over a century. Given that visual scenes often contain distinct individual objects, here we examine how regularities between individual objects prioritize local or global processing. Participants viewed Navon-like figures consisting of small local objects making up a global object, and were asked to identify either the shape of the local objects or the shape of the global object, as fast and accurately as possible. Unbeknown to the participants, local regularities (i.e., triplets) or global regularities (i.e., quadruples) were embedded among the objects. We found that the identification of the local shape was faster when individual objects reliably co-occurred immediately next to each other as triplets (local regularities, Experiment 1). This result suggested that local regularities draw attention to the local scale. Moreover, the identification of the global shape was faster when objects co-occurred at the global scale as quadruples (global regularities, Experiment 2). This result suggested that global regularities draw attention to the global scale. No participant was explicitly aware of the regularities in the experiments. The results suggest that statistical regularities can determine whether attention is directed to the individual objects or to the entire scene. The findings provide evidence that regularities guide the spatial scale of attention in the absence of explicit awareness.     

Optimal and preferred eye landing positions in objects and scenes

Viewing position effects are commonly observed in reading, but they have only rarely been investigated in object perception or in the realistic context of a natural scene. In two experiments, we explored where people fixate within photorealistic objects and the effects of this landing position on recognition and subsequent eye movements. The results demonstrate an optimal viewing position—objects are processed more quickly when fixation is in the centre of the object. Viewers also prefer to saccade to the centre of objects within a natural scene, even when making a large saccade. A central landing position is associated with an increased likelihood of making a refixation, a result that differs from previous reports and suggests that multiple fixations within objects, within scenes, occur for a range of reasons. These results suggest that eye movements within scenes are systematic and are made with reference to an early parsing of the scene into constituent objects.     

Attentional priming releases crowding

Views of natural scenes unfold over time, and objects of interest that were present a moment ago tend to remain present. While visual crowding places a fundamental limit on object recognition in cluttered scenes, most studies of crowding have suffered from the limitation that they typically involved static scenes. The role of temporal continuity in crowding has therefore been unaddressed. We investigated intertrial effects upon crowding in visual scenes, showing that crowding is considerably diminished when objects remain constant on consecutive visual search trials. Repetition of both the target and distractors decreases the critical distance for crowding from flankers. More generally, our results show how object continuity through between-trial priming releases objects that would otherwise be unidentifiable due to crowding. Crowding, although it is a significant bottleneck on object recognition, can be mitigated by statistically likely temporal continuity of the objects. Crowding therefore depends not only on what is momentarily present, but also on what was previously attended.