Semantic memory for contextual regularities within and across scene categories: Evidence from eye movements

When encountering familiar scenes, observers can use item-specific memory to facilitate the guidance of attention to objects appearing in known locations or configurations. Here, we investigated how memory for relational contingencies that emerge across different scenes can be exploited to guide attention. Participants searched for letter targets embedded in pictures of bedrooms. In a between-subjects manipulation, targets were either always on a bed pillow or randomly positioned. When targets were systematically located within scenes, search for targets became more efficient. Importantly, this learning transferred to bedrooms without pillows, ruling out learning that is based on perceptual contingencies. Learning also transferred to living room scenes, but it did not transfer to kitchen scenes, even though both scene types contained pillows. These results suggest that statistical regularities abstracted across a range of stimuli are governed by semantic expectations regarding the presence of target-predicting local landmarks. Moreover, explicit awareness of these contingencies led to a central tendency bias in recall memory for precise target positions that is similar to the spatial category effects observed in landmark memory. These results broaden the scope of conditions under which contextual cuing operates and demonstrate how semantic memory plays a causal and independent role in the learning of associations between objects in real-world scenes.     

Prior knowledge on the illumination position

Visual perception is fundamentally ambiguous because an infinite number of three-dimensional scenes are consistent with our retinal images. To circumvent these ambiguities, the visual system uses prior knowledge such as the assumption that light is coming from above our head. The use of such assumptions is rational when these assumptions are related to statistical regularities of our environment. In confirmation of previous visual search experiments, we demonstrate here that the assumption on the illumination position is in fact biased to the above-left rather than directly above. This bias to the left reaches 26 degrees on average in a more direct shape discrimination task. Both right-handed and left-handed observers have a similar leftward bias. We discuss the possible origins of this singular bias on the illumination position.     

Incidental learning of real-world regularities

The large literature on incidental learning relies almost exclusively on laboratory experiments. Whenever researchers have attempted to demonstrate incidental learning of real-world regularities, they have typically failed to show learning. For example, it is well established that people do not learn regularities in everyday objects, such as the left-right orientation of faces on coins, despite a very large exposure to them. In this report, we examine this apparent contradiction. We argue that most studies exploring real-life incidental learning use tests that are not as sensitive to low-confidence information as those traditionally used in laboratory tasks. Using more sensitive measures, we show that it is possible to learn regularities from British and Japanese cultural life as a direct result of exposure to these regularities. Further, confidence measures suggest that although the information may be acquired incidentally, it can be expressed with and without concomitant awareness of that knowledge.     

Recognition and attention guidance during contextual cueing in real-world scenes: evidence from eye movements

When confronted with a previously encountered scene, what information is used to guide search to a known target? We contrasted the role of a scene's basic-level category membership with its specific arrangement of visual properties. Observers were repeatedly shown photographs of scenes that contained consistently but arbitrarily located targets, allowing target positions to be associated with scene content. Learned scenes were then unexpectedly mirror reversed, spatially translating visual features as well as the target across the display while preserving the scene's identity and concept. Mirror reversals produced a cost as the eyes initially moved toward the position in the display in which the target had previously appeared. The cost was not complete, however; when initial search failed, the eyes were quickly directed to the target's new position. These results suggest that in real-world scenes, shifts of attention are initially based on scene identity, and subsequent shifts are guided by more detailed information regarding scene and object layout.     

Recognition and attention guidance during contextual cueing in real-world scenes: Evidence from eye movements

When confronted with a previously encountered scene, what information is used to guide search to a known target? We contrasted the role of a scene's basic-level category membership with its specific arrangement of visual properties. Observers were repeatedly shown photographs of scenes that contained consistently but arbitrarily located targets, allowing target positions to be associated with scene content. Learned scenes were then unexpectedly mirror reversed, spatially translating visual features as well as the target across the display while preserving the scene's identity and concept. Mirror reversals produced a cost as the eyes initially moved toward the position in the display in which the target had previously appeared. The cost was not complete, however; when initial search failed, the eyes were quickly directed to the target's new position. These results suggest that in real-world scenes, shifts of attention are initially based on scene identity, and subsequent shifts are guided by more detailed information regarding scene and object layout.     

Visual exploration pattern in hemineglect

The analysis of eye movement parameters in visual neglect such as cumulative fixation duration, saccade amplitude, or the numbers of saccades has been used to probe attention deficits in neglect patients, since the pattern of exploratory eye movements has been taken as a strong index of attention distribution. The current overview of the literature of visual neglect has its emphasis on studies dealing with eye movement and exploration analysis. We present our own results in 15 neglect patients. The free exploration behavior was analyzed in these patients presenting 32 naturalistic color photographs of everyday scenes. Cumulative fixation duration, spatial distribution of fixations in the horizontal and vertical plane, the number and amplitude of exploratory saccades was analyzed and compared with the results of an age-matched control group. A main result of our study was that in neglect patients, fixation distribution of free exploration of natural scenes is not only influenced by the left–right bias in the horizontal direction but also by the vertical direction.     

空间工作记忆负载对真实场景情境提示效应的影响

重复的画面布局能够促进观察者对目标项的搜索 (情境提示效应)。本研究采用双任务范式,分别在视觉搜索任务的学习阶段 (实验2a) 和测验阶段 (实验2b) 加入空间工作记忆任务, 并与单任务基线 (实验1)进行比较, 考察空间工作记忆负载对真实场景搜索中情境线索学习和情境提示效应表达的影响。结果发现: 空间负载会增大学习阶段的情境提示效应量, 同时削弱测验阶段的情境提示效应量, 而不影响情境线索的外显提取。由此可见, 真实场景中情境线索的学习和提示效应的表达均会受到有限的工作记忆资源的影响, 但情境线索提取的外显性不变。     

The role of color in visual search in real-world scenes: evidence from contextual cuing

Because the importance of color in visual tasks such as object identification and scene memory has been debated, we sought to determine whether color is used to guide visual search in contextual cuing with real-world scenes. In Experiment 1, participants searched for targets in repeated scenes that were shown in one of three conditions: natural colors, unnatural colors that remained consistent across repetitions, and unnatural colors that changed on every repetition. We found that the pattern of learning was the same in all three conditions. In Experiment 2, we did a transfer test in which the repeating scenes were shown in consistent colors that suddenly changed on the last block of the experiment. The color change had no effect on search times, relative to a condition in which the colors did not change. In Experiments 3 and 4, we replicated Experiments 1 and 2, using scenes from a color-diagnostic category of scenes, and obtained similar results. We conclude that color is not used to guide visual search in real-world contextual cuing, a finding that constrains the role of color in scene identification and recognition processes.     

Categorical implicit learning in real-world scenes: evidence from contextual cueing

The present study examined the extent to which learning mechanisms are deployed on semantic-categorical regularities during a visual searching within real-world scenes. The contextual cueing paradigm was used with photographs of indoor scenes in which the semantic category did or did not predict the target position on the screen. No evidence of a facilitation effect was observed in the predictive condition compared to the nonpredictive condition when participants were merely instructed to search for a target T or L (Experiment 1). However, a rapid contextual cueing effect occurred when each display containing the search target was preceded by a preview of the scene on which participants had to make a decision regarding the scene's category (Experiment 2). A follow-up explicit memory task indicated that this benefit resulted from implicit learning. Similar implicit contextual cueing effects were also obtained when the scene to categorize was different from the subsequent search scene (Experiment 3) and when a mere preview of the search scene preceded the visual searching (Experiment 4). These results suggested that if enhancing the processing of the scene was required with the present material, such implicit semantic learning can nevertheless take place when the category is task irrelevant.     

Object representations are biased toward each other through statistical learning

The visual system is remarkably efficient at extracting regularities from the environment through statistical learning. While such extraction has extensive consequences on cognition, it is unclear how statistical learning shapes the representations of the individual objects that comprise the regularities. Here we examine how statistical learning alters object representations. In three experiments, participants were exposed to either random arrays containing objects in a random order, or structured arrays containing object pairs where two objects appeared next to each other in fixed spatial or temporal configurations. After exposure, one object in each pair was briefly presented and participants judged the location or the orientation of the object without seeing the other object in the pair. We found that when an object reliably appeared next to another object in space, it was judged as being closer to the other object in space even though the other object was never presented (Experiments 1 and 2). Likewise, when an object reliably preceded another object in time, its orientation was biased toward the orientation of the other object even though the other object was never presented (Experiment 3). These results demonstrated that statistical learning fundamentally shapes how individual objects are represented in visual memory, by biasing the representation of one object toward its co-occurring partner. Importantly, participants in all experiments were not explicitly aware of the regularities. Thus, the bias in object representations was implicit. The current study reveals a novel impact of statistical learning on object representation: spatially co-occurring objects are represented as being closer in space, and temporally co-occurring objects are represented as having more similar features.     

仿真场景下类别空间关系判断中的注意分配

研究采用“提示-目标”的范式,并结合单、双任务的方法探讨仿真场景下类别空间关系判断中的空间注意分配。结果发现：提示影响类别空间关系判断效率,不同类型提示对类别空间关系判断的影响不同;类别空间关系判断还受到任务类型和判断类型的影响。由此得出以下结论：提示有助于注意资源的集中,提示越有效注意资源分配效率越高;注意分配受到第二任务要求的影响;决定注意分配策略的认知判断过程可能是持续存在的     

A computational framework for attentional object discovery in RGB-D videos

We present a computational framework for attention-guided visual scene exploration in sequences of RGB-D data. For this, we propose a visual object candidate generation method to produce object hypotheses about the objects in the scene. An attention system is used to prioritise the processing of visual information by (1) localising candidate objects, and (2) integrating an inhibition of return (IOR) mechanism grounded in spatial coordinates. This spatial IOR mechanism naturally copes with camera motions and inhibits objects that have already been the target of attention. Our approach provides object candidates which can be processed by higher cognitive modules such as object recognition. Since objects are basic elements for many higher level tasks, our architecture can be used as a first layer in any cognitive system that aims at interpreting a stream of images. We show in the evaluation how our framework finds most of the objects in challenging real-world scenes.     

Low-level orientation information for social evaluation in face images

Observers make a range of social evaluations based on facial appearance, including judgments of trustworthiness, warmth, competence, and other aspects of personality. What visual information do people use to make these judgments? While links have been made between perceived social characteristics and other high-level properties of facial appearance (e.g., attractiveness, masculinity), there has been comparatively little effort to link social evaluations to low-level visual features, like spatial frequency and orientation sub-bands, known to be critically important for face processing. We explored the extent to which different social evaluations depended critically on horizontal orientation energy vs. vertical orientation energy, as is the case for face identification and emotion recognition. We found that while trustworthiness judgments exhibited this bias for horizontal orientations, competence and dominance did not, suggesting that social evaluations may depend on a multi-channel representation of facial appearance at early stages of visual processing.     

Statistical learning using real-world scenes: extracting categorical regularities without conscious intent

ABSTRACT— Recent work has shown that observers can parse streams of syllables, tones, or visual shapes and learn statistical regularities in them without conscious intent (e.g., learn that A is always followed by B). Here, we demonstrate that these statistical-learning mechanisms can operate at an abstract, conceptual level. In Experiments 1 and 2, observers incidentally learned which semantic categories of natural scenes covaried (e.g., kitchen scenes were always followed by forest scenes). In Experiments 3 and 4, category learning with images of scenes transferred to words that represented the categories. In each experiment, the category of the scenes was irrelevant to the task. Together, these results suggest that statistical-learning mechanisms can operate at a categorical level, enabling generalization of learned regularities using existing conceptual knowledge. Such mechanisms may guide learning in domains as disparate as the acquisition of causal knowledge and the development of cognitive maps from environmental exploration.     

Bringing color to emotion: The influence of color on attentional bias to briefly presented emotional images

Is color a critical feature in emotional content extraction and involuntary attentional orienting toward affective stimuli? Here we used briefly presented emotional distractors to investigate the extent to which color information can influence the time course of attentional bias in early visual cortex. While participants performed a demanding visual foreground task, complex unpleasant and neutral background images were displayed in color or grayscale format for a short period of 133 ms and were immediately masked. Such a short presentation poses a challenge for visual processing. In the visual detection task, participants attended to flickering squares that elicited the steady-state visual evoked potential (SSVEP), allowing us to analyze the temporal dynamics of the competition for processing resources in early visual cortex. Concurrently we measured the visual event-related potentials (ERPs) evoked by the unpleasant and neutral background scenes. The results showed (a) that the distraction effect was greater with color than with grayscale images and (b) that it lasted longer with colored unpleasant distractor images. Furthermore, classical and mass-univariate ERP analyses indicated that, when presented in color, emotional scenes elicited more pronounced early negativities (N1–EPN) relative to neutral scenes, than when the scenes were presented in grayscale. Consistent with neural data, unpleasant scenes were rated as being more emotionally negative and received slightly higher arousal values when they were shown in color than when they were presented in grayscale. Taken together, these findings provide evidence for the modulatory role of picture color on a cascade of coordinated perceptual processes: by facilitating the higher-level extraction of emotional content, color influences the duration of the attentional bias to briefly presented affective scenes in lower-tier visual areas.     

Using real-world scenes as contextual cues for search

Research on contextual cueing has demonstrated that with simple arrays of letters and shapes, search for a target increases in efficiency as associations between a search target and its surrounding visual context are learned. We investigated whether the visual context afforded by repeated exposure to real-world scenes can also guide attention when the relationship between the scene and a target position is arbitrary. Observers searched for and identified a target letter embedded in photographs of real-world scenes. Although search time within novel scenes was consistent across trials, search time within repeated scenes decreased across repetitions. Unlike previous demonstrations of contextual cueing, however, memory for scene-target covariation was explicit. In subsequent memory tests, observers recognized repeated contexts more often than those that were presented once and displayed superior recall of target position within the repeated scenes. In addition, repetition of inverted scenes, which made the scene more difficult to identify, produced a markedly reduced rate of learning, suggesting semantic information concerning object and scene identity are used to guide attention.     

The Correlational Structure of Natural Images and the Calibration of Spatial Representations

Physiologists have long proposed that correlated input activity is important in normal sensory development. Here it is postulated that the visual system is sensitive to the correlation in image intensity across the visual field, and that these correlations are used to help calibrate spatial representations. Since measurements made near to each other in the visual field are more correlated than measurements made at a distance, the degree of correlation can be used as an estimate of the distance between two measurements and can therefore be used to calibrate a roughly organized spatial representation. We therefore explored the hypothesis that low level spatial representations are calibrated using a signal based on image intensity correlation. If the visual system uses input statistics to calibrate its spatial representation, then any distortions and anisotropies in these input statistics should be mirrored by distortions in the representation of space. To test the psychological implications of this hypothesis, a collection of 81 images of open and urban landscapes were used to estimate the degree of correlation between image intensity measurement pairs as a function of both distance and orientation. Doing this we show that a system that used the statistics measured to calibrate its representation would show: (1) a horizontal-vertical illusion;(2) the magnitude of this illusion would depend on the amount of open and urban landscapes in the environment;(3) there would be a nontrivial relationship between line orientation and judged length. Analogues of all these distortions and regularities can be found in the psychophysical literature on distance estimation. This gives strength to the proposal that spatial representations are calibrated using input statistics.     

Evidence for preserved representations in change blindness

People often fail to detect large changes to scenes, provided that the changes occur during a visual disruption. This phenomenon, known as "change blindness," occurs both in the laboratory and in real-world situations in which changes occur unexpectedly. The pervasiveness of the inability to detect changes is consistent with the theoretical notion that we internally represent relatively little information from our visual world from one glance at a scene to the next. However, evidence for change blindness does not necessarily imply the absence of such a representation---people could also miss changes if they fail to compare an existing representation of the pre-change scene to the post-change scene. In three experiments, we show that people often do have a representation of some aspects of the pre-change scene even when they fail to report the change. And, in fact, they appear to "discover" this memory and can explicitly report details of a changed object in response to probing questions. The results of these real-world change detection studies are discussed in the context of broader claims about change blindness.     

Colour vision is aligned with natural scene statistics at 4 months of age

Visual perception in adult humans is thought to be tuned to represent the statistical regularities of natural scenes. For example, in adults, visual sensitivity to different hues shows an asymmetry which coincides with the statistical regularities of colour in the natural world. Infants are sensitive to statistical regularities in social and linguistic stimuli, but whether or not infants’ visual systems are tuned to natural scene statistics is currently unclear. We measured colour discrimination in infants to investigate whether or not the visual system can represent chromatic scene statistics in very early life. Our results reveal the earliest association between vision and natural scene statistics that has yet been found: even as young as 4 months of age, colour vision is aligned with the distributions of colours in natural scenes.

Research Highlights

1. We find infants’ colour sensitivity is aligned with the distribution of colours in the natural world, as it is in adults.
2. At just 4 months, infants’ visual systems are tailored to extract and represent the statistical regularities of the natural world.
3. This points to a drive for the human brain to represent statistical regularities even at a young age.

     

Local and global contextual constraints on the identification of objects in scenes.

Objects likely to appear in a given real-world scene are frequently found to be easier to recognize. Two different sources of contextual information have been proposed as the basis for this effect: global scene background and individual companion objects. The present paper examines the relative importance of these two elements in explaining the context-sensitivity of object identification in full scenes. Specific sequences of object fixations were elicited during free scene exploration, while fixation times on designated target objects were recorded as a measure of ease of target identification. Episodic consistency between the target, the global scene background, and the object fixated just prior to the target (the prime), were manipulated orthogonally. Target fixation times were examined for effects of prime and background. Analyses show effects of both factors, which are modulated by the chronology and spatial extent of scene exploration. The results are discussed in terms of their implications for a model of visual object recognition in the context of real-world scenes.