The role of higher order image statistics in masking scene gist recognition

In the present article, we investigated whether higher order image statistics, which are known to be carried by the Fourier phase spectrum, are sufficient to affect scene gist recognition. In Experiment 1, we compared the scene gist masking strength of four masking image types that varied in their degrees of second- and higher order relationships: normal scene images, scene textures, phase-randomized scene images, and white noise. Masking effects were the largest for masking images that possessed significant higher order image statistics (scene images and scene textures) as compared with masking images that did not (phase-randomized scenes and white noise), with scene image masks yielding the largest masking effects. In a control study, we eliminated all differences in the second-order statistics of the masks, while maintaining differences in their higher order statistics by comparing masking by scene textures rather than by their phase-randomized versions, and showed that the former produced significantly stronger gist masking. Experiments 2 and 3 were designed to test whether conceptual masking could account for the differences in the strength of the scene texture and phase-randomized masks used in Experiment 1, and revealed that the recognizability of scene texture masks explained just 1% of their masking variance. Together, the results suggest that (1) masks containing the higher order statistical structure of scenes are more effective at masking scene gist processing than are masks lacking such structure, and (2) much of the disruption of scene gist recognition that one might be tempted to attribute to conceptual masking is due to spatial masking.     

The influence of color on the perception of scene gist

In 3 experiments the authors used a new contextual bias paradigm to explore how quickly information is extracted from a scene to activate gist, whether color contributes to this activation, and how color contributes, if it does. Participants were shown a brief presentation of a scene followed by the name of a target object. The target object could be consistent or inconsistent with scene gist but was never actually present in the scene. Scene gist activation was operationalized as the degree to which participants respond "yes" to consistent versus inconsistent objects, reflecting a response bias produced by scene gist. Experiment 1 demonstrated that scene gist is activated after a 42-ms exposure and that the strength of the activation increases with longer presentation durations. Experiments 2 and 3 explored the contribution of color to the activation of scene gist. The results revealed that color has an influence across a wide variety of scenes and is directly associated with scene gist.     

Recognition of facial expressions is influenced by emotional scene gist

Recognition of facial expressions has traditionally been investigated by presenting facial expressions without any context information. However, we rarely encounter an isolated facial expression; usually, we perceive a person's facial reaction as part of the surrounding context. In the present study, we addressed the question of whether emotional scenes influence the explicit recognition of facial expressions. In three experiments, participants were required to categorize facial expressions (disgust, fear, happiness) that were shown against backgrounds of natural scenes with either a congruent or an incongruent emotional significance. A significant interaction was found between facial expressions and the emotional content of the scenes, showing a response advantage for facial expressions accompanied by congruent scenes. This advantage was robust against increasing task load. Taken together, the results show that the surrounding scene is an important factor in recognizing facial expressions.     

View combination in scene recognition

Becoming familiar with an environment requires the ability to integrate spatial information from different views. We provide evidence that view combination, a mechanism believed to underlie the ability to recognize novel views of familiar objects, is also used to recognize coherent, real-world scenes. In two experiments, we trained participants to recognize a real-world scene from two perspectives. When the angular difference between the learned views was relatively small, the participants subsequently recognized novel views from locations between the learned views about as well as they recognized the learned views and better than novel views situated outside of the shortest distance between the learned views. In contrast, with large angles between training views, all the novel views were recognized less well than the trained views. These results extend the view combination approach to scenes and are difficult to reconcile with models proposing that scenes are recognized by transforming them to match only the nearest stored view.     

Diagnostic colors mediate scene recognition

In this research, we aim to ground scene recognition on information other than the identity of component objects. Specifically we seek to understand the structure of color cues that allows the express recognition of scene gists. Using the L*a*b* color space we examined the conditions under which chromatic cues concur with brightness to allow a viewer to recognize scenes at a glance. Using different methods, Experiments 1 and 2 tested the hypothesis that colors do contribute when they are diagnostic (i.e., predictive) of a scene category. Experiment 3 examined the structure of colored cues at different spatial scales that are responsible for the effects of color diagnosticity reported in Experiments 1 and 2. Together, the results suggest that colored blobs at a coarse spatial scale concur with luminance cues to form the relevant spatial layout that mediates express scene recognition.     

Scene context influences without scene gist: Eye movements guided by spatial associations in visual search

Although the use of semantic information about the world seems ubiquitous in every task we perform, it is not clear whether we rely on a scene’s semantic information to guide attention when searching for something in a specific scene context (e.g., keys in one’s living room). To address this question, we compared contribution of a scene’s semantic information (i.e., scene gist) versus learned spatial associations between objects and context. Using the flash-preview–moving-window paradigm Castelhano and Henderson (Journal of Experimental Psychology: Human Perception and Performance 33:753–763, 2007), participants searched for target objects that were placed in either consistent or inconsistent locations and were semantically consistent or inconsistent with the scene gist. The results showed that learned spatial associations were used to guide search even in inconsistent contexts, providing evidence that scene context can affect search performance without consistent scene gist information. We discuss the results in terms of hierarchical organization of top-down influences of scene context.     

View dependence in scene recognition after active learning

Human spatial encoding of three-dimensional navigable space was studied, using a virtual environment simulation. This allowed subjects to become familiar with a realistic scene by making simulated rotational and translational movements during training. Subsequent tests determined whether subjects could generalize their recognition ability by identifying novel-perspective views and topographic floor plans of the scene. Results from picture recognition tests showed that familiar direction views were most easily recognized, although significant generalization to novel views was observed. Topographic floor plans were also easily identified. In further experiments, novel-view performance diminished when active training was replaced by passive viewing of static images of the scene. However, the ability to make self-initiated movements, as opposed to watching dynamic movie sequences, had no effect on performance. These results suggest that representation of navigable space is view dependent and highlight the importance of spatial-temporal continuity during learning.     

Coincident orientation of objects and viewpoint-dependence in scene recognition

Viewpoint-dependence is a well-known phenomenon in which participants' spatial memory is better for previously experienced points of view than for novel ones. In the current study, partial-scene-recognition was used to examine the effect of coincident orientation of all the objects on viewpoint-dependence in spatial memory. When objects in scenes had no clear orientations (e.g., balls), participants' recognition of experienced directions was better than that of novel ones, indicating that there was viewpoint-dependence. However, when the objects in scenes were toy bears with clear orientations, the coincident orientation of objects (315 degrees), which was not experienced, shared the advantage of the experienced direction (0 degrees), and participants were equally likely to choose either direction when reconstructing the spatial representation in memory. These findings suggest that coincident orientation of objects may affect egocentric representations in spatial memory.     

Scene recognition following locomotion around a scene

Effects of locomotion on scene-recognition reaction time (RT) and accuracy were studied. In experiment 1, observers memorized an 11-object scene and made scene-recognition judgments on subsequently presented scenes from the encoded view or different views (ie scenes were rotated or observers moved around the scene, both from 40 degrees to 360 degrees). In experiment 2, observers viewed different 5-object scenes on each trial and made scene-recognition judgments from the encoded view or after moving around the scene, from 36 degrees to 180 degrees. Across experiments, scene-recognition RT increased (in experiment 2 accuracy decreased) with angular distance between encoded and judged views, regardless of how the viewpoint changes occurred. The findings raise questions about conditions in which locomotion produces spatially updated representations of scenes.     

Real-world schemata and scene recognition in adults and children

Recognition memory for previously seen multiobject scenes was examined for different types of contextual arrangements between objects in the scenes. It was found that organized scenes with novel but possible interobject relations were recognized more accurately than either organized scenes with familiar interobject relations or unorganized scenes with impossible interobject relations. This finding was obtained for adults, 8- to 10-year-old children, and 5- to 8-year-old children who indicated concrete-operational ability in Piaget’s conservation-of-liquid quantity task. The results were interpreted in conjunction with a two-stage model of scene processing involving the formation of a schema to represent a scene (Stage 1), and the operation of the schema in governing the further processing of detailed information in the scene (Stage 2). It was concluded that preoperational children can form schemata to represent organized scenes (Stage 1), but it is not until the emergence of concrete operations that these schemata become operational with respect to guiding the further processing of information in the scene (Stage 2).     

Intrinsic frames of reference and egocentric viewpoints in scene recognition

Three experiments investigated the roles of intrinsic directions of a scene and observer's viewing direction in recognizing the scene. Participants learned the locations of seven objects along an intrinsic direction that was different from their viewing direction and then recognized spatial arrangements of three or six of these objects from different viewpoints. The results showed that triplets with two objects along the intrinsic direction (intrinsic triplets) were easier to recognize than triplets with two objects along the study viewing direction (non-intrinsic triplets), even when the intrinsic triplets were presented at a novel test viewpoint and the non-intrinsic triplets were presented at the familiar test viewpoint. The results also showed that configurations with the same three or six objects were easier to recognize at the familiar test viewpoint than other viewpoints. These results support and develop the model of spatial memory and navigation proposed by Mou, McNamara, Valiquette, and Rump [Mou, W., McNamara, T. P., Valiquiette C. M., & Rump, B. (2004). Allocentric and egocentric updating of spatial memories. Journal of Experimental Psychology: Learning, Memory, and Cognition, 30, 142-157].     

Rapid detection of person information in a naturalistic scene

A preferential-looking paradigm was used to investigate how gaze is distributed in naturalistic scenes. Two scenes were presented side by side: one contained a single person (person-present) and one did not (person-absent). Eye movements were recorded, the principal measures being the time spent looking at each region of the scenes, and the latency and location of the first fixation within each trial. We studied gaze patterns during free viewing, and also in a task requiring gender discrimination of the human figure depicted. Results indicated a strong bias towards looking to the person-present scene. This bias was present on the first fixation after image presentation, confirming previous findings of ultra-rapid processing of complex information. Faces attracted disproportionately many fixations, the preference emerging in the first fixation and becoming stronger in the following ones. These biases were exaggerated in the gender-discrimination task. A tendency to look at the object being fixated by the person in the scene was shown to be strongest at a slightly later point in the gaze sequence. We conclude that human bodies and faces are subject to special perceptual processing when presented as part of a naturalistic scene.     

自然场景与身体动作对面孔表情识别的影响

本研究以消极情绪间感知相似性较低的厌恶、恐惧面孔表情为材料,提供5个情绪性语言标签减少文字背景对面孔识别的促进作用,通过2个实验对自然场景以及身体动作对面孔表情识别的影响进行了研究,旨在考察面孔表情与自然场景间的情绪一致性对情绪面孔识别和自然场景加工的影响,以及加入与自然场景情绪相冲突的身体动作对面孔表情识别可能产生的影响。研究结果表明:(1)尽管增加了情绪性语言标签选项数量,自然场景的情绪对面孔表情识别的影响依旧显著;(2)当面孔表情与自然场景情绪不一致时,面孔识别需要更多依赖对自然场景的加工,因此对自然场景的加工程度更高;(3)身体动作会在一定程度上干扰自然场景对面孔表情识别的影响,但自然场景依然对情绪面孔的表情识别有重要作用。     

A simplified conjoint recognition paradigm for the measurement of gist and verbatim memory

The distinction between verbatim and gist memory traces has furthered the understanding of numerous phenomena in various fields, such as false memory research, research on reasoning and decision making, and cognitive development. To measure verbatim and gist memory empirically, an experimental paradigm and multinomial measurement model has been proposed but rarely applied. In the present article, a simplified conjoint recognition paradigm and multinomial model is introduced and validated as a measurement tool for the separate assessment of verbatim and gist memory processes. A Bayesian metacognitive framework is applied to validate guessing processes. Extensions of the model toward incorporating the processes of phantom recollection and erroneous recollection rejection are discussed.     

Rapid extraction of gist from visual text and its influence on word recognition

Two experiments explored rapid extraction of gist from a visual text and its influence on word recognition. In both, a short text (sentence) containing a target word was presented for 200 ms and was followed by a target recognition task. Results showed that participants recognized contextually anomalous word targets less frequently than contextually consistent counterparts (Experiment 1). This context effect was obtained when sentences contained the same semantic content but with disrupted syntactic structure (Experiment 2). Results demonstrate that words in a briefly presented visual sentence are processed in parallel and that rapid extraction of sentence gist relies on a primitive representation of sentence context (termed protocontext) that is semantically activated by the simultaneous presentation of multiple words (i.e., a sentence) before syntactic processing.     

Active and passive scene recognition across views.

Recent evidence suggests that scene recognition across views is impaired when an array of objects rotates relative to a stationary observer, but not when the observer moves relative to a stationary display [Simons, D.J., Wang, R.F., 1998. Perceiving real-world viewpoint changes. Psychological Science 9, 315-320]. The experiments in this report examine whether the relatively poorer performance by stationary observers across view changes results from a lack of perceptual information for the rotation or from the lack of active control of the perspective change, both of which are present for viewpoint changes. Three experiments compared performance when observers passively experienced the view change and when they actively caused the change. Even with visual information and active control over the display rotation, change detection performance was still worse for orientation changes than for viewpoint changes. These findings suggest that observers can update a viewer-centered representation of a scene when they move to a different viewing position, but such updating does not occur during display rotations even with visual and motor information for the magnitude of the change. This experimental approach, using arrays of real objects rather than computer displays of isolated individual objects, can shed light on mechanisms that allow accurate recognition despite changes in the observer's position and orientation.     

The influence of surface color information and color knowledge information in object recognition

In order to clarify whether the influence of color knowledge information in object recognition depends on the presence of the appropriate surface color, we designed a name-object verification task. The relationship between color and shape information provided by the name and by the object photo was manipulated in order to assess color interference independently of shape interference. We tested three different versions for each object: typically colored, black and white, and nontypically colored. The response times on the nonmatching trials were used to measure the interference between the name and the photo. We predicted that the more similar the name and the photo are, the longer it would take to respond. Overall, the color similarity effect disappeared in the black-and-white and nontypical color conditions, suggesting that the influence of color knowledge on object recognition depends on the presence of the appropriate surface color information.     

Using what we know: Consequences of intentionally retrieving gist versus item-specific information

The effect of a prior gist-based versus item-specific retrieval orientation on recognition of objects and words was examined. Prior item-specific retrieval increased item-specific recognition of episodically related but not previously tested objects relative to both conceptual- and perceptual-gist retrieval. An item-specific retrieval advantage also was found when the stimuli were words (synonyms) rather than objects but not when participants overtly named objects during gist-based recognition testing, which suggests that they did not always label objects under general gist-retrieval instructions. Unlike verbal overshadowing, labeling objects during recognition attenuated (but did not eliminate) test- and interference-related forgetting. A full understanding of how retrieval affects subsequent memory, even for events or facts that are not themselves retrieved, must take into account the specificity with which that retrieval occurs.     

Primacy of emotional vs. semantic scene recognition in peripheral vision

Emotional scenes were presented peripherally (5.2° away from fixation) or foveally (at fixation) for 150 ms. In affective evaluation tasks viewers judged whether a scene was unpleasant or not, or whether it was pleasant or not. In semantic categorisation tasks viewers judged whether a scene involved animals or humans (superordinate-level task), or whether it portrayed females or males (subordinate-level task). The same stimuli were used for the affective and the semantic task. Results indicated that in peripheral vision affective evaluation was less accurate and slower than animal/human discrimination, and did not show any advantage over gender discrimination. In addition, performance impairment in the peripheral relative to the foveal condition was greater or equivalent for affective than for semantic categorisation. These findings cast doubts on the specialness and the primacy of affective over semantic recognition. The findings are also relevant when considering the role of the subcortical "low route" in emotional processing.     

Domain knowledge moderates the influence of visual saliency in scene recognition

Is the sequence of eye‐movements made when viewing a picture related to encoding the image into memory? The suggestion of a relationship is supported by studies that have found that scanpaths are more similar over multiple viewings of a stimulus than would be expected by chance. It has also been found that low‐level visual saliency contributes to the initial formation of these scanpaths, and has lead to formation of theories such as the saliency map hypothesis. However, bottom‐up processes such as these can be overridden by top‐down cognitive knowledge in the form of domain proficiency. Domain specialists were asked to look at a set of photographs of real‐world scenes in preparation for a memory test. Then they were given a second set of stimuli and were asked to identify the picture as old (from the previous set) or new (never seen before). Eye tracking analyses (including scanpath comparison using a string editing algorithm) revealed that saliency did influence where participants looked and in what sequence. However, this was reliably reduced when participants viewed pictures from their specialist domain. This effect is shown to be robust in a repeated viewing of the stimuli.