Pictures in mind: initial encoding of object properties varies with the realism of the scene stimulus

A fundamental question in perception is how we visually encode and retain information about a complex scene in order to allow effective operation within it. Interestingly, the stimuli used to investigate scene perception have varied greatly between studies, ranging from line drawings to coloured drawings, computer-generated scenes, photographs, and real scenes. Are findings from these different types of scene stimulus equally ecologically valid? Two experiments are reported that address this issue. In the first we compared photographic and non-photographic scenes and found that observers perform better in questions testing object memory when viewing photographs, suggesting an initial benefit for encoding information from photographs. In the second we found that whether or not non-photographic scenes obeyed realistic scene-organising properties influenced object-memory formation. Effects varied for the different question types, but again were most prominent early in viewing. We conclude that in the search for an understanding of everyday scene perception we must be very careful in our choice of scene stimuli and in our interpretation of findings from the laboratory.     

Detection and Identification of Change in Naturalistic Scenes

Research using change detection paradigms has demonstrated that only limited scene information remains available for conscious report following initial inspection of a scene. Previous researchers have found higher change identification rates for deletions of parts of objects in line drawings of scenes than additions. Other researchers, however, have found an asymmetry in the opposite direction for addition/deletion of whole objects in line drawings of scenes. Experiment 1 investigated subjects' accuracy in detecting and identifying changes made to successive views of high quality photographs of naturalistic scenes that involved the addition and deletion of objects, colour changes to objects, and changes to the spatial location of objects. Identification accuracy for deletions from scenes was highest, with lower identification rates for object additions and colour changes, and the lowest rates for identification of location changes. Data further suggested that change identification rates for the presence/absence of objects were a function of the number of identical items present in the scene. Experiment 2 examined this possibility further, and also investigated whether the higher identification rates for deletions found in Experiment 1 were found for changes involving whole objects or parts of objects. Results showed higher identification rates for deletions, but only where a unique object was deleted from a scene. The presence of an identical object in the scene abolished this deletion identification advantage. Results further showed that the deletion/addition asymmetry occurs both when the objects are parts of a larger object and when they are entire objects in the scene.     

Scene articulation: dependence of illuminant estimates on number of surfaces

The ability of observers to detect changes in illuminant over two scenes containing different random samples of reflecting surfaces was determined in an experiment with Mondrian-like patterns containing different numbers of coloured patches. Performance was found to improve as the number of patches increased from 9 to 49. In principle, observers could have used space-average scene colour as the cue ('grey-world' hypothesis) or the colour of the brightest surface in the scene ('bright-is-white' hypothesis), as the two cues generally covary. In a second experiment, observers matched illuminants across different patterns in which the space-average cue and the brightest-patch cue were independently manipulated. The articulation of the patterns was varied: the number of patches increased from 49 (patch width 1 deg visual angle) to over 50000 (patch width 0.03 deg), while the gamut of colours was held constant. Space-average colour was found to be the dominant cue with all patterns except for those with the largest patches.     

How Are Different Properties of a Scene Encoded in Visual Memory?

Recent studies in scene perception suggest that much of what observers believe they see is not retained in visual memory. Depending on the roles they play in organizing the perception of a scene, different visual properties may require different amounts of attention to be incorporated into a mental representation of the scene. The goal of this study was to compare how three visual properties of scenes, colour, object position, and object presence, are encoded in visual memory. We used a variation on the change detection “flicker” task and measured the time to detect scene changes when: (1) a cue was provided regarding the type of change; and, (2) no cue was provided. We hypothesized that cueing would enhance the processing of visual properties that require more attention to be encoded into scene representations, whereas cueing would not have an effect for properties that are readily or automatically encoded in visual memory. In Experiment 1, we found that there was a cueing advantage for colour changes, but not for position or presence changes. In Experiment 2, we found the same cueing effect regardless of whether the colour change altered the configuration of the scene or not. These results are consistent with the idea that properties that typically help determine the configuration of the scene, for example, position and presence, are better encoded in scene representations than are surface properties such as colour.     

“Filling-in” colour in natural scenes

Our subjective experience of the world as being in full colour across the entire visual field is at odds with the highly fovea-biased distribution of cones in the retina. It is unclear how this percept of “pan-field colour” comes about. We use novel stimuli—“colour chimeras”—to demonstrate a related visual phenomenon in which observers perceive rich colour throughout images with large achromatic regions. This percept appears to critically depend on natural scene statistics. By separately manipulating chromatic and structural content in such images, we demonstrate that both the spatial distribution of colour and the presence of recognizable scene structure contribute to the experience of pan-field colour in these stimuli. Our results suggest that this percept is unlikely to be due to a low-level colour spreading process. Instead, we suggest that mechanisms dependent on natural scenes’ chromatic and luminance statistics provide the basis for the phenomenon.     

Judgement of lateral orientation for previously seen and unseen photographs of faces

In Study 1, 56 undergraduates judged whether each member of two sets of unseen photographs of faces was shown in the original or reversed orientation. Performance was at chance level, indicating that they could not perform this task. In Studies 2 and 3, the first of which involved unanalyzed data from previously-published work, an investigation was made of identification of orientation for faces which had been laterally reversed in the course of experiments on recognition memory. For a total of 406 subjects, over-all identification accuracy was about 60%, which was above chance. However, subjects were correct more often on normal (unchanged) than on reversed (changed) faces and were generally more likely to identify a face as normal when they were certain than when they were uncertain in their initial recognition judgement. It was concluded that identification performance could largely be accounted for by a response strategy model in which subjects judged orientation on the basis of their subjective familiarity with the face. Together, these studies demonstrate that subjects could not usually detect the lateral orientation of previously-seen or of unseen photographs of faces.     

多目标觉知范式检验视觉意识容量

采用一种新的多目标觉知范式检验视觉意识容量。实验中首先迅速呈现包含8个刺激的圆形阵列（刺激位置随机分布）,然后所有刺激被灰色的圆覆盖,并随机选取阵列中的一个刺激作为目标,要求被试用尽可能少的点击次数找到目标。结果发现,即使被试不能一次击中目标,其也可能拥有对目标不精确的视觉信息。这些不精确的视觉信息能够帮助被试用更少的点击次数和更短的点击距离找到目标,且被试对这些不精确的视觉信息是有意识的。研究结果表明视觉意识能够以概率性的表征存在,并支持视觉整体统计假说,也能够进一步解释视觉意识容量的主观测量和客观测量之间存在的冲突。     

Information limits on identification of natural surfaces by apparent colour

By adaptational and other mechanisms, the visual system can compensate for moderate changes in the colour of the illumination on a scene. Although the colours of most surfaces are perceived to be constant ('colour constancy'), some are not. The effect of these residual colour changes on the ability of observers to identify surfaces by their apparent colour was determined theoretically from high-resolution hyperspectral images of natural scenes under different daylights with correlated colour temperatures 4,300 K, 6,500 K, and 25,000 K. Perceived differences between colours were estimated with an approximately uniform colour-distance measure. The information preserved under illuminant changes increased with the number of surfaces in the sample, but was limited to a relatively low asymptotic value, indicating the importance of physical factors in constraining identification by apparent colour.     

Developmental changes in children's understanding of the similarity between photographs and their referents

In a series of three experiments, we investigated the development of children's understanding of the similarities between photographs and their referents. Based on prior work on the development of analogical understanding (e.g. Gentner & Rattermann, 1991), we suggest that the appreciation of this relation involves multiple levels. Photographs are similar to their referents both in terms of the constituent objects and in terms of the relations among these objects. We predicted that children would appreciate object similarity (whether photographs depict the same objects as in the referent scene) before they would appreciate relational similarity (whether photographs depict the objects in the same spatial positions as in the referent scene). To test this hypothesis, we presented 3-, 4-, 5-, 6-, and 7-year-old children and adults with several candidate photographs of an arrangement of objects. Participants were asked to choose which of the photographs was 'the same' as the arrangement. We manipulated the types of information the photographs preserved about the referent objects. One set of photographs did not preserve the object properties of the scene. Another set of photographs preserved the object properties of the scene, but not the relational similarity, such that the original objects were depicted but occupied different spatial positions in the arrangement. As predicted, younger children based their choices of the photographs largely on object similarity, whereas older children and adults also took relational similarity into account. Results are discussed in terms of the development of children's appreciation of different levels of similarity.     

Diagnostic colours contribute to the early stages of scene categorization: Behavioural and neurophysiological evidence

We examined the effects of colour cues on the express categorization of natural scenes. Using a go/no-go paradigm sensitive to fast recognition processes, we measured early event-related potential (ERP) correlates of scene categorization to elucidate the processing stage at which colour contributes to scene recognition. Observers were presented with scenes belonging to four colour-diagnostic categories (desert, forest, canyon and coastline). Scenes were presented in one of three forms: Diagnostically coloured, nondiagnostically coloured, or greyscale images. In a verification task, observers were instructed to respond whenever the presented stimulus matched a previously presented category name. Reaction times and accuracy were optimal when the stimuli were presented as their original diagnostically coloured version, followed by their greyscale version, and lastly by their nondiagostically coloured version. These effects were mirrored in the early (i.e., 150 ms following stimulus onset) ERP frontal correlates. Their onset was delayed for greyscale scenes compared to diagnostically coloured scenes, and for nondiagnostically coloured scenes compared to the other two conditions. Frontal ERP amplitudes also decreased for greyscale and nondiagnostically coloured scenes. Together, the results suggest that diagnostic colours are part of the scene gist responsible for express scene categorization.     

Postsaccadic processing of the retinal image during picture scanning

Eye movements were monitored while observers inspected photographs of natural scenes. At the end of each saccade (i.e., at the beginning of each period of steady fixation), the stimulus was replaced for a certain period of time by a uniform field (Experiment 1) or a blurred version of the stimulus scene (Experiment 2). Total fixation duration was measured as a function of the duration of the initial uniform field or the blurred image that followed the saccade. It was found that fixation duration increased proportionally with the duration of the initial replacement field, even for durations as short as 25 msec. These results suggest that the visual system uses information on the retina right after each saccade is completed and that the blurred, low-resolution information used in Experiment 2 (cutoff frequency of 0.8 cpd) is not sufficient for the requirements of picture processing in this task.     

Visual discrimination of spectral distributions

Human observers seem to be able to use different features that classify materials with a large degree of accuracy. In this paper, we look at human perception of statistical properties of the spectral distribution in a scene. We investigated whether human observers can discriminate just as accurately between coloured textures that have a spectral distribution due either to shading only or to both shading and specular reflectance as between uniform colours. Thresholds for the discrimination of coloured textures are about 15 times as high as thresholds for the discrimination of uniform colours, provided there is a sharp transition between the two colours. However, the coloured texture thresholds are only 1.5 times higher when we introduce a gradual transition between the two colours. There are also distinct qualitative differences in discrimination thresholds for different base colours. These differences cannot be predicted from discrimination thresholds for uniform colours. Human observers are surprisingly good at discriminating between a material edge and a shadow edge in complex scenes. Statistical differences in the orientation of the colour distributions in colour space might be used to accomplish this. In a second experiment we investigated how well observers can discriminate between two linear distributions in colour space that have the same base colour but different orientations. When we vary the line-length in R, G, B space, thresholds cannot be predicted completely by the conservation of the average distance between the two distributions. This means that observers use not only the maximum colour difference in the stimulus to do the task, but other cues are also involved.     

Minimalist surface-colour matching

Some theories of surface-colour perception assume that observers estimate the illuminant on a scene so that its effects can be discounted. A critical test of this interpretation of colour constancy is whether surface-colour matching is worse when the number of surfaces in a scene is so small that any illuminant estimate is unreliable. In the experiment reported here, observers made asymmetric colour matches between pairs of simultaneously presented Mondrian-like patterns under different daylights. The patterns had either 49 surfaces or a minimal 2 surfaces. No significant effect of number was found, suggesting that illuminant estimates are unnecessary for surface-colour matching.     

Mid-level feature contributions to category-specific gaze guidance

Our research has previously shown that scene categories can be predicted from observers’ eye movements when they view photographs of real-world scenes. The time course of category predictions reveals the differential influences of bottom-up and top-down information. Here we used these known differences to determine to what extent image features at different representational levels contribute toward guiding gaze in a category-specific manner. Participants viewed grayscale photographs and line drawings of real-world scenes while their gaze was tracked. Scene categories could be predicted from fixation density at all times over a 2-s time course in both photographs and line drawings. We replicated the shape of the prediction curve found previously, with an initial steep decrease in prediction accuracy from 300 to 500 ms, representing the contribution of bottom-up information, followed by a steady increase, representing top-down knowledge of category-specific information. We then computed the low-level features (luminance contrasts and orientation statistics), mid-level features (local symmetry and contour junctions), and Deep Gaze II output from the images, and used that information as a reference in our category predictions in order to assess their respective contributions to category-specific guidance of gaze. We observed that, as expected, low-level salience contributes mostly to the initial bottom-up peak of gaze guidance. Conversely, the mid-level features that describe scene structure (i.e., local symmetry and junctions) split their contributions between bottom-up and top-down attentional guidance, with symmetry contributing to both bottom-up and top-down guidance, while junctions play a more prominent role in the top-down guidance of gaze.     

Einstein’s jacket: Evidence for long-term perceptual specificity in mental imagery

To what extent are visual fantasies constrained by our perceptual experience of the real world? Our study exploits the fact that people’s knowledge of the appearance of individuals from the early 20th Century (e.g., Albert Einstein, Winston Churchill) derives predominantly from viewing black-and-white media images. An initial experiment shows that mental imagery for individuals from this period are experienced as significantly less colourful than imagery for individuals from the era of colour media. A second experiment manipulated whether participants were instructed to explicitly imagine using colour or not (i.e., “imagine Albert Einstein wearing a green jacket” vs. “imagine Albert Einstein wearing a jacket”). Results show that colour manipulation only influences imagery for black-and-white era individuals, with no comparable effect on imagery for colour era individuals. This finding is replicated in a third experiment that includes an additional control condition of imagining generic characters (i.e., “Imagine a knight wearing a cloak” vs. “imagine a knight wearing a red cloak”). Again, only imagery for black-and-white era individuals is affected by the colour manipulation. Overall these results provide evidence for long-term perceptual specificity effects in mental imagery. We argue that visual fantasies can be constrained by surface features of underlying representations in memory, even when imagining something we have never directly perceived.     

Natural scenes can be identified as rapidly as individual features

Can observers determine the gist of a natural scene in a purely feedforward manner, or does this process require deliberation and feedback? Observers can recognise images that are presented for very brief periods of time before being masked. It is unclear whether this recognition process occurs in a purely feedforward manner or whether feedback from higher cortical areas to lower cortical areas is necessary. The current study revealed that the minimum presentation time required to identify or to determine the gist of a natural scene was no different from that required to determine the orientation or colour of an isolated line. Conversely, a visual task that would be expected to necessitate feedback (determining whether an image contained exactly six lines) required a significantly greater minimum presentation time. Assuming that the orientation or colour of an isolated line can be determined in a purely feedforward manner, these results indicate that the identification and the determination of the gist of a natural scene can also be performed in a purely feedforward manner. These results challenge a number of theories of visual recognition that require feedback.     

Racial categorization and preference among older children in The Netherlands

In this study the pervasiveness of racial categorization is investigated among children (10–12 years of age) in multi-racial schools. Subjects were asked to sort photographs of unknown contemporaries and to indicate preferences. Skin colour, sex and facial expression were used as three characteristics which varied systematically in the pictures. The results show, first, that children preferred to use different features simultaneously instead of a single feature only. Second, in a dichotomous classification task skin colour and sex were the most obvious visible features used for categorization. Gender was also used for explaining socially undesirable behaviour, and for indicating preferences. while race was not used in these tasks. In indicating preferences facial expression and not skin colour was used as a subordinate category. There were very few differences between ethnic Dutch, coloured and ethnic minority children in the use of skin colour or other features.     

Representational drawings of solid objects by young children

Two groups of children aged 6 and 8 years were given three tasks requiring graphical representations of solid geometric forms. These tasks were drawing from life models, copying from photographs, and copying from line drawings of these objects. Performance was assessed on the basis of level of approximation to correct perspective. Older children used more perspective features than younger children in their drawings. At all ages, the drawings from life were most difficult. Results on the two copying tasks were not consistent. Drawings made by copying photographs were either as advanced as or poorer than copies of line drawings. The results are explained in terms of the difficulties exhibited by young children in translating the three-dimensional scene to a two-dimensional picture plane and strategies adopted by them to cope with these problems.     

The role of context on boundary extension

Boundary extension (BE) is a memory error in which observers remember more of a scene than they actually viewed. This error reflects one’s prediction that a scene naturally continues and is driven by scene schema and contextual knowledge. In two separate experiments we investigated the necessity of context and scene schema in BE. In Experiment 1, observers viewed scenes that either contained semantically consistent or inconsistent objects as well as objects on white backgrounds. In both types of scenes and in the no-background condition there was a BE effect; critically, semantic inconsistency in scenes reduced the magnitude of BE. In Experiment 2 when we used abstract shapes instead of meaningful objects, there was no BE effect. We suggest that although scene schema is necessary to elicit BE, contextual consistency is not required.     

Looking at scenes while searching for numbers: dividing attention multiplies space

Observers tend to remember seeing a greater expanse of a scene than was shown (boundary extension [BE]). Is undivided visual attention necessary for BE? In Experiment 1, 108 observers viewed photographs with superimposed numerals (2s and 5s). Each appeared for 750 msec, followed by a masked interval and a test picture (same, closer up, or wider angled). Test pictures were rated as the same, closer, or wider angled on a 5-point scale. Visual attention was manipulated with a search task: The observers reported the number of 5s (zero, one, or two). The observers performed search only, picture rating only, or both (giving search priority). Search accuracy was unaffected by condition. BE occurred in both conditions but was greater with divided attention. The results were replicated using incidental BE tests (Experiments 2 and 3). We propose that anticipatory representation of layout occurs automatically during scene perception, with focal attention serving to constrain the boundary error.