“Filling-in” colour in natural scenes

Our subjective experience of the world as being in full colour across the entire visual field is at odds with the highly fovea-biased distribution of cones in the retina. It is unclear how this percept of “pan-field colour” comes about. We use novel stimuli—“colour chimeras”—to demonstrate a related visual phenomenon in which observers perceive rich colour throughout images with large achromatic regions. This percept appears to critically depend on natural scene statistics. By separately manipulating chromatic and structural content in such images, we demonstrate that both the spatial distribution of colour and the presence of recognizable scene structure contribute to the experience of pan-field colour in these stimuli. Our results suggest that this percept is unlikely to be due to a low-level colour spreading process. Instead, we suggest that mechanisms dependent on natural scenes’ chromatic and luminance statistics provide the basis for the phenomenon.     

Infants look longer at colours that adults like when colours are highly saturated

The extent to which aesthetic preferences are ‘innate’ has been highly debated (Reber, Schwarz, & Winkielman, Personality and Social Psychology Review, 8(4), 364–382, 2004). For some types of visual stimuli infants look longer at those that adults prefer. It is unclear whether this is also the case for colour. A lack of relationship in prior studies between how long infants look at different colours and how much adults like those colours might be accounted for by stimulus limitations. For example, stimuli may have been too desaturated for infant vision. In the current study, using saturated colours more suitable for infants, we aim to quantify the relationship between infant looking and adult preference for colour. We take infant looking times at multiple hues from a study of infant colour categorization (Skelton, Catchpole, Abbott, Bosten, & Franklin, Proceedings of the National Academy of Sciences of the United States of America, 114(21), 5545–5550, 2017) and then measure adult preferences and compare these to infant looking. When colours are highly saturated, infants look longer at colours that adults prefer. Both infant looking time and adult preference are greatest for blue hues and are least for green-yellow. Infant looking and adult preference can be partly summarized by activation of the blue-yellow dimension in the early encoding of human colour vision. These findings suggest that colour preference is at least partially rooted in the sensory mechanisms of colour vision, and more broadly that aesthetic judgements may in part be due to underlying sensory biases.

     

Colour perception changes with basic colour word comprehension

Recent work has investigated the origin of infant colour categories, showing pre-linguistic infants categorise colour even in the absence of colour words. These infant categories are similar but not identical to adult categories, giving rise to an important question about how infant colour perception changes with the learning of colour words. Here we present two novel paradigms in which 12- and 19-month-old participants learning English as their first language were assessed on their perception of colour, while data on their colour word comprehension were also collected. Results indicate that participants' perception of colours close to the colour category boundaries dramatically change after colour word learning. The results highlight the shift made from infant colour categories to adult-like linguistically mediated colour categories that accompanies colour word learning.

Research Highlights

• We aimed to test whether colour perception is linguistically mediated in infants.
• We used novel eye-tracking and pupillometry paradigms to test infant colour perception either side of learning colour words.
• Infants' discrimination of colour changes after learning colour words, suggesting a shift due to colour word learning.
• A shift from pre-linguistic colour representation to linguistically mediated colour representation is discussed.

     

Information limits on identification of natural surfaces by apparent colour

By adaptational and other mechanisms, the visual system can compensate for moderate changes in the colour of the illumination on a scene. Although the colours of most surfaces are perceived to be constant ('colour constancy'), some are not. The effect of these residual colour changes on the ability of observers to identify surfaces by their apparent colour was determined theoretically from high-resolution hyperspectral images of natural scenes under different daylights with correlated colour temperatures 4,300 K, 6,500 K, and 25,000 K. Perceived differences between colours were estimated with an approximately uniform colour-distance measure. The information preserved under illuminant changes increased with the number of surfaces in the sample, but was limited to a relatively low asymptotic value, indicating the importance of physical factors in constraining identification by apparent colour.     

Does colour constancy exist?

For a stable visual world, the colours of objects should appear the same under different lights. This property of colour constancy has been assumed to be fundamental to vision, and many experimental attempts have been made to quantify it. I contend here, however, that the usual methods of measurement are either too coarse or concentrate not on colour constancy itself, but on other, complementary aspects of scene perception. Whether colour constancy exists other than in nominal terms remains unclear.     

Colour vision brings clarity to shadows

We have revealed a new role for colour vision in visual scene analysis: colour vision facilitates shadow identification. Shadows are important features of the visual scene, providing information about the shape, depth, and movement of objects. To be useful for perception, however, shadows must be distinguished from other types of luminance variation, principally the variation in object reflectance. A potential cue for distinguishing shadows from reflectance variations is colour, since chromatic changes typically occur at object but not shadow boundaries. We tested whether colour cues were exploited by the visual system for shadow identification, by comparing the ability of human test subjects to identify simulated shadows on chromatically variegated versus achromatically variegated backgrounds with identical luminance compositions. Performance was superior with the chromatically variegated backgrounds. Furthermore, introducing random colour contrast across the shadow boundaries degraded their identification. These findings demonstrate that the visual system exploits inbuilt assumptions about the relationships between colour and luminance in the natural visual world.     

Toward ecologically realistic theories in visual short-term memory research

Recent evidence from neuroimaging and psychophysics suggests common neural and representational substrates for visual perception and visual short-term memory (VSTM). Visual perception is adapted to a rich set of statistical regularities present in the natural visual environment. Common neural and representational substrates for visual perception and VSTM suggest that VSTM is adapted to these same statistical regularities too. This article discusses how the study of VSTM can be extended to stimuli that are ecologically more realistic than those commonly used in standard VSTM experiments and what the implications of such an extension could be for our current view of VSTM. We advocate for the development of unified models of visual perception and VSTM—probabilistic and hierarchical in nature—incorporating prior knowledge of natural scene statistics.     

The role of context in object recognition

In the real world, objects never occur in isolation; they co-vary with other objects and particular environments, providing a rich source of contextual associations to be exploited by the visual system. A natural way of representing the context of an object is in terms of its relationship to other objects. Alternately, recent work has shown that a statistical summary of the scene provides a complementary and effective source of information for contextual inference, which enables humans to quickly guide their attention and eyes to regions of interest in natural scenes. A better understanding of how humans build such scene representations, and of the mechanisms of contextual analysis, will lead to a new generation of computer vision systems.     

Diagnostic colours contribute to the early stages of scene categorization: Behavioural and neurophysiological evidence

We examined the effects of colour cues on the express categorization of natural scenes. Using a go/no-go paradigm sensitive to fast recognition processes, we measured early event-related potential (ERP) correlates of scene categorization to elucidate the processing stage at which colour contributes to scene recognition. Observers were presented with scenes belonging to four colour-diagnostic categories (desert, forest, canyon and coastline). Scenes were presented in one of three forms: Diagnostically coloured, nondiagnostically coloured, or greyscale images. In a verification task, observers were instructed to respond whenever the presented stimulus matched a previously presented category name. Reaction times and accuracy were optimal when the stimuli were presented as their original diagnostically coloured version, followed by their greyscale version, and lastly by their nondiagostically coloured version. These effects were mirrored in the early (i.e., 150 ms following stimulus onset) ERP frontal correlates. Their onset was delayed for greyscale scenes compared to diagnostically coloured scenes, and for nondiagnostically coloured scenes compared to the other two conditions. Frontal ERP amplitudes also decreased for greyscale and nondiagnostically coloured scenes. Together, the results suggest that diagnostic colours are part of the scene gist responsible for express scene categorization.     

Statistical regularities guide the spatial scale of attention

Visual scenes contain information on both a local scale (e.g., a tree) and a global scale (e.g., a forest). The question of whether the visual system prioritizes local or global elements has been debated for over a century. Given that visual scenes often contain distinct individual objects, here we examine how regularities between individual objects prioritize local or global processing. Participants viewed Navon-like figures consisting of small local objects making up a global object, and were asked to identify either the shape of the local objects or the shape of the global object, as fast and accurately as possible. Unbeknown to the participants, local regularities (i.e., triplets) or global regularities (i.e., quadruples) were embedded among the objects. We found that the identification of the local shape was faster when individual objects reliably co-occurred immediately next to each other as triplets (local regularities, Experiment 1). This result suggested that local regularities draw attention to the local scale. Moreover, the identification of the global shape was faster when objects co-occurred at the global scale as quadruples (global regularities, Experiment 2). This result suggested that global regularities draw attention to the global scale. No participant was explicitly aware of the regularities in the experiments. The results suggest that statistical regularities can determine whether attention is directed to the individual objects or to the entire scene. The findings provide evidence that regularities guide the spatial scale of attention in the absence of explicit awareness.     

Influence of scale and orientation on the visual perception of natural scenes

It is well known that the distribution of spatial content with respect to spatial scale in real-world scenes falls in accordance with a 1/ f α relation. Equally well known is the tendency for an orientation bias in scene content with the predominant bias in content at the horizontal and vertical orientations. This has led to the suggestion of a relationship in which the mechanisms of the human visual system are optimized for processing such regularities. Here we review current literature concerning the measurement of these regularities (via Fourier analysis) of natural scenes in the context of other work that has psychophysically assessed the extent to which visual perception exploits such regularities of spatial content. In addition, 2 psychophysical experiments are presented that extend this literature and argue for the importance of these regularities in perceiving orientation in real-world visual stimuli.     

Learning the association between a context and a target location in infancy

Extracting the statistical regularities present in the environment is a central learning mechanism in infancy. For instance, infants are able to learn the associations between simultaneously or successively presented visual objects (Fiser & Aslin, 2002 ; Kirkham, Slemmer & Johnson, 2002 ). The present study extends these results by investigating whether infants can learn the association between a target location and the context in which it is presented. With this aim, we used a visual associative learning procedure inspired by the contextual cuing paradigm, with infants from 8 to 12 months of age. In two experiments, in which we varied the complexity of the stimuli, we first habituated infants to several scenes where the location of a target (a cartoon character) was consistently associated with a context, namely a specific configuration of geometrical shapes. Second, we examined whether infants learned the covariation between the target location and the context by measuring looking times at scenes that either respected or violated the association. In both experiments, results showed that infants learned the target–context associations, as they looked longer at the familiar scenes than at the novel ones. In particular, infants selected clusters of co‐occurring contextual shapes and learned the covariation between the target location and this subset. These results support the existence of a powerful and versatile statistical learning mechanism that may influence the orientation of infants’ visual attention toward areas of interest in their environment during early developmental stages. A video abstract of this article can be viewed at: https://www.youtube.com/watch?v=9Hm1unyLBn0     

Exposure to behavioral regularities in everyday life predicts infants’ acquisition of mental state vocabulary

There are two broad views of children's theory of mind. The mentalist view is that it emerges in infancy and is possibly innate. The minimalist view is that it emerges more gradually in childhood and is heavily dependent on learning. According to minimalism, children initially understand behaviors rather than mental states, and they are assisted in doing so by recognizing repeating patterns in behavior. The regularities in behavior allow them to predict future behaviors, succeed on theory-of-mind tasks, acquire mental state words, and eventually, understand the mental states underlying behavior. The present study provided the first clear evidence for the plausibility of this view by fitting head cameras to 54 infants aged 6 to 25 months, and recording their view of the world in their daily lives. At 6 and 12 months, infants viewed an average of 146.5 repeated behaviors per hour, a rate consistent with approximately 560,000 repetitions in their first year, and with repetitions correlating with children's acquisition of mental state words, even after controlling for their general vocabulary and a range of variables indexing social interaction. We also recorded infants’ view of people searching or searching for and retrieving objects. These were 92 times less common and did not correlate with mental state vocabulary. Overall, the findings indicate that repeated behaviors provide a rich source of information for children that would readily allow them to recognize patterns in behavior and help them acquire mental state words, providing the first clear evidence for this claim of minimalism.

Research Highlights

• Six- to 25-month-olds wore head cameras to record home life from infants’ point-of-view and help adjudicate between nativist and minimalist views of theory-of-mind (ToM).
• Nativists say ToM is too early developing to enable learning, whereas minimalists say infants learn to predict behaviors from behavior patterns in environment.
• Consistent with minimalism, infants had an incredibly rich exposure (146.5/h, >560,000 in first year) to repeated behaviors (e.g., drinking from a cup repeatedly).
• Consistent with minimalism, more repeated behaviors correlated with infants’ mental state vocabulary, even after controlling for gender, age, searches witnessed and non-mental state vocabulary.

     

Developmental changes in colour constancy in a naturalistic object selection task

When the illumination falling on a surface change, so does the reflected light. Despite this, adult observers are good at perceiving surfaces as relatively unchanging—an ability termed colour constancy. Very few studies have investigated colour constancy in infants, and even fewer in children. Here we asked whether there is a difference in colour constancy between children and adults; what the developmental trajectory is between six and 11 years; and whether the pattern of constancy across illuminations and reflectances differs between adults and children. To this end, we developed a novel, child-friendly computer-based object selection task. In this, observers saw a dragon's favourite sweet under a neutral illumination and picked the matching sweet from an array of eight seen under a different illumination (blue, yellow, red, or green). This set contained a reflectance match (colour constant; perfect performance) and a tristimulus match (colour inconstant). We ran two experiments, with two-dimensional scenes in one and three-dimensional renderings in the other. Twenty-six adults and 33 children took part in the first experiment; 26 adults and 40 children took part in the second. Children performed better than adults on this task, and their performance decreased with age in both experiments. We found differences across illuminations and sweets, but a similar pattern across both age groups. This unexpected finding might reflect a real decrease in colour constancy from childhood to adulthood, explained by developmental changes in the perceptual and cognitive mechanisms underpinning colour constancy, or differences in task strategies between children and adults.

Highlights

• Six- to 11-year-old children demonstrated better performance than adults on a colour constancy object selection task.
• Performance decreased with age over childhood.
• These findings may indicate development of cognitive strategies used to overcome automatic colour constancy mechanisms.

     

Scene articulation: dependence of illuminant estimates on number of surfaces

The ability of observers to detect changes in illuminant over two scenes containing different random samples of reflecting surfaces was determined in an experiment with Mondrian-like patterns containing different numbers of coloured patches. Performance was found to improve as the number of patches increased from 9 to 49. In principle, observers could have used space-average scene colour as the cue ('grey-world' hypothesis) or the colour of the brightest surface in the scene ('bright-is-white' hypothesis), as the two cues generally covary. In a second experiment, observers matched illuminants across different patterns in which the space-average cue and the brightest-patch cue were independently manipulated. The articulation of the patterns was varied: the number of patches increased from 49 (patch width 1 deg visual angle) to over 50000 (patch width 0.03 deg), while the gamut of colours was held constant. Space-average colour was found to be the dominant cue with all patterns except for those with the largest patches.     

Variations in intensity statistics for representational and abstract art, and for art from the Eastern and Western hemispheres

Two recent studies suggest that natural scenes and paintings show similar statistical properties. But does the content or region of origin of an artwork affect its statistical properties? We addressed this question by having judges place paintings from a large, diverse collection of paintings into one of three subject-matter categories using a forced-choice paradigm. Basic statistics for images whose caterogization was agreed by all judges showed no significant differences between those judged to be 'landscape' and 'portrait/still-life', but these two classes differed from paintings judged to be 'abstract'. All categories showed basic spatial statistical regularities similar to those typical of natural scenes. A test of the full painting collection (140 images) with respect to the works' place of origin (provenance) showed significant differences between Eastern works and Western ones, differences which we find are likely related to the materials and the choice of background color. Although artists deviate slightly from reproducing natural statistics in abstract art (compared to representational art), the great majority of human art likely shares basic statistical limitations. We argue that statistical regularities in art are rooted in the need to make art visible to the eye, not in the inherent aesthetic value of natural-scene statistics, and we suggest that variability in spatial statistics may be generally imposed by manufacture.     

Contributions of Head-Mounted Cameras to Studying the Visual Environments of Infants and Young Children

Head-mounted video cameras (with and without an eye camera to track gaze direction) are being increasingly used to study infants’ and young children's visual environments and provide new and often unexpected insights about the visual world from a child's point of view. The challenge in using head cameras is principally conceptual and concerns the match between what these cameras measure and the research question. Head cameras record the scene in front of faces and thus answer questions about those head-centered scenes. In this “Tools of the Trade” article, we consider the unique contributions provided by head-centered video, the limitations and open questions that remain for head-camera methods, and the practical issues of placing head cameras on infants and analyzing the generated video.     

Wavelength Theory of Colour Strikes Back: The Return of the Physical

There have been a number of criticisms, based on visual processes, of the Australian view that colour is an objective property of the world. These criticisms have led to subjective theories about colour. These visual processes (metamers, retinex theory, opponent processes, simultaneous contrast, colour constancy, subjective colours) have been examined and it is suggested that they do not carry their supposed critical weight against an objective theory. In particular, it is argued that metamers don't occur in nature and primate colour vision evolved without metamers. Thus normal colour vision occurs without the problem of metamers. This argument, in conjunction with evidence against the critical roles of opponent processes and retinex theory in colour vision, is taken to suggest that colour can be given a photon energy/wavelengthrealism explanation. This proposal allows an account of the many microstructural bases of colour generation put forward by Nassau (1983). It is argued that neither disjunctive realism or reflectance realism are adequate objective explanations of colour.     

Visual discrimination of spectral distributions

Human observers seem to be able to use different features that classify materials with a large degree of accuracy. In this paper, we look at human perception of statistical properties of the spectral distribution in a scene. We investigated whether human observers can discriminate just as accurately between coloured textures that have a spectral distribution due either to shading only or to both shading and specular reflectance as between uniform colours. Thresholds for the discrimination of coloured textures are about 15 times as high as thresholds for the discrimination of uniform colours, provided there is a sharp transition between the two colours. However, the coloured texture thresholds are only 1.5 times higher when we introduce a gradual transition between the two colours. There are also distinct qualitative differences in discrimination thresholds for different base colours. These differences cannot be predicted from discrimination thresholds for uniform colours. Human observers are surprisingly good at discriminating between a material edge and a shadow edge in complex scenes. Statistical differences in the orientation of the colour distributions in colour space might be used to accomplish this. In a second experiment we investigated how well observers can discriminate between two linear distributions in colour space that have the same base colour but different orientations. When we vary the line-length in R, G, B space, thresholds cannot be predicted completely by the conservation of the average distance between the two distributions. This means that observers use not only the maximum colour difference in the stimulus to do the task, but other cues are also involved.