Saliency on a natural scene background: Effects of color and luminance contrast add linearly

In natural vision, shifts in spatial attention are associated with shifts of gaze. Computational models of such overt attention typically use the concept of a saliency map: Normalized maps of center-surround differences are computed for individual stimulus features and added linearly to obtain the saliency map. Although the predictions of such models correlate with fixated locations better than chance, their mechanistic assumptions are less well investigated. Here, we tested one key assumption: Do the effects of different features add linearly or according to a max-type of interaction? We measured the eye position of observers viewing natural stimuli whose luminance contrast and/or color contrast (saturation) increased gradually toward one side. We found that these feature gradients biased fixations toward regions of high contrasts. When two contrast gradients (color and luminance) were superimposed, linear summation of their individual effects predicted their combined effect. This demonstrated that the interaction of color and luminance contrast with respect to human overt attention is—irrespective of the precise model—consistent with the assumption of linearity, but not with a max-type interaction of these features.     

Contrast constancy revisited: The perceived contrast of sinusoidal gratings above threshold

The contrast sensitivity function of the human visual system, measured with sinusoidal luminance gratings, has an inverted U shape with a peak around 2–4?c/deg. Above threshold, it is thought that luminance gratings of equal physical contrasts but of distinguishably different spatial frequencies are all perceived as having similar contrasts, a phenomenon that has been termed contrast constancy. However, when suprathreshold contrast matches were measured for pairs of luminance gratings whose spatial frequencies were indistinguishable, the matching curves were not flat and followed a similar inverted U shape form as the contrast sensitivity function at threshold. It was therefore suggested that contrast constancy may only be revealed when matching the contrasts of clearly distinguishable spatial frequencies. Here, observers matched the perceived contrasts of suprathreshold luminance gratings of similar but visibly different spatial frequencies between 0.25 and 16?c/deg. The results show that, much like the contrast sensitivity function at threshold, observers are more sensitive to intermediate spatial frequencies (1–6?c/deg) than they are to either higher or to lower spatial frequencies. This tuning is evident when matching reference contrasts of 30–80%, implying a significant role in everyday vision. To demonstrate that these results were not due to local adaptation, the experiment was repeated with shorter stimulus duration, producing the same results. The extent of departure from contrast constancy found in the present study is compared to previously reported suprathreshold measurements. The results are also discussed with consideration to limitations with display apparatus such as monitor blur.     

What does second-order vision see in an image?

The human visual system is sensitive to both first-order variations in luminance and second-order variations in local contrast and texture. Although there is some debate about the nature of second-order vision and its relationship to first-order processing, there is now a body of results showing that they are processed separately. However, the amount, and nature, of second-order structure present in the natural environment is unclear. This is an important question because, if natural scenes contain little second-order structure in addition to first-order signals, the notion of a separate second-order system would lack ecological validity. Two models of second-order vision were applied to a number of well-calibrated natural images. Both models consisted of a first stage of oriented spatial filters followed by a rectifying nonlinearity and then a second set of filters. The models differed in terms of the connectivity between first-stage and second-stage filters. Output images taken from the models indicate that natural images do contain useful second-order structure. Specifically, the models reveal variations in texture and features defined by such variations. Areas of high contrast (but not necessarily high luminance) are also highlighted by the models. Second-order structure--as revealed by the models--did not correlate with the first-order profile of the images, suggesting that the two types of image 'content' may be statistically independent.     

Interaction of color and geometric cues in depth perception: When does “red” mean “near”?

Luminance and color are strong and self-sufficient cues to pictorial depth in visual scenes and images. The present study investigates the conditions under which luminance or color either strengthens or overrides geometric depth cues. We investigated how luminance contrast associated with the color red and color contrast interact with relative height in the visual field, partial occlusion, and interposition to determine the probability that a given figure presented in a pair is perceived as nearer than the other. Latencies of near responses were analyzed to test for effects of attentional selection. Figures in a pair were supported by luminance contrast (Experiment 1) or isoluminant color contrast (Experiment 2) and combined with one of the three geometric cues. The results of Experiment 1 show that the luminance contrast of a color (here red), when it does not interact with other colors, produces the same effects as achromatic luminance contrasts. The probability of near increases with the luminance contrast of the color stimulus, the latencies for near responses decrease with increasing luminance contrast. Partial occlusion is found to be a strong enough pictorial cue to support a weaker red luminance contrast. Interposition cues lose out against cues of spatial position and partial occlusion. The results of Experiment 2, with isoluminant displays of varying color contrast, reveal that red color contrast on a light background supported by any of the three geometric cues wins over green or white supported by any of the three geometric cues. On a dark background, red color contrast supported by the interposition cue loses out against green or white color contrast supported by partial occlusion. These findings reveal that color is not an independent depth cue, but is strongly influenced by luminance contrast and stimulus geometry. Systematically shorter response latencies for stronger near percepts demonstrate that selective visual attention reliably detects the most likely depth cue combination in a given configuration.     

Attention-based texture segregation

Luminance- or color-defined +/- 45 degrees-oriented bars were arranged to yield single-feature or double-conjunction texture pairs. In the former, the global edge between two regions is formed by differences in one attribute (orientation, or color, or luminance). In the color/orientation double-conjunction pair, one region has +45 degrees red and -45 degrees green textels, the other -45 degrees red and +45 degrees green textels (the luminance/orientation double-conjunction pair is similar); such a pair contains a single-feature orientation edge in the subset of red (or green) textels, and a color edge in the subset of +45 degrees (or -45 degrees) textels. We studied whether edge detection improved when observers were instructed to attend to such subsets. Two groups of observers participated: in the test group, the stimulus construction was explained to observers, and they were cued to attend to one subset. The control group ran through the same total number of sessions without explanations/cues. The effect of cuing was week but statistically significant. Feature cuing was more effective for color/orientation than for luminance/orientation conjunctions. Within each stimulus category, performance was nearly the same no matter which subset was attended to. On average, a global performance improvement occurred over time even without cuing, but some observers did not improve with either cuing or practice. We discuss these results in the context of one-versus two-stage segregation theories, as well as by reference to signal enhancement versus noise suppression. We conclude that texture segregation can be improved by attentional strategies aimed to isolate specific stimulus features.     

Transient covert attention does alter appearance: a reply to Schneider (2006)

We recently demonstrated that transient covert attention increases the apparent contrast of a stimulus (Carrasco, Ling, & Read, 2004). Schneider (2006) proposes that the observed increase in apparent contrast is largely due to sensory interactions occurring between the precue a nd stimulus,rather than to attention. Specifically, hereports that cuing effects only occur at contrasts ne ar detection threshold, and that there a re confounding sensoryinteractions between the cue and stimulus at suprathreshold detection contrasts. Our response is twofold. First, we outline the key methodological differences between our original study and Schneider's that are likely to account for the different results, and explain how we had ruled out the sensory interaction explanation of the cue. Second, we directly test the prediction put forth by Schneider. If the effects were due to sensory interactions, reversing the luminance polarity of the precue in our paradigm should lead to differential cuing effects. We replicate one of the experiments of our original study and add a condition in which the cue luminance is either black or white. Our results replicated our previous findings-they showed an increase in apparent contrast to a high-contrast stimulus when it was precued. Moreover, we found that the black cue and the white cue had the same effect, thus ruling out the alternative explanation proposed by Schneider. Transient attention does alter contrast appearance.     

Binocularity and visual search—Revisited

Binocular rivalry is a phenomenon of visual competition in which perception alternates between two monocular images. When two eye’s images only differ in luminance, observers may perceive shininess, a form of rivalry called binocular luster. Does dichoptic information guide attention in visual search? Wolfe and Franzel (Perception & Psychophysics, 44(1), 81–93, 1988) reported that rivalry could guide attention only weakly, but that luster (shininess) “popped out,” producing very shallow Reaction Time (RT) × Set Size functions. In this study, we have revisited the topic with new and improved stimuli. By using a checkerboard pattern in rivalry experiments, we found that search for rivalry can be more efficient (16 ms/item) than standard, rivalrous grating (30 ms/item). The checkerboard may reduce distracting orientation signals that masked the salience of rivalry between simple orthogonal gratings. Lustrous stimuli did not pop out when potential contrast and luminance artifacts were reduced. However, search efficiency was substantially improved when luster was added to the search target. Both rivalry and luster tasks can produce search asymmetries, as is characteristic of guiding features in search. These results suggest that interocular differences that produce rivalry or luster can guide attention, but these effects are relatively weak and can be hidden by other features like luminance and orientation in visual search tasks.     

Infants' perception of subjective contours from apparent motion

We examined infants' perception of subjective contours in Subjective-Contour-from-Apparent-Motion (SCAM) stimuli [e.g., Cicerone, C. M., Hoffman, D. D., Gowdy, P. D., & Kim, J. S. (1995). The perception of color from motion. Perception & Psychophysics, 57, 761-777] using the preferential looking technique. The SCAM stimulus is composed of random dots which are assigned two different colors. Circular region assigned one color moved apparently, keeping all dots' location unchanged. In the SCAM stimulus, adults can perceive subjective color spreading and subjective contours in apparent motion (http://c-faculty.chuo-u.ac.jp/ approximately ymasa/okamura/ibd_demo.html). In the present study, we conducted two experiments by using this type of SCAM stimulus. A total of thirty-six 3-8-month-olds participated. In experiment 1, we presented two stimuli to the infants side by side: a SCAM stimulus consisting of different luminance, and a non-SCAM stimulus consisting of isoluminance dots. The results indicated that the 5-8-month-olds showed preference for the SCAM stimuli. In experiments 2 and 3, we confirmed that the infants' preference for the SCAM stimulus was not generated by the local difference and local change made by luminance of dots but by the subjective contours. These results suggest that 5-8-month-olds were able to perceive subjective contours in the SCAM stimuli.     

Isoluminance and contingent color aftereffects

In the typical induction of the orientation-contingent color aftereffect (CCAE), the stimuli are composed of elements that differ in both color and luminance. Three experiments are reported that show that chromatic contrast between stimulus elements is insufficient for the induction of the orientation-CCAE and that luminance contrast is necessary. These experiments expand on previous research concerned with the role of luminance contrast in the induction of orientation-CCAEs by eliminating alternative explanations.     

Color correspondence in apparent motion

To maintain figural identity during motion perception, the visual system must match images over space and time. Correct matching requires a metric for identifying "corresponding" images, those representing the same physical object. To test whether matching is based on achromatic (black/white) polarity and chromatic (red/green) color, observers viewed an ambiguous motion display and judged the path of apparent motion. Matching preserved black/white identity regardless of whether frames were viewed binocularly or dichoptically. Red/green identity was also preserved, but coherence of motion depended in part on the number of frames in the motion sequence and on the background luminance. These results suggest that correspondence is computed by a weighted metric containing terms for image features coded early in visual processing.     

Coherence and transparency of moving plaids composed of Fourier and non-Fourier gratings.

We examined the perceptual coherence of two-component moving plaids. The gratings that constituted the plaids were either standard Fourier gratings (F), in which luminance was determined by a drifting sinusoid, or non-Fourier gratings (NF), in which the contrast of a random background was modulated by a drifting sinusoid. These NF gratings are examples of stimuli that generate a compelling percept of motion, even though they fail to elicit a motion signal from motion analyzers based on standard cross-correlation (Chubb & Sperling, 1988). Naive observers viewed three types of stimuli consisting of superpositions of these two components: (1) two standard drifting gratings (F/F), (2) two non-Fourier drifting gratings (NF/NF), and (3) one standard and one non-Fourier drifting grating (F/NF). As expected, the F/F stimulus yielded a compelling percept of coherent motion. The dominant percept of all the observers for the NF/NF stimulus was one of coherent motion, provided that both gratings were visible and of approximately equal contrast. None of the observers reported a dominant percept of coherent motion for the F/NF condition, over a wide range of contrasts for the two grating components and across two varieties of NF gratings. In view of the results of Albright (1992) and Albright and Chaudhuri (1989), that show that single cells in macaque V1 and MT respond to both F and NF motion, one cannot interpret our findings as evidence that F and NF motion are processed independently. Alternative, "higher level" interpretations based on the intrinsically ambiguous nature of the stimuli and physical laws governing the appearance of transparent objects are discussed.     

The speed and accuracy of material recognition in natural images

We studied the time course of material categorization in natural images relative to superordinate and basic-level object categorization, using a backward-masking paradigm. We manipulated several low-level features of the images—including luminance, contrast, and color—to assess their potential contributions. The results showed that the speed of material categorization was roughly comparable to the speed of basic-level object categorization, but slower than that of superordinate object categorization. The performance seemed to be crucially mediated by low-level factors, with color leading to a solid increase in performance for material categorization. At longer presentation durations, material categorization was less accurate than both types of object categorization. Taken together, our results show that material categorization can be as fast as basic-level object categorization, but is less accurate.     

Image quality of a mobile display under different illuminations

This study constructed the image quality models for a small mobile display under different ambient illumination levels using Group Method and Data Handling (GMDH) and described the relationship between perceived image quality and physical measurements. 33 college students took part in this experiment and were asked to evaluate the image quality under 1500 lux (typical indoor office illumination) and 7000 lux (simulated outdoor environment) in Stage One and Stage Two, respectively. In each stage, the participants had to evaluate 21 images. 17 sets of the data as a training set were used to build the model and four sets of the data as a testing set were used to verify the model. The results indicated that the effects of luminance, contrast, correlated color temperature (CCT), and resolution were significant on perceived image quality under 1500 lux. However, color temperature was not a significant physical characteristic, and an interaction between luminance and contrast was found below 7000 lux. From the results of the experiment, it is considered that the outdoor environment (7000 lux) is not suitable for using mobile displays. Finally, once a valid image quality model is built, the subjective image quality can be established when the measurements of significant physical characteristics are provided. The results of subjective ratings can also be provided for mobile display manufacturers to improve the product quality so that their products can meet customers' requirements.     

Focusing attention on overlapping and non-overlapping figures with subjective contours

In a series of five experiments we investigated whether observers could focus attention on a restricted visual area that was demarcated by Kanizsa-like subjective contours, and whether this effect also occurred in the case of overlapping figures. The task was a simple reaction time to a luminance increment and the basic finding was that reaction time was faster when the imperative stimulus fell inside the focus of attention than when it fell outside. The first two experiments showed that the extent of the attentional focus could be adapted to a region that was demarcated by subjective contours, thus extending the results of previous studies that used regions demarcated by real contours. The last three experiments showed that, regardless of the type of margins, focusing was more efficient for the figure that was perceived as lying in front in a pair of overlapping figures. Received: 21 December 1999  / Accepted: 23 August 2000     

Effect of luminance contrast on the motion aftereffect

The effects of luminance contrast and spatial frequency on the motion aftereffect were investigated. The point of subjective equality for velocity was measured as an index of the motion aftereffect. The largest effect was observed when a low contrast grating (5%) was presented as a test stimulus after adaptation to a high contrast grating (100%) in the low spatial frequency condition (0.8 cycle deg.-1). On the whole, the effect increased with increasing adapting contrast and with decreasing test contrast or spatial frequency. Small effects were observed at high test contrasts. These results were inconsistent with those of Keck, Palella, and Pantle in 1976. Analysis showed that there was no saturation on velocity of the motion aftereffect above 5% of the contrast although Keck, et al. (1976) found that the incremental increases of the effect above 3% adapting contrast were small.     

Mislocalization of a target toward subjective contours: attentional modulation of location signals

This study examined whether a briefly presented target was mislocalized toward a subjective contour. Observers manually reproduced the position of a briefly presented peripheral target circle above a central fixation cross. A luminance contour, a subjective contour, or a no-contour stimulus was presented in either the left of right visual field, and a no-contour control was presented in the opposite visual field. After these stimuli vanished, a target circle was then presented. Consequently, the degree of mislocalization toward the subjective and luminance contours was the same; this indicated that image integration at a coarse spatial scale cannot explain mislocalization. Experiment 2 revealed that the mislocalization in Experiment 1 was not a result of eye movements. Experiment 3 found that the spatial attention allocated at the location of the luminance and subjective contours was more than that allocated at the no-contour stimulus. An attentional shift toward the task-irrelevant stimulus resulted in a mislocalization of the target.     

Local contrast in natural images: normalisation and coding efficiency

The visual system employs a gain control mechanism in the cortical coding of contrast whereby the response of each cell is normalised by the integrated activity of neighbouring cells. While restricted in space, the normalisation pool is broadly tuned for spatial frequency and orientation, so that a cell's response is adapted by stimuli which fall outside its 'classical' receptive field. Various functions have been attributed to divisive gain control: in this paper we consider whether this output nonlinearity serves to increase the information carrying capacity of the neural code. 46 natural scenes were analysed with the use of oriented, frequency-tuned filters whose bandwidths were chosen to match those of mammalian striate cortical cells. The images were logarithmically transformed so that the filters responded to a luminance ratio or contrast. In the first study, the response of each filter was calibrated relative to its response to a grating stimulus, and local image contrast was expressed in terms of the familiar Michelson metric. We found that the distribution of contrasts in natural images is highly kurtotic, peaking at low values and having a long exponential tail. There is considerable variability in local contrast, both within and between images. In the second study we compared the distribution of response activity before and after implementing contrast normalisation, and noted two major changes. Response variability, both within and between scenes, is reduced by normalisation, and the entropy of the response distribution is increased after normalisation, indicating a more efficient transfer of information.     

No blindness for things that do not change

Abstract— It is well known that under normal circumstances, human observers are able to detect a visual change (a luminance transient) in the outside world very easily. This study demonstrated that observers are also easily able to detect a nonchanging element if it is located in a display containing multiple elements that do change. That is, a nonchanging element popped out from a display containing multiple changing elements (luminance transients). The efficient detection of the nonchanging element may be due to temporal grouping created by the dynamic character of the stimulus display.     

“Filling-in” colour in natural scenes

Our subjective experience of the world as being in full colour across the entire visual field is at odds with the highly fovea-biased distribution of cones in the retina. It is unclear how this percept of “pan-field colour” comes about. We use novel stimuli—“colour chimeras”—to demonstrate a related visual phenomenon in which observers perceive rich colour throughout images with large achromatic regions. This percept appears to critically depend on natural scene statistics. By separately manipulating chromatic and structural content in such images, we demonstrate that both the spatial distribution of colour and the presence of recognizable scene structure contribute to the experience of pan-field colour in these stimuli. Our results suggest that this percept is unlikely to be due to a low-level colour spreading process. Instead, we suggest that mechanisms dependent on natural scenes’ chromatic and luminance statistics provide the basis for the phenomenon.     

Why don't we see changes?: The role of attentional bottlenecks and limited visual memory

Seven experiments explore the role of bottlenecks in selective attention and access to visual short-term memory in the failure of observers to identify clearly visible changes in otherwise stable visual displays. Experiment One shows that observers fail to register a color change in an object even if they are cued to the location of the object by a transient at that location as the change is occurring. Experiment Two shows the same for orientation change. In Experiments Three and Four, attention is directed to specific objects prior to making changes in those objects. Observers have only a very limited memory for the status of recently attended items. Experiment Five reveals that observers have no ability to detect changes that happen after attention has been directed to an object and before attention returns to that object. In Experiment Six, attention is cued at rates that more closely resemble natural rates and Experiment Seven uses natural images. Memory capacity remains very small (<4 items).