Relative salience affects the process of detecting changes in orientation and luminance

This study tests the effect of relative saliency on perceptual comparison and decision processes in the context of change detection in which distinct visual mechanisms process two features (e.g., luminance and orientation). Townsend and Nozawa's (1995) systems factorial technology was used to investigate the process architecture and stopping rule when deciding whether luminance or orientation of a Gabor patch had changed. Experiment 1 found individual differences in decision strategies when we did not control relative saliency. One group of participants adopted co-active processing, and the other group adopted serial self-terminating processing to detect the change signals. When Experiment 2 eliminated the relative saliency, all but one observer adopted parallel processing and followed a self-terminating rule. These results support the relative saliency hypothesis and highlight the fact that observers adopt different change-detection strategies for two features, especially when relative saliency exists between the two feature dimensions.     

The interplay of bottom-up and top-down mechanisms in visual guidance during object naming

An ongoing issue in visual cognition concerns the roles played by low- and high-level information in guiding visual attention, with current research remaining inconclusive about the interaction between the two. In this study, we bring fresh evidence into this long-standing debate by investigating visual saliency and contextual congruency during object naming (Experiment 1), a task in which visual processing interacts with language processing. We then compare the results of this experiment to data of a memorization task using the same stimuli (Experiment 2). In Experiment 1, we find that both saliency and congruency influence visual and naming responses and interact with linguistic factors. In particular, incongruent objects are fixated later and less often than congruent ones. However, saliency is a significant predictor of object naming, with salient objects being named earlier in a trial. Furthermore, the saliency and congruency of a named object interact with the lexical frequency of the associated word and mediate the time-course of fixations at naming. In Experiment 2, we find a similar overall pattern in the eye-movement responses, but only the congruency of the target is a significant predictor, with incongruent targets fixated less often than congruent targets. Crucially, this finding contrasts with claims in the literature that incongruent objects are more informative than congruent objects by deviating from scene context and hence need a longer processing. Overall, this study suggests that different sources of information are interactively used to guide visual attention on the targets to be named and raises new questions for existing theories of visual attention.     

The interaction between social saliency and perceptual saliency

The ability to select visual targets in hierarchical stimuli can be affected by both perceptual saliency and social saliency. However, the functional relations between the effects are not understood. Here we examined whether these two factors interact or combine in an additive way. Participants first learnt to associate geometric shapes with three people (e.g., triangle–self, square–stranger). After learning the associations, participants were presented with compound stimuli (e.g., a global triangle formed by a set of local squares) and had to select a target at the global or local level. In Experiment 1 the task was to identify the person associated with the local or global shape. In Experiment 2 the task was simply to identify the shape. We manipulated perceptual saliency by blurring local elements to form perceptually global salient stimuli or by contrasting the colours of neighbouring local elements (red vs. white) to form perceptually local salient stimuli. In Experiment 1 (person discrimination) there was a strong effect of saliency on local targets (there were faster and more accurate responses to high than to low saliency targets) when social and perceptual saliency occurred at same level. However, both perceptual and social saliency effects were eliminated when the effect of saliency at one level competed with that at the other level. In Experiment 2 (shape discrimination), there were only effects of perceptual saliency. The data indicate that social saliency interacts with perceptual saliency when explicit social categorizations are made, consistent with both factors modulating a common process of visual selection.     

Additional-singleton interference in efficient visual search: A common salience route for detection and compound tasks

In efficient search for feature singleton targets, additional singletons (ASs) defined in a nontarget dimension are frequently found to interfere with performance. All search tasks that are processed via a spatial saliency map of the display would be predicted to be subject to such AS interference. In contrast, dual-route models, such as feature integration theory, assume that singletons are detected not via a saliency map, but via a nonspatial route that is immune to interference from cross-dimensional ASs. Consistent with this, a number of studies have reported absent interference effects in detection tasks. However, recent work suggests that the failure to find such effects may be due to the particular frequencies at which ASs were presented, as well as to their relative saliency. These two factors were examined in the present study. In contrast to previous reports, cross-dimensional ASs were found to slow detection (target-present and target-absent) responses, modulated by both their frequency of occurrence and saliency (relative to the target). These findings challenge dual-route models and support single-route models, such as dimension weighting and guided search.     

Something Overlooked? How experts in change detection use visual saliency

How does expertise in the analysis of particular images influence the effects of visual saliency upon attention? Expert analysts of aerial photographs and untrained viewers undertook change‐detection and location memory tasks using aerial photographs with eye movements recorded throughout. Experts were more accurate in both tasks. Significant differences were also seen in the scanpaths: Untrained viewers fixated preferentially upon salient features throughout stimulus presentation whereas experts did not. However, both groups showed a strong influence of saliency in change detection and memory tasks. We interpret this apparent contradiction by: (i) assuming that the use of saliency in visual search is discretionary, and experts can use semantic information to prioritise where to fixate next; whereas, (ii) in tasks requiring spatial memory, analysis of visual saliency delivers easily acquired landmarks to reference the location of items in an image; a previously overlooked function used by expert and untrained viewers alike. Copyright © 2009 John Wiley & Sons, Ltd.     

Influence of semantic consistency and perceptual features on visual attention during scene viewing in toddlers

Conceptual representations of everyday scenes are built in interaction with visual environment and these representations guide our visual attention. Perceptual features and object-scene semantic consistency have been found to attract our attention during scene exploration. The present study examined how visual attention in 24-month-old toddlers is attracted by semantic violations and how perceptual features (i. e. saliency, centre distance, clutter and object size) and linguistic properties (i. e. object label frequency and label length) affect gaze distribution. We compared eye movements of 24-month-old toddlers and adults while exploring everyday scenes which either contained an inconsistent (e.g., soap on a breakfast table) or consistent (e.g., soap in a bathroom) object. Perceptual features such as saliency, centre distance and clutter of the scene affected looking times in the toddler group during the whole viewing time whereas looking times in adults were affected only by centre distance during the early viewing time. Adults looked longer to inconsistent than consistent objects either if the objects had a high or a low saliency. In contrast, toddlers presented semantic consistency effect only when objects were highly salient. Additionally, toddlers with lower vocabulary skills looked longer to inconsistent objects while toddlers with higher vocabulary skills look equally long to both consistent and inconsistent objects. Our results indicate that 24-month-old children use scene context to guide visual attention when exploring the visual environment. However, perceptual features have a stronger influence in eye movement guidance in toddlers than in adults. Our results also indicate that language skills influence cognitive but not perceptual guidance of eye movements during scene perception in toddlers.     

Quantifying the contribution of low-level saliency to human eye movements in dynamic scenes

We investigated the contribution of low-level saliency to human eye movements in complex dynamic scenes. Eye movements were recorded while naive observers viewed a heterogeneous collection of 50 video clips (46,489 frames; 4-6 subjects per clip), yielding 11,916 saccades of amplitude ≥2°. A model of bottom-up visual attention computed instantaneous saliency at the instant each saccade started and at its future endpoint location. Median model-predicted saliency was 45% the maximum saliency, a significant factor 2.03 greater than expected by chance. Motion and temporal change were stronger predictors of human saccades than colour, intensity, or orientation features, with the best predictor being the sum of all features. There was no significant correlation between model-predicted saliency and duration of fixation. A majority of saccades were directed to a minority of locations reliably marked as salient by the model, suggesting that bottom-up saliency may provide a set of candidate saccade target locations, with the final choice of which location of fixate more strongly determined top-down.     

Detection of emotional faces: salient physical features guide effective visual search

In this study, the authors investigated how salient visual features capture attention and facilitate detection of emotional facial expressions. In a visual search task, a target emotional face (happy, disgusted, fearful, angry, sad, or surprised) was presented in an array of neutral faces. Faster detection of happy and, to a lesser extent, surprised and disgusted faces was found both under upright and inverted display conditions. Inversion slowed down the detection of these faces less than that of others (fearful, angry, and sad). Accordingly, the detection advantage involves processing of featural rather than configural information. The facial features responsible for the detection advantage are located in the mouth rather than the eye region. Computationally modeled visual saliency predicted both attentional orienting and detection. Saliency was greatest for the faces (happy) and regions (mouth) that were fixated earlier and detected faster, and there was close correspondence between the onset of the modeled saliency peak and the time at which observers initially fixated the faces. The authors conclude that visual saliency of specific facial features--especially the smiling mouth--is responsible for facilitated initial orienting, which thus shortens detection. (PsycINFO Database Record (c) 2008 APA, all rights reserved).     

A biologically based model for recognition of 2-D occluded patterns

In this work, we present a biologically inspired model for recognition of occluded patterns. The general architecture of the model is based on the two visual information processing pathways of the human visual system, i.e. the ventral and the dorsal pathways. The proposed hierarchically structured model consists of three parallel processing channels. The main channel learns invariant representations of the input patterns and is responsible for pattern recognition task. But, it is limited to process one pattern at a time. The direct channel represents the biologically based direct connection from the lower to the higher processing level in the human visual cortex. It computes rapid top-down pattern-specific cues to modulate processing in the other two channels. The spatial channel mimics the dorsal pathway of the visual cortex. It generates a combined saliency map of the input patterns and, later, segments the part of the map representing the occluded pattern. This segmentation process is based on our hypothesis that the dorsal pathway, in addition to encoding spatial properties, encodes the shape representations of the patterns as well. The lateral interaction between the main and the spatial channels at appropriate processing levels and top-down, pattern-specific modulation of the these two channels by the direct channel strengthen the locations and features representing the occluded pattern. Consequently, occluded patterns become focus of attention in the ventral channel and also the pattern selected for further processing along this channel for final recognition.     

Contextual guidance of eye movements and attention in real-world scenes: the role of global features in object search

Many experiments have shown that the human visual system makes extensive use of contextual information for facilitating object search in natural scenes. However, the question of how to formally model contextual influences is still open. On the basis of a Bayesian framework, the authors present an original approach of attentional guidance by global scene context. The model comprises 2 parallel pathways; one pathway computes local features (saliency) and the other computes global (scene-centered) features. The contextual guidance model of attention combines bottom-up saliency, scene context, and top-down mechanisms at an early stage of visual processing and predicts the image regions likely to be fixated by human observers performing natural search tasks in real-world scenes.     

SUN: Top-down saliency using natural statistics

When people try to find particular objects in natural scenes they make extensive use of knowledge about how and where objects tend to appear in a scene. Although many forms of such "top-down" knowledge have been incorporated into saliency map models of visual search, surprisingly, the role of object appearance has been infrequently investigated. Here we present an appearance-based saliency model derived in a Bayesian framework. We compare our approach with both bottom-up saliency algorithms as well as the state-of-the-art Contextual Guidance model of Torralba et al. (2006) at predicting human fixations. Although both top-down approaches use very different types of information, they achieve similar performance; each substantially better than the purely bottom-up models. Our experiments reveal that a simple model of object appearance can predict human fixations quite well, even making the same mistakes as people.     

Domain knowledge moderates the influence of visual saliency in scene recognition

Is the sequence of eye‐movements made when viewing a picture related to encoding the image into memory? The suggestion of a relationship is supported by studies that have found that scanpaths are more similar over multiple viewings of a stimulus than would be expected by chance. It has also been found that low‐level visual saliency contributes to the initial formation of these scanpaths, and has lead to formation of theories such as the saliency map hypothesis. However, bottom‐up processes such as these can be overridden by top‐down cognitive knowledge in the form of domain proficiency. Domain specialists were asked to look at a set of photographs of real‐world scenes in preparation for a memory test. Then they were given a second set of stimuli and were asked to identify the picture as old (from the previous set) or new (never seen before). Eye tracking analyses (including scanpath comparison using a string editing algorithm) revealed that saliency did influence where participants looked and in what sequence. However, this was reliably reduced when participants viewed pictures from their specialist domain. This effect is shown to be robust in a repeated viewing of the stimuli.     

A clash of bottom-up and top-down processes in visual search: the reversed letter effect revisited

It is harder to find the letter "N" among its mirror reversals than vice versa, an inconvenient finding for bottom-up saliency accounts based on primary visual cortex (V1) mechanisms. However, in line with this account, we found that in dense search arrays, gaze first landed on either target equally fast. Remarkably, after first landing, gaze often strayed away again and target report was delayed. This delay was longer for target "N" We suggest that the delay arose because bottom-up saliency clashed with top-down shape recognition. Thus, although gaze landed accurately and quickly to the distinctive feature in the target shape (the orientation of the diagonal bar in "N" or "И"), the identical zigzag shape of target and distractors was registered, leading to temporary confusion. In sparser search arrays with smaller set sizes, top-down target shape recognition occurs earlier and bottom-up saliency is weaker. The clash in this case causes search asymmetry even before target location at first gaze landing. Our findings rule out previous suggestions that search asymmetry stems from stronger preattentive salience for the reversed target and/or faster rejection of familiar distractors.     

Neural enhancement and pre-emptive perception: the genesis of attention and the attentional maintenance of the cortical salience map

One of the stable hypotheses in systems neuroscience is the relationship between attention and the enhancement of visual responses when an animal attends to the stimulus in its receptive field (Goldberg and Wurtz, 1972 Journal of Neurophysiology 35 560-574). This was first discovered in the superior colliculus of the monkey: neurons in the superficial layers of the superior colliculus responded more intensely to the onset of a stimulus during blocks of trials in which the monkey had to make a saccade to it than they did during blocks of trials in which the monkey had to continue fixating a central point and not respond to the stimulus. This enhancement has been found in many brain regions, including prefrontal cortex (Boch and Goldberg, 1987 Investigative Ophthalmology 28 Supplement, 124), V4 (Moran and Desimone, 1985 Science 229 782-784), and lateral intraparietal area (Colby et al, 1996 Journal of Neurophysiology 76 2841-2852; Colby and Goldberg, 1999 Annual Review of Neuroscience 22 319-349), and even V1 (Lamme et al, 2000 Vision Research 40 1507-1521). In these studies the assumption has been that the monkey attended to the stimulus because the stimulus evoked an enhanced response. In the experiments described here we show that for abruptly appearing stimuli, attention is not related to the initial response evoked by the stimulus, but by the activity present on the salience map in the parietal cortex when the stimulus appears. Attention to the stimulus may subsequently, by a top down signal, sustain the map, but stimuli can as easily be suppressed by top down features as they can be enhanced.     

Saliency on a natural scene background: Effects of color and luminance contrast add linearly

In natural vision, shifts in spatial attention are associated with shifts of gaze. Computational models of such overt attention typically use the concept of a saliency map: Normalized maps of center-surround differences are computed for individual stimulus features and added linearly to obtain the saliency map. Although the predictions of such models correlate with fixated locations better than chance, their mechanistic assumptions are less well investigated. Here, we tested one key assumption: Do the effects of different features add linearly or according to a max-type of interaction? We measured the eye position of observers viewing natural stimuli whose luminance contrast and/or color contrast (saturation) increased gradually toward one side. We found that these feature gradients biased fixations toward regions of high contrasts. When two contrast gradients (color and luminance) were superimposed, linear summation of their individual effects predicted their combined effect. This demonstrated that the interaction of color and luminance contrast with respect to human overt attention is—irrespective of the precise model—consistent with the assumption of linearity, but not with a max-type interaction of these features.     

Adult visual experience promotes recovery of primary visual cortex from long-term monocular deprivation

Prolonged visual deprivation from early childhood to maturity is believed to cause permanent visual impairment. However, there have been case reports of substantial improvement of binocular vision in human adults following lifelong visual impairment or deprivation. These observations, together with recent findings of adult ocular dominance plasticity in rodents, led us to re-examine whether adult primary visual cortex (V1) is capable of any recovery following long-term monocular deprivation starting in development. Using mice as a model, we find that monocular deprivation from early development to mature ages (well past the critical period) severely impaired binocular vision by reducing the amplitude of responses elicited by stimulation of the deprived eye. Surprisingly, we find little effect on nondeprived eye responses. Restoration of binocular vision in mature adults yields modest but significant improvement of visual responses in V1. Remarkably, we find that when binocular vision is followed by occlusion of the nondeprived eye, visual responses in V1 recover almost fully, as measured by visual evoked potential amplitude, spatial frequency threshold, and single-unit activity. We conclude that adult V1 can recover from long-term deprivation when provided with an optimal regimen of visual experience.     

A continuous smooth map of space in the primary visual cortex of the common marmoset

We examined the fine-scale mapping of the visual world within the primary visual cortex of the marmoset monkey (Callithrix jacchus) using differential optical imaging. We stimulated two sets of complementary stripe-like locations in turn, subtracting them to generate the cortical representations of continuous bands of visual space. Rotating this stimulus configuration makes it possible to map different spatial axes within the primary visual cortex. In a similar manner, shifting the stimulated locations between trials makes it possible to map retinotopy at an even finer scale. Using these methods we found no evidence of any local anisotropies or distortions in the cortical representation of visual space. This is despite the fact that orientation preference is mapped in a discontinuous manner across the surface of marmoset V1. Overall, our results indicate that space is mapped in a continuous and smooth manner in the primary visual cortex of the common marmoset.     

Reversing the hands of time: changing the mapping from seeing to saying

To describe a scene, speakers must map visual information to a linguistic plan. Eye movements capture features of this linkage in a tendency for speakers to fixate referents just before they are mentioned. The current experiment examined whether and how this pattern changes when speakers create atypical mappings. Eye movements were monitored as participants told the time from analog clocks. Half of the participants did this in the usual manner. For the other participants, the denotations of the clock hands were reversed, making the big hand the hour and the little hand the minute. Eye movements revealed that it was not the visual features or configuration of the hands that determined gaze patterns, but rather top-down control from upcoming referring expressions. Differences in eye-voice spans further suggested a process in which scene elements are relationally structured before a linguistic plan is executed. This provides evidence for structural rather than lexical incrementality in planning and supports a "seeing-for-saying" hypothesis in which the visual system is harnessed to the linguistic demands of an upcoming utterance.     

The detection of feature singletons defined in two dimensions is based on salience summation, rather than on serial exhaustive or interactive race architectures

Influential models of visual search assume that dimension-specific feature contrast signals are summed into a master saliency map in a coactive fashion. The main source of evidence for coactivation models, and against parallel race models, is violations of the race model inequality (RMI; Miller, 1982) by redundantly defined singleton feature targets. However, RMI violations do not rule out serial exhaustive (Townsend & Nozawa, 1997) or interactive race (Mordkoff & Yantis, 1991) architectures. These alternatives were tested in two experiments. In Experiment 1, we used a double-factorial design with singleton targets defined in two dimensions and at two levels of intensity, to distinguish between serial versus parallel models and self-terminating versus exhaustive stopping rules. In Experiment 2, we manipulated contingency benefits that are expected to affect the magnitude of redundancy gains and/or RMI violations on the assumption of an interactive race. The results of both experiments revealed redundancy gains as well as violations of the RMI, but the data pattern excluded serial-exhaustive and interactive race models as possible explanations for RMI violations. This result supports saliency summation (coactivation) models of search for singleton feature targets.     

Relative visual saliency differences induce sizable bias in consumer choice

Consumers often need to make very rapid choices among multiple brands (e.g., at a supermarket shelf) that differ both in their reward value (e.g., taste) and in their visual properties (e.g., color and brightness of the packaging). Since the visual properties of stimuli are known to influence visual attention, and attention is known to influence choices, this gives rise to a potential visual saliency bias in choices. We utilize experimental design from visual neuroscience in three real food choice experiments to measure the size of the visual saliency bias and how it changes with decision speed and cognitive load. Our results show that at rapid decision speeds visual saliency influences choices more than preferences do, that the bias increases with cognitive load, and that it is particularly strong when individuals do not have strong preferences among the options.