A saliency map in primary visual cortex

I propose that pre-attentive computational mechanisms in primary visual cortex create a saliency map. This map awards higher responses to more salient image locations; these responses are those of conventional V1 cells tuned to input features, such as orientation and color. Hence no separate feature maps, or any subsequent combination of them, is needed to create a saliency map. I use a model to show that this saliency map accounts for the way that the relative difficulties of visual search tasks depend on the features and spatial configurations of targets and distractors. This proposal links psychophysical behavior to V1 physiology and anatomy, and thereby makes testable predictions.     

Why is “blindsight” blind? A new perspective on primary visual cortex,recurrent activity and visual awareness

The neuropsychological phenomenon of blindsight has been taken to suggest that the primary visual cortex (V1) plays a unique role in visual awareness, and that extrastriate activation needs to be fed back to V1 in order for the content of that activation to be consciously perceived. The aim of this review is to evaluate this theoretical framework and to revisit its key tenets. Firstly, is blindsight truly a dissociation of awareness and visual detection? Secondly, is there sufficient evidence to rule out the possibility that the loss of awareness resulting from a V1 lesion simply reflects reduced extrastriate responsiveness, rather than a unique role of V1 in conscious experience? Evaluation of these arguments and the empirical evidence leads to the conclusion that the loss of phenomenal awareness in blindsight may not be due to feedback activity in V1 being the hallmark awareness. On the basis of existing literature, an alternative explanation of blindsight is proposed. In this view, visual awareness is a “global” cognitive function as its hallmark is the availability of information to a large number of perceptual and cognitive systems; this requires inter-areal long-range synchronous oscillatory activity. For these oscillations to arise, a specific temporal profile of neuronal activity is required, which is established through recurrent feedback activity involving V1 and the extrastriate cortex. When V1 is lesioned, the loss of recurrent activity prevents inter-areal networks on the basis of oscillatory activity. However, as limited amount of input can reach extrastriate cortex and some extrastriate neuronal selectivity is preserved, computations involving comparison of neural firing rates within a cortical area remain possible. This enables “local” read-out from specific brain regions, allowing for the detection and discrimination of basic visual attributes. Thus blindsight is blind due to lack of “global” long-range synchrony, and it functions via “local” neural readout from extrastriate areas.     

Orientation illusions induced by briefly flashed plaids

Orientation illusions which occur when a vertical grating is surrounded by a plaid can be induced either by one of the plaid's orthogonal component gratings or by a virtual axis of symmetry of the plaid, whichever is nearest vertical. In six experiments in which such illusion displays were flashed for durations between 15 and 405 ms, it was found that when these two-dimensional illusions are induced by a component grating (direct effects) the illusions increase monotonically as duration decreases, from 1 degree-2 degrees to about 6 degrees-7 degrees, over this range. Effects induced by axes of symmetry (indirect effects), in contrast, begin to occur only at longer durations: for short exposures, illusions are directionally opposite and large, similar to direct effects. These results suggest that only plaid-component-selective mechanisms operate at the shortest exposure durations and that additional time is required to extract more global higher-order pattern structure. The data are discussed in relation to sustained and transient mechanisms and also with respect to recent reports of more global processing mechanisms in extrastriate cortex and related data on one-dimensional tilt illusions.     

The differential effects of brief exposures and surrounding contours on direct and indirect tilt illusions

Four experiments in which logarithmic intervals between 25 and 1600 ms were used for stimulus duration in tests for the tilt illusion are reported. It is demonstrated that the direct and the indirect tilt illusions both increase in magnitude inversely with length of stimulus presentation. The data suggest that whereas the direct effect peaks with a value of about +7 degrees at the shortest flash duration used (25 ms), peak indirect effects (of about +2 degrees) do not occur at this duration. In addition, whereas direct effects level out after 100 ms stimulus exposure times, to the usual magnitude obtained with long presentations (about +2 degrees), indirect effects reach their standard magnitude (-0.5 degrees to -1.0 degrees) later, at exposures of about 400 ms. Even at very short flash durations, a luminance square frame surrounding the illusion display reduces the indirect effect by two thirds of its magnitude but has no effect at all on the direct effect. It is suggested that direct effects arise early in visual processing, in area V1, where there are transient mechanisms and where corruption of orientation analysis by the inducing grating would occur prior to later, extrastriate, global analysis of the surrounding peripheral frame. Indirect effects, on the other hand, may arise later, along the sustained parvocellular colour-form pathway, where more global processing occurs and susceptibility to surrounding fields might be expected.     

Perceptual grouping in Gabor lattices: proximity and alignment

We propose the Gabor lattice as a new stimulus designed to deal with multiple organizations in perceptual grouping, allowing both comparison between psychophysical data and neural findings and a systematic investigation of grouping based on several low-level characteristics and their interactions. A Gabor lattice is a geometric lattice with Gabor patches, evoking a multistable global orientation percept. Visual grouping in Gabor lattices with elements aligned in a global orientation was compared with grouping of nonaligned Gabor patches and of Gaussian blobs. The effect sizes of proximity and alignment were estimated in logistic regression analyses. The results confirmed the importance of proximity and local element alignment as factors in dynamic grouping. We also found a small but consistent enhancement of grouping along the global vector orthogonal to the local patch orientations. In light of these results, we further motivate the relevance of these stimuli and the associated experimental paradigm.     

Does contour classification precede contour grouping in perception of partially visible figures?

When a figure is only partially visible and its contours represent a small fraction of total image contours (as when there is much background clutter), a fast contour classification mechanism may filter non-figure contours in order to restrict the size of the input to subsequent contour grouping mechanisms. The results of two psychophysical experiments suggest that the human visual system can classify figure from non-figure contours on the basis of a difference in some contour property (e.g. length, orientation, curvature, etc). While certain contour properties (e.g. orientation, curvature) require only local analysis for classification, other contour properties (e.g. length) may require more global analysis of the retinal image. We constructed a pyramid-based computational model based on these observations and performed two simulations of experiment 1: one simulation with classification enabled and the other simulation with classification disabled. The classification-based simulation gave the superior account of human performance in experiment 1. When a figure is partially visible, with few contours relative to the number of non-figure contours, contour classification followed by contour grouping can be more efficient than contour grouping alone, owing to smaller input to grouping mechanisms.     

The role of the human extrastriate visual cortex in mirror symmetry discrimination: a TMS-adaptation study

The human visual system is able to efficiently extract symmetry information from the visual environment. Prior neuroimaging evidence has revealed symmetry-preferring neuronal representations in the dorsolateral extrastriate visual cortex; the objective of the present study was to investigate the necessity of these representations in symmetry discrimination. This was accomplished by the use of state-dependent transcranial magnetic stimulation, which combines the fine resolution of adaptation paradigms with the assessment of causality. Subjects were presented with adapters and targets consisting of dot configurations that could be symmetric along either the vertical or horizontal axis (or they could be non-symmetric), and they were asked to perform a symmetry discrimination task on the targets while fixating the center of the screen. TMS was applied during the delay between the adapter and the test stimulus over one of four different sites: Left or Right V1/V2, or left or right dorsolateral extrastriate cortex (DLO). TMS over both Left and Right DLO reduced the adaptation effect in detecting vertical and horizontal symmetry, although the Left DLO effect on horizontal symmetry and the Right DLO effect on both vertical and horizontal symmetry were present only when considering subjects who showed a behavioral adaptation effect in the baseline No-TMS condition. Application of TMS over the Left or Right V1/V2 did not modulate the adaptation effect. Overall, these data suggest that both the Left and Right DLO contain neuronal representations tuned to mirror symmetry which play a causal role in symmetry discrimination.     

Brain areas sensitive to coherent visual motion

Detection of coherent motion versus noise is widely used as a measure of global visual-motion processing. To localise the human brain mechanisms involved in this performance, functional magnetic resonance imaging (fMRI) was used to compare brain activation during viewing of coherently moving random dots with that during viewing spatially and temporally comparable dynamic noise. Rates of reversal of coherent motion and coherent-motion velocities (5 versus 20 deg s-1) were also compared. Differences in local activation between conditions were analysed by statistical parametric mapping. Greater activation by coherent motion compared to noise was found in V5 and putative V3A, but not in V1. In addition there were foci of activation on the occipital ventral surface, the intraparietal sulcus, and superior temporal sulcus. Thus, coherent-motion information has distinctive effects in a number of extrastriate visual brain areas. The rate of motion reversal showed only weak effects in motion-sensitive areas. V1 was better activated by noise than by coherent motion, possibly reflecting activation of neurons with a wider range of motion selectivities. This activation was at a more anterior location in the comparison of noise with the faster velocity, suggesting that 20 deg s-1 is beyond the velocity range of the V1 representation of central visual field. These results support the use of motion-coherence tests for extrastriate as opposed to V1 function. However, sensitivity to motion coherence is not confined to V5, and may extend beyond the classically defined dorsal stream.     

Influence of scale and orientation on the visual perception of natural scenes

It is well known that the distribution of spatial content with respect to spatial scale in real-world scenes falls in accordance with a 1/ f α relation. Equally well known is the tendency for an orientation bias in scene content with the predominant bias in content at the horizontal and vertical orientations. This has led to the suggestion of a relationship in which the mechanisms of the human visual system are optimized for processing such regularities. Here we review current literature concerning the measurement of these regularities (via Fourier analysis) of natural scenes in the context of other work that has psychophysically assessed the extent to which visual perception exploits such regularities of spatial content. In addition, 2 psychophysical experiments are presented that extend this literature and argue for the importance of these regularities in perceiving orientation in real-world visual stimuli.     

The modulatory impact of reward and attention on global feature selection in human visual cortex

Reward powerfully influences human behaviour and perception, with reward effects being observed already on the level of basic sensory processing. Although reward-related modulations generally resemble those related to attentional selection, it is debated whether these effects indeed reflect the same selection operations. Here we focus on neuromagnetic indices of global colour-based attention in visual cortex, and ask whether reward elicits the same or separable underlying modulation effects. Observers performed a colour/orientation selection task where colour served to define the target as well as reward prospect. On each trial a target containing the target colour and one other colour was presented in the left visual field (VF) together with a bicoloured distractor in the right VF. Reward was delivered on correctly performed trials when the reward colour appeared in the target but not when it appeared in the distractor. The effect of global colour selection was assessed by comparing the brain response to the distractor depending on whether it contained the target colour, the reward colour, both, or neither. We observed that both the reward and target colour led to similar increases of the neuromagnetic response between ~200–260 ms originating from the same ventral extrastriate visual cortex areas, albeit slightly temporally lagged. Importantly, the response to the target and reward colour alone always added up to match the response size of their combined presentation. These results suggest that while reward and attention recruit the same global feature selection effects in extrastriate visual cortex, they are likely controlled by independent top-down influences.     

Cortical dynamics of lateral inhibition: Visual persistence and ISI

Psychophysical studies show that increasing the interstimulus interval (ISI) between two stimuli decreases persistence of the first stimulus. While some researchers account for these results with interactions of transient and sustained inhibition, this paper describes an alternative explanation. In a neural-network model of boundary detection called the boundary contour system, persistence is the result of feedback-generated reverberations. Mechanisms to control these reverberations include lateral inhibition, which computer simulations show allows persistence in the network to qualitatively match the psychophysical data. Additional simulations predict that increasing the duration of the second stimulus should cause persistence to increase with ISI. The model links psychophysical data on visual persistence with computational requirements of spatial vision and properties of cells in visual cortex.     

Neural dynamics of motion perception: Direction fields,apertures, and resonant grouping

A neural network model of global motion segmentation by visual cortex is described. Called the motion boundary contour system (BCS), the model clarifies how ambiguous local movements on a complex moving shape are actively reorganized into a coherent global motion signal, Unlike many previous researchers, we analyze how a coherent motion signal is imparted to all regions of a moving figure, not only to regions at which unambiguous motion signals exist. The model hereby suggests a solution to the global aperture problem. The motion BCS describes how preprocessing of motion signals by a motion oriented contrast (MOC) filter is joined to long-range cooperative grouping mechanisms in a motion cooperative-competitive (MOCC) loop to control phenomena such as motion capture. The motion BCS is computed in parallel with the static BCS of Grossberg and Mingolla (1985a, 1985b, 1987). Homologous properties of the motion BCS and the static BCS, specialized to process motion directions and static orientations, respectively, support a unified explanation of many data about static form perception and motion form perception that have heretofore been unexplained or treated separately. Predictions about microscopic computational differences of the parallel cortical streams V1 → MT and V1 → V2 → MT are made— notably, the magnocellular thick stripe and parvocellular interstripe streams. It is shown how the motion BCS can compute motion directions that may be synthesized from multiple orientations with opposite directions of contrast. Interactions of model simple cells, complex cells, hyper-complex cells, and bipole cells are described, with special emphasis given to new functional roles in direction disambiguation for endstopping at multiple processing stages and to the dynamic interplay of spatially short-range and long-range interactions.     

Estimating local shape from shading in the presence of global shading

In theory, global shading may help with the estimation of local surface structurefrom shading (e.g., in specifying the illuminant direction). Empirically, we do not know whether human observers combine the information given by the local and global shading to estimate local shape. Observers had to indicate the orientation of a local elongated perturbation with or without global shading information provided by a background surface. Our psychophysical results show the following:

1. Observers do not estimate the orientation of the local perturbation more accuratelywith global shading information than they do in the absence of such information.
2. Responses depend dramatically on the inclination between the illuminant direction and the viewing direction. For an inclination of 20°, observers indicate more or less the orientation of the local ridge; however, for an inclination of 40°, they indicate either the direction of the illuminant or an orientation close to the shadow edge of the perturbation. Most subjects show some combination of these behaviors. This behavior is not altered by global shading information.

We conclude that in our paradigm, global shading information does not aid the estimation of local shape.     

Psychophysical evidence for an extrastriate contribution to a pattern-selective motion aftereffect

A stationary vertical test grating appears to drift to the left after adaptation to an inducing grating drifting to the right, this being known as the motion aftereffect (MAE). Pattern-specific motion aftereffects (PSMAEs) induced by superimposed pairs of gratings in which the component gratings drift up and down but the observer sees a single coherent plaid drifting to the right have been investigated. Two experiments are reported in which it is demonstrated that the PSMAE is tuned more to the motion of the pattern than to the orientation and direction of motion of the component gratings. However, when subjects adapt to the component gratings in alternation, aftereffect magnitude is dependent upon the individual grating orientations and motion directions. These results can be interpreted in terms of extrastriate contributions to the PSMAE, possibly arising from the middle temporal area, where some cells, unlike those in striate cortex (V1), are tuned to pattern motion rather than to component motion.     

Spatial and orientation specific integration in the tilt illusion

Lateral inhibition across a population of cells in visual cortex which are tuned to local orientation has been proposed and widely accepted as a basic process in the analysis of contour in the visual field. The tilt illusion is usually explained in terms of this inhibition. Experiments are reported which cast new light on the analysis of visual orientation. It is shown that tilt illusions may be obtained with very thin inducing annuli which are spatially remote from the test figure. In experiments in which remote crossed-grating plaids were used, an illusion which was pattern (global) rather than component (local) selective was seen. It is difficult to account for these observations in terms of local inhibitory mechanisms. Rather, the results support the existence of a secondary mechanism which is involved in basic orientation analytic processes. The relevance of these observations to models of visual contour analysis is discussed.     

Common mechanisms in apparent motion perception and visual pattern matching

There are close functional similarities between apparent motion perception and visual pattern matching. In particular, striking functional similarities have been demonstrated between perception of rigid objects in apparent motion and purely mental transformations of visual size and orientation used in comparisons of objects with respect to shape but regardless of size and orientation. In both cases, psychophysical data suggest that differences in visual size are resolved as differences in depth, such that transformation of size is done by translation in depth. Furthermore, the process of perceived or imagined translation appears to be stepwise additive such that a translation over a long distance consists of a sequence of smaller translations, the durations of these steps being additive. Both perceived and imagined rotation also appear to be stepwise additive, and combined transformations of size and orientation appear to be done by alternation of small steps of pure translation and small steps of pure rotation. The functional similarities suggest that common mechanisms underlie perception of apparent motion and purely mental transformations. In line with this suggestion, functional brain imaging has isolated neural structures in motion area MT used in mental transformation of size.     

A distributed intercortical processing of binocular rivalry: psychophysical evidence

When dissimilar visual scenes are viewed dichoptically, the observer perceives several different representations of the scene over time. To reveal that a distributed intercortical network mediates this phenomenon of binocular rivalry, we used a Kanizsa square-like display consisting of four pairs of color-rivalry-inducing elements. We found that when all four dominant elements had the same color, regardless of whether they were from the same or different eyes, the visual system ably integrated them into a larger subjective surface. Once formed, the same-colored subjective surface enjoyed a relatively longer predominance than mixed-colored patterns. During rivalry alternation, this same-colored surface was more likely to be replaced by a complementary same-colored surface, rather than by mixed-colored patterns (cohesive effect). Further, surface integration, which is mainly an extrastriate cortical function, was stronger when the same eye viewed the same-colored rivalry stimuli. Since the eye-of-origin signature is explicitly represented in V1, these findings together suggest that rivalry is processed along a distributed network including V1 and the extrastriate cortices.     

The effects of exposure duration and surrounding frames on direct and indirect tilt aftereffects and illusions

Direct and indirect tilt illusions (TIs) have been shown to have different mechanisms, because spatial parameters that affect the one do not affect the other, and vice versa. The indirect TI, for example, is reduced markedly by a surrounding vertical square frame, a manipulandum that has no effect on the direct TI (Wenderoth & Johnstone, 1988a). In six experiments, we show that both direct and indirect TIs are enhanced in magnitude with short (10-60 msec) exposures; that tilt aftereffects (TAEs) induced with short test exposures are entirely comparable in magnitude; that a surrounding square frame reduces indirect TAEs but not direct TAEs, just as occurs with the TI; and that when the surrounding frame is present during adaptation only, test only, and both or neither, the greatest indirect TAE reduction occurs when the frame is present during the test. These results are consistent with the view (Wenderoth & Johnstone, 1987, 1988a, 1988b) that indirect TIs and TAEs may not reflect temporary neural modification based on V1 lateral inhibitory processes but rather the operation of more global, possibly extrastriate, orientation-constancy mechanisms.     

The use of chromatic information for motion segmentation: differences between psychophysical and eye-movement measures

Previous psychophysical studies have shown that chromatic (red/green) information can be used as a segmentation cue for motion integration. We investigated the mechanisms mediating this phenomenon by comparing chromatic effects (and, for comparison, luminance effects) on motion integration between two measures: (i) directional eye movements with the notion that these responses are mediated mainly by low-level motion mechanisms, and (ii) psychophysical reports, with the notion that subjects' reports should employ higher-level (attention-based) mechanisms if available. To quantify chromatic (and luminance) effects on motion integration, coherent motion thresholds were obtained for two conditions, one in which the signal and noise dots were the same 'red' or 'green' chromaticity (or the same 'bright' or 'dark' luminance), referred to as homogeneous, and the other in which the signal and noise dots were of different chromaticities (or luminances), referred to as heterogeneous. 'Benefit ratios' (theta(HOM)/theta(HET)) were then computed, with values significantly greater than 1.0 indicating that chromatic (or luminance) information serves as a segmentation cue for motion integration. The results revealed a high and significant chromatic benefit ratio when the measure was based on psychophysical report, but not when it was based on an eye-movement measure. By contrast, luminance benefit ratios were roughly the same (and significant) for both measures. For comparison to adults, eye-movement data were also obtained from 3-month-old infants. Infants showed marginally significant benefit ratios in the luminance, but not in the chromatic, condition. In total, these results suggest that the use of chromatic information as a segmentation cue for motion integration relies on higher-level mechanisms, whereas luminance information works mainly through low-level motion mechanisms.