The perceptual root of object-based storage: an interactive model of perception and visual working memory

Mainstream theories of visual perception assume that visual working memory (VWM) is critical for integrating online perceptual information and constructing coherent visual experiences in changing environments. Given the dynamic interaction between online perception and VWM, we propose that how visual information is processed during visual perception can directly determine how the information is going to be selected, consolidated, and maintained in VWM. We demonstrate the validity of this hypothesis by investigating what kinds of perceptual information can be stored as integrated objects in VWM. Three criteria for object-based storage are introduced: (a) automatic selection of task-irrelevant features, (b) synchronous consolidation of multiple features, and (c) stable maintenance of feature conjunctions. The results show that the outputs of parallel perception meet all three criteria, as opposed to the outputs of serial attentive processing, which fail all three criteria. These results indicate that (a) perception and VWM are not two sequential processes, but are dynamically intertwined; (b) there are dissociated mechanisms in VWM for storing information identified at different stages of perception; and (c) the integrated object representations in VWM originate from the "preattentive" or "proto" objects created by parallel perception. These results suggest how visual perception, attention, and VWM can be explained by a unified framework.     

EPS Mid-Career Award 2004: brain mechanisms of attention

There are many varieties of "attention", to some extent separate yet working together to produce coherent perception, thought, and behaviour. Using data from human behaviour, functional neuroimaging, and single-cell recording in the behaving monkey, I consider different levels of attention and their basis in physiological mechanisms of biased competition. Beginning with visual attention, I suggest that processing is competitive in many brain systems that code visual input. Competition is biased towards stimuli that match task requirements and is integrated between systems coding different object properties. The result is flexible, object-based attentional selection. In the second part of the paper, I describe recent experiments on attentional competition within and between sensory modalities. Though competition is often modality specific, more global levels of interference are also easy to demonstrate. In the third part of the paper, I move to frontoparietal cortex and to a pattern of similar brain regions recruited by many different cognitive demands. This multiple-demand (MD) pattern, I suggest, reflects neurons with highly flexible response properties, adapting to represent the information and events of many different tasks. Biased competition in MD regions may play a central role in broad attentional capacity limits and attentional focusing. More generally, I suggest that biased competition is characteristic of many different cognitive domains and brain systems. Coherent "attention" develops as different systems converge to work on related cognitive content.     

大脑视知觉调控的神经机制

大脑的知觉加工并非单纯由外部刺激驱动,而是存在自上而下的知觉调控。尽管这一现象被大量实验研究证实,但其神经机制仍然是认知神经科学研究的重要问题。本研究系统介绍了知觉调控的神经基础、实现形式、研究范式,及其理论模型,分析指出了当前研究面临的主要问题,并对未来的研究进行了展望,以期促进该问题研究的进一步开展。     

Modularity and attention

Abstract

There is abundant evidence that the visual information-processing systems of higher animals are modular systems-systems consisting of relatively isolated networks of neurons that take the visual world apart (for example, Felleman & Van Essen, 1991, for an overview). Within contemporary psychology this modularity is often seen as a problem that requires a solution. The problem is: How does the brain put together all these different activities to produce a unified picture, so that, for example, for any object the right colour is associated with the right shape? The solution is that focal attention integrates the initially separable features in unitary objects.

In this contribution the opposite point of view is defended-not modularity as a problem solved by attention, but attention as a problem solved by modularity. The starting-point is that contemporary experimental psychology has to explain observed behaviour, not visual perception as a subjective experience. In these explanations the theoretical construct “focal attention” forms a major theoretical or conceptual problem. That problem can be solved in terms of modularity. It is argued that the modular character of the information-processing system is at the basis of the solutions nature has offered to solve two major selection problems in vision: the selection of an action and the selection of the object to act upon.     

Where perception meets memory: A review of repetition priming in visual search tasks

What we have recently seen and attended to strongly influences how we subsequently allocate visual attention. A clear example is how repeated presentation of an object’s features or location in visual search tasks facilitates subsequent detection or identification of that item, a phenomenon known as priming. Here, we review a large body of results from priming studies that suggest that a short-term implicit memory system guides our attention to recently viewed items. The nature of this memory system and the processing level at which visual priming occurs are still debated. Priming might be due to activity modulations of low-level areas coding simple stimulus characteristics or to higher level episodic memory representations of whole objects or visual scenes. Indeed, recent evidence indicates that only minor changes to the stimuli used in priming studies may alter the processing level at which priming occurs. We also review recent behavioral, neuropsychological, and neurophysiological evidence that indicates that the priming patterns are reflected in activity modulations at multiple sites along the visual pathways. We furthermore suggest that studies of priming in visual search may potentially shed important light on the nature of cortical visual representations. Our conclusion is that priming occurs at many different levels of the perceptual hierarchy, reflecting activity modulations ranging from lower to higher levels, depending on the stimulus, task, and context—in fact, the neural loci that are involved in the analysis of the stimuli for which priming effects are seen.     

Neural correlates of object indeterminacy in art compositions

Indeterminate art invokes a perceptual dilemma in which apparently detailed and vivid images resist identification. We used event-related fMRI to study visual perception of representational, indeterminate and abstract paintings. We hypothesized increased activation along a gradient of posterior-to-anterior ventral visual areas with increased object resolution, and postulated that object resolution would be associated with visual imagery. Behaviorally, subjects were faster to recognize familiar objects in representational than in both indeterminate and abstract paintings. We found activation within a distributed cortical network that includes visual, parietal, limbic and prefrontal regions. Representational paintings, which depict scenes cluttered with familiar objects, evoked stronger activation than indeterminate and abstract paintings in higher-tier visual areas. Perception of scrambled paintings was associated with imagery-related activation in the precuneus and prefrontal cortex. Finally, representational paintings evoked stronger activation than indeterminate paintings in the temporoparietal junction. Our results suggest that perception of familiar content in art works is mediated by object recognition, memory recall and mental imagery, cognitive processes that evoke activation within a distributed cortical network.     

Action history influences eye movements

Recent research has revealed that simple actions can have a profound effect on subsequent perception – people are faster to find a target that shares features with a previously acted on object even when those features are irrelevant to their task (the action effect). However, the majority of the evidence for this interaction between action and perception has come from manual response data. Therefore, it is unknown whether action affects early visual search processes, if it modulates post-attentional-selection processes, or both. To investigate this, we tracked participants’ spontaneous eye movements as they performed an action effect task. In two experiments we found that participants looked more quickly to the colour of an object they had previously acted on, compared to if they had viewed but not acted on the object, showing that action influenced early visual search processes. Additionally, there was evidence for post-selection effects as well. The results suggest that prior action affects both pre-selection and post-selection processes – spontaneously guiding attention to, and maintaining it on, objects that were previously important to the observer.     

Incremental grouping of image elements in vision

One important task for the visual system is to group image elements that belong to an object and to segregate them from other objects and the background. We here present an incremental grouping theory (IGT) that addresses the role of object-based attention in perceptual grouping at a psychological level and, at the same time, outlines the mechanisms for grouping at the neurophysiological level. The IGT proposes that there are two processes for perceptual grouping. The first process is base grouping and relies on neurons that are tuned to feature conjunctions. Base grouping is fast and occurs in parallel across the visual scene, but not all possible feature conjunctions can be coded as base groupings. If there are no neurons tuned to the relevant feature conjunctions, a second process called incremental grouping comes into play. Incremental grouping is a time-consuming and capacity-limited process that requires the gradual spread of enhanced neuronal activity across the representation of an object in the visual cortex. The spread of enhanced neuronal activity corresponds to the labeling of image elements with object-based attention.     

Feature confirmation in object perception: Feature integration theory 26 years on from the Treisman Bartlett lecture

The Treisman Bartlett lecture, reported in the Quarterly Journal of Experimental Psychology in 1988, provided a major overview of the feature integration theory of attention. This has continued to be a dominant account of human visual attention to this day. The current paper provides a summary of the work reported in the lecture and an update on critical aspects of the theory as applied to visual object perception. The paper highlights the emergence of findings that pose significant challenges to the theory and which suggest that revisions are required that allow for (a) several rather than a single form of feature integration, (b) some forms of feature integration to operate preattentively, (c) stored knowledge about single objects and interactions between objects to modulate perceptual integration, (d) the application of feature-based inhibition to object files where visual features are specified, which generates feature-based spreading suppression and scene segmentation, and (e) a role for attention in feature confirmation rather than feature integration in visual selection. A feature confirmation account of attention in object perception is outlined.     

Location perception: the X-Files parable

Three aspects of visual object location were investigated: (1) how the visual system integrates information for locating objects, (2) how attention operates to affect location perception, and (3) how the visual system deals with locating an object when multiple objects are present. The theories were described in terms of a parable (the X-Files parable). Then, computer simulations were developed. Finally, predictions derived from the simulations were tested. In the scenario described in the parable, we ask how a system of detectors might locate an alien spaceship, how attention might be implemented in such a spaceship detection system, and how the presence of one spaceship might influence the location perception of another alien spaceship. Experiment 1 demonstrated that location information is integrated with a spatial average rule. In Experiment 2, this rule was applied to a more-samples theory of attention. Experiment 3 demonstrated how the integration rule could account for various visual illusions.     

Location perception: TheX-Files parable

Three aspects of visual object location were investigated: (1) how the visual system integrates information for locating objects, (2) how attention operates to affect location perception, and (3) how the visual system deals with locating an object when multiple objects are present. The theories were described in terms of a parable (theX-Files parable). Then, computer simulations were developed. Finally, predictions derived from the simulations were tested. In the scenario described in the parable, we ask how a system of detectors might locate an alien spaceship, how attention might be implemented in such a spaceship detection system, and how the presence of one spaceship might influence the location perception of another alien spaceship. Experiment 1 demonstrated that location information is integrated with a spatial average rule. In Experiment 2, this rule was applied to a more-samples theory of attention. Experiment 3 demonstrated how the integration rule could account for various visual illusions.     

Evidence of fixed capacity in visual object categorization

How is visual object perception limited by divided attention? Whereas some theories have proposed that it is not limited at all (unlimited capacity), others have proposed that divided attention introduces restrictive capacity limitations or serial processing (fixed capacity). We addressed this question using a task in which observers searched for instances of particular object categories, such as a moose or squirrel. We applied an extended simultaneous-sequential paradigm to test the fixed-capacity and unlimited-capacity models (Experiment 1). The results were consistent with fixed capacity and rejected unlimited capacity. We ascertained that these results were due to attention, and not to sensory interactions such as crowding, by repeating the experiment using a cuing paradigm with physically identical displays (Experiment 2). The results from both experiments were consistent with theories of object perception that have fixed capacity, and they rejected theories with unlimited capacity. Both serial and parallel models with fixed capacity remain viable alternatives.     

Influence of semantic consistency and perceptual features on visual attention during scene viewing in toddlers

Conceptual representations of everyday scenes are built in interaction with visual environment and these representations guide our visual attention. Perceptual features and object-scene semantic consistency have been found to attract our attention during scene exploration. The present study examined how visual attention in 24-month-old toddlers is attracted by semantic violations and how perceptual features (i. e. saliency, centre distance, clutter and object size) and linguistic properties (i. e. object label frequency and label length) affect gaze distribution. We compared eye movements of 24-month-old toddlers and adults while exploring everyday scenes which either contained an inconsistent (e.g., soap on a breakfast table) or consistent (e.g., soap in a bathroom) object. Perceptual features such as saliency, centre distance and clutter of the scene affected looking times in the toddler group during the whole viewing time whereas looking times in adults were affected only by centre distance during the early viewing time. Adults looked longer to inconsistent than consistent objects either if the objects had a high or a low saliency. In contrast, toddlers presented semantic consistency effect only when objects were highly salient. Additionally, toddlers with lower vocabulary skills looked longer to inconsistent objects while toddlers with higher vocabulary skills look equally long to both consistent and inconsistent objects. Our results indicate that 24-month-old children use scene context to guide visual attention when exploring the visual environment. However, perceptual features have a stronger influence in eye movement guidance in toddlers than in adults. Our results also indicate that language skills influence cognitive but not perceptual guidance of eye movements during scene perception in toddlers.     

Top-down control of visual perception: attention in natural vision

Top-down perceptual influences can bias (or pre-empt) perception. In natural scenes, the receptive fields of neurons in the inferior temporal visual cortex (IT) shrink to become close to the size of objects. This facilitates the read-out of information from the ventral visual system, because the information is primarily about the object at the fovea. Top-down attentional influences are much less evident in natural scenes than when objects are shown against blank backgrounds, though are still present. It is suggested that the reduced receptive-field size in natural scenes, and the effects of top-down attention contribute to change blindness. The receptive fields of IT neurons in complex scenes, though including the fovea, are frequently asymmetric around the fovea, and it is proposed that this is the solution the IT uses to represent multiple objects and their relative spatial positions in a scene. Networks that implement probabilistic decision-making are described, and it is suggested that, when in perceptual systems they take decisions (or 'test hypotheses'), they influence lower-level networks to bias visual perception. Finally, it is shown that similar processes extend to systems involved in the processing of emotion-provoking sensory stimuli, in that word-level cognitive states provide top-down biasing that reaches as far down as the orbitofrontal cortex, where, at the first stage of affective representations, olfactory, taste, flavour, and touch processing is biased (or pre-empted) in humans.     

Object-based warping: an illusory distortion of space within objects

Visual objects are high-level primitives that are fundamental to numerous perceptual functions, such as guidance of attention. We report that objects warp visual perception of space in such a way that spatial distances within objects appear to be larger than spatial distances in ground regions. When two dots were placed inside a rectangular object, they appeared farther apart from one another than two dots with identical spacing outside of the object. To investigate whether this effect was object based, we measured the distortion while manipulating the structure surrounding the dots. Object displays were constructed with a single object, multiple objects, a partially occluded object, and an illusory object. Nonobject displays were constructed to be comparable to object displays in low-level visual attributes. In all cases, the object displays resulted in a more powerful distortion of spatial perception than comparable non-object-based displays. These results suggest that perception of space within objects is warped.     

The detection of temporally defined objects does not require focused attention

Perceptual grouping is crucial to distinguish objects from their background. Recent studies have shown that observers can detect an object that does not have any unique qualities other than unique temporal properties. A crucial question is whether focused attention is needed for this type of grouping. In two visual search experiments, we show that searching for an object defined by temporal grouping can occur in parallel. These findings suggest that focused attention is not needed for temporal grouping to occur. It is proposed that temporal grouping may occur because the neurons representing the changing object elements adopt firing frequencies that cause the visual system to bind these elements together without the need for focused attention.     

Object-based auditory and visual attention

Theories of visual attention argue that attention operates on perceptual objects, and thus that interactions between object formation and selective attention determine how competing sources interfere with perception. In auditory perception, theories of attention are less mature and no comprehensive framework exists to explain how attention influences perceptual abilities. However, the same principles that govern visual perception can explain many seemingly disparate auditory phenomena. In particular, many recent studies of 'informational masking' can be explained by failures of either auditory object formation or auditory object selection. This similarity suggests that the same neural mechanisms control attention and influence perception across different sensory modalities.     

The detection of temporally defined objects does not require focused attention

Perceptual grouping is crucial to distinguish objects from their background. Recent studies have shown that observers can detect an object that does not have any unique qualities other than unique temporal properties. A crucial question is whether focused attention is needed for this type of grouping. In two visual search experiments, we show that searching for an object defined by temporal grouping can occur in parallel. These findings suggest that focused attention is not needed for temporal grouping to occur. It is proposed that temporal grouping may occur because the neurons representing the changing object elements adopt firing frequencies that cause the visual system to bind these elements together without the need for focused attention.     

Mirror neurons, the representation of word meaning, and the foot of the third left frontal convolution

Previous neuroimaging research has attempted to demonstrate a preferential involvement of the human mirror neuron system (MNS) in the comprehension of effector-related action word (verb) meanings. These studies have assumed that Broca's area (or Brodmann's area 44) is the homologue of a monkey premotor area (F5) containing mouth and hand mirror neurons, and that action word meanings are shared with the mirror system due to a proposed link between speech and gestural communication. In an fMRI experiment, we investigated whether Broca's area shows mirror activity solely for effectors implicated in the MNS. Next, we examined the responses of empirically determined mirror areas during a language perception task comprising effector-specific action words, unrelated words and nonwords. We found overlapping activity for observation and execution of actions with all effectors studied, i.e., including the foot, despite there being no evidence of foot mirror neurons in the monkey or human brain. These "mirror" areas showed equivalent responses for action words, unrelated words and nonwords, with all of these stimuli showing increased responses relative to visual character strings. Our results support alternative explanations attributing mirror activity in Broca's area to covert verbalisation or hierarchical linearisation, and provide no evidence that the MNS makes a preferential contribution to comprehending action word meanings.