Neural dynamics of object-based multifocal visual spatial attention and priming: object cueing, useful-field-of-view, and crowding

How are spatial and object attention coordinated to achieve rapid object learning and recognition during eye movement search? How do prefrontal priming and parietal spatial mechanisms interact to determine the reaction time costs of intra-object attention shifts, inter-object attention shifts, and shifts between visible objects and covertly cued locations? What factors underlie individual differences in the timing and frequency of such attentional shifts? How do transient and sustained spatial attentional mechanisms work and interact? How can volition, mediated via the basal ganglia, influence the span of spatial attention? A neural model is developed of how spatial attention in the where cortical stream coordinates view-invariant object category learning in the what cortical stream under free viewing conditions. The model simulates psychological data about the dynamics of covert attention priming and switching requiring multifocal attention without eye movements. The model predicts how “attentional shrouds” are formed when surface representations in cortical area V4 resonate with spatial attention in posterior parietal cortex (PPC) and prefrontal cortex (PFC), while shrouds compete among themselves for dominance. Winning shrouds support invariant object category learning, and active surface-shroud resonances support conscious surface perception and recognition. Attentive competition between multiple objects and cues simulates reaction-time data from the two-object cueing paradigm. The relative strength of sustained surface-driven and fast-transient motion-driven spatial attention controls individual differences in reaction time for invalid cues. Competition between surface-driven attentional shrouds controls individual differences in detection rate of peripheral targets in useful-field-of-view tasks. The model proposes how the strength of competition can be mediated, though learning or momentary changes in volition, by the basal ganglia. A new explanation of crowding shows how the cortical magnification factor, among other variables, can cause multiple object surfaces to share a single surface-shroud resonance, thereby preventing recognition of the individual objects.     

Cortical dynamics of contextually cued attentive visual learning and search: spatial and object evidence accumulation

How do humans use target-predictive contextual information to facilitate visual search? How are consistently paired scenic objects and positions learned and used to more efficiently guide search in familiar scenes? For example, humans can learn that a certain combination of objects may define a context for a kitchen and trigger a more efficient search for a typical object, such as a sink, in that context. The ARTSCENE Search model is developed to illustrate the neural mechanisms of such memory-based context learning and guidance and to explain challenging behavioral data on positive-negative, spatial-object, and local-distant cueing effects during visual search, as well as related neuroanatomical, neurophysiological, and neuroimaging data. The model proposes how global scene layout at a first glance rapidly forms a hypothesis about the target location. This hypothesis is then incrementally refined as a scene is scanned with saccadic eye movements. The model simulates the interactive dynamics of object and spatial contextual cueing and attention in the cortical What and Where streams starting from early visual areas through medial temporal lobe to prefrontal cortex. After learning, model dorsolateral prefrontal cortex (area 46) primes possible target locations in posterior parietal cortex based on goal-modulated percepts of spatial scene gist that are represented in parahippocampal cortex. Model ventral prefrontal cortex (area 47/12) primes possible target identities in inferior temporal cortex based on the history of viewed objects represented in perirhinal cortex.     

Visual working memory retains movement information within an allocentric reference frame

What frame of reference do we use to remember observed movements? One possibility is that visual working memory (VWM) retains movement information using a retinotopic frame of reference: A coordinate system with respect to the retina that retains view-dependent information. Alternatively, VWM might retain movement information using an allocentric frame of reference: A coordinate system with respect to the scene that retains view-invariant information. To address this question, I examined whether VWM retains view-dependent or view-invariant movement information. Results show that (1) observers have considerable difficulty remembering from which viewpoints they observed movements after a few seconds' delay, and (2) the same number of movements can be retained in VWM whether the movements are encoded and tested from the same viewpoint or from different viewpoints. Thus, movement representations contain little to no view-dependent information, which suggests that VWM uses an allocentric reference frame to retain movement information.     

Infants' emerging ability to represent occluded object motion

The emerging ability to represent an oscillating moving object over occlusions was studied in 7-21-week-old infants. The object moved at 0.25 Hz and was either occluded at the center of the trajectory (for 0.3 s) or at one turning point (for 0.7 s). Each trial lasted for 20 s. Both eye and head movements were measured. By using two kinds of motion, sinusoidal (varying velocity) and triangular (constant velocity), infants' ability to take velocity change into account when predicting the reappearance of the moving object was tested. Over the age period studied, performance at the central occluder progressed from almost total ignorance of what happened to consistent predictive behavior. From around 12 weeks of age, infants began to form representations of the moving object that persisted over temporary occlusions. At around 5 months of age these representations began to incorporate the dynamics of the represented motion. Strong learning effects were obtained over single trials, but there was no evidence of retention between trials. The individual differences were profound.     

The effect of spatial and nonspatial contextual information on visual object memory

Recent research has found visual object memory can be stored as part of a larger scene representation rather than independently of scene context. The present study examined how spatial and nonspatial contextual information modulate visual object memory. Two experiments tested participants’ visual memory by using a change detection task in which a target object's orientation was either the same as it appeared during initial viewing or changed. In addition, we examined the effect of spatial and nonspatial contextual manipulations on change detection performance. The results revealed that visual object representations can be maintained reliably after viewing arrays of objects. Moreover, change detection performance was significantly higher when either spatial or nonspatial contextual information remained the same in the test image. We concluded that while processing complex visual stimuli such as object arrays, visual object memory can be stored as part of a comprehensive scene representation, and both spatial and nonspatial contextual changes modulate visual memory retrieval and comparison.     

Subliminal access to abstract face representations does not rely on attention

The present study used masked repetition priming to examine whether face representations can be accessed without attention. Two experiments using a face recognition task (fame judgement) presented masked repetition and control primes in spatially unattended locations prior to target onset. Experiment 1 (n=20) used the same images as primes and as targets and Experiment 2 (n=17) used different images of the same individual as primes and targets. Repetition priming was observed across both experiments regardless of whether spatial attention was cued to the location of the prime. Priming occurred for both famous and non-famous targets in Experiment 1 but was only reliable for famous targets in Experiment 2, suggesting that priming in Experiment 1 indexed access to view-specific representations whereas priming in Experiment 2 indexed access to view-invariant, abstract representations. Overall, the results indicate that subliminal access to abstract face representations does not rely on attention.     

Attention to distinguishing features in object recognition

This study advances the hypothesis that, in the course of object recognition, attention is directed to distinguishing features: visual information that is diagnostic of object identity in a specific context. In five experiments, observers performed an object categorization task involving drawings of fish (Experiments 1–4) and photographs of natural sea animals (Experiment 5). Allocation of attention to distinguishing and non-distinguishing features was examined using primed-matching (Experiment 1) and visual probe (Experiments 2, 4, 5) methods, and manipulated by spatial precuing (Experiment 3). Converging results indicated that in performing the object categorization task, attention was allocated to the distinguishing features in a context-dependent manner, and that such allocation facilitated performance. Based on the view that object recognition, like categorization, is essentially a process of discrimination between probable alternatives, the implications of the findings for the role of attention to distinguishing features in object recognition are discussed.     

Attending to the parts of a single object: part-based selection limitations

Studies of object-based attention have demonstrated poorer performance in dividing attention between two objects in a scene than in focusing attention on a single object. However, objects often are composed of several parts, and parts are central to theories of object recognition. Are parts also important for visual attention? That is, can attention be limited in the number of parts processed simultaneously? We addressed this question in four experiments. In Experiments 1 and 2, participants reported two attributes that appeared on the same part or on different parts of a single multipart object. Participants were more accurate in reporting the attributes on the same part than attributes on different parts. This part-based effect was not influenced by the spatial distance between the parts, ruling out a simple spatial attention interpretation of our results. A control study demonstrated that our spatial manipulation was sufficient to observe shifts of spatial attention. This study revealed an effect of spatial distance, indicating that our spatial manipulation was adequate for observing spatial attention. The absence of a distance effect in Experiments 1 and 2 suggests that part-based attention may not rely entirely on simple shifts of spatial attention. Finally, in Experiment 4 we found evidence for part-based attention, using stimuli controlled for the distance between the parts of an object. The results of these experiments indicate that visual attention can selectively process the parts of an object. We discuss the relationship between parts and objects and the locus of part-based attentional selection.     

Is a Pink Cow Still a Cow? Individual Differences in Toddlers' Vocabulary Knowledge and Lexical Representations

When a toddler knows a word, what does she actually know? Many categories have multiple relevant properties; for example, shape and color are relevant to membership in the category banana . How do toddlers prioritize these properties when recognizing familiar words, and are there systematic differences among children? In this study, toddlers viewed pairs of objects associated with prototypical colors. On some trials, objects were typically colored (e.g., Holstein cow and pink pig); on other trials, colors were switched (e.g., pink cow and Holstein‐patterned pig). On each trial, toddlers were directed to find a target object. Overall, recognition was disrupted when colors were switched, as measured by eye movements. Moreover, individual differences in vocabularies predicted recognition differences: Toddlers who say fewer shape‐based words were more disrupted by color switches. “Knowing” a word may not mean the same thing for all toddlers; different toddlers prioritize different facets of familiar objects in their lexical representations.     

Initial scene representations facilitate eye movement guidance in visual search

What role does the initial glimpse of a scene play in subsequent eye movement guidance? In 4 experiments, a brief scene preview was followed by object search through the scene via a small moving window that was tied to fixation position. Experiment 1 demonstrated that the scene preview resulted in more efficient eye movements compared with a control preview. Experiments 2 and 3 showed that this scene preview benefit was not due to the conceptual category of the scene or identification of the target object in the preview. Experiment 4 demonstrated that the scene preview benefit was unaffected by changing the size of the scene from preview to search. Taken together, the results suggest that an abstract (size invariant) visual representation is generated in an initial scene glimpse and that this representation can be retained in memory and used to guide subsequent eye movements.     

The Development of Invariant Object Recognition Requires Visual Experience With Temporally Smooth Objects

How do newborns learn to recognize objects? According to temporal learning models in computational neuroscience, the brain constructs object representations by extracting smoothly changing features from the environment. To date, however, it is unknown whether newborns depend on smoothly changing features to build invariant object representations. Here, we used an automated controlled‐rearing method to examine whether visual experience with smoothly changing features facilitates the development of view‐invariant object recognition in a newborn animal model—the domestic chick (Gallus gallus). When newborn chicks were reared with a virtual object that moved smoothly over time, the chicks created view‐invariant representations that were selective for object identity and tolerant to viewpoint changes. Conversely, when newborn chicks were reared with a temporally non‐smooth object, the chicks developed less selectivity for identity features and less tolerance to viewpoint changes. These results provide evidence for a “smoothness constraint” on the development of invariant object recognition and indicate that newborns leverage the temporal smoothness of natural visual environments to build abstract mental models of objects.     

An integrated model of eye movements and visual encoding

Recent computational models of cognition have made good progress in accounting for the visual processes needed to encode external stimuli. However, these models typically incorporate simplified models of visual processing that assume a constant encoding time for all visual objects and do not distinguish between eye movements and shifts of attention. This paper presents a domain-independent computational model, EMMA, that provides a more rigorous account of eye movements and visual encoding and their interaction with a cognitive processor. The visual-encoding component of the model describes the effects of frequency and foveal eccentricity when encoding visual objects as internal representations. The eye-movement component describes the temporal and spatial characteristics of eye movements as they arise from shifts of visual attention. When integrated with a cognitive model, EMMA generates quantitative predictions concerning when and where the eyes move, thus serving to relate higher-level cognitive processes and attention shifts with lower-level eye-movement behavior. The paper evaluates EMMA in three illustrative domains — equation solving, reading, and visual search — and demonstrates how the model accounts for aspects of behavior that simpler models of cognitive and visual processing fail to explain.     

How category learning affects object representations: not all morphspaces stretch alike

How does learning to categorize objects affect how people visually perceive them? Behavioral, neurophysiological, and neuroimaging studies have tested the degree to which category learning influences object representations, with conflicting results. Some studies have found that objects become more visually discriminable along dimensions relevant to previously learned categories, while others have found no such effect. One critical factor we explore here lies in the structure of the morphspaces used in different studies. Studies finding no increase in discriminability often use blended morphspaces, with morphparents lying at corners of the space. By contrast, studies finding increases in discriminability use factorial morphspaces, defined by separate morphlines forming axes of the space. Using the same 4 morphparents, we created both factorial and blended morphspaces matched in pairwise discriminability. Category learning caused a selective increase in discriminability along the relevant dimension of the factorial space, but not in the blended space, and led to the creation of functional dimensions in the factorial space, but not in the blended space. These findings demonstrate that not all morphspaces stretch alike: Only some morphspaces support enhanced discriminability to relevant object dimensions following category learning. Our results have important implications for interpreting neuroimaging studies reporting little or no effect of category learning on object representations in the visual system: Those studies may have been limited by their use of blended morphspaces.     

Identification of objects in scenes: the role of scene background in object naming.

Research with brief presentations of scenes has indicated that scene context facilitates object identification. In the present experiments we used a paradigm in which an object in a scene is "wiggled"--drawing both attention and an eye fixation to itself--and then named. Thus the effect of scene context on object identification can be examined in a situation in which the target object is fixated and hence is fully visible. Experiment 1 indicated that a scene background that was episodically consistent with a target object facilitated the speed of naming. In Experiments 2 and 3, we investigated the time course of scene background information acquisition using display changes contingent on eye movements to the target object. The results from Experiment 2 were inconclusive; however, Experiment 3 demonstrated that scene background information present only on either the first or second fixation on a scene significantly affected naming time. Thus background information appears to be both extracted and able to affect object identification continuously during scene viewing.     

Global transsaccadic change blindness during scene perception

Each time the eyes are spatially reoriented via a saccadic eye movement, the image falling on the retina changes. How visually specific are the representations that are functional across saccades during active scene perception? This question was investigated with a saccade-contingent display-change paradigm in which pictures of complex real-world scenes were globally changed in real time during eye movements. The global changes were effected by presenting each scene as an alternating set of scene strips and occluding gray bars, and by reversing the strips and bars during specific saccades. The results from two experiments demonstrated a global transsaccadic change-blindness effect, suggesting that point-by-point visual representations are not functional across saccades during complex scene perception.     

Object identification in context: the visual processing of natural scenes.

When we view a natural visual scene, we seem able to determine effortlessly the scene's semantic category, constituent objects, and spatial relations. How do we accomplish this visual-cognitive feat? The commonly held explanation is known as the schema hypothesis, according to which a visual scene is rapidly identified as a member of a semantic category, and predictions generated from the scene category are then used to aid subsequent object identification. In this paper I will first outline and offer a critique of the evidence that has been taken to support the schema hypothesis. I will then offer an alternative framework for understanding scene processing, which I will call the local-processing hypothesis. This hypothesis assumes a modular, informationally-encapsulated architecture, and explicitly includes the role of covert visual attention in scene processing.     

Influence of semantic consistency and perceptual features on visual attention during scene viewing in toddlers

Conceptual representations of everyday scenes are built in interaction with visual environment and these representations guide our visual attention. Perceptual features and object-scene semantic consistency have been found to attract our attention during scene exploration. The present study examined how visual attention in 24-month-old toddlers is attracted by semantic violations and how perceptual features (i. e. saliency, centre distance, clutter and object size) and linguistic properties (i. e. object label frequency and label length) affect gaze distribution. We compared eye movements of 24-month-old toddlers and adults while exploring everyday scenes which either contained an inconsistent (e.g., soap on a breakfast table) or consistent (e.g., soap in a bathroom) object. Perceptual features such as saliency, centre distance and clutter of the scene affected looking times in the toddler group during the whole viewing time whereas looking times in adults were affected only by centre distance during the early viewing time. Adults looked longer to inconsistent than consistent objects either if the objects had a high or a low saliency. In contrast, toddlers presented semantic consistency effect only when objects were highly salient. Additionally, toddlers with lower vocabulary skills looked longer to inconsistent objects while toddlers with higher vocabulary skills look equally long to both consistent and inconsistent objects. Our results indicate that 24-month-old children use scene context to guide visual attention when exploring the visual environment. However, perceptual features have a stronger influence in eye movement guidance in toddlers than in adults. Our results also indicate that language skills influence cognitive but not perceptual guidance of eye movements during scene perception in toddlers.     

4.5-month-old infants’ learning, retention and use of object boundary information

4.5-month-old infants can use information learned from prior experience with objects to help determine the boundaries of objects in a complex visual scene (Needham, 1998; Needham, Dueker, & Lockhead, 2002). The present studies investigate the effect of delay (between prior experience and test) on infant use of such experiential knowledge. Results indicate that infants can use experience with an object to help them to parse a scene containing that object 24 (Experiment 1). Experiment 2 suggests that after 24 h infants have begun to forget some object attributes, and that this forgetting promotes generalization from one similar object to another. After a 72-h delay, infants did not show any beneficial effect of prior experience with one of the objects in the scene (Experiments 3A and B). However, prior experience with multiple objects, similar to an object in the scene, facilitated infant segregation of the scene 72 h later, suggesting that category information remains available in infant memory longer than experience with a single object. The results are discussed in terms of optimal infant benefit from prior experiences with objects.     

Object-based eye movements: The eyes prefer to stay within the same object

The present study addressed the question of whether we prefer to make eye movements within or between objects. More specifically, when fixating one end of an object, are we more likely to make the next saccade within that same object or to another object? Observers had to discriminate small letters placed on rectangles similar to those used by Egly, Driver, and Rafal (1994). Following an exogenous cue, observers made a saccade to one end of one of the rectangles. The small target letter, which could be discriminated only after it had been fixated, could appear either within the same or at a different object. Consistent with object-based attention, we show that observers prefer to make an eye movement to the other end of the fixated same object, rather than to the equidistant end of a different object. It is concluded that there is a preference to make eye shifts within the same object, rather than between objects.     

Content,not context,facilitates memory for real-world scenes

ABSTRACT

Can eye movements tell us whether people will remember a scene? In order to investigate the link between eye movements and memory encoding and retrieval, we asked participants to study photographs of real-world scenes while their eye movements were being tracked. We found eye gaze patterns during study to be predictive of subsequent memory for scenes. Moreover, gaze patterns during study were more similar to gaze patterns during test for remembered than for forgotten scenes. Thus, eye movements are indeed indicative of scene memory. In an explicit test for context effects of eye movements on memory, we found recognition rate to be unaffected by the disruption of spatial and/or temporal context of repeated eye movements. Therefore, we conclude that eye movements cue memory by selecting and accessing the most relevant scene content, regardless of its spatial location within the scene or the order in which it was selected.