Neural dynamics of 3-D surface perception: figure-ground separation and lightness perception

This article develops the FACADE theory of three-dimensional (3-D) vision to simulate data concerning how two-dimensional pictures give rise to 3-D percepts of occluded and occluding surfaces. The theory suggests how geometrical and contrastive properties of an image can either cooperate or compete when forming the boundary and surface representations that subserve conscious visual percepts. Spatially long-range cooperation and short-range competition work together to separate boundaries of occluding figures from their occluded neighbors, thereby providing sensitivity to T-junctions without the need to assume that T-junction "detectors" exist. Both boundary and surface representations of occluded objects may be amodally completed, whereas the surface representations of unoccluded objects become visible through modal processes. Computer simulations include Bregman-Kanizsa figure-ground separation, Kanizsa stratification, and various lightness percepts, including the Münker-White, Benary cross, and checkerboard percepts.     

Does scene context always facilitate retrieval of visual object representations?

An object-to-scene binding hypothesis maintains that visual object representations are stored as part of a larger scene representation or scene context, and that scene context facilitates retrieval of object representations (see, e.g., Hollingworth, Journal of Experimental Psychology: Learning, Memory and Cognition, 32, 58-69, 2006). Support for this hypothesis comes from data using an intentional memory task. In the present study, we examined whether scene context always facilitates retrieval of visual object representations. In two experiments, we investigated whether the scene context facilitates retrieval of object representations, using a new paradigm in which a memory task is appended to a repeated-flicker change detection task. Results indicated that in normal scene viewing, in which many simultaneous objects appear, scene context facilitation of the retrieval of object representations-henceforth termed object-to-scene binding-occurred only when the observer was required to retain much information for a task (i.e., an intentional memory task).     

动、静态视觉信息在真实世界视觉搜索中的作用

真实环境中的视觉搜索是人和动物赖以生存的重要能力。目前的视觉搜索研究多使用静态的观察者和静止的二维搜索对象, 侧重于探究注意在搜索中的作用; 现有的视觉搜索理论模型主要概括了影响搜索的自上而下的注意因素, 而将自下而上影响因素简单归结为影像显著性, 然而在真实环境中, 观察者或搜索对象是可以运动的, 搜索时可利用的视觉信息包括动态光流和静态影像结构信息。已有的视觉识别研究发现这两种信息相结合可以使观察者准确持久地识别场景、事件和三维结构。在现有视觉搜索理论模型中引入两种视觉信息可以较好还原真实环境中的搜索任务。我们提出研究构想和实验方案,探究利用动、静态视觉信息的视觉搜索过程, 从而完善现有的视觉搜索模型。我们认为充分利用环境信息可以提高搜索效率, 且在视觉搜索训练和智能搜索设计等方面有重要的应用价值。     

The effect of spatial and nonspatial contextual information on visual object memory

Recent research has found visual object memory can be stored as part of a larger scene representation rather than independently of scene context. The present study examined how spatial and nonspatial contextual information modulate visual object memory. Two experiments tested participants’ visual memory by using a change detection task in which a target object's orientation was either the same as it appeared during initial viewing or changed. In addition, we examined the effect of spatial and nonspatial contextual manipulations on change detection performance. The results revealed that visual object representations can be maintained reliably after viewing arrays of objects. Moreover, change detection performance was significantly higher when either spatial or nonspatial contextual information remained the same in the test image. We concluded that while processing complex visual stimuli such as object arrays, visual object memory can be stored as part of a comprehensive scene representation, and both spatial and nonspatial contextual changes modulate visual memory retrieval and comparison.     

On the temporal dynamics of language-mediated vision and vision-mediated language

Recent converging evidence suggests that language and vision interact immediately in non-trivial ways, although the exact nature of this interaction is still unclear. Not only does linguistic information influence visual perception in real-time, but visual information also influences language comprehension in real-time. For example, in visual search tasks, incremental spoken delivery of the target features (e.g., “Is there a red vertical?”) can increase the efficiency of conjunction search because only one feature is heard at a time. Moreover, in spoken word recognition tasks, the visual presence of an object whose name is similar to the word being spoken (e.g., a candle present when instructed to “pick up the candy”) can alter the process of comprehension. Dense sampling methods, such as eye-tracking and reach-tracking, richly illustrate the nature of this interaction, providing a semi-continuous measure of the temporal dynamics of individual behavioral responses. We review a variety of studies that demonstrate how these methods are particularly promising in further elucidating the dynamic competition that takes place between underlying linguistic and visual representations in multimodal contexts, and we conclude with a discussion of the consequences that these findings have for theories of embodied cognition.     

Contextual cueing of visual attention

Visual context information constrains what to expect and where to look, facilitating search for and recognition of objects embedded in complex displays. This article reviews a new paradigm called contextual cueing, which presents well-defined, novel visual contexts and aims to understand how contextual information is learned and how it guides the deployment of visual attention. In addition, the contextual cueing task is well suited to the study of the neural substrate of contextual learning. For example, amnesic patients with hippocampal damage are impaired in their learning of novel contextual information, even though learning in the contextual cueing task does not appear to rely on conscious retrieval of contextual memory traces. We argue that contextual information is important because it embodies invariant properties of the visual environment such as stable spatial layout information as well as object covariation information. Sensitivity to these statistical regularities allows us to interact more effectively with the visual world.     

See an object,hear an object file: Object correspondence transcends sensory modality

An important task of perceptual processing is to parse incoming information into distinct units and to keep track of those units over time as the same, persisting representations. Within the study of visual perception, maintaining such persisting object representations is helped by “object files”—episodic representations that store (and update) information about objects' properties and track objects over time and motion via spatiotemporal information. Although object files are typically discussed as visual, here we demonstrate that object–file correspondence can be computed across sensory modalities. An object file can be initially formed with visual input and later accessed with corresponding auditory information, suggesting that object files may be able to operate at a multimodal level of perceptual processing.     

Differential effects of shared attention on perception of heading and 3-D object motion

When a person moves in a straight line through a stationary environment, the images of object surfaces move in a radial pattern away from a single point. This point, known as thefocus of expansion (FOE), corresponds to the person’s direction of motion. People judge their heading from image motion quite well in this situation. They perform most accurately when they can see the region around the FOE, which contains the most useful information for this task. Furthermore, a large moving object in the scene has no effect on observer heading judgments unless it obscures the FOE. Therefore, observers may obtain the most accurate heading judgments by focusing their attention on the region around the FOE. However, in many situations (e.g., driving), the observer must pay attention to other moving objects in the scene (e.g., cars and pedestrians) to avoid collisions. These objects may be located far from the FOE in the visual field. We tested whether people can accurately judge their heading and the three-dimensional (3-D) motion of objects while paying attention to one or the other task. The results show that differential allocation of attention affects people’s ability to judge 3-D object motion much more than it affects their ability to judge heading. This suggests that heading judgments are computed globally, whereas judgments about object motion may require more focused attention.     

Divide and conquer: how object files adapt when a persisting object splits into two

Coherent visual experience requires not only segmenting incoming visual input into a structured scene of objects, but also binding discrete views of objects into dynamic representations that persist across time and motion. However, surprisingly little work has explored the principles that guide the construction and maintenance of such persisting object representations. What causes a part of the visual field to be treated as the same object over time? In the cognitive development literature, a key principle of object persistence is cohesion: An object must always maintain a single bounded contour. Here we demonstrate for the first time that mechanisms of adult midlevel vision are affected by cohesion violations. Using the object-file framework, we tested whether object-specific preview benefits-a hallmark of persisting object representations-are obtained for dynamic objects that split into two during their motion. We found that these preview benefits do not fully persist through such cohesion violations without incurring significant performance costs. These results illustrate how cohesion is employed as a constraint that guides the maintenance of object representations in adult midlevel vision.     

The effect of colour congruency on shape discriminations of novel objects

Although visual object recognition is primarily shape driven, colour assists the recognition of some objects. It is unclear, however, just how colour information is coded with respect to shape in long-term memory and how the availability of colour in the visual image facilitates object recognition. We examined the role of colour in the recognition of novel, 3-D objects by manipulating the congruency of object colour across the study and test phases, using an old/new shape-identification task. In experiment 1, we found that participants were faster at correctly identifying old objects on the basis of shape information when these objects were presented in their original colour, rather than in a different colour. In experiments 2 and 3, we found that participants were faster at correctly identifying old objects on the basis of shape information when these objects were presented with their original part-colour conjunctions, rather than in different or in reversed part-colour conjunctions. In experiment 4, we found that participants were quite poor at the verbal recall of part-colour conjunctions for correctly identified old objects, presented as grey-scale images at test. In experiment 5, we found that participants were significantly slower at correctly identifying old objects when object colour was incongruent across study and test, than when background colour was incongruent across study and test. The results of these experiments suggest that both shape and colour information are stored as part of the long-term representation of these novel objects. Results are discussed in terms of how colour might be coded with respect to shape in stored object representations.     

Differential effects of shared attention on perception of heading and 3-D object motion

When a person moves in a straight line through a stationary environment, the images of object surfaces move in a radial pattern away from a single point. This point, known as the focus of expansion (FOE), corresponds to the person's direction of motion. People judge their heading from image motion quite well in this situation. They perform most accurately when they can see the region around the FOE, which contains the most useful information for this task. Furthermore, a large moving object in the scene has no effect on observer heading judgments unless it obscures the FOE. Therefore, observers may obtain the most accurate heading judgments by focusing their attention on the region around the FOE. However, in many situations (e.g., driving), the observer must pay attention to other moving objects in the scene (e.g., cars and pedestrians) to avoid collisions. These objects may be located far from the FOE in the visual field. We tested whether people can accurately judge their heading and the three-dimensional (3-D) motion of objects while paying attention to one or the other task. The results show that differential allocation of attention affects people's ability to judge 3-D object motion much more than it affects their ability to judge heading. This suggests that heading judgments are computed globally, whereas judgments about object motion may require more focused attention.     

Explicit and implicit memory for rotating objects

Although both the object and the observer often move in natural environments, the effect of motion on visual object recognition ha not been well documented. The authors examined the effect of a reversal in the direction of rotation on both explicit and implicit memory for novel, 3-dimensional objects. Participants viewed a series of continuously rotating objects and later made either an old-new recognition judgment or a symmetric-asymmetric decision. For both tasks, memory for rotating objects was impaired when the direction of rotation was reversed at test. These results demonstrate that dynamic information can play a role in visual object recognition and suggest that object representations can encode spatiotemporal information.     

When do spatial and visual working memory interact?

This study examined how spatial working memory and visual (object) working memory interact, focusing on two related questions: First, can these systems function independently from one another? Second, under what conditions do they operate together? In a dual-task paradigm, participants attempted to remember locations in a spatial working memory task and colored objects in a visual working memory task. Memory for the locations and objects was subject to independent working memory storage limits, which indicates that spatial and visual working memory can function independently from one another. However, additional experiments revealed that spatial working memory and visual working memory interact in three memory contexts: when retaining (1) shapes, (2) integrated color-shape objects, and (3) colored objects at specific locations. These results suggest that spatial working memory is needed to bind colors and shapes into integrated object representations in visual working memory. Further, this study reveals a set of conditions in which spatial and visual working memory can be isolated from one another.     

Learning sequence of views of three-dimensional objects: the effect of temporal coherence on object memory

How humans recognize objects remains a contentious issue in current research on high-level vision. Here, I test the proposal by Wallis and Bülthoff (1999 Trends in Cognitive Sciences 3 22-31) suggesting that object representations can be learned through temporal association of multiple views of the same object. Participants first studied image sequences of novel, three-dimensional objects in a study block. On each trial, the images were from either an orderly sequence of depth-rotated views of the same object (SS), a scrambled sequence of those views (SR), or a sequence of different objects (RR). Recognition memory was assessed in a following test block. A within-object advantage was consistently observed --greater accuracy in the SR than the RR condition in all four experiments, greater accuracy in the SS than the RR condition in two experiments. Furthermore, spatiotemporal coherence did not produce better recognition than temporal coherence alone (similar or less accuracy in the SS compared to the SR condition). These results suggest that the visual system can use temporal regularity to build invariant object representations, via the temporal-association mechanism.     

Object representations maintain attentional control settings across space and time

Previous research has revealed that we create and maintain mental representations for perceived objects on the basis of their spatiotemporal continuity. An important question is what type of information can be maintained within these so-called object files. We provide evidence that object files retain specific attentional control settings for items presented inside the object, even when it disappears from vision. The objects were entire visual search displays consisting of multiple items moving into and out of view. It was demonstrated that search was speeded when the search target position was repeated from trial to trial, but especially so when spatiotemporal continuity suggested that the entire display was the same object. We conclude that complete spatial attentional biases can be stored in an object file.     

Coordinate systems in the long-term memory representation of three-dimensional shapes

Visual information can be stored relative to a particular point of view or independently of any particular point of view. Research on mental rotation has shown that people can store and use viewer-centered visual representations of objects and scenes. Some theories of object recognition posit that object-centered representations are also stored and used to represent the shape of three-dimensional objects. In this paper a series of experiments provides evidence that people can store and use both viewer-centered and object-centered representations of three-dimensional objects.     

The experience of force: the role of haptic experience of forces in visual perception of object motion and interactions, mental simulation, and motion-related judgments

Forces are experienced in actions on objects. The mechanoreceptor system is stimulated by proximal forces in interactions with objects, and experiences of force occur in a context of information yielded by other sensory modalities, principally vision. These experiences are registered and stored as episodic traces in the brain. These stored representations are involved in generating visual impressions of forces and causality in object motion and interactions. Kinematic information provided by vision is matched to kinematic features of stored representations, and the information about forces and causality in those representations then forms part of the perceptual interpretation. I apply this account to the perception of interactions between objects and to motions of objects that do not have perceived external causes, in which motion tends to be perceptually interpreted as biological or internally caused. I also apply it to internal simulations of events involving mental imagery, such as mental rotation, trajectory extrapolation and judgment, visual memory for the location of moving objects, and the learning of perceptual judgments and motor skills. Simulations support more accurate judgments when they represent the underlying dynamics of the event simulated. Mechanoreception gives us whatever limited ability we have to perceive interactions and object motions in terms of forces and resistances; it supports our practical interventions on objects by enabling us to generate simulations that are guided by inferences about forces and resistances, and it helps us learn novel, visually based judgments about object behavior.     

Object-based auditory facilitation of visual search for pictures and words with frequent and rare targets

Auditory and visual processes demonstrably enhance each other based on spatial and temporal coincidence. Our recent results on visual search have shown that auditory signals also enhance visual salience of specific objects based on multimodal experience. For example, we tend to see an object (e.g., a cat) and simultaneously hear its characteristic sound (e.g., “meow”), to name an object when we see it, and to vocalize a word when we read it, but we do not tend to see a word (e.g., cat) and simultaneously hear the characteristic sound (e.g., “meow”) of the named object. If auditory–visual enhancements occur based on this pattern of experiential associations, playing a characteristic sound (e.g., “meow”) should facilitate visual search for the corresponding object (e.g., an image of a cat), hearing a name should facilitate visual search for both the corresponding object and corresponding word, but playing a characteristic sound should not facilitate visual search for the name of the corresponding object. Our present and prior results together confirmed these experiential association predictions. We also recently showed that the underlying object-based auditory–visual interactions occur rapidly (within 220 ms) and guide initial saccades towards target objects. If object-based auditory–visual enhancements are automatic and persistent, an interesting application would be to use characteristic sounds to facilitate visual search when targets are rare, such as during baggage screening. Our participants searched for a gun among other objects when a gun was presented on only 10% of the trials. The search time was speeded when a gun sound was played on every trial (primarily on gun-absent trials); importantly, playing gun sounds facilitated both gun-present and gun-absent responses, suggesting that object-based auditory–visual enhancements persistently increase the detectability of guns rather than simply biasing gun-present responses. Thus, object-based auditory–visual interactions that derive from experiential associations rapidly and persistently increase visual salience of corresponding objects.     

Binding feature dimensions in visual short-term memory

We explored several possible influences on binding in visual short-term memory (VSTM) performance. The task was to report whether a test object was the same (“old” trials) or different (“new” trials) from any of the sample objects seen a second ago. The objects were composed of two features that varied from continuous to discrete shapes and colors. In “old” trials the test object appeared either in the same or different position. In “new” trials the test object differed along both features, requiring storage of only one feature per object; along one feature, requiring no binding but storage of all features; or it was created by recombining features from the sample, which requires binding. Existing storage hypotheses are unable to explain the similar sensitivity (d′) obtained in the two last conditions when position remained the same and may suggest that links are created between positions and features. Highest sensitivity occurred when the test object remained at the same position, required no binding, and discrete features were used. Object-type × position, and feature combination × position interactions occurred, suggesting different storage modes depending on whether objects change position during retention.     

Stored representations of three-dimensional objects in the absence of two-dimensional cues

Object recognition was studied in human subjects to determine whether the storage of the visual objects was in a two-dimensional or a three-dimensional representation. Novel motion-based and disparity-based stimuli were generated in which three-dimensional and two-dimensional form cues could be manipulated independently. Subjects were required to generate internal representations from motion stimuli that lacked explicit two-dimensional cues. These stored internal representations were then matched against internal three-dimensional representations constructed from disparity stimuli. These new stimuli were used to confirm prior studies that indicated the primacy of two-dimensional cues for view-based object storage. However, under tightly controlled conditions for which only three-dimensional cues were available, human subjects were also able to match an internal representation derived from motion of that of disparity. This last finding suggests that there is an internal storage of an object's representations in three dimensions, a tenet that has been rejected by view-based theories. Thus, any complete theory of object recognition that is based on primate vision must incorporate three-dimensional stored representations.