Causal Cognitive Architecture 3: A solution to the binding problem

The binding problem is considered in terms of how a brain-inspired cognitive system can recognize multiple sensory features from an object which may be among many objects, process those features individually and then bind the multiple features to the object they belong to. The Causal Cognitive Architecture 3 (CCA3) is a brain-inspired cognitive architecture using a multi-dimensional navigation map as its basic store of information, and capable of pre-causal as well as fully causal behavior. Objects within an input sensory scene are segmented, and sensory features (e.g., visual, auditory, etc.) of each segmented object are spatially mapped onto a variety of navigation maps. It is shown that to provide efficient, flexible, causal solutions to real-world problems, it is not sufficient to bind space (i.e., objects spatially) but it is necessary to also bind time (i.e., change and rate of change of objects within a sensory scene). The CCA3 binds both space and time onto a navigation map as physical features, and is better able to function in real-world environments. As the CCA3 is brain-inspired, the Causal Cognitive Architecture can help to better hypothesize and understand biological mammalian brain function, including solutions to the binding problem. The CCA3 architecture allows it to work in different knowledge domains, possess continual lifelong learning, and demonstrate reasonable explainability.     

Impossible shadows and the shadow correspondence problem

Shadows cast by objects contain potentially useful information about the location of these objects in the scene as well as their surface reflectance. However, before the visual system can use this information, it has to solve the shadow correspondence problem, that is to match the objects with their respective shadows. In the first experiment, it is shown that the estimate of the light source position is affected by a gradual luminance ramp added to the image. In the second experiment, it is shown that observers process impossible shadow images as if they ignored the local features of the objects. All together, the results suggest that the visual system solves the shadow correspondence problem by relying on a coarse representation of the scene.     

视觉工作记忆中的特征捆绑

特征捆绑是认知科学和神经科学研究的前沿问题,近来已经成为研究者们关于意识问题争论的焦点所在。很多心理活动涉及复杂对象各种特征的捆绑,在工作记忆中保持这些捆绑的机制是心理加工得以有效进行的基础。视觉工作记忆中的特征捆绑近来成为研究热点之一,有关研究大多凭借变化检测范式和单刺激探测范式,探讨了视觉工作记忆中捆绑特征的保持是否需要注意参与,以及分离特征存储和捆绑特征存储的关系等问题,但目前这些问题仍都没有明确的答案。     

Integrating faces, houses, motion, and action: spontaneous binding across ventral and dorsal processing streams

Perceiving an event requires the integration of its features across numerous brain maps and modules. Visual object perception is thought to be mediated by a ventral processing stream running from occipital to inferotemporal cortex, whereas most spatial processing and action control is attributed to the dorsal stream connecting occipital, parietal, and frontal cortex. Here we show that integration operates not only on ventral features and objects, such as faces and houses, but also across ventral and dorsal pathways, binding faces and houses to motion and manual action. Furthermore, these bindings seem to persist over time, as they influenced performance on future task-relevant visual stimuli. This is reflected by longer reaction times for repeating one, but alternating other features in a sequence, compared to complete repetition or alternation of features. Our findings are inconsistent with the notion that the dorsal stream is operating exclusively online and has no access to memory.     

Between-object binding and visual attention

Many previous studies have found that we can attend pairs of visual features (e.g., colour, orientation) more efficiently when they belong to the same “object” compared to when they belong to separate, neighbouring objects (e.g., Behrmann, Zemel, & Mozer, 1998; Egly, Rafal, & Driver, 1994). This advantage for attending features from the same object may reflect stronger binding between these features than arises for pairs of features belonging to separate objects. However, recent findings described by Davis, Welch, Holmes, and Shepherd (in press) suggest that under specific conditions this same-object advantage can be reversed, such that attention now spreads more readily between features belonging to separate neighbouring objects than between features of the same object. In such cases it would appear that features belonging to separate visual objects are more strongly bound than features of the same object. Here I review these findings and present the results of a new study. Together these data suggest that magnocellular processes in the human visual system bind together features from separate objects, whereas parvocellular processes bind together features from the same object.     

Storage of features, conjunctions and objects in visual working memory

Working memory can be divided into separate subsystems for verbal and visual information. Although the verbal system has been well characterized, the storage capacity of visual working memory has not yet been established for simple features or for conjunctions of features. The authors demonstrate that it is possible to retain information about only 3-4 colors or orientations in visual working memory at one time. Observers are also able to retain both the color and the orientation of 3-4 objects, indicating that visual working memory stores integrated objects rather than individual features. Indeed, objects defined by a conjunction of four features can be retained in working memory just as well as single-feature objects, allowing many individual features to be retained when distributed across a small number of objects. Thus, the capacity of visual working memory must be understood in terms of integrated objects rather than individual features.     

Visual search, visual streams, and visual architectures.

Most psychological, physiological, and computational models of early vision suggest that retinal information is divided into a parallel set of feature modules. The dominant theories of visual search assume that these modules form a "blackboard" architecture: a set of independent representations that communicate only through a central processor. A review of research shows that blackboard-based theories, such as feature-integration theory, cannot easily explain the existing data. The experimental evidence is more consistent with a "network" architecture, which stresses that: (1) feature modules are directly connected to one another, (2) features and their locations are represented together, (3) feature detection and integration are not distinct processing stages, and (4) no executive control process, such as focal attention, is needed to integrate features. Attention is not a spotlight that synthesizes objects from raw features. Instead, it is better to conceptualize attention as an aperture which masks irrelevant visual information.     

Visual search,visual streams,and visual architectures

Most psychological, physiological, and computational models of early vision suggest that retinal information is divided into a parallel set of feature modules. The dominant theories of visual search assume that these modules form a “blackboard“ architecture: a set of independent representations that communicate only through a central processor. A review of research shows that blackboard-based theories, such as feature-integration theory, cannot easily explain the existing data. The experimental evidence is more consistent with a “network” architecture, which stresses that: (1) feature modules are directly connected to one another, (2)features and their locations are represented together, (3) feature detection and integration are not distinct processing stages, and (4) no executive control process, such as focal attention, is needed to integrate features. Attention is not a spotlight that synthesizes objects from raw features. Instead, it is better to conceptualize attention as an aperture which masks irrelevant visual information.     

Crossmodal correspondences: A tutorial review

In many everyday situations, our senses are bombarded by many different unisensory signals at any given time. To gain the most veridical, and least variable, estimate of environmental stimuli/properties, we need to combine the individual noisy unisensory perceptual estimates that refer to the same object, while keeping those estimates belonging to different objects or events separate. How, though, does the brain “know” which stimuli to combine? Traditionally, researchers interested in the crossmodal binding problem have focused on the roles that spatial and temporal factors play in modulating multisensory integration. However, crossmodal correspondences between various unisensory features (such as between auditory pitch and visual size) may provide yet another important means of constraining the crossmodal binding problem. A large body of research now shows that people exhibit consistent crossmodal correspondences between many stimulus features in different sensory modalities. For example, people consistently match high-pitched sounds with small, bright objects that are located high up in space. The literature reviewed here supports the view that crossmodal correspondences need to be considered alongside semantic and spatiotemporal congruency, among the key constraints that help our brains solve the crossmodal binding problem.     

A computational framework for attentional object discovery in RGB-D videos

We present a computational framework for attention-guided visual scene exploration in sequences of RGB-D data. For this, we propose a visual object candidate generation method to produce object hypotheses about the objects in the scene. An attention system is used to prioritise the processing of visual information by (1) localising candidate objects, and (2) integrating an inhibition of return (IOR) mechanism grounded in spatial coordinates. This spatial IOR mechanism naturally copes with camera motions and inhibits objects that have already been the target of attention. Our approach provides object candidates which can be processed by higher cognitive modules such as object recognition. Since objects are basic elements for many higher level tasks, our architecture can be used as a first layer in any cognitive system that aims at interpreting a stream of images. We show in the evaluation how our framework finds most of the objects in challenging real-world scenes.     

What Can Spatial Deficits Teach Us About Feature Binding and Spatial Maps?

The “binding problem” is discussed with reference to feature integration and visual search. Neuropsychological and neurophysiological findings support the existence of multiple visual areas within the primate cortex that respond to primary features of objects and spatial locations. Evidence from studies with hemineglect and Balint's syndrome supports the role of spatial attention in feature integration and demonstrates the necessity for an intact, explicitspatial representation. Some implicit spatial maps remain intact but are not sufficient to support the perception of properly bound features. The evidence suggests strong interactions between parietal spatial representations and temporal feature representations in feature integration.     

Binding in short-term visual memory

The integration of complex information in working memory, and its effect on capacity, shape the limits of conscious cognition. The literature conflicts on whether short-term visual memory represents information as integrated objects. A change-detection paradigm using objects defined by color with location or shape was used to investigate binding in short-term visual memory. Results showed that features from the same dimension compete for capacity, whereas features from different dimensions can be stored in parallel. Binding between these features can occur, but focused attention is required to create and maintain the binding over time, and this integrated format is vulnerable to interference. In the proposed model, working memory capacity is limited both by the independent capacity of simple feature stores and by demands on attention networks that integrate this distributed information into complex but unified thought objects.     

Binding across space and time in visual working memory

Recent studies of visual short-term memory have suggested that the binding of features such as color and shape into remembered objects is relatively automatic. A series of seven experiments broadened this investigation by comparing the immediate retention of colored shapes with performance when color and shape were separated either spatially or temporally, with participants required actively to form the bound object. Attentional load was manipulated with a demanding concurrent task, and retention in working memory was then tested using a single recognition probe. Both spatial and temporal separation of features tended to impair performance, as did the concurrent task. There was, however, no evidence for greater attentional disruption of performance as a result of either spatial or temporal separation of features. Implications for the process of binding in visual working memory are discussed, and an interpretation is offered in terms of the episodic buffer component of working memory, which is assumed to be a passive store capable of holding bound objects, but not of performing the binding.     

Visual working memory for dynamic objects: Impaired binding between object feature and location

In daily life, people frequently need to observe dynamic objects and temporarily maintain their representations in visual working memory (VWM). The present study explored the mechanism underlying the binding between perceptual features and locations of dynamic objects in VWM. In three experiments, we measured and compared the memory performance for feature-location binding of multiple dynamic and static objects. The results showed that the feature-location binding was impaired for the dynamic objects compared with the static objects. The impairment persisted when the global spatial configuration of the objects remained intact during the motion, as well as when the binding task was relatively easy, such as binding between single-feature objects and coarse locations. The results indicate that object features and locations are not maintained in VWM as well-integrated object files; rather, the formation of feature-location binding may require additional processes, which are disrupted by the constant change of locations in dynamic circumstances. We propose a consolidation process as possible underlying mechanism, and discuss factors that may influence the strength of feature-location binding in dynamic circumstances.     

视觉工作记忆中特征绑定关系的记忆机制

弱客体理论认为, 视觉工作记忆系统是由众多特征存储的子系统组成, 不同纬度的特征独立存储在相应的有限容量的子系统中, 相互之间不会竞争记忆资源。而对于特征之间绑定关系的记忆存在不同的观点。一种观点认为绑定关系的记忆需要注意维持, 因此绑定关系的记忆会占用到特征记忆的资源; 另一种观点认为绑定关系的记忆是自动发生的, 不需要注意的维持。本实验的目的是探究特征绑定关系的记忆是否是自动发生的。在实验中, 我们设计了两种任务, 一种任务是只记忆颜色, 另一种任务是记忆颜色跟位置及其之间的绑定关系, 并测试分析视觉工作记忆相关的ERP成分CDA。结果发现两种任务条件下的CDA波幅之间没有显著差异, 说明视觉工作记忆中的特征绑定是自动发生的。     

视觉工作记忆中的储存单位--特征还是客体?

视觉工作记忆中存储的基本单位是客体还是特征,目前尚无定论。本实验试图进一步探索这一问题。被试为30名大学生。实验1的条件是:颜色特征数不同,客体总数相同。实验2的条件是:颜色特征数相同,客体总数不同。结果表明:在记忆由同一属性的特征组成的客体时,视觉工作记忆存储的基本单位是特征。     

Visual representation in analogical problem solving

Analogical reasoning has been shown to be effective in the process of solving Dunker’s radiation problem. The spatial nature of the solution to this problem suggests that a visually represented analogue should be particularly effective. However, recent work seems to indicate that a visual analogue does not assist in solving the radiation problem. This paper reports a detailed experimental analysis of the effectiveness of visually represented analogues to the radiation problem. The results show that visual analogues can be effective ff they represent the appropriate features of the problem-solution relationship. The paper also reports on the use of an appropriate visual representation within the problem as a facilitator of analogical reasoning. The results indicate that a visual representation within the problem can act as a facilitator of analogical reasoning, possibly by acting as a retrieval cue.     

The privileged role of location in visual working memory

Reports have conflicted about the possible special role of location in visual working memory (WM). One important question is: Do we maintain the locations of objects in WM even when they are irrelevant to the task at hand? Here we used a continuous response scale to study the types of reporting errors that participants make when objects are presented at the same or at different locations in space. When several objects successively shared the same location, participants exhibited a higher tendency to report features of the wrong object in memory; that is, they responded with features that belonged to objects retained in memory but not probed at retrieval. On the other hand, a similar effect was not observed when objects shared a nonspatial feature, such as color. Furthermore, the effect of location on reporting errors was present even when its manipulation was orthogonal to the task at hand. These findings are consistent with the view that binding together different nonspatial features of an object in memory might be mediated through an object’s location. Hence, spatial location may have a privileged role in WM. The relevance of these findings to conceptual models, as well as to neural accounts of visual WM, is discussed.     

Visual development in human infants: Binding features,surfaces, and objects

The development of visual binding in humans has been investigated with psychophysical tasks assessing the extent to which young infants achieve perceptual completion of partly occluded objects. These experiments lead to two conclusions. First, neonates are capable of figure-ground segregation, but do not perceive the unity of a centre-occluded object; the ability to perceive object unity emerges over the first several postnatal months. Second, by 4 months, infants rely on a range of Gestalt visual information in perceiving unity, including common motion, alignment, and good form. This developmental pattern is thought to be built on the ability to detect, and then utilize, appropriate visual information in support of the binding of features into surfaces and objects. Evidence from changes in infant attention, computational modelling, and developmental neurophysiology is cited that is consistent with this view.