Visual working memory for dynamic objects: Impaired binding between object feature and location

In daily life, people frequently need to observe dynamic objects and temporarily maintain their representations in visual working memory (VWM). The present study explored the mechanism underlying the binding between perceptual features and locations of dynamic objects in VWM. In three experiments, we measured and compared the memory performance for feature-location binding of multiple dynamic and static objects. The results showed that the feature-location binding was impaired for the dynamic objects compared with the static objects. The impairment persisted when the global spatial configuration of the objects remained intact during the motion, as well as when the binding task was relatively easy, such as binding between single-feature objects and coarse locations. The results indicate that object features and locations are not maintained in VWM as well-integrated object files; rather, the formation of feature-location binding may require additional processes, which are disrupted by the constant change of locations in dynamic circumstances. We propose a consolidation process as possible underlying mechanism, and discuss factors that may influence the strength of feature-location binding in dynamic circumstances.     

Does maintaining bindings in visual working memory require more attention than maintaining features?

ABSTRACT

Whether more attention is required for maintaining bindings than maintaining features in visual working memory (VWM) remains an open question. If maintaining bindings in VWM does not require more attention than maintaining features, is it related to the stability of binding representations? In this study, we explored whether maintaining bindings requires more attention than maintaining features for similar and dissimilar objects by inserting a feature report task into the maintenance phase of VWM in Experiments 1 and 2. We also investigated whether the effect of similarity on the attentional requirement for maintaining bindings and features is due to the stability of VWM representations by inserting a suffix during the maintenance phase of VWM in Experiment 3. The results showed that when object-based attention was consumed, bindings were more impaired than features for dissimilar objects but not for similar objects. We also found that the bindings of similar objects were less interfered by the suffix than those of dissimilar objects. Our findings suggest that maintaining bindings does not require more attention than maintaining features when the binding representations are stable in VWM and similarity improves the stability of binding representations.     

Asymmetrical access to color and location in visual working memory

Models of visual working memory (VWM) have benefitted greatly from the use of the delayed-matching paradigm. However, in this task, the ability to recall a probed feature is confounded with the ability to maintain the proper binding between the feature that is to be reported and the feature (typically location) that is used to cue a particular item for report. Given that location is typically used as a cue-feature, we used the delayed-estimation paradigm to compare memory for location to memory for color, rotating which feature was used as a cue and which was reported. Our results revealed several novel findings: 1) the likelihood of reporting a probed object’s feature was superior when reporting location with a color cue than when reporting color with a location cue; 2) location report errors were composed entirely of swap errors, with little to no random location reports; and 3) both colour and location reports greatly benefitted from the presence of nonprobed items at test. This last finding suggests that it is uncertainty over the bindings between locations and colors at memory retrieval that drive swap errors, not at encoding. We interpret our findings as consistent with a representational architecture that nests remembered object features within remembered locations.     

客体在视觉工作记忆中的存储机制

采用变化觉察范式,通过两个实验对不同类型视觉信息在工作记忆中的存储机制进行了探讨。实验一中要求记忆的客体由颜色和形状两基本特征构成;实验二中的客体为具有不同颜色及开口朝向的兰道环。通过观察检测项变化方式对记忆绩效的影响,获得以下结论:（1）对颜色和形状两基本特征的存储以整合客体为单位;（2）兰道环的颜色与开口朝向信息难以以整合客体方式存储。笔者推测,视觉工作记忆中存储的整合客体并非如现有理论假设的,是在集中注意参与下创建的,而是由并行加工阶段所获信息构成;客体中所含需集中注意提取的细节特征信息难以与其基本特征信息整合于该客体表征     

Encoding strategies in self-initiated visual working memory

During a typical day, visual working memory (VWM) is recruited to temporarily maintain visual information. Although individuals often memorize external visual information provided to them, on many other occasions they memorize information they have constructed themselves. The latter aspect of memory, which we term self-initiated WM, is prevalent in everyday behavior but has largely been overlooked in the research literature. In the present study we employed a modified change detection task in which participants constructed the displays they memorized, by selecting three or four abstract shapes or real-world objects and placing them at three or four locations in a circular display of eight locations. Half of the trials included identical targets that participants could select. The results demonstrated consistent strategies across participants. To enhance memory performance, participants reported selecting abstract shapes they could verbalize, but they preferred real-world objects with distinct visual features. Furthermore, participants constructed structured memory displays, most frequently based on the Gestalt organization cue of symmetry, and to a lesser extent on cues of proximity and similarity. When identical items were selected, participants mostly placed them in close proximity, demonstrating the construction of configurations based on the interaction between several Gestalt cues. The present results are consistent with recent findings in VWM, showing that memory for visual displays based on Gestalt organization cues can benefit VWM, suggesting that individuals have access to metacognitive knowledge on the benefit of structure in VWM. More generally, this study demonstrates how individuals interact with the world by actively structuring their surroundings to enhance performance.     

The privileged role of location in visual working memory

Reports have conflicted about the possible special role of location in visual working memory (WM). One important question is: Do we maintain the locations of objects in WM even when they are irrelevant to the task at hand? Here we used a continuous response scale to study the types of reporting errors that participants make when objects are presented at the same or at different locations in space. When several objects successively shared the same location, participants exhibited a higher tendency to report features of the wrong object in memory; that is, they responded with features that belonged to objects retained in memory but not probed at retrieval. On the other hand, a similar effect was not observed when objects shared a nonspatial feature, such as color. Furthermore, the effect of location on reporting errors was present even when its manipulation was orthogonal to the task at hand. These findings are consistent with the view that binding together different nonspatial features of an object in memory might be mediated through an object’s location. Hence, spatial location may have a privileged role in WM. The relevance of these findings to conceptual models, as well as to neural accounts of visual WM, is discussed.     

Action relevance induces an attentional weighting of representations in visual working memory

Information maintained in visual working memory (VWM) can be strategically weighted according to its task-relevance. This is typically studied by presenting cues during the maintenance interval, but under natural conditions, the importance of certain aspects of our visual environment is mostly determined by intended actions. We investigated whether representations in VWM are also weighted with respect to their potential action relevance. In a combined memory and movement task, participants memorized a number of items and performed a pointing movement during the maintenance interval. The test item in the memory task was subsequently presented either at the movement goal or at another location. We found that performance was better for test items presented at a location that corresponded to the movement goal than for test items presented at action-irrelevant locations. This effect was sensitive to the number of maintained items, suggesting that preferential maintenance of action relevant information becomes particularly important when the demand on VWM is high. We argue that weighting according to action relevance is mediated by the deployment of spatial attention to action goals, with representations spatially corresponding to the action goal benefitting from this attentional engagement. Performance was also better at locations next to the action goal than at locations farther away, indicating an attentional gradient spreading out from the action goal. We conclude that our actions continue to influence visual processing at the mnemonic level, ensuring preferential maintenance of information that is relevant for current behavioral goals.     

No evidence for binding of items to task-irrelevant backgrounds in visual working memory

When representing visual features such as color and shape in visual working memory (VWM), participants also represent the locations of those features as a spatial configuration of the locations of those features in the display. In everyday life, we encounter objects against some background, yet it is unclear whether the configural representation in memory obligatorily constitutes the entire display, including that (often task-irrelevant) background information. In three experiments, participants completed a change detection task on color and shape; the memoranda were presented in front of uniform gray backgrounds, a textured background (Exp. 1), or a background containing location placeholders (Exps. 2 and 3). When whole-display probes were presented, changes to the objects’ locations or feature bindings impacted memory performance—implying that the spatial configuration of the probes influenced participants’ change decisions. Furthermore, when only a single item was probed, the effect of changing its location or feature bindings was either diminished or completely extinguished, implying that single probes do not necessarily elicit the entire spatial configuration. Critically, when task-irrelevant backgrounds were also presented that may have provided a spatial configuration for the single probes, the effect of location or bindings was not moderated. These findings suggest that although the spatial configuration of a display guides VWM-based recognition, this information does not necessarily always influence the decision process during change detection.     

Visual working memory for simple and complex visual stimuli

Does the magical number four characterize our visual working memory (VWM) capacity for all kinds of objects, or is the capacity of VWM inversely related to the perceptual complexity of those objects? To find out how perceptual complexity affects VWM, we used a change detection task to measure VWM capacity for six types of stimuli of different complexity: colors, letters, polygons, squiggles, cubes, and faces. We found that the estimated capacity decreased for more complex stimuli, suggesting that perceptual complexity was an important factor in determining VWM capacity. However, the considerable correlation between perceptual complexity and VWM capacity declined significantly if subjects were allowed to view the sample memory display longer. We conclude that when encoding limitations are minimized, perceptual complexity affects, but does not determine, VWM capacity.     

客体同维度特征的视觉工作记忆存储机制

客体理论认为视觉工作记忆的存储单位是客体, 人类能够把组成物体的所有特征整合成一个单元进行记忆, 不管这些特征是来自不同维度还是同一维度。然而同一维度多个特征可以被整合成客体记忆的结果只被少数研究证实, 大量研究发现同一维度的多个特征不能被整合记忆, 这形成了弱客体理论的基本观点。为了解决两大理论之间的争议, 本研究主要探究同维度特征能否被整合记忆。通过分析以往研究, 我们认为实验范式和物体意义两个因素可能是导致之前大部分研究没有发现维度内特征能够被整合记忆的原因。因此, 实验1采用回忆报告范式替代以往研究经常使用的变化觉察范式, 结果发现, 同一维度的多个特征难以被整合记忆。实验2以记忆无意义物体的特征为基线, 探究在记忆有意义物体的特征时是否可能将多个特征进行整合记忆。结果发现, 对于有意义物体的特征的记忆成绩并没有显著好于记忆无意义物体特征的成绩, 说明即使采用具有较强整合线索的有意义物体作为载体, 同一维度多个特征也难以被整合记忆。两个实验结果进一步支持了弱客体理论。     

Associative learning improves visual working memory performance

The ability to remember visual stimuli over a short delay period is limited by the small capacity of visual working memory (VWM). Here the authors investigate the role of learning in enhancing VWM. Participants saw 2 spatial arrays separated by a 1-s interval. The 2 arrays were identical except for 1 location. Participants had to detect the difference. Unknown to the participants, some spatial arrays would repeat once every dozen trials or so for up to 32 repetitions. Spatial VWM performance increased significantly when the same location changed across display repetitions, but not at all when different locations changed from one display repetition to another. The authors suggest that a major role of learning in VWM is to mediate which information gets retained, rather than to directly increase VWM capacity.     

Feature binding in children and young adults

The authors measured memory for individual features (objects only or locations only) and the combination of those features (objects and locations) in 9-, 12-, and 21-year-old students with a yes or no recognition task. Analysis of recognition memory performance (d' scores) revealed that although age differences existed in memory for individual features, age differences were greater in the tasks that required memory for combined features (objects and locations). Hierarchical multiple regression analyses indicated that age remained a significant predictor of memory performance in the combination condition even after the authors statistically removed memory performance in object and location conditions and the interaction effects of object and location. These results provide evidence for developmental differences in the binding of features in memory.     

Does high memory load kick task-irrelevant information out of visual working memory?

The limited capacity of visual working memory (VWM) requires the existence of an efficient information selection mechanism. While it has been shown that under low VWM load, an irrelevant simple feature can be processed, its fate under high load (e.g., six objects) remains unclear. We explored this issue by probing the “irrelevant-change distracting effect,” in which the change of a stored irrelevant feature affects performance. Simple colored shapes were used as stimuli, with color as the target. Using a whole-probe method (presenting six objects in both the memory and test arrays), in Experiment 1 we found that a change to one of the six shapes led to a significant distracting effect. Using a partial-probe method (presenting the probe either at the screen center or at a location selected from the memory array), in Experiment 2 we showed the distracting effect again. These results suggest that irrelevant simple features can be stored into VWM, regardless of memory load.     

Current directions in visual working memory research: An introduction and emerging insights

Visual working memory (VWM) is a core construct in the cognitive (neuro‐)sciences, assumed to serve as a hub for information exchange and thus supporting a multitude of cognitive functions related to processing visual information. Here, we give an introduction into key terms and paradigms and an overview of ongoing debates in the field, to which the articles collected in this Special Issue on ‘Current Directions in Visual Working Memory Research’ contribute. Our aim is to extract, from this overview, some ‘emerging’ theoretical insights concerning questions such as the optimal way to examine VWM, which types of mental representations contribute to performance on VWM tasks, and how VWM keeps features from the same object together and apart from features of concurrently maintained objects (the binding problem).     

视觉工作记忆的同类别存储优势

概念规律如记忆项间的类别关系如何影响视觉工作记忆容量是一个有争议的问题。针对该问题, 学界存在两种预测截然不同的假说：(1)混合类别优势假说, (2)同类别优势假说。综述文献发现, 该类研究均采用带有细节特征的真实客体作为实验材料, 因此前人研究中发现的混合类别优势效应或同类别优势效应中必然混有低水平知觉特征的影响。故本研究采用去除细节信息的动物剪影作为记忆材料来排除上述因素的影响, 旨在厘清上述两种假设, 并采用对侧延迟活动作为神经指标, 来进一步探讨概念规律影响工作记忆容量的内在机制。两个行为实验发现, 不论记忆项同时呈现还是序列呈现, 均存在同类别记忆优势效应。脑电实验结果发现相比记忆不同类别客体, 记忆同等数量的同类别客体诱发的对侧延迟活动的幅值更小。上述结果一致表明, 视觉工作记忆可借助概念的方式将同类别客体加以组织, 从而有效扩大视觉工作记忆容量, 支持了同类别优势假说。     

Pupillary correlates of covert shifts of attention during working memory maintenance

The pupillary light reflex (PLR) was used to track covert shifts of attention to items maintained in visual working memory (VWM). In three experiments, participants performed a change detection task in which rectangles appeared on either side of fixation and at test participants indicated if the cued rectangle changed its orientation. Prior to presentation or during the delay, participants were cued to the light or dark side of the screen. When cued to the light side, the pupil constricted, and when cued to the dark side, the pupil dilated, suggesting that the PLR tracked covert shifts of attention. Similar covert shifts of attention were seen when the target stimuli remained onscreen and during a blank delay period, suggesting similar effects for attention to perceptual stimuli and attention to stimuli maintained in VWM. Furthermore, similar effects were demonstrated when participants were pre-cued or retro-cued to the prioritized location, suggesting that shifts of covert attention can occur both before and after target presentation. These results are consistent with prior research, suggesting an important role of covert shifts of attention during VWM maintenance and that the PLR can be used to track these covert shifts of attention.     

Action relations, semantic relations, and familiarity of spatial position in Balint’s syndrome: Crossover effects on perceptual report and on localization

We contrasted effects of the action relationship between objects, relative to effects of semantic relation and familiarity of spatial location, in a patient with Balint’s syndrome. We found enhanced perceptual report of objects placed in the correct colocations for action in comparison with when the objects (1) were placed in incorrect locations for action (Experiment 1), (2) were associatively related (Experiment 2), or (3) fell in familiar locations but were not action related (Experiment 4). In contrast, the ability to localize the objects was affected by whether the objects were in familiar as opposed to unfamiliar locations with respect to one another. Action relations between objects facilitate joint attention to both objects but do not facilitate binding to location. Familiar spatial relations facilitate spatial binding, but do not cue joint attention to the stimuli. Effects of action relation on attention and localization can be dissociated from effects of semantic relationship and familiarity of spatial location on attention and localization.     

The perceptual root of object-based storage: an interactive model of perception and visual working memory

Mainstream theories of visual perception assume that visual working memory (VWM) is critical for integrating online perceptual information and constructing coherent visual experiences in changing environments. Given the dynamic interaction between online perception and VWM, we propose that how visual information is processed during visual perception can directly determine how the information is going to be selected, consolidated, and maintained in VWM. We demonstrate the validity of this hypothesis by investigating what kinds of perceptual information can be stored as integrated objects in VWM. Three criteria for object-based storage are introduced: (a) automatic selection of task-irrelevant features, (b) synchronous consolidation of multiple features, and (c) stable maintenance of feature conjunctions. The results show that the outputs of parallel perception meet all three criteria, as opposed to the outputs of serial attentive processing, which fail all three criteria. These results indicate that (a) perception and VWM are not two sequential processes, but are dynamically intertwined; (b) there are dissociated mechanisms in VWM for storing information identified at different stages of perception; and (c) the integrated object representations in VWM originate from the "preattentive" or "proto" objects created by parallel perception. These results suggest how visual perception, attention, and VWM can be explained by a unified framework.     

多特征刺激在视觉工作记忆中的存储模式

使用事件相关电位技术, 采用变化觉察范式, 以双色块和单色块为刺激材料, 通过两个实验探讨了多特征刺激在视觉工作记忆中的存储模式。实验1通过对双色块和单色块的比较来考察特征数量对视觉工作记忆资源分配的影响, 通过操控双色方块的变化幅度来考察精度需求对资源分配的影响。结果发现, 特征数量的增加诱发了更强的对侧延迟活动(CDA), 而精度需求的提高并没有引起对侧延迟活动上的差异, 却诱发了指示自上而下的控制的晚期正成分(LPC)的增强。实验2通过在测验阶段加入指示可能变化的部分的提示符, 发现提示符并未能促进变化觉察的行为反应。整个研究表明, 特征数量影响视觉工作记忆资源的分配, 而精度需求并不起作用, 同时多特征刺激在视觉工作记忆中的存储单位并不是单个的独立特征, 而是整合的客体。     

The benefit of surface uniformity for encoding boundary features in visual working memory

Using a change detection paradigm, the present study examined an object-based encoding benefit in visual working memory (VWM) for two boundary features (two orientations in Experiments 1-2 and two shapes in Experiments 3-4) assigned to a single object. Participants remembered more boundary features when they were conjoined into a single object of uniform color than when they were not bound together into an object. However, such an object-based benefit diminished or disappeared when those features occurred on different parts of an object, as defined by surface discontinuity or by negative minima of curvature. These results suggest that the number of boundary features that can be stored in VWM is influenced by the hierarchical structure of objects: surface uniformity facilitates the integration of two boundary features into a single object representation, but this integration is weakened when the features occur on different parts of an object.