Beyond slots and resources: Grounding cognitive concepts in neural dynamics

Research over the past decade has suggested that the ability to hold information in visual working memory (VWM) may be limited to as few as three to four items. However, the precise nature and source of these capacity limits remains hotly debated. Most commonly, capacity limits have been inferred from studies of visual change detection, in which performance declines systematically as a function of the number of items that participants must remember. According to one view, such declines indicate that a limited number of fixed-resolution representations are held in independent memory “slots.” Another view suggests that such capacity limits are more apparent than real, but emerge as limited memory resources are distributed across more to-be-remembered items. Here we argue that, although both perspectives have merit and have generated and explained impressive amounts of empirical data, their central focus on the representations—rather than processes—underlying VWM may ultimately limit continuing progress in this area. As an alternative, we describe a neurally grounded, process-based approach to VWM: the dynamic field theory. Simulations demonstrate that this model can account for key aspects of behavioral performance in change detection, in addition to generating novel behavioral predictions that have been confirmed experimentally. Furthermore, we describe extensions of the model to recall tasks, the integration of visual features, cognitive development, individual differences, and functional imaging studies of VWM. We conclude by discussing the importance of grounding psychological concepts in neural dynamics, as a first step toward understanding the link between brain and behavior.     

Visual working memory supports the inhibition of previously processed information: evidence from preview search

In four experiments we assessed whether visual working memory (VWM) maintains a record of previously processed visual information, allowing old information to be inhibited, and new information to be prioritized. Specifically, we evaluated whether VWM contributes to the inhibition (i.e., visual marking) of previewed distractors in a preview search. We evaluated this proposal by testing three predictions. First, Experiments 1 and 2 demonstrate that preview inhibition is more effective when the number of previewed distractors is below VWM capacity than above; an effect that can only be observed at small preview set sizes (Experiment 2A) and when observers are allowed to move their eyes freely (Experiment 2B). Second, Experiment 3 shows that, when quantified as the number of inhibited distractors, the magnitude of the preview effect is stable across different search difficulties. Third, Experiment 4 demonstrates that individual differences in preview inhibition are correlated with individual differences in VWM capacity. These findings provide converging evidence that VWM supports the inhibition of previewed distractors. More generally, these findings demonstrate how VWM contributes to the efficiency of human visual information processing--VWM prioritizes new information by inhibiting old information from being reselected for attention.     

Orienting attention in visual working memory reduces interference from memory probes

Given a changing visual environment, and the limited capacity of visual working memory (VWM), the contents of VWM must be in constant flux. Using a change detection task, the authors show that VWM is subject to obligatory updating in the face of new information. Change detection performance is enhanced when the item that may change is retrospectively cued 1 s after memory encoding and 0.5 s before testing. The retro-cue benefit cannot be explained by memory decay or by a reduction in interference from other items held in VWM. Rather, orienting attention to a single memory item makes VWM more resistant to interference from the test probe. The authors conclude that the content of VWM is volatile unless it receives focused attention, and that the standard change detection task underestimates VWM capacity.     

Development of visual working memory precision in childhood

Visual working memory (VWM) is the facility to hold in mind visual information for brief periods of time. Developmental studies have suggested an increase during childhood in the maximum number of complete items that can simultaneously be stored in VWM. Here, we exploit a recent theoretical and empirical innovation to investigate instead the precision with which items are stored in VWM, where precision is a continuous measure reflecting VWM resolution. Ninety boys aged 7 to 13 years completed one-item and three-item VWM tasks in which stimuli were coloured bars varying in orientation. On each trial, participants used a rotating dial to reproduce the probed stimulus from memory. Results show linear age-related improvement in recall precision for both one-item and three-item VWM tasks. However, even the youngest age group stored a significant amount of information about all three items on the difficult 3-item VWM task. Importantly, the development of VWM precision was not accounted for by development on a sensorimotor control task. Whereas storage of a single complete item was previously thought to be well within the capacity limitations of the current age range, these results suggest protracted development during childhood and early adolescence in the resolution with which single and multiple items are stored in VWM. Probabilistic modelling of response distribution data suggests that improvement in VWM performance is attributable to a specific decrease in variability of stored feature representations, rather than to a decrease in misbinding or random noise. As such, we highlight a novel, potentially developmentally plausible mechanism that may underlie developmental improvement in VWM performance, independent of any alterations in the maximum number of complete items which can be stored.     

The perceptual root of object-based storage: an interactive model of perception and visual working memory

Mainstream theories of visual perception assume that visual working memory (VWM) is critical for integrating online perceptual information and constructing coherent visual experiences in changing environments. Given the dynamic interaction between online perception and VWM, we propose that how visual information is processed during visual perception can directly determine how the information is going to be selected, consolidated, and maintained in VWM. We demonstrate the validity of this hypothesis by investigating what kinds of perceptual information can be stored as integrated objects in VWM. Three criteria for object-based storage are introduced: (a) automatic selection of task-irrelevant features, (b) synchronous consolidation of multiple features, and (c) stable maintenance of feature conjunctions. The results show that the outputs of parallel perception meet all three criteria, as opposed to the outputs of serial attentive processing, which fail all three criteria. These results indicate that (a) perception and VWM are not two sequential processes, but are dynamically intertwined; (b) there are dissociated mechanisms in VWM for storing information identified at different stages of perception; and (c) the integrated object representations in VWM originate from the "preattentive" or "proto" objects created by parallel perception. These results suggest how visual perception, attention, and VWM can be explained by a unified framework.     

Searching while loaded: Visual working memory does not interfere with hybrid search efficiency but hybrid search uses working memory capacity

In “hybrid search” tasks, such as finding items on a grocery list, one must search the scene for targets while also searching the list in memory. How is the representation of a visual item compared with the representations of items in the memory set? Predominant theories would propose a role for visual working memory (VWM) either as the site of the comparison or as a conduit between visual and memory systems. In seven experiments, we loaded VWM in different ways and found little or no effect on hybrid search performance. However, the presence of a hybrid search task did reduce the measured capacity of VWM by a constant amount regardless of the size of the memory or visual sets. These data are broadly consistent with an account in which VWM must dedicate a fixed amount of its capacity to passing visual representations to long-term memory for comparison to the items in the memory set. The data cast doubt on models in which the search template resides in VWM or where memory set item representations are moved from LTM through VWM to earlier areas for comparison to visual items.     

Encoding strategy and not visual working memory capacity correlates with intelligence

There is conflicting evidence on whether the capacity of visual working memory (VWM) reflects a central capacity limit that also influences intelligence. We propose that encoding strategy and, more specifically, attentional selection, underlie the correlation of some VWM tasks and IQ, and not variations in VWM itself. In Experiment 1, change detection measures of VWM were found to be contaminated by some cognitive process that depressed performance at higher set sizes, so that fewer items were remembered when eight rather than just four were presented. Measuring VWM using whole report instead gave a less variable estimate that was higher, particularly for larger set sizes. Nonverbal IQ did not correlate with this estimate of VWM capacity, but instead with the additional factor that contaminates change detection estimates. We propose that this phenomenon reflects a lack of selection during encoding. In Experiment 2, we investigated the role of rehearsal using articulatory suppression and showed that this could not account for the key differences between the procedures.     

视觉工作记忆对前注意阶段注意定向的调节

在注意阶段,视觉工作记忆与注意共享某些特征时,视觉工作记忆内容会引导注意定向。在前注意阶段的影响又是如何？对此问题的研究却很少见,本研究采用工作记忆与基于特征的视觉搜索相结合的任务深入探讨了此问题。研究结果发现：即使在前注意阶段,注意定向也会受到视觉工作记忆自上而下的调节。然而与注意阶段不同的是：当视觉工作记忆内容与随后视觉搜索任务的目标相关时,与记忆内容匹配的搜索项目会引导注意定向到该项目所在的位置,但只有此项目作为分心物时,才会明显地延迟对靶子的加工,而此项目作为靶子时,这种影响并不明显;当视觉工作记忆中保持的项目与随后视觉搜索任务的目标无关时,注意会忽略视觉工作记忆中保持的项目,有效地检测靶子     

Identifying a “default” visual search mode with operant conditioning

The presence of a singleton in a task-irrelevant domain can impair visual search. This impairment, known as the attentional capture depends on the set of participants. When narrowly searching for a specific feature (the feature search mode), only matching stimuli capture attention. When searching broadly (the singleton detection mode), any oddball captures attention. The present study examined which strategy represents the “default” mode using an operant conditioning approach in which participants were trained, in the absence of explicit instructions, to search for a target in an ambiguous context in which one of two modes was available. The results revealed that participants behaviorally adopted the singleton detection as the default mode but reported using the feature search mode. Conscious strategies did not eliminate capture. These results challenge the view that a conscious set always modulates capture, suggesting that the visual system tends to rely on stimulus salience to deploy attention.     

Tuning perception: Visual working memory biases the quality of visual awareness

Studies of consciousness reveal that it is possible to manipulate subjective awareness of a visual stimulus. For example, items held in visual working memory (VWM) that match target features increase the speed with which the target reaches visual awareness. To examine the effect of VWM on perception, previous studies have mainly used coarse measures of awareness, such as present/absent or forced-choice judgments. These methods can reveal whether or not an individual has seen an item, but they do not provide information about the quality with which the item was seen. Using continuous report methods it has been shown that the fidelity of a perceived item can be affected by whether or not that item is masked. In the present study, we used an object-substitution masking task to examine whether items held in VWM would influence the quality with which a masked target reached awareness, or whether the threshold for awareness was instead affected by stimuli held in memory. We observed that targets matching the contents of VWM were recalled with greater precision compared to items that did not match the contents of VWM. Importantly, this effect occurred without affecting the likelihood of the target being perceived. These results suggest that VWM plays a greater role in modulating the fidelity of perceived representations than in lowering the overall threshold of awareness.     

Salience drives overt selection of two equally relevant visual targets

In the present study, we investigated whether salience determines the sequence of selection when participants search for two equally relevant visual targets. To do this, attentional selection was tracked overtly as observers inspected two items of differing physical salience: one a highly salient color singleton, and the other a less salient shape singleton. Participants were instructed to make natural eye movements in order to determine whether two line segments contained within the two singletons were oriented in the same or in different directions. Because both singleton items were task-relevant, participants had no reason to inspect one item before the other. As expected, observers fixated both targets on the majority of trials. Critically, saccades to the color singleton preceded saccades to the less salient shape singleton on the majority of trials. This demonstrates that the order of attentional object selection is largely determined by stimulus salience when task relevance is equated.     

Looking sharp: Becoming a search template boosts precision and stability in visual working memory

Visual working memory (VWM) plays a central role in visual cognition, and current work suggests that there is a special state in VWM for items that are the goal of visual searches. However, whether the quality of memory for target templates differs from memory for other items in VWM is currently unknown. In this study, we measured the precision and stability of memory for search templates and accessory items to determine whether search templates receive representational priority in VWM. Memory for search templates exhibited increased precision and probability of recall, whereas accessory items were remembered less often. Additionally, while memory for Templates showed benefits when instances of the Template appeared in search, this benefit was not consistently observed for Accessory items when they appeared in search. Our results show that becoming a search template can substantially affect the quality of a representation in VWM.     

Does visual working memory work as a few fixed slots?

The present study investigated whether visual working memory (VWM) functions as a few (about 3?～ 4) fixed slots by examining how the distribution of VWM is adjusted. Adopting a change-detection paradigm, we required subjects to memorize four items, one of which was prioritized. If VWM functions as 3?～?4 slots, allocating multiple slots to the prioritized item would leave no slot for some other items; consequently no information would be stored for them, leading to a substantial decrease in change-detection performance no matter whether small or large changes occurred. The result showed that small changes on the unfavoured items were detected less accurately, indicating that more VWM was allocated to the favoured item. Yet meanwhile large changes that occurred on those unfavoured items could still be detected as well as those on the favoured one, indicating that each of them was still stored to some extent rather than completely discarded. The results suggested that VWM may not work as 3?～?4 fixed slots. Possible mechanisms were discussed based on the present results, including a modified slot model with more available slots, a continuous resource model, and a hierarchical model that assumes storage of ensemble information in addition to the information of individual items.     

The role of top-down suppression in mitigating the disruptive effects of task-irrelevant feature changes in visual working memory

Studies of change detection have shown that changing the task-irrelevant features of remembered objects impairs change detection for task-relevant features, a phenomenon known as the irrelevant change effect. Although this effect is pronounced at short study-test intervals, it is eliminated at longer delays. This has prompted the proposal that although all features of attended objects are initially stored together in visual working memory (VWM), top-down control can be used to suppress task-irrelevant features over time. The present study reports the results of three experiments aimed at testing the top-down suppression hypothesis. Experiments 1 and 2 tested whether the magnitude or time course of the irrelevant change effect was affected by the concurrent performance of a demanding executive load task (counting backwards by threes). Contrary to the top-down suppression view, the decreased availability of executive resources did not prolong the duration of the irrelevant change effect in either experiment, as would be expected if these resources were necessary to actively suppress task-irrelevant features. Experiment 3 showed that a visual pattern mask eliminates the irrelevant change effect and suggests that the source of the effect may lie in the use a high-resolution, sensory memory representation to match the memory and test displays when no task-irrelevant feature changes are present. These results suggest that the dissipation of the irrelevant change effect over time likely does not depend on the use of top-down control and raises questions about what can be inferred about the nature of storage in VWM from studies of this effect.     

Influence of attentional capture on oculomotor control

Previous research has shown that when searching for a color singleton, top-down control cannot prevent attentional capture by an abrupt visual onset. The present research addressed whether a task-irrelevant abrupt onset would affect eye movement behavior when searching for a color singleton. Results show that in many instances the eye moved in the direction of the task-irrelevant abrupt onset. There was evidence that top-down control could neither entirely prevent attentional capture by visual onsets nor prevent the eye from starting to move in the direction of the onset. Results suggest parallel programming of 2 saccades: 1 voluntary goal-directed eye movement toward the color singleton target and 1 stimulus-driven eye movement reflexively elicited by the abrupt onset. A neurophysiologically plausible model that can account for the current findings is discussed.     

视觉工作记忆负载类型对注意选择的影响

通过操纵Flanker任务相对于视觉工作记忆任务的呈现位置, 探讨在视觉工作记忆编码和保持阶段, 精度负载和容量负载对注意选择的影响。行为结果发现, Flanker任务呈现位置和视觉工作记忆负载类型影响注意选择; ERP结果发现, 在保持阶段, 当搜索目标和干扰项不一致时, 负载类型影响N2成分。研究表明, 在编码阶段, 视觉工作记忆负载主要通过占用更多知觉资源降低干扰效应, 支持知觉负载理论; 而在保持阶段, 当Flanker任务位于记忆项内部时, 两类负载在工作记忆表征过程中不同的神经活动导致投入到注意选择的认知控制资源不同, 可能是两类负载影响保持阶段注意选择的机制。     

视觉工作记忆的同类别存储优势

概念规律如记忆项间的类别关系如何影响视觉工作记忆容量是一个有争议的问题。针对该问题, 学界存在两种预测截然不同的假说：(1)混合类别优势假说, (2)同类别优势假说。综述文献发现, 该类研究均采用带有细节特征的真实客体作为实验材料, 因此前人研究中发现的混合类别优势效应或同类别优势效应中必然混有低水平知觉特征的影响。故本研究采用去除细节信息的动物剪影作为记忆材料来排除上述因素的影响, 旨在厘清上述两种假设, 并采用对侧延迟活动作为神经指标, 来进一步探讨概念规律影响工作记忆容量的内在机制。两个行为实验发现, 不论记忆项同时呈现还是序列呈现, 均存在同类别记忆优势效应。脑电实验结果发现相比记忆不同类别客体, 记忆同等数量的同类别客体诱发的对侧延迟活动的幅值更小。上述结果一致表明, 视觉工作记忆可借助概念的方式将同类别客体加以组织, 从而有效扩大视觉工作记忆容量, 支持了同类别优势假说。     

Visual mental imagery influences attentional guidance in a visual-search task

Visual mental imagery resembles visual working memory (VWM). Because both visual mental imagery and VWM involve the representation and manipulation of visual information, it was hypothesized that they would exert similar effects on visual attention. Several previous studies have demonstrated that working-memory representations guide attention toward a memory-matching task-irrelevant stimulus during visual-search tasks. Therefore, mental imagery may also guide attention toward imagery-matching stimuli. In the present study, five experiments were conducted to investigate the effects of visual mental imagery on visual attention during a visual-search task. Participants were instructed to visualize a color or an object clearly associated with a specific color, after which they were asked to detect a colored target in the visual-search task. Reaction times for target detection were shorter when the color of the target matched the imagined color, and when the color of the target was similar to that strongly associated with the imagined object, than when the color of the target did not match that of the mental representation. This effect was not observed when participants were not instructed to imagine a color. These results suggest that similar to VWM, visual mental imagery guides attention toward imagery-matching stimuli.     

Investigating the other-race effect in working memory

People have difficulties in remembering other-race faces; this so-called other-race effect (ORE) has been frequently observed in long-term recognition memory (LTM). Several theories argue that the ORE in LTM is caused by differences in earlier processing stages, such as encoding of ingroup and outgroup faces. We test this hypothesis by exploring whether the ORE can already be observed in visual working memory (VWM)—an intermediate system located between encoding processes and LTM storage. In four independent experiments, we observed decreased performance for outgroup faces compared to ingroup faces using three different VWM tasks: an adaptive N-back task, a self-ordered pointing task, and a change detection task. Also, we found that the number of items stored in VWM is smaller for outgroup faces than for ingroup faces. Further, we explored whether performance differences in the change detection task are related to the classic ORE in recognition memory. Our results provide further evidence that the ORE originates during earlier stages of cognitive processing. We discuss that (how) future ORE research may benefit from considering theories and evidence from the VWM literature.     

The contents of visual working memory reduce uncertainty during visual search

Information held in visual working memory (VWM) influences the allocation of attention during visual search, with targets matching the contents of VWM receiving processing benefits over those that do not. Such an effect could arise from multiple mechanisms: First, it is possible that the contents of working memory enhance the perceptual representation of the target. Alternatively, it is possible that when a target is presented among distractor items, the contents of working memory operate postperceptually to reduce uncertainty about the location of the target. In both cases, a match between the contents of VWM and the target should lead to facilitated processing. However, each effect makes distinct predictions regarding set-size manipulations; whereas perceptual enhancement accounts predict processing benefits regardless of set size, uncertainty reduction accounts predict benefits only with set sizes larger than 1, when there is uncertainty regarding the target location. In the present study, in which briefly presented, masked targets were presented in isolation, there was a negligible effect of the information held in VWM on target discrimination. However, in displays containing multiple masked items, information held in VWM strongly affected target discrimination. These results argue that working memory representations act at a postperceptual level to reduce uncertainty during visual search.