Evidence for working memory storage operations in perceptual cortex

Isolating the short-term storage component of working memory (WM) from the myriad of associated executive processes has been an enduring challenge. Recent efforts have identified patterns of activity in visual regions that contain information about items being held in WM. However, it remains unclear (1) whether these representations withstand intervening sensory input and (2) how communication between multimodal association cortex and the unimodal perceptual regions supporting WM representations is involved in WM storage. We present evidence that the features of a face held in WM are stored within face-processing regions, that these representations persist across subsequent sensory input, and that information about the match between sensory input and a memory representation is relayed forward from perceptual to prefrontal regions. Participants were presented with a series of probe faces and indicated whether each probe matched a target face held in WM. We parametrically varied the feature similarity between the probe and target faces. Activity within face-processing regions scaled linearly with the degree of feature similarity between the probe face and the features of the target face, suggesting that the features of the target face were stored in these regions. Furthermore, directed connectivity measures revealed that the direction of information flow that was optimal for performance was from sensory regions that stored the features of the target face to dorsal prefrontal regions, supporting the notion that sensory input is compared to representations stored within perceptual regions and is subsequently relayed forward. Together, these findings indicate that WM storage operations are carried out within perceptual cortex.     

Bindings in working memory: The role of object-based attention

Over the past decade, it has been debated whether retaining bindings in working memory (WM) requires more attention than retaining constituent features, focusing on domain-general attention and space-based attention. Recently, we proposed that retaining bindings in WM needs more object-based attention than retaining constituent features (Shen, Huang, & Gao, 2015, Journal of Experimental Psychology: Human Perception and Performance, doi: 10.1037/xhp0000018). However, only unitized visual bindings were examined; to establish the role of object-based attention in retaining bindings in WM, more emperical evidence is required. We tested 4 new bindings that had been suggested requiring no more attention than the constituent features in the WM maintenance phase: The two constituent features of binding were stored in different WM modules (cross-module binding, Experiment 1), from auditory and visual modalities (cross-modal binding, Experiment 2), or temporally (cross-time binding, Experiments 3) or spatially (cross-space binding, Experiments 4–6) separated. In the critical condition, we added a secondary object feature-report task during the delay interval of the change-detection task, such that the secondary task competed for object-based attention with the to-be-memorized stimuli. If more object-based attention is required for retaining bindings than for retaining constituent features, the secondary task should impair the binding performance to a larger degree relative to the performance of constituent features. Indeed, Experiments 1–6 consistently revealed a significantly larger impairment for bindings than for the constituent features, suggesting that object-based attention plays a pivotal role in retaining bindings in WM.     

Dynamic visual noise interferes with storage in visual working memory

Several studies have demonstrated that dynamic visual noise (DVN) does not interfere with memory for random matrices. This has led to suggestions that (a) visual working memory is distinct from imagery, and (b) visual working memory is not a gateway between sensory input and long-term storage. A comparison of the interference effects of DVN with memory for matrices and colored textures shows that DVN can interfere with visual working memory, probably at a level of visual detail not easily supported by long-term memory structures or the recoding of the visual pattern elements. The results support a gateway model of visuospatial working memory and raise questions about the most appropriate ways to measure and model the different levels of representation of information that can be held in visual working memory.     

Fragile visual short-term memory is an object-based and location-specific store

Fragile visual short-term memory (FM) is a recently discovered form of visual short-term memory. Evidence suggests that it provides rich and high-capacity storage, like iconic memory, yet it exists, without interference, almost as long as visual working memory. In the present study, we sought to unveil the functional underpinnings of this memory storage. We found that FM is only completely erased when the new visual scene appears at the same location and consists of the same objects as the to-be-recalled information. This result has two important implications: First, it shows that FM is an object- and location-specific store, and second, it suggests that FM might be used in everyday life when the presentation of visual information is appropriately designed.     

Tuning perception: Visual working memory biases the quality of visual awareness

Studies of consciousness reveal that it is possible to manipulate subjective awareness of a visual stimulus. For example, items held in visual working memory (VWM) that match target features increase the speed with which the target reaches visual awareness. To examine the effect of VWM on perception, previous studies have mainly used coarse measures of awareness, such as present/absent or forced-choice judgments. These methods can reveal whether or not an individual has seen an item, but they do not provide information about the quality with which the item was seen. Using continuous report methods it has been shown that the fidelity of a perceived item can be affected by whether or not that item is masked. In the present study, we used an object-substitution masking task to examine whether items held in VWM would influence the quality with which a masked target reached awareness, or whether the threshold for awareness was instead affected by stimuli held in memory. We observed that targets matching the contents of VWM were recalled with greater precision compared to items that did not match the contents of VWM. Importantly, this effect occurred without affecting the likelihood of the target being perceived. These results suggest that VWM plays a greater role in modulating the fidelity of perceived representations than in lowering the overall threshold of awareness.     

A bilateral advantage for storage in visual working memory

Various studies have demonstrated enhanced visual processing when information is presented across both visual hemifields rather than in a single hemifield (the bilateral advantage). For example, Alvarez and Cavanagh (2005) reported that observers were able to track twice as many moving visual stimuli when the tracked items were presented bilaterally rather than unilaterally, suggesting that independent resources enable tracking in the two visual fields. Motivated by similarities in the apparent capacity and neural substrates that mediate tracking and visual working memory (WM), the present work examined whether or not a bilateral advantage also arises during storage in visual WM. Using a recall procedure to assess working memory for orientation information, we found a reliable bilateral advantage; recall error was smaller with bilateral sample displays than with unilateral displays. To demonstrate that the bilateral advantage influenced storage per se rather than just encoding efficiency, we replicated the observed bilateral advantage using sequentially presented stimuli. Finally, to further characterize how bilateral presentations enhanced storage in working memory, we measured both the number and the resolution of the stored items and found that bilateral presentations lead to an increased probability of storage, rather than enhanced mnemonic resolution. Thus, the bilateral advantage extends beyond the initial selection and encoding of visual information to influence online maintenance in visual working memory.     

Visual memory and visual perception: when memory improves visual search

This study examined the relationship between memory and perception in order to identify the influence of a memory dimension in perceptual processing. Our aim was to determine whether the variation of typical size between items (i.e., the size in real life) affects visual search. In two experiments, the congruency between typical size difference and perceptual size difference was manipulated in a visual search task. We observed that congruency between the typical and perceptual size differences decreased reaction times in the visual search (Exp. 1), and noncongruency between these two differences increased reaction times in the visual search (Exp. 2). We argue that these results highlight that memory and perception share some resources and reveal the intervention of typical size difference on the computation of the perceptual size difference.     

Working memory, perceptual priming, and the perception of hierarchical forms: Opposite effects of priming and working memory without memory refreshing

Previous research has shown that stimuli held in working memory (WM) can influence spatial attention. Using Navon stimuli, we explored whether and how items in WM affect the perception of visual targets at local and global levels in compound letters. Participants looked for a target letter presented at a local or global level while holding a regular block letter as a memory item. An effect of holding the target’s identity in WM was found. When memory items and targets were the same, performance was better than in a neutral condition when the memory item did not appear in the hierarchical letter (a benefit from valid cuing). When the memory item matched the distractor in the hierarchical stimulus, performance was worse than in the neutral baseline (a cost on invalid trials). These effects were greatest when the WM cue matched the global level of the hierarchical stimulus, suggesting that WM biases attention to the global level of form. Interestingly, in a no-memory priming condition, target perception was faster in the invalid condition than in the neutral baseline, reversing the effect in the WM condition. A further control experiment ruled out the effects of WM being due to participants’ refreshing their memory from the hierarchical stimulus display. The data show that information in WM biases the selection of hierarchical forms, whereas priming does not. Priming alters the perceptual processing of repeated stimuli without biasing attention.     

Perceptual completion and object-based representations in short-term visual memory

Object-based representations in visual short-term memory (VSTM) were examined using a change detection memory task. A display comprising two rows of four differently colored elements was followed by a probe display in which only one of the rows reappeared. On same trials, the probed row was identical to the corresponding row in the memory display. On different trials, two of the elements in the probed row had their colors exchanged. In each memory display, a task-irrelevant visual element appeared between the two rows, with the potential to function as an occluder. Performance was enhanced when perceptual completion meant that four, rather than eight, objects were perceived in the memory display and when the probe display revealed that the occluded elements continued behind the occluder. It appears that several forms of representation can co-occur to support VSTM, one of which is object based.     

工作记忆与知觉负载对工作记忆表征引导注意的调节

本研究采用4个眼动实验探讨不同知觉负载条件下的视觉搜索任务中工作记忆负载对基于工作记忆表征的注意引导效应的影响。实验1和实验3采用低知觉负载的视觉搜索任务,结果在视觉工作记忆负载为1和2时观察到了显著的注意引导效应,但当负载增加到4时注意引导效应消失了;实验2和实验4采用高知觉负载的视觉搜索任务,结果发现注意引导效应在工作记忆负载增加到2时就已经消失了。上述结果表明:工作记忆负载和知觉负载都能够通过调控认知资源的方式来影响工作记忆表征对注意的引导,当认知资源充足时,工作记忆能够同时保持多个记忆表征对视觉注意的引导。     

工作记忆对知觉类别学习的影响:问题与构想

人们学习将感知到的客体按照不同的标准或规则分类存储,即类别学习。而工作记忆作为多种认知加工的基础,对类别学习具有重要作用。基于已有研究,分别梳理了言语工作记忆和视空工作记忆对基于规则和信息整合类别结构的影响。目前,执行功能是否影响内隐类别学习系统、工作记忆负荷对内隐外显类别学习系统的影响是否一致还存在争议。此外,工作记忆影响知觉类别学习的认知加工阶段尚不清晰。未来的研究还需明确工作记忆负荷对内隐和外显类别学习的影响,进一步验证工作记忆影响类别学习的认知加工阶段假设的合理性。     

Limitations of object-based feature encoding in visual short-term memory

The present study investigated object-based feature encoding in visual short-term memory for 2 features within the same dimension that occur on different parts of an object. Using the change-detection paradigm, this experiment studied objects with 2 colors and objects with 2 orientations. Participants found it easier to monitor 1 rather than both features of such objects, even when decision noise was properly controlled for. However, no object-based benefit was observed for encoding the 2 features of each object that were of the same dimension. When similar stimuli were used but the 2 features of each object were from different dimensions (color and orientation), an object-based benefit was observed. These results thus impose a major constraint on object-based feature encoding theories by showing that only features from different dimensions can benefit from object-based encoding.     

The contributions of visual and central attention to visual working memory

We investigated the role of two kinds of attention—visual and central attention—for the maintenance of visual representations in working memory (WM). In Experiment 1 we directed attention to individual items in WM by presenting cues during the retention interval of a continuous delayed-estimation task, and instructing participants to think of the cued items. Attending to items improved recall commensurate with the frequency with which items were attended (0, 1, or 2 times). Experiments 1 and 3 further tested which kind of attention—visual or central—was involved in WM maintenance. We assessed the dual-task costs of two types of distractor tasks, one tapping sustained visual attention and one tapping central attention. Only the central attention task yielded substantial dual-task costs, implying that central attention substantially contributes to maintenance of visual information in WM. Experiment 2 confirmed that the visual-attention distractor task was demanding enough to disrupt performance in a task relying on visual attention. We combined the visual-attention and the central-attention distractor tasks with a multiple object tracking (MOT) task. Distracting visual attention, but not central attention, impaired MOT performance. Jointly, the three experiments provide a double dissociation between visual and central attention, and between visual WM and visual object tracking: Whereas tracking multiple targets across the visual filed depends on visual attention, visual WM depends mostly on central attention.     

The effects of autobiographical memory and visual perspective on working memory

The present research aims to explore whether recalling and writing about autobiographical memory from different perspectives (first-person perspective vs. third-person perspective) could affect cognitive function. The participants first performed a working memory task to evaluate their working memory capacity as a baseline and then were instructed to recall (Study 1) or write about (Study 2) personal events (failures vs. successes) from the first-person perspective or the third-person perspective. Finally, they performed the working memory task again. The results suggested that autobiographical memory and perspective influence working memory interactively. When recalling a success, the participants who recalled from the third-person perspective performed better than those who recalled from the first-person perspective on the working memory capacity task; when recalling a failure, the opposite was true.     

视觉工作记忆的过滤效能

过滤效能反映了视觉工作记忆的干扰抑制功能, 研究者可基于储存容量或表征精度对其进行测量, 其神经加工过程主要分为觉察分心项目、过滤启动、实现过滤或储存, 涉及前额叶皮层和基底核、后顶叶皮层的协同作用。过滤效能的变化方向受到年龄、特殊障碍、情绪、认知特点等因素的影响。未来研究仍需解决的问题包括厘清过滤效能与工作记忆容量的关系, 辨明过滤效能的心理实现过程, 探索不同年龄、特殊障碍和职业等群体过滤效能的脑机制以及提升基础研究范式的生态学效度。     

The reliability and stability of visual working memory capacity

Because of the central role of working memory capacity in cognition, many studies have used short measures of working memory capacity to examine its relationship to other domains. Here, we measured the reliability and stability of visual working memory capacity, measured using a single-probe change detection task. In Experiment 1, the participants (N = 135) completed a large number of trials of a change detection task (540 in total, 180 each of set sizes 4, 6, and 8). With large numbers of both trials and participants, reliability estimates were high (α > .9). We then used an iterative down-sampling procedure to create a look-up table for expected reliability in experiments with small sample sizes. In Experiment 2, the participants (N = 79) completed 31 sessions of single-probe change detection. The first 30 sessions took place over 30 consecutive days, and the last session took place 30 days later. This unprecedented number of sessions allowed us to examine the effects of practice on stability and internal reliability. Even after much practice, individual differences were stable over time (average between-session r = .76).     

The many faces of working memory and short-term storage

The topic of working memory (WM) is ubiquitous in research on cognitive psychology and on individual differences. According to one definition, it is a small amount of information kept in a temporary state of heightened accessibility; it is used in most types of communication and problem solving. Short-term storage has been defined as the passive (i.e., non-attention-based, nonstrategic) component of WM or, alternatively, as a passive store separate from an attention-based WM. Here I note that much confusion has been created by the use by various investigators of many, subtly different definitions of WM and short-term storage. The definitions are sometimes made explicit and sometimes implied. As I explain, the different definitions may have stemmed from the use of a wide variety of techniques to explore WM, along with differences in theoretical orientation. By delineating nine previously used definitions of WM and explaining how additional ones may emerge from combinations of these nine, I hope to improve scientific discourse on WM. The potential advantages of clarity about definitions of WM and short-term storage are illustrated with respect to several ongoing research controversies.     

Auditory and visual spatial working memory

A series of experiments compared short-term memory for object locations in the auditory and visual modalities. The stimulus materials consisted of sounds and pictures presented at different locations in space. Items were presented in pure- or mixed-modality lists of increasing length. At test, participants responded to renewed presentation of the objects by indicating their original position. If two independent modality-specific and resource-limited short-term memories support the remembering of locations, memory performance should be higher in the mixed-modality than in the pure-modality condition. Yet, memory performance was the same for items in both types of list. In addition, responses to the memory load manipulation in both modalities showed very similar declines in performance. The results are interpreted in terms of object files binding object and location information in episodic working memory, independently of the input modality.     

Emotional influences on perception and working memory

Although there has been steady progress elucidating the influence of emotion on cognition, it remains unclear precisely when and why emotion impairs or facilitates cognition. The present study investigated the mechanisms involved in the influence of emotion on perception and working memory (WM), using modified 0-back and 2-back tasks, respectively. First, results showed that attentional focus modulated the impact of emotion on perception. Specifically, emotion facilitated perceptual task performance when it was relevant to the task, but it impaired performance when it was irrelevant to the task. The differential behavioural effect of emotion on perception as a function of attentional focus diminished under high WM load. Second, attentional focus did not directly modulate the impact of emotion on WM, but rather its influence depended on the dynamic relationship between internal representations. Specifically, WM performance was worse when the material already being held online and the new input were of matching emotions (e.g. both were negative), compared to when they were not. We propose that the competition between “bottom-up” and “top-down” processing for limited cognitive resources explains the nature of the influence of emotion on both perception and WM.