The aftermath of memory retrieval for recycling visual working memory representations

We examined the aftermath of accessing and retrieving a subset of information stored in visual working memory (VWM)—namely, whether detection of a mismatch between memory and perception can impair the original memory of an item while triggering recognition-induced forgetting for the remaining, untested items. For this purpose, we devised a consecutive-change detection task wherein two successive testing probes were displayed after a single set of memory items. Across two experiments utilizing different memory-testing methods (whole vs. single probe), we observed a reliable pattern of poor performance in change detection for the second test when the first test had exhibited a color change. The impairment after a color change was evident even when the same memory item was repeatedly probed; this suggests that an attention-driven, salient visual change made it difficult to reinstate the previously remembered item. The second change detection, for memory items untested during the first change detection, was also found to be inaccurate, indicating that recognition-induced forgetting had occurred for the unprobed items in VWM. In a third experiment, we conducted a task that involved change detection plus continuous recall, wherein a memory recall task was presented after the change detection task. The analyses of the distributions of recall errors with a probabilistic mixture model revealed that the memory impairments from both visual changes and recognition-induced forgetting are explained better by the stochastic loss of memory items than by their degraded resolution. These results indicate that attention-driven visual change and recognition-induced forgetting jointly influence the “recycling” of VWM representations.     

The effect of items in working memory on the deployment of attention and the eyes during visual search

Paying attention to an object facilitates its storage in working memory. The authors investigate whether the opposite is also true: whether items in working memory influence the deployment of attention. Participants performed a search for a prespecified target while they held another item in working memory. In some trials this memory item was present in the search display as a distractor. Such a distractor has no effect on search time if the search target is in the display. In that case, the item in working memory is unlikely to be selected as a target for an eye movement, and if the eyes do land on it, fixation duration is short. In the absence of the target, however, there is a small but significant effect of the memory item on search time. The authors conclude that the target for visual search has a special status in working memory that allows it to guide attention. Guidance of attention by other items in working memory is much weaker and can be observed only if the search target is not present in the display.     

Dynamic states in working memory modulate guidance of visual attention: Evidence from an n-back paradigm

Items held in working memory (WM) can automatically bias attention when they reappear in visual displays. Recent evidence, however, suggests that WM biases of attention may be reduced under certain conditions, for example with increasing memory load. We employed a dual task paradigm to investigate how WM biases are affected by dynamic updating of memory contents. 1-back and 2-back versions of a memory task with colour stimuli were interrupted at intervals by an unrelated visual search task. Reappearance in the search display of the item that was currently active in WM guided attention, while suppressed or inactive items did not. We conclude that the rapid updating of memory contents facilitates the shifting of memory representations into different activity states on a moment-to-moment basis. The finding is consistent with models that propose that only one item can be “active” in WM at any one time to guide attention.     

Searching while loaded: Visual working memory does not interfere with hybrid search efficiency but hybrid search uses working memory capacity

In “hybrid search” tasks, such as finding items on a grocery list, one must search the scene for targets while also searching the list in memory. How is the representation of a visual item compared with the representations of items in the memory set? Predominant theories would propose a role for visual working memory (VWM) either as the site of the comparison or as a conduit between visual and memory systems. In seven experiments, we loaded VWM in different ways and found little or no effect on hybrid search performance. However, the presence of a hybrid search task did reduce the measured capacity of VWM by a constant amount regardless of the size of the memory or visual sets. These data are broadly consistent with an account in which VWM must dedicate a fixed amount of its capacity to passing visual representations to long-term memory for comparison to the items in the memory set. The data cast doubt on models in which the search template resides in VWM or where memory set item representations are moved from LTM through VWM to earlier areas for comparison to visual items.     

Hybrid search in the temporal domain: Evidence for rapid,serial logarithmic search through memory

In hybrid search, observers memorize a number of possible targets and then search for any of these in visual arrays of items. Wolfe (2012) has previously shown that the response times in hybrid search increase with the log of the memory set size. What enables this logarithmic search of memory? One possibility is a series of steps in which subsets of the memory set are compared to all items in the visual set simultaneously. In the present experiments, we presented single visual items sequentially in a rapid serial visual presentation (RSVP) display, eliminating the possibility of simultaneous testing of all items. We used a staircasing procedure to estimate the time necessary to effectively detect the target in the RSVP stream. Processing time increased in a log–linear fashion with the number of potential targets. This finding eliminates the class of models that require simultaneous comparison of some memory items to all (or many) items in the visual display. Experiment 3 showed that, similar to visual search, memory search efficiency in this paradigm is influenced by the similarity between the target set and the distractors. These results indicate that observers perform separate memory searches on each eligible item in the visual display. Moreover, it appears that memory search for one item can proceed while other items are being categorized as “eligible” or “not eligible.”     

Perceptual grouping boosts visual working memory capacity and reduces effort during retention

Consistent, robust boosts to visual working memory capacity are observed when colour–location arrays contain duplicate colours. The prevailing explanation suggests that duplicated colours are encoded as one perceptual group. If so, then we should observe not only higher working memory capacity overall for displays containing duplicates, but specifically an improved ability to remember unique colours from displays including duplicates compared with displays comprising all uniquely coloured items. Furthermore, less effort should be required to retain displays as colour redundancy increases. I recorded gaze position and pupil sizes during a visual change detection task including displays of 4–6 items with either all unique colours, two items with a common colour, or three items with a common colour in samples of young and healthy elderly adults. Increased redundancy was indeed associated with higher estimated working memory capacity, both for tests of duplicates and uniquely coloured items. Redundancy was also associated with decreased pupil size during retention, especially in young adults. While elderly adults also benefited from colour redundancy, spillover to unique items was less obvious with low redundancy than in young adults. This experiment confirms previous findings and presents complementary novel evidence linking perceptual grouping via colour redundancy with decreased mental effort.     

Resources masquerading as slots: Flexible allocation of visual working memory

Whether the capacity of visual working memory is better characterized by an item-based or a resource-based account continues to be keenly debated. Here, we propose that visual working memory is a flexible resource that is sometimes deployed in a slot-like manner. We develop a computational model that can either encode all items in a memory set, or encode only a subset of those items. A fixed-capacity mnemonic resource is divided among the items in memory. When fewer items are encoded, they are each remembered with higher fidelity, but at the cost of having to rely on an explicit guessing process when probed about an item that is not in memory. We use the new model to test the prediction that participants will more often encode the entire set of items when the demands on memory are predictable.     

The size of an attentional window affects working memory guidance

We ask whether attentional guidance from working memory (WM) is influenced by the size of an attentional window. Participants adopted either a focused or a diffuse attentional window when responding to a search display. Prior to the search display an initial cue had to be held in memory (Experiment 1A, visual WM; Experiment 1C, verbal WM) or merely identified (Experiment 1B, identification). In all cases, search performance was affected by the re-presentation of the cue in the search display, with the cuing effects (either cost or benefit) being larger when the cue was held in memory than when it was merely identified. Critically, the magnitude of the cuing benefit increased when participants adopted a diffuse attentional window. This held for effects that are based on items held in WM and for effects that are based only on item priming. The results suggest that variations in the size of an attentional window modulate top-down (both WM conditions) as well as bottom-up guidance of attention (identification condition).     

Guided search through memory

In “hybrid” search, observers search a visual display for any of several targets held in memory. It is known that the contents of the memory set can guide visual search (e.g., if the memorized targets are all animals, visual attention can be guided away from signs). It is not known if the visual display can guide memory search (e.g., if the memory set is composed of signs and animals, can a visual display of signs restrict memory search to just the signs?). In three hybrid search experiments, participants memorized sets of items that belonged to either one or several categories. Participants were then presented with visual displays containing multiple items, also drawn from one or several categories. Participants were asked to determine if any of the items from their current memory set were present in the visual display. We replicate the finding that visual search can be guided by the contents of memory. We find weaker, novel evidence that memory search can be guided by the contents of the visual display.     

Influence of emotion on memory for temporal information

Contextual information, such as color and spatial location, has been found to be better remembered for emotional than for neutral items. The current study examined whether the influence of emotion extends to memory for another fundamental feature of episodic memory: temporal information. Results from a list-discrimination paradigm showed that (a) item memory was enhanced for both negative and positive pictures compared with neutral ones and was better for negative than for positive pictures and (b) temporal information was better remembered for negative than for positive and neutral pictures, whereas positive and neutral pictures did not differ from each other. These findings are discussed in relation to the processes involved in memory for temporal information.     

Working memory for relations among objects

Across many areas of study in cognition, the capacity of working memory (WM) is widely agreed to be roughly three to five items: three to five objects (i.e., bound collections of object features) in the literature on visual WM or three to five role bindings (i.e., objects in specific relational roles) in the literature on memory and reasoning. Three experiments investigated the capacity of observers’ WM for the spatial relations among objects in a visual display, and the results suggest that the “items” in WM are neither simply objects nor simply role bindings. The results of Experiment 1 are most consistent with a model that treats an “item” in visual WM as an object, along with the roles of all its relations to one other object. Experiment 2 compared observers’ WM for object size with their memory for relative size and provided evidence that observers compute and store objects’ relations per se (rather than just absolute size) in WM. Experiment 3 tested and confirmed several more nuanced predictions of the model supported by Experiment 1. Together, these findings suggest that objects are stored in visual WM in pairs (along with all the relations between the objects in a pair) and that, from the perspective of WM, a given object in one pair is not the same “item” as that same object in a different pair.     

Electrophysiological correlates of encoding processes in a full-report visual working memory paradigm

Why are some visual stimuli remembered, whereas others are forgotten? A limitation of recognition paradigms is that they measure aggregate behavioral performance and/or neural responses to all stimuli presented in a visual working memory (VWM) array. To address this limitation, we paired an electroencephalography (EEG) frequency-tagging technique with two full-report VWM paradigms. This permitted the tracking of individual stimuli as well as the aggregate response. We recorded high-density EEG (256 channel) while participants viewed four shape stimuli, each flickering at a different frequency. At retrieval, participants either recalled the location of all stimuli in any order (simultaneous full report) or were cued to report the item in a particular location over multiple screen displays (sequential full report). The individual frequency tag amplitudes evoked for correctly recalled items were significantly larger than the amplitudes of subsequently forgotten stimuli, regardless of retrieval task. An induced-power analysis examined the aggregate neural correlates of VWM encoding as a function of items correctly recalled. We found increased induced power across a large number of electrodes in the theta, alpha, and beta frequency bands when more items were successfully recalled. This effect was more robust for sequential full report, suggesting that retrieval demands can influence encoding processes. These data are consistent with a model in which encoding-related resources are directed to a subset of items, rather than a model in which resources are allocated evenly across the array. These data extend previous work using recognition paradigms and stress the importance of encoding in determining later VWM retrieval success.     

Culture impacts the magnitude of the emotion-induced memory trade-off effect

The present study assessed the extent to which culture impacts the emotion-induced memory trade-off effect. This trade-off effect occurs because emotional items are better remembered than neutral ones, but this advantage comes at the expense of memory for backgrounds such that neutral backgrounds are remembered worse when they occurred with an emotional item than with a neutral one. Cultures differ in their prioritisation of focal object versus contextual background information, with Westerners focusing more on objects and Easterners focusing more on backgrounds. Americans, a Western culture, and Turks, an Eastern-influenced culture, incidentally encoded positive, negative, and neutral items placed against neutral backgrounds, and then completed a surprise memory test with the items and backgrounds tested separately. Results revealed a reduced trade-off for Turks compared to Americans. Although both groups exhibited an emotional enhancement in item memory, Turks did not show a decrement in memory for backgrounds that had been paired with emotional items. These findings complement prior ones showing reductions in trade-off effects as a result of task instructions. Here, we suggest that a contextual-focus at the level of culture can mitigate trade-off effects in emotional memory.     

Search through complex motion displays does not break down under spatial memory load

Recent work has shown that observers are remarkably effective in searching displays of randomly moving items. In two experiments, we combined working memory tasks with visual search, to test whether search through such complex motion displays, as compared with search through static items, places an extra burden on spatial working memory. In our first experiment, we show that the dual-task interference observed for motion search is specific to spatial working memory, in line with earlier work for static search. In our second experiment, we found dual-task interference for both static and motion search, but no difference between them. The results support the suggestion that the same search process is active during search among static and search among moving items.     

Working memory, perceptual priming, and the perception of hierarchical forms: Opposite effects of priming and working memory without memory refreshing

Previous research has shown that stimuli held in working memory (WM) can influence spatial attention. Using Navon stimuli, we explored whether and how items in WM affect the perception of visual targets at local and global levels in compound letters. Participants looked for a target letter presented at a local or global level while holding a regular block letter as a memory item. An effect of holding the target’s identity in WM was found. When memory items and targets were the same, performance was better than in a neutral condition when the memory item did not appear in the hierarchical letter (a benefit from valid cuing). When the memory item matched the distractor in the hierarchical stimulus, performance was worse than in the neutral baseline (a cost on invalid trials). These effects were greatest when the WM cue matched the global level of the hierarchical stimulus, suggesting that WM biases attention to the global level of form. Interestingly, in a no-memory priming condition, target perception was faster in the invalid condition than in the neutral baseline, reversing the effect in the WM condition. A further control experiment ruled out the effects of WM being due to participants’ refreshing their memory from the hierarchical stimulus display. The data show that information in WM biases the selection of hierarchical forms, whereas priming does not. Priming alters the perceptual processing of repeated stimuli without biasing attention.     

The contents of visual working memory reduce uncertainty during visual search

Information held in visual working memory (VWM) influences the allocation of attention during visual search, with targets matching the contents of VWM receiving processing benefits over those that do not. Such an effect could arise from multiple mechanisms: First, it is possible that the contents of working memory enhance the perceptual representation of the target. Alternatively, it is possible that when a target is presented among distractor items, the contents of working memory operate postperceptually to reduce uncertainty about the location of the target. In both cases, a match between the contents of VWM and the target should lead to facilitated processing. However, each effect makes distinct predictions regarding set-size manipulations; whereas perceptual enhancement accounts predict processing benefits regardless of set size, uncertainty reduction accounts predict benefits only with set sizes larger than 1, when there is uncertainty regarding the target location. In the present study, in which briefly presented, masked targets were presented in isolation, there was a negligible effect of the information held in VWM on target discrimination. However, in displays containing multiple masked items, information held in VWM strongly affected target discrimination. These results argue that working memory representations act at a postperceptual level to reduce uncertainty during visual search.     

Expertise for upright faces improves the precision but not the capacity of visual working memory

Considerable research has focused on how basic visual features are maintained in working memory, but little is currently known about the precision or capacity of visual working memory for complex objects. How precisely can an object be remembered, and to what extent might familiarity or perceptual expertise contribute to working memory performance? To address these questions, we developed a set of computer-generated face stimuli that varied continuously along the dimensions of age and gender, and we probed participants’ memories using a method-of-adjustment reporting procedure. This paradigm allowed us to separately estimate the precision and capacity of working memory for individual faces, on the basis of the assumptions of a discrete capacity model, and to assess the impact of face inversion on memory performance. We found that observers could maintain up to four to five items on average, with equally good memory capacity for upright and upside-down faces. In contrast, memory precision was significantly impaired by face inversion at every set size tested. Our results demonstrate that the precision of visual working memory for a complex stimulus is not strictly fixed but, instead, can be modified by learning and experience. We find that perceptual expertise for upright faces leads to significant improvements in visual precision, without modifying the capacity of working memory.     

Visual short-term memory operates more efficiently on boundary features than on surface features

A change detection task was used to estimate the visual short-term memory storage capacity for either the orientation or the size of objects. On each trial, several,objects were briefly presented, followed by a blank interval and then by a second display of objects that either was identical to the first display or had a single object that was different (the object changed either orientation or size, in separate experiments). The task was to indicate whether the two displays were the same or different, and the number of objects remembered was estimated from the percent correct on this task. Storage capacity for a feature was nearly twice as large when that feature was defined by the object boundary, rather than by the surface texture of the object. This dramatic difference in storage capacity suggests that a particular feature (e.g., right tilted or small) is not stored in memory with an invariant abstract code. Instead, there appear to be different codes for the boundary and surface features of objects, and memory operates on boundary features more efficiently than it operates on surface features.     

Reconstructing the recent visual past: Hierarchical knowledge-based effects in visual working memory

This paper presents two experiments that examine the influence of multiple levels of knowledge on visual working memory (VWM). Experiment 1 focused on memory for faces. Faces were selected from continua that were constructed by morphing two face photographs in 100 steps; half of the continua morphed a famous face into an unfamiliar one, while the other half used two unfamiliar faces. Participants studied six sequentially presented faces each from a different continuum, and at test they had to locate one of these within its continuum. Experiment 2 examined immediate memory for object sizes. On each trial, six images were shown; these were either all vegetables or all random shapes. Immediately after each list, one item was presented again, in a new random size, and participants reproduced its studied size. Results suggested that two levels of knowledge influenced VWM. First, there was an overall central-tendency bias whereby items were remembered as being closer to the overall average or central tokens (averaged across items and trials) than they actually were. Second, when object knowledge was available for the to-be-remembered items (i.e., famous face or typical size of a vegetable) a further bias was introduced in responses. The results extend the findings of Hemmer and Steyvers (Psychonomic Bulletin & Review, 16, 80–87, 2009a) from episodic memory to VWM and contribute to the growing literature which illustrates the complexity and flexibility of the representations subtending VWM performance (e.g., Bae, Olkkonen, Allred, & Flombaum, Journal of Experimental Psychology: General, 144(4):744–63, 2015).     

Asymmetrical access to color and location in visual working memory

Models of visual working memory (VWM) have benefitted greatly from the use of the delayed-matching paradigm. However, in this task, the ability to recall a probed feature is confounded with the ability to maintain the proper binding between the feature that is to be reported and the feature (typically location) that is used to cue a particular item for report. Given that location is typically used as a cue-feature, we used the delayed-estimation paradigm to compare memory for location to memory for color, rotating which feature was used as a cue and which was reported. Our results revealed several novel findings: 1) the likelihood of reporting a probed object’s feature was superior when reporting location with a color cue than when reporting color with a location cue; 2) location report errors were composed entirely of swap errors, with little to no random location reports; and 3) both colour and location reports greatly benefitted from the presence of nonprobed items at test. This last finding suggests that it is uncertainty over the bindings between locations and colors at memory retrieval that drive swap errors, not at encoding. We interpret our findings as consistent with a representational architecture that nests remembered object features within remembered locations.