Visual working memory for simple and complex visual stimuli

Does the magical number four characterize our visual working memory (VWM) capacity for all kinds of objects, or is the capacity of VWM inversely related to the perceptual complexity of those objects? To find out how perceptual complexity affects VWM, we used a change detection task to measure VWM capacity for six types of stimuli of different complexity: colors, letters, polygons, squiggles, cubes, and faces. We found that the estimated capacity decreased for more complex stimuli, suggesting that perceptual complexity was an important factor in determining VWM capacity. However, the considerable correlation between perceptual complexity and VWM capacity declined significantly if subjects were allowed to view the sample memory display longer. We conclude that when encoding limitations are minimized, perceptual complexity affects, but does not determine, VWM capacity.     

Updating positive and negative stimuli in working memory in depression

Difficulties in the ability to update stimuli in working memory (WM) may underlie the problems with regulating emotions that lead to the development and perpetuation of mood disorders such as depression. To examine the ability to update affective material in WM, the authors had diagnosed depressed and never-disordered control participants perform an emotion 2-back task in which participants were presented with a series of happy, sad, and neutral faces and were asked to indicate whether the current face had the same (match-set) or different (break-set or no-set) emotional expression as that presented 2 faces earlier. Participants also performed a 0-back task with the same emotional stimuli to serve as a control for perceptual processing. After transforming reaction times to control for baseline group differences, depressed and nondepressed participants exhibited biases in updating emotional content that reflects the tendency to keep negative information and positive information, respectively, active in WM. Compared with controls, depressed participants were both slower to disengage from sad stimuli and faster to disengage from happy facial expressions. In contrast, nondepressed controls took longer to disengage from happy stimuli than from neutral or sad stimuli. These group differences in reaction times may reflect both protective and maladaptive biases in WM that underlie the ability to effectively regulate negative affect.     

Trade-offs in visual attention and the enhancement of memory specificity for positive and negative emotional stimuli

Emotional influences on memory can lead to trade-offs in memory for gist or detail and trade-offs in memory for central or peripheral aspects of an event. Attentional narrowing has often been proposed as a theoretical explanation for this pattern of findings with negative emotion. These trade-offs have been less extensively investigated with positive emotion. In three experiments, we investigate memory for specific visual details of positive and negative stimuli, examine central–peripheral trade-offs in memory, and assess the hypothesis that attentional narrowing contributes to emotional enhancement of memory specificity. We found that memory for details was enhanced by negative and positive emotion. Central–peripheral trade-offs were found in memory for negative emotional stimuli but not in memory for positive emotional stimuli. These trade-offs with negative emotion were associated with attentional narrowing at the time of encoding, as measured by eye movements. There were no attentional effects at the time of encoding found with positive emotional stimuli. Evidence was found for the attentional narrowing hypothesis of memory specificity and central–peripheral trade-offs in memory for negative emotional events. Alternative explanations for the positive emotional enhancement of memory specificity are required.     

Familiarity enhances visual working memory for faces

Although it is intuitive that familiarity with complex visual objects should aid their preservation in visual working memory (WM), empirical evidence for this is lacking. This study used a conventional change-detection procedure to assess visual WM for unfamiliar and famous faces in healthy adults. Across experiments, faces were upright or inverted and a low- or high-load concurrent verbal WM task was administered to suppress contribution from verbal WM. Even with a high verbal memory load, visual WM performance was significantly better and capacity estimated as significantly greater for famous versus unfamiliar faces. Face inversion abolished this effect. Thus, neither strategic, explicit support from verbal WM nor low-level feature processing easily accounts for the observed benefit of high familiarity for visual WM. These results demonstrate that storage of items in visual WM can be enhanced if robust visual representations of them already exist in long-term memory.     

Coordination in visual working memory

Coordination of mental procedures is considered in terms of control processes (Baddeley, 1989) in visual working memory and appears to be a separable aspect of the demand imposed by cascaded serial processes (Carlson & Lundy, 1992). The main task required subjects to indicate whether symbolically suggested rotations and reflections correctly describe the difference between matrix patterns of filled-in squares within a 3 x 3 grid or between line drawings. Experiments were carried out to show that coordination is a separable component in this transformation task. A marker for coordination is the difference between the time taken to execute two transformations as a whole and the sum of the component transformations in isolation. The separate coordination demand was found in an experiment with matrix patterns mentioned, in an experiment with letter-like line drawings, and also in an experiment that forced subjects to maintain whole-pattern representations. A last experiment checked whether coordination is carried out by an autonomous control unit. There was a self-paced control of serial presentation of transformation symbols instead of a simultaneous presentation of those symbols. This additional external triggering resulted in a substantial decrease in the demand for coordination. Coordination of mental procedures and temporary representations is a fundamental constraint on the use of working-memory processes.     

Aging and guided visual search: the role of visual working memory

In the present study, we examined age-related differences in utilizing learned associations to guide visual search. Participants viewed an object cue that was associated with 1 or 2 target colors, followed by a search array. Older adults showed slower RTs, larger visual search slopes, and reduced cue–target association knowledge than did younger adults. We also found increased search RTs and higher error rates when the cue was associated with 2 colors instead of 1. However, visual search slopes did not vary with the number of associated colors. This indicates that participants could activate multiple templates simultaneously to guide the search or retrieve associative information directly from the long-term memory. Furthermore, regression analyses showed that VWMC can predict both visual search performances and cue–target association knowledge. More broadly, our results demonstrate that VWMC can modify the effects of age on cued visual search performance.     

Infants' recognition memory for a series of visual stimuli

The tendency of infants to distribute attention selectively to novel and familiar visual stimuli was employed to study infants' recognition memory for a series of visual targets. Infants 5 months of age demonstrated an unequal distribution of visual fixation to novel and familiar stimuli, with more attention to the novel, on both immediate and delayed stimulus-recognition tests for each of three novelty problems administered during a single testing session. The degree of differential fixation to novel targets exhibited no reliable decline from immediate to delayed testing and was not significantly altered by the serial order which the problem occupied during immediate recognition testing.     

The aftermath of memory retrieval for recycling visual working memory representations

We examined the aftermath of accessing and retrieving a subset of information stored in visual working memory (VWM)—namely, whether detection of a mismatch between memory and perception can impair the original memory of an item while triggering recognition-induced forgetting for the remaining, untested items. For this purpose, we devised a consecutive-change detection task wherein two successive testing probes were displayed after a single set of memory items. Across two experiments utilizing different memory-testing methods (whole vs. single probe), we observed a reliable pattern of poor performance in change detection for the second test when the first test had exhibited a color change. The impairment after a color change was evident even when the same memory item was repeatedly probed; this suggests that an attention-driven, salient visual change made it difficult to reinstate the previously remembered item. The second change detection, for memory items untested during the first change detection, was also found to be inaccurate, indicating that recognition-induced forgetting had occurred for the unprobed items in VWM. In a third experiment, we conducted a task that involved change detection plus continuous recall, wherein a memory recall task was presented after the change detection task. The analyses of the distributions of recall errors with a probabilistic mixture model revealed that the memory impairments from both visual changes and recognition-induced forgetting are explained better by the stochastic loss of memory items than by their degraded resolution. These results indicate that attention-driven visual change and recognition-induced forgetting jointly influence the “recycling” of VWM representations.     

Distinct capacity limits for attention and working memory: Evidence from attentive tracking and visual working memory paradigms

A hallmark of both visual attention and working memory is their severe capacity limit: People can attentively track only about four objects in a multiple object tracking (MOT) task and can hold only up to four objects in visual working memory (VWM). It has been proposed that attention underlies the capacity limit of VWM. We tested this hypothesis by determining the effect of varying the load of a MOT task performed during the retention interval of a VWM task and comparing the resulting dual-task costs with those observed when a VWM task was performed concurrently with another VWM task or with a verbal working memory task. Instead of supporting the view that the capacity limit of VWM is solely attention based, the results indicate that VWM capacity is set by the interaction of visuospatial attentional, central amodal, and local task-specific sources of processing.     

Interactions between visual working memory representations

We investigated whether the representations of different objects are maintained independently in working memory or interact with each other. Observers were shown two sequentially presented orientations and required to reproduce each orientation after a delay. The sequential presentation minimized perceptual interactions so that we could isolate interactions between memory representations per se. We found that similar orientations were repelled from each other whereas dissimilar orientations were attracted to each other. In addition, when one of the items was given greater attentional priority by means of a cue, the representation of the high-priority item was not influenced very much by the orientation of the low-priority item, but the representation of the low-priority item was strongly influenced by the orientation of the high-priority item. This indicates that attention modulates the interactions between working memory representations. In addition, errors in the reported orientations of the two objects were positively correlated under some conditions, suggesting that representations of distinct objects may become grouped together in memory. Together, these results demonstrate that working-memory representations are not independent but instead interact with each other in a manner that depends on attentional priority.     

Auditory and visual spatial working memory

A series of experiments compared short-term memory for object locations in the auditory and visual modalities. The stimulus materials consisted of sounds and pictures presented at different locations in space. Items were presented in pure- or mixed-modality lists of increasing length. At test, participants responded to renewed presentation of the objects by indicating their original position. If two independent modality-specific and resource-limited short-term memories support the remembering of locations, memory performance should be higher in the mixed-modality than in the pure-modality condition. Yet, memory performance was the same for items in both types of list. In addition, responses to the memory load manipulation in both modalities showed very similar declines in performance. The results are interpreted in terms of object files binding object and location information in episodic working memory, independently of the input modality.     

Perceptual organization influences visual working memory

Previous studies have demonstrated that top-down factors can bias the storage of information in visual working memory. However, relatively little is known about the role that bottom-up stimulus characteristics play in visual working memory storage. In the present study, subjects performed a change detection task in which the to-be-remembered objects were organized in accordance with Gestalt grouping principles. When an attention-capturing cue was presented at the location of one object, other objects that were perceptually grouped with the cued object were more likely to be stored in working memory than were objects that were not grouped with the cued object. Thus, objects that are grouped together tend to be stored together, indicating that bottom-up perceptual organization influences the storage of information in visual working memory.     

Similarity effects in visual working memory

Perceptual similarity is an important property of multiple stimuli. Its computation supports a wide range of cognitive functions, including reasoning, categorization, and memory recognition. It is important, therefore, to determine why previous research has found conflicting effects of inter-item similarity on visual working memory. Studies reporting a similarity advantage have used simple stimuli whose similarity varied along a featural continuum. Studies reporting a similarity disadvantage have used complex stimuli from either a single or multiple categories. To elucidate stimulus conditions for similarity effects in visual working memory, we tested memory for complex stimuli (faces) whose similarity varied along a morph continuum. Participants encoded 3 morphs generated from a single face identity in the similar condition, or 3 morphs generated from different face identities in the dissimilar condition. After a brief delay, a test face appeared at one of the encoding locations for participants to make a same/different judgment. Two experiments showed that similarity enhanced memory accuracy without changing the response criterion. These findings support previous computational models that incorporate featural variance as a component of working memory load. They delineate limitations of models that emphasize cortical resources or response decisions.     

Distractor devaluation requires visual working memory

Visual stimuli seen previously as distractors in a visual search task are subsequently evaluated more negatively than those seen as targets. An attentional inhibition account for this distractor-devaluation effect posits that associative links between attentional inhibition and to-be-ignored stimuli are established during search, stored, and then later reinstantiated, implying that distractor devaluation may require visual working memory (WM) resources. To assess this, we measured distractor devaluation with and without a concurrent visual WM load. Participants viewed a memory array, performed a simple search task, evaluated one of the search items (or a novel item), and then viewed a memory test array. Although distractor devaluation was observed with low (and no) WM load, it was absent when WM load was increased. This result supports the notions that active association of current attentional states with stimuli requires WM and that memory for these associations plays a role in affective response.     

Reconstructing the recent visual past: Hierarchical knowledge-based effects in visual working memory

This paper presents two experiments that examine the influence of multiple levels of knowledge on visual working memory (VWM). Experiment 1 focused on memory for faces. Faces were selected from continua that were constructed by morphing two face photographs in 100 steps; half of the continua morphed a famous face into an unfamiliar one, while the other half used two unfamiliar faces. Participants studied six sequentially presented faces each from a different continuum, and at test they had to locate one of these within its continuum. Experiment 2 examined immediate memory for object sizes. On each trial, six images were shown; these were either all vegetables or all random shapes. Immediately after each list, one item was presented again, in a new random size, and participants reproduced its studied size. Results suggested that two levels of knowledge influenced VWM. First, there was an overall central-tendency bias whereby items were remembered as being closer to the overall average or central tokens (averaged across items and trials) than they actually were. Second, when object knowledge was available for the to-be-remembered items (i.e., famous face or typical size of a vegetable) a further bias was introduced in responses. The results extend the findings of Hemmer and Steyvers (Psychonomic Bulletin & Review, 16, 80–87, 2009a) from episodic memory to VWM and contribute to the growing literature which illustrates the complexity and flexibility of the representations subtending VWM performance (e.g., Bae, Olkkonen, Allred, & Flombaum, Journal of Experimental Psychology: General, 144(4):744–63, 2015).     

Hierarchical encoding in visual working memory: ensemble statistics bias memory for individual items

Influential models of visual working memory treat each item to be stored as an independent unit and assume that there are no interactions between items. However, real-world displays have structure that provides higher-order constraints on the items to be remembered. Even in the case of a display of simple colored circles, observers can compute statistics, such as mean circle size, to obtain an overall summary of the display. We examined the influence of such an ensemble statistic on visual working memory. We report evidence that the remembered size of each individual item in a display is biased toward the mean size of the set of items in the same color and the mean size of all items in the display. This suggests that visual working memory is constructive, encoding displays at multiple levels of abstraction and integrating across these levels, rather than maintaining a veridical representation of each item independently.     

Clear evidence for item limits in visual working memory

There is a consensus that visual working memory (WM) resources are sharply limited, but debate persists regarding the simple question of whether there is a limit to the total number of items that can be stored concurrently. Zhang and Luck (2008) advanced this debate with an analytic procedure that provided strong evidence for random guessing responses, but their findings can also be described by models that deny guessing while asserting a high prevalence of low precision memories. Here, we used a whole report memory procedure in which subjects reported all items in each trial and indicated whether they were guessing with each response. Critically, this procedure allowed us to measure memory performance for all items in each trial. When subjects were asked to remember 6 items, the response error distributions for about 3 out of the 6 items were best fit by a parameter-free guessing model (i.e. a uniform distribution). In addition, subjects’ self-reports of guessing precisely tracked the guessing rate estimated with a mixture model. Control experiments determined that guessing behavior was not due to output interference, and that there was still a high prevalence of guessing when subjects were instructed not to guess. Our novel approach yielded evidence that guesses, not low-precision representations, best explain limitations in working memory. These guesses also corroborate a capacity-limited working memory system – we found evidence that subjects are able to report non-zero information for only 3–4 items. Thus, WM capacity is constrained by an item limit that precludes the storage of more than 3–4 individuated feature values.     

A bilateral advantage for storage in visual working memory

Various studies have demonstrated enhanced visual processing when information is presented across both visual hemifields rather than in a single hemifield (the bilateral advantage). For example, Alvarez and Cavanagh (2005) reported that observers were able to track twice as many moving visual stimuli when the tracked items were presented bilaterally rather than unilaterally, suggesting that independent resources enable tracking in the two visual fields. Motivated by similarities in the apparent capacity and neural substrates that mediate tracking and visual working memory (WM), the present work examined whether or not a bilateral advantage also arises during storage in visual WM. Using a recall procedure to assess working memory for orientation information, we found a reliable bilateral advantage; recall error was smaller with bilateral sample displays than with unilateral displays. To demonstrate that the bilateral advantage influenced storage per se rather than just encoding efficiency, we replicated the observed bilateral advantage using sequentially presented stimuli. Finally, to further characterize how bilateral presentations enhanced storage in working memory, we measured both the number and the resolution of the stored items and found that bilateral presentations lead to an increased probability of storage, rather than enhanced mnemonic resolution. Thus, the bilateral advantage extends beyond the initial selection and encoding of visual information to influence online maintenance in visual working memory.     

Comparisons of memory for nonverbal auditory and visual sequential stimuli

Properties of auditory and visual sensory memory were compared by examining subjects' recognition performance of randomly generated binary auditory sequential frequency patterns and binary visual sequential color patterns within a forced-choice paradigm. Experiment 1 demonstrated serial-position effects in auditory and visual modalities consisting of both primacy and recency effects. Experiment 2 found that retention of auditory and visual information was remarkably similar when assessed across a 10 s interval. Experiments 3 and 4, taken together, showed that the recency effect in sensory memory is affected more by the type of response required (recognition vs. reproduction) than by the sensory modality employed. These studies suggest that auditory and visual sensory memory stores for nonverbal stimuli share similar properties with respect to serial-position effects and persistence over time.