Organization of visual short-term memory

The authors examined the organization of visual short-term memory (VSTM). Using a change-detection task, they reported that VSTM stores relational information between individual items. This relational processing is mediated by the organization of items into spatial configurations. The spatial configuration of visual objects is important for VSTM of spatial locations, colors, and shapes. When color VSTM is compared with location VSTM, spatial configuration plays an integral role because configuration is important for color VSTM, whereas color is not important for location VSTM. The authors also examined the role of attention and found that the formation of configuration is modulated by both top-down and bottom-up attentional factors. In summary, the authors proposed that VSTM stores the relational information of individual visual items on the basis of global spatial configuration.     

Visual short-term memory load suppresses temporo-parietal junction activity and induces inattentional blindness

The right temporo-parietal junction (TPJ) is critical for stimulus-driven attention and visual awareness. Here we show that as the visual short-term memory (VSTM) load of a task increases, activity in this region is increasingly suppressed. Correspondingly, increasing VSTM load impairs the ability of subjects to consciously detect the presence of a novel, unexpected object in the visual field. These results not only demonstrate that VSTM load suppresses TPJ activity and induces inattentional blindness, but also offer a plausible neural mechanism for this perceptual deficit: suppression of the stimulus-driven attentional network.     

Visual short-term memory benefit for objects on different 3-D surfaces

Visual short-term memory (VSTM) plays an important role in visual cognition. Although objects are located on different 3-dimensional (3-D) surfaces in the real world, how VSTM capacity may be influenced by the presence of multiple 3-D surfaces has never been examined. By manipulating binocular disparities of visual displays, the authors found that more colored objects could be held in VSTM when they were placed on 2 rather than on 1 planar 3-D surfaces. This between-surface benefit in VSTM was present only when binding of objects' colors to their 3-D locations was required (i.e., when observers needed to remember which color appeared where). When binding was not required, no between-surface benefit in VSTM was observed. This benefit in VSTM could not be attributed to the number of spatial locations attended within a given surface. It was not due to a general perceptual grouping effect either, because grouping by motion and grouping by different regions of the same surface did not yield the same benefit. This increment in capacity indicates that VSTM benefits from the placement of objects in a 3-D scene.     

Visual short-term memory load strengthens selective attention

Perceptual load theory accounts for many attentional phenomena; however, its mechanism remains elusive because it invokes underspecified attentional resources. Recent dual-task evidence has revealed that a concurrent visual short-term memory (VSTM) load slows visual search and reduces contrast sensitivity, but it is unknown whether a VSTM load also constricts attention in a canonical perceptual load task. If attentional selection draws upon VSTM resources, then distraction effects—which measure attentional “spill-over”—will be reduced as competition for resources increases. Observers performed a low perceptual load flanker task during the delay period of a VSTM change detection task. We observed a reduction of the flanker effect in the perceptual load task as a function of increasing concurrent VSTM load. These findings were not due to perceptual-level interactions between the physical displays of the two tasks. Our findings suggest that perceptual representations of distractor stimuli compete with the maintenance of visual representations held in memory. We conclude that access to VSTM determines the degree of attentional selectivity; when VSTM is not completely taxed, it is more likely for task-irrelevant items to be consolidated and, consequently, affect responses. The “resources” hypothesized by load theory are at least partly mnemonic in nature, due to the strong correspondence they share with VSTM capacity.     

Testing visual short-term memory of pigeons (Columba livia) and a rhesus monkey (Macaca mulatta) with a location change detection task

Change detection is commonly used to assess capacity (number of objects) of human visual short-term memory (VSTM). Comparisons with the performance of non-human animals completing similar tasks have shown similarities and differences in object-based VSTM, which is only one aspect (“what”) of memory. Another important aspect of memory, which has received less attention, is spatial short-term memory for “where” an object is in space. In this article, we show for the first time that a monkey and pigeons can be accurately trained to identify location changes, much as humans do, in change detection tasks similar to those used to test object capacity of VSTM. The subject’s task was to identify (touch/peck) an item that changed location across a brief delay. Both the monkey and pigeons showed transfer to delays longer than the training delay, to greater and smaller distance changes than in training, and to novel colors. These results are the first to demonstrate location-change detection in any non-human species and encourage comparative investigations into the nature of spatial and visual short-term memory.     

Visual short-term memory is not improved by training

A critical question in visual working or short-term memory (VSTM) research is whether the ability to remember briefly presented visual stimuli can be increased. Here we test whether VSTM for locations and shapes is improved by training that allows one to utilize another memory system, visual longterm memory (VLTM). Training was done by repeatedly presenting a subset of memory displays, creating long-term memory traces for these displays. Surprisingly, VSTM performance for repeated displays was not higher than for nonrepeated ones, even though participants recognized repeated displays on a forced-choice test given at the end of the experiment. We suggest that the fidelity of information held by VLTM is inferior to that of information held by VSTM and thus provides no additional benefit over what is extracted on the fly by VSTM.     

Object-position binding in visual short-term memory for sequentially presented unfamiliar stimuli

The effect of spatial position on visual short-term memory (VSTM) for sequentially presented objects has been investigated relatively little, despite the fact that vision in natural environments is characterised by frequent changes in object position and gaze location. We investigated the effect of reusing previously examined spatial positions on VSTM for object appearance. Observers performed a yes-no recognition task following a memory display comprising briefly presented 1/f noise discs (ie possessing spectral properties akin to natural images) shown sequentially at random coordinates. At test, single stimuli were presented either at original spatial positions, new positions, or at a fixed central position. Results, interpreted in terms of appearance and position preview effects, indicate that, where original spatial positions were reused at test, memory performance was elevated by more than 25%, despite that spatial position was task-irrelevant (in the sense that it could not be used to facilitate a correct response per se). This study generalises object-spatial-position binding theory to a sequential display scenario in which the influences of extrafoveal processing, spatial context cues, and long-term memory support were minimised, thereby eliminating the hypothesis that object priming is the principal cause of the 'same-position advantage' in VSTM.     

Distributing versus focusing attention in visual short-term memory

Visual short-term memory (VSTM) is traditionally considered a robust form of visual memory resistant to interference from subsequent visual input. This study shows that the robustness of VSTM depends on the way attention is allocated in VSTM. When attention is distributed across multiple memory items, VSTM for these items is vulnerable to interference from subsequent input, including passively viewed images and the postchange testing displays. Yet attention can readjust its focus in VSTM on the basis of an attentional orienting cue presented long after encoding. When attention is oriented to a particular memorized item, the memory is resistant to subsequent interference. This effect, however, is eliminated when the subset of items demanding focal attention exceeds one, suggesting that orienting attention in VSTM is less flexible than orienting attention in perception. We propose that the robustness of VSTM is influenced by whether attention is focused or distributed in VSTM.     

Eye movements and visual memory: detecting changes to saccade targets in scenes

Saccade-contingent change detection provides a powerful tool for investigating scene representation and scene memory. In the present study, critical objects presented within color images of naturalistic scenes were changed during a saccade toward or away from the target. During the saccade,the critical object was changed to another object type, to a visually different token of the same object type, or was deleted from the scene. There were three main results. First, the deletion of a saccade target was special: Detection performance for saccade target deletions was very good, and this level of performance did not decline with the amplitude of the saccade. In contrast, detection of type and token changes at the saccade target, and of all changes including deletions at a location that had just been fixated but was not the saccade target, decreased as the amplitude of the saccade increased. Second, detection performance for type and token changes, both when the changing object was the target of the saccade and when the object had just been fixated but was not the saccade target, was well above chance. Third, mean gaze durations were reliably elevated for those trials in which the change was not overtly detected. The results suggest that the presence of the saccade target plays a special role in trassaccadic integration, and together with other recent findings, suggest more generally that a relatively rich scene representation is retained across saccades and stored in visual memory.     

Sequential dynamics in visual short-term memory

Visual short-term memory (VSTM) is thought to help bridge across changes in visual input, and yet many studies of VSTM employ static displays. Here we investigate how VSTM copes with sequential input. In particular, we characterize the temporal dynamics of several different components of VSTM performance, including: storage probability, precision, variability in precision, guessing, and swapping. We used a variant of the continuous-report VSTM task developed for static displays, quantifying the contribution of each component with statistical likelihood estimation, as a function of serial position and set size. In Experiments 1 and 2, storage probability did not vary by serial position for small set sizes, but showed a small primacy effect and a robust recency effect for larger set sizes; precision did not vary by serial position or set size. In Experiment 3, the recency effect was shown to reflect an increased likelihood of swapping out items from earlier serial positions and swapping in later items, rather than an increased rate of guessing for earlier items. Indeed, a model that incorporated responding to non-targets provided a better fit to these data than alternative models that did not allow for swapping or that tried to account for variable precision. These findings suggest that VSTM is updated in a first-in-first-out manner, and they bring VSTM research into closer alignment with classical working memory research that focuses on sequential behavior and interference effects.     

Posterior parietal cortex activity predicts individual differences in visual short-term memory capacity

Humans show a severe capacity limit in the number of objects they can store in visual short-term memory (VSTM). We recently demonstrated with functional magnetic resonance imaging that VSTM storage capacity estimated in averaged group data correlated strongly with posterior parietal/superior occipital cortex activity (Todd & Marois, 2004). However, individuals varied widely in their VSTM capacity. Here, we examined the neural basis of these individual differences. A voxelwise, individualdifferences analysis revealed a significant correlation between posterior parietal cortex (PPC) activity and individuals’ VSTM storage capacity. In addition, a region-of-interest analysis indicated that other brain regions, particularly visual occipital cortex, may contribute to individual differences in VSTM capacity. Thus, although not ruling out contributions from other brain regions, the individual-differences approach supports a key role for the PPC in VSTM by demonstrating that its activity level predicts individual differences in VSTM storage capacity.     

How memory mechanisms are a key component in the guidance of our eye movements: Evidence from the global effect

Investigating eye movements has been a promising approach to uncover the role of visual working memory in early attentional processes. Prior research has already demonstrated that eye movements in search tasks are more easily drawn toward stimuli that show similarities to working memory content, as compared with neutral stimuli. Previous saccade tasks, however, have always required a selection process, thereby automatically recruiting working memory. The present study was an attempt to confirm the role of working memory in oculomotor selection in an unbiased saccade task that rendered memory mechanisms irrelevant. Participants executed a saccade in a display with two elements, without any instruction to aim for one particular element. The results show that when two objects appear simultaneously, a working memory match attracts the first saccade more profoundly than do mismatch objects, an effect that was present throughout the saccade latency distribution. These findings demonstrate that memory plays a fundamental biasing role in the earliest competitive processes in the selection of visual objects, even when working memory is not recruited during selection.     

Visual cognition influences early vision: the role of visual short-term memory in amodal completion

A partly occluded visual object is perceptually filled in behind the occluding surface, a process known as amodal completion or visual interpolation. Previous research focused on the image-based properties that lead to amodal completion. In the present experiments, we examined the role of a higher-level visual process-visual short-term memory (VSTM)-in amodal completion. We measured the degree of amodal completion by asking participants to perform an object-based attention task on occluded objects while maintaining either zero or four items in visual working memory. When no items were stored in VSTM, participants completed the occluded objects; when four items were stored in VSTM, amodal completion was halted (Experiment 1). These results were not caused by the influence of VSTM on object-based attention per se (Experiment 2) or by the specific location of to-be-remembered items (Experiment 3). Items held in VSTM interfere with amodal completion, which suggests that amodal completion may not be an informationally encapsulated process, but rather can be affected by high-level visual processes.     

Visual short-term memory for sequential arrays

The capacity of visual short-term memory (VSTM) for a single visual display has been investigated in past research, but VSTM for multiple sequential arrays has been explored only recently. In this study, we investigate the capacity of VSTM across two sequential arrays separated by a variable stimulus onset asynchrony (SOA). VSTM for spatial locations (Experiment 1), colors (Experiments 2-4), orientations (Experiments 3 and 4), and conjunction of color and orientation (Experiment 4) were tested, with the SOA across the two sequential arrays varying from 100 to 1,500 msec. We find that VSTM for the trailing array is much better than VSTM for the leading array, but when averaged across the two arrays VSTM has a constant capacity independent of the SOA. We suggest that multiple displays compete for retention in VSTM and that separating information into two temporally discrete groups does not enhance the overall capacity of VSTM.     

Visual short-term memory during smooth pursuit eye movements

Visual short-term memory (VSTM) was probed while observers performed smooth pursuit eye movements. Smooth pursuit keeps a moving object stabilized in the fovea. VSTM capacity for position was reduced during smooth pursuit compared with a condition with eye fixation. There was no difference between a condition in which the items were approximately stabilized on the retina because they moved with the pursuit target and a condition in which the items moved across the retina because they were stationary in space. The reduction of capacity for position was eliminated when miniature items were presented on the pursuit target. Similarly, VSTM capacity for color did not differ between smooth pursuit and fixation. The results suggest that visuospatial attention is tied to the target during smooth pursuit, which impairs VSTM for the position of peripheral objects. Sensory memory during smooth pursuit was only slightly impaired.     

Enhanced visual short-term memory in action video game players

Visual short-term memory (VSTM) is critical for acquiring visual knowledge and shows marked individual variability. Previous work has illustrated a VSTM advantage among action video game players (Boot et al. Acta Psychologica 129:387–398, 2008). A growing body of literature has suggested that action video game playing can bolster visual cognitive abilities in a domain-general manner, including abilities related to visual attention and the speed of processing, providing some potential bases for this VSTM advantage. In the present study, we investigated the VSTM advantage among video game players and assessed whether enhanced processing speed can account for this advantage. Experiment 1, using simple colored stimuli, revealed that action video game players demonstrate a similar VSTM advantage over nongamers, regardless of whether they are given limited or ample time to encode items into memory. Experiment 2, using complex shapes as the stimuli to increase the processing demands of the task, replicated this VSTM advantage, irrespective of encoding duration. These findings are inconsistent with a speed-of-processing account of this advantage. An alternative, attentional account, grounded in the existing literature on the visuo-cognitive consequences of video game play, is discussed.     

The role of central attention in retrieval from visual short-term memory

The role of central attention in visual short-term memory (VSTM) encoding and maintenance is well established, yet its role in retrieval has been largely unexplored. This study examined the involvement of central attention in retrieval from VSTM using a dual-task paradigm. Participants performed a color change-detection task. Set size varied between 1 and 3 items, and the memory sample was maintained for either a short or a long delay period. A secondary tone discrimination task was introduced at the end of the delay period, shortly before the appearance of a central probe, and occupied central attention while participants were searching within VSTM representations. Similarly to numerous previous studies, reaction time increased as a function of set size reflecting the occurrence of a capacity-limited memory search. When the color targets were maintained over a short delay, memory was searched for the most part without the involvement of central attention. However, with a longer delay period, the search relied entirely on the operation of central attention. Taken together, this study demonstrates that central attention is involved in retrieval from VSTM, but the extent of its involvement depends on the duration of the delay period. Future studies will determine whether the type of memory search (parallel or serial) carried out during retrieval depends on the nature of the attentional mechanism involved the task.     

The capacity of visual short-term memory is not a fixed number of objects

Luck and Vogel (1997) have reported several striking results in support of the view that visual short-term memory (VSTM) has a fixed capacity of four objects, irrespective of how many relevant features those objects comprise. However, more recent studies have challenged this account, indicating only a weak effect of the number of objects once other factors are more evenly equated across conditions. Here, we employed a symmetry manipulation to verify object segmentation in our displays, to demonstrate that when spatial and masking factors are held constant, the number of objects per se has no effect on VSTM. Instead, VSTM capacity may reflect the number of object "parts" or feature conjunctions in a given display.     

Is visual short-term memory object based? Rejection of the “strong-object” hypothesis

Is the capacity of visual short-term memory (VSTM) limited by the number of objects or by the number of features? VSTM for objects with either one feature or two color features was tested. Results show that capacity was limited primarily by the number of colors to be memorized, not by the number of objects. This result held up with variations in color saturation, blocked or mixed conditions, duration of memory image, and absence or presence of verbal load. However, conjoining features into objects improved VSTM capacity when size-orientation and color-orientation conjunctions were tested. Nevertheless, the number of features still mattered. When feature heterogeneity was controlled, VSTM for conjoined objects was worse than VSTM for objects made of single features. Our results support a weak-object hypothesis of VSTM capacity that suggests that VSTM is limited by both the number of objects and the feature composition of those objects.     

Toward ecologically realistic theories in visual short-term memory research

Recent evidence from neuroimaging and psychophysics suggests common neural and representational substrates for visual perception and visual short-term memory (VSTM). Visual perception is adapted to a rich set of statistical regularities present in the natural visual environment. Common neural and representational substrates for visual perception and VSTM suggest that VSTM is adapted to these same statistical regularities too. This article discusses how the study of VSTM can be extended to stimuli that are ecologically more realistic than those commonly used in standard VSTM experiments and what the implications of such an extension could be for our current view of VSTM. We advocate for the development of unified models of visual perception and VSTM—probabilistic and hierarchical in nature—incorporating prior knowledge of natural scene statistics.