A bilateral advantage for maintaining objects in visual short term memory

Research has shown that attentional pre-cues can subsequently influence the transfer of information into visual short term memory (VSTM) (Schmidt, B., Vogel, E., Woodman, G., & Luck, S. (2002). Voluntary and automatic attentional control of visual working memory. Perception & Psychophysics, 64(5), 754–763). However, studies also suggest that those effects are constrained by the hemifield alignment of the pre-cues (Holt, J. L., & Delvenne, J.-F. (2014). A bilateral advantage in controlling access to visual short-term memory. Experimental Psychology, 61(2), 127–133), revealing better recall when distributed across hemifields relative to within a single hemifield (otherwise known as a bilateral field advantage). By manipulating the duration of the retention interval in a colour change detection task (1 s, 3 s), we investigated whether selective pre-cues can also influence how information is later maintained in VSTM. The results revealed that the pre-cues influenced the maintenance of the colours in VSTM, promoting consistent performance across retention intervals (Experiments 1 & 4). However, those effects were only shown when the pre-cues were directed to stimuli displayed across hemifields relative to stimuli within a single hemifield. Importantly, the results were not replicated when participants were required to memorise colours (Experiment 2) or locations (Experiment 3) in the absence of spatial pre-cues. Those findings strongly suggest that attentional pre-cues have a strong influence on both the transfer of information in VSTM and its subsequent maintenance, allowing bilateral items to better survive decay.     

Visual short-term memory is not improved by training

A critical question in visual working or short-term memory (VSTM) research is whether the ability to remember briefly presented visual stimuli can be increased. Here we test whether VSTM for locations and shapes is improved by training that allows one to utilize another memory system, visual longterm memory (VLTM). Training was done by repeatedly presenting a subset of memory displays, creating long-term memory traces for these displays. Surprisingly, VSTM performance for repeated displays was not higher than for nonrepeated ones, even though participants recognized repeated displays on a forced-choice test given at the end of the experiment. We suggest that the fidelity of information held by VLTM is inferior to that of information held by VSTM and thus provides no additional benefit over what is extracted on the fly by VSTM.     

Task-irrelevant distractors in the delay period interfere selectively with visual short-term memory for spatial locations

Visual short-term memory (VSTM) enables the representation of information in a readily accessible state. VSTM is typically conceptualized as a form of “active” storage that is resistant to interference or disruption, yet several recent studies have shown that under some circumstances task-irrelevant distractors may indeed disrupt performance. Here, we investigated how task-irrelevant visual distractors affected VSTM by asking whether distractors induce a general loss of remembered information or selectively interfere with memory representations. In a VSTM task, participants recalled the spatial location of a target visual stimulus after a delay in which distractors were presented on 75% of trials. Notably, the distractor’s eccentricity always matched the eccentricity of the target, while in the critical conditions the distractor’s angular position was shifted either clockwise or counterclockwise relative to the target. We then computed estimates of recall error for both eccentricity and polar angle. A general interference model would predict an effect of distractors on both polar angle and eccentricity errors, while a selective interference model would predict effects of distractors on angle but not on eccentricity errors. Results showed that for stimulus angle there was an increase in the magnitude and variability of recall errors. However, distractors had no effect on estimates of stimulus eccentricity. Our results suggest that distractors selectively interfere with VSTM for spatial locations.     

Internal attention to features in visual short-term memory guides object learning

Attending to objects in the world affects how we perceive and remember them. What are the consequences of attending to an object in mind? In particular, how does reporting the features of a recently seen object guide visual learning? In three experiments, observers were presented with abstract shapes in a particular color, orientation, and location. After viewing each object, observers were cued to report one feature from visual short-term memory (VSTM). In a subsequent test, observers were cued to report features of the same objects from visual long-term memory (VLTM). We tested whether reporting a feature from VSTM: (1) enhances VLTM for just that feature (practice-benefit hypothesis), (2) enhances VLTM for all features (object-based hypothesis), or (3) simultaneously enhances VLTM for that feature and suppresses VLTM for unreported features (feature-competition hypothesis). The results provided support for the feature-competition hypothesis, whereby the representation of an object in VLTM was biased towards features reported from VSTM and away from unreported features (Experiment 1). This bias could not be explained by the amount of sensory exposure or response learning (Experiment 2) and was amplified by the reporting of multiple features (Experiment 3). Taken together, these results suggest that selective internal attention induces competitive dynamics among features during visual learning, flexibly tuning object representations to align with prior mnemonic goals.     

Improving visual short-term memory by sequencing the stimulus array

When multiple objects are presented briefly and simultaneously in a visual array, visual short-term memory (VSTM) can maintain only a limited number of these items. The present research report reveals that splitting the to-be-remembered items into two sequential arrays significantly increases VSTM performance relative to the simultaneous presentation of the same items. A memory benefit also emerges when the full object array is flashed twice (repeated) rather than being presented continuously for the same duration. Moreover, the sequential and repetition benefits are specifically pronounced for individuals with low performance for simultaneously presented items. Our results suggest that the conventional, simultaneous presentation mode may underestimate VSTM performance due to attentional limitations and/or competition between stimulus representations. In contrast, temporal segregation of the stimulus input may help participants maximize their performance and utilize their full VSTM capacity.     

Orienting attention to locations in mental representations

Many cognitive processes depend on our ability to hold information in mind, often well beyond the offset of the original sensory input. The capacity of this visual short-term memory (VSTM) is limited to around three to four items. Recent research has demonstrated that the content of VSTM can be modulated by top-down attentional biases. This has been demonstrated using retrodictive spatial cues, termed "retro-cues," which orient subjects' attention to spatial locations within VSTM. In the present article, we tested whether the use of these cues is modulated by memory load and cue delay. There are a number of important conclusions: (1) Top-down biases can operate on very brief iconic traces as well as on older VSTM representations (Exp. 1). (2) When operating within capacity, subjects use the cue to prioritise where they initiate their memory search, rather than to discard uncued items (Exps. 2 and 3). (3) When capacity is exceeded, there is little benefit to top-down biasing relative to a neutral condition; however, unattended items are lost, with there being a substantial cost of invalid spatial cueing (Exp. 3). (4) These costs and benefits of orienting spatial attention differ across iconic memory and VSTM representations when VSTM capacity is exceeded (Exp. 4).     

Hemifield asymmetries differentiate VSTM for single- and multiple-feature objects

Visual short-term memory (VSTM) is a capacity-limited system for maintaining visual information across brief durations. Limits in the amount of information held in memory reflect processing constraints in the intraparietal sulcus (IPS), a region of the frontoparietal network also involved in visual attention. During VSTM and visual attention, areas of IPS demonstrate hemispheric asymmetries. Whereas the left hemisphere represents information in only the right hemifield, the right hemisphere represents information across the visual field. In visual attention, hemispheric asymmetries are associated with differences in behavioral performance across the visual field. In order to assess the degree of hemifield asymmetries in VSTM, we measured memory performance across the visual field for both single- and two-feature objects. Consistent with theories of right-hemisphere dominance, there was a memory benefit for single-feature items in the left visual hemifield. However, when the number of features increased, the behavioral bias reversed, demonstrating a benefit for remembering two-feature objects in the right hemifield. On an individual basis, the cost of remembering an additional feature in the hemifields was correlated, suggesting that the shift in hemifield biases reflected a redistribution of resources across the visual field. Furthermore, we demonstrate that these results cannot be explained by differences in perceptual or decision-making load. Our results are consistent with a flexible resource model of VSTM in which attention and/or working memory demands result in representation of items in the right hemifield by both the left and right hemispheres.     

Manual tapping enhances visual short‐term memory performance where visual and motor coordinates correspond

Visuo‐manual interaction in visual short‐term memory (VSTM) has been investigated little, despite its importance in everyday tasks requiring the coordination of visual perception and manual action. This study examines the influence of a manual action performed during stimulus learning on a subsequent VSTM test for object appearance. The memory display comprised a sequence of briefly presented 1/f noise discs (i.e., possessing spectral properties akin to natural images), wherein each new stimulus was presented at a unique screen location. Participants either did (or did not) perform a concurrent manual action (spatial tapping) task requiring that a hand‐held stylus be moved to a position on a touch tablet that corresponded (or did not correspond) to the screen position of each new stimulus as it appeared. At test, a single stimulus was presented, either at one of the original screen positions, or at a new position. Two factors were examined: the execution (or otherwise) of spatial tapping at a corresponding or non‐corresponding position, and the presentation of test stimuli either at their original spatial positions, or at new positions. We find that spatial tapping at corresponding positions elevates VSTM performance by more than 15%, but this occurs only when stimulus positions are matched from memory to test display. Our findings suggest that multimodal attentional focus during stimulus encoding (incorporating visual, spatial, and manual components) leads to stronger, more robust memory representations. We posit several possible explanations for this effect.     

The influence of spatial pattern on visual short-term memory for contrast

Several psychophysical studies of visual short-term memory (VSTM) have shown high-fidelity storage capacity for many properties of visual stimuli. On judgments of the spatial frequency of gratings, for example, discrimination performance does not decrease significantly, even for memory intervals of up to 30 s. For other properties, such as stimulus orientation and contrast, however, such “perfect storage” behavior is not found, although the reasons for this difference remain unresolved. Here, we report two experiments in which we investigated the nature of the representation of stimulus contrast in VSTM using spatially complex, two-dimensional random-noise stimuli. We addressed whether information about contrast per se is retained during the memory interval by using a test stimulus with the same spatial structure but either the same or the opposite local contrast polarity, with respect to the comparison (i.e., remembered) stimulus. We found that discrimination thresholds got steadily worse with increasing duration of the memory interval. Furthermore, performance was better when the test and comparison stimuli had the same local contrast polarity than when they were contrast-reversed. Finally, when a noise mask was introduced during the memory interval, its disruptive effect was maximal when the spatial configuration of its constituent elements was uncorrelated with those of the comparison and test stimuli. These results suggest that VSTM for contrast is closely tied to the spatial configuration of stimuli and is not transformed into a more abstract representation.     

Surface assignment modulates object formation for visual short-term memory

Recent results have suggested that the operational units of visual short-term memory (VSTM) are whole objects, rather than features or the total amount of information to be remembered. Here, for the first time, the influence of surface assignment on object formation for VSTM was investigated. The observers had to memorize the features of four briefly presented (300 ms) two-part objects, followed by a mask and a cue indicating which object to report on. The experiments contrasted whether there were any apparent depth differences between the two parts of each object, and whether observers had to report on only one or both features of the post-cued target object. Depth differences induced with stereoscopic disparity, and with a pictorial depth cue (simple interposition of object features), interfered strongly with performance when both features of an object needed to be memorized, but aided performance when only a single feature needed to be remembered. Furthermore, there was considerable within-feature interference consistent with some previous findings, but contradicting others. The potential implications for conceptions of VSTM are discussed in the light of two hypothesized stages: an early feature-based stage, as well as a higher-level object-based stage where the depth manipulations exert their effects. The results argue for a strong modulatory influence of surface assignment on object formation for a VSTM task.     

Perceptual asymmetries are preserved in short-term memory tasks

Visual performance is heterogeneous at isoeccentric locations; it is better on the horizontal than on the vertical meridian and worse at the upper than at the lower region of the vertical meridian (Carrasco, Talgar, & Cameron, 2001; Talgar & Carrasco, 2002). It is unknown whether these performance inhomogeneities are also present in spatial frequency tasks and whether asymmetries present during encoding of visual information also emerge in visual short-term memory (VSTM) tasks. Here, we investigated the similarity of the perceptual and VSTM tasks in spatial frequency discrimination (Experiments 1 and 2) and perceived spatial frequency (Experiments 3 and 4). We found that (1) performance in both simultaneous (perceptual) and delayed (VSTM) spatial frequency discrimination tasks varies as a function of location; it is better along the horizontal than along the vertical meridian; and (2) perceived spatial frequency in both tasks is higher along the horizontal than along the vertical meridian. These results suggest that perceived spatial frequency may mediate performance differences in VSTM tasks across the visual field, implying that the quality with which we encode information affects VSTM.     

Task set induces dynamic reallocation of resources in visual short-term memory

Successful interaction with the environment requires the ability to flexibly allocate resources to different locations in the visual field. Recent evidence suggests that visual short-term memory (VSTM) resources are distributed asymmetrically across the visual field based upon task demands. Here, we propose that context, rather than the stimulus itself, determines asymmetrical distribution of VSTM resources. To test whether context modulates the reallocation of resources to the right visual field, task set, defined by memory-load, was manipulated to influence visual short-term memory performance. Performance was measured for single-feature objects embedded within predominantly single- or two-feature memory blocks. Therefore, context was varied to determine whether task set directly predicts changes in visual field biases. In accord with the dynamic reallocation of resources hypothesis, task set, rather than aspects of the physical stimulus, drove improvements in performance in the right- visual field. Our results show, for the first time, that preparation for upcoming memory demands directly determines how resources are allocated across the visual field.     

Eye movements and the integration of visual memory and visual perception

Because visual perception has temporal extent, temporally discontinuous input must be linked in memory. Recent research has suggested that this may be accomplished by integrating the active contents of visual short-term memory (VSTM) with subsequently perceived information. In the present experiments, we explored the relationship between VSTM consolidation and maintenance and eye movements, in order to discover how attention selects the information that is to be integrated. Specifically, we addressed whether stimuli needed to be overtly attended in order to be included in the memory representation or whether covert attention was sufficient. Results demonstrated that in static displays in which the to-be-integrated information was presented in the same spatial location, VSTM consolidation proceeded independently of the eyes, since subjects made few eye movements. In dynamic displays, however, in which the to-be-integrated information was presented in different spatial locations, eye movements were directly related to task performance. We conclude that these differences are related to different encoding strategies. In the static display case, VSTM was maintained in the same spatial location as that in which it was generated. This could apparently be accomplished with covert deployments of attention. In the dynamic case, however, VSTM was generated in a location that did not overlap with one of the to-be-integrated percepts. In order to "move" the memory trace, overt shifts of attention were required.     

Shifting attention among working memory representations: testing cue type, awareness, and strategic control

It is well known that visual working memory (VWM) performance is modulated by attentional cues presented during encoding. Interestingly, retrospective cues presented after encoding, but prior to the test phase also improve performance. This improvement in performance is termed the retro-cue benefit. We investigated whether the retro-cue benefit is sensitive to cue type, whether participants were aware of their improvement in performance due to the retro-cue, and whether the effect was under strategic control. Experiment 1 compared the potential cueing benefits of abrupt onset retro-cues relying on bottom-up attention, number retro-cues relying on top-down attention, and arrow retro-cues, relying on a mixture of both. We found a significant retro-cue effect only for arrow retro-cues. In Experiment 2, we tested participants' awareness of their use of the informative retro-cue and found that they were aware of their improved performance. In Experiment 3, we asked whether participants have strategic control over the retro-cue. The retro-cue was difficult to ignore, suggesting that strategic control is low. The retro-cue effect appears to be within conscious awareness but not under full strategic control.     

Visual short-term memory for sequential arrays

The capacity of visual short-term memory (VSTM) for a single visual display has been investigated in past research, but VSTM for multiple sequential arrays has been explored only recently. In this study, we investigate the capacity of VSTM across two sequential arrays separated by a variable stimulus onset asynchrony (SOA). VSTM for spatial locations (Experiment 1), colors (Experiments 2-4), orientations (Experiments 3 and 4), and conjunction of color and orientation (Experiment 4) were tested, with the SOA across the two sequential arrays varying from 100 to 1,500 msec. We find that VSTM for the trailing array is much better than VSTM for the leading array, but when averaged across the two arrays VSTM has a constant capacity independent of the SOA. We suggest that multiple displays compete for retention in VSTM and that separating information into two temporally discrete groups does not enhance the overall capacity of VSTM.     

Feature binding in visual short term memory: A General Recognition Theory analysis

Creating and maintaining accurate bindings of elementary features (e.g., color and shape) in visual short-term memory (VSTM) is fundamental for veridical perception. How are low-level features bound in memory? The present work harnessed a multivariate model of perception – the General Recognition Theory (GRT) – to unravel the internal representations underlying feature binding in VSTM. On each trial, preview and target colored shapes were presented in succession, appearing in either repeated or altered spatial locations. Participants gave two same/different responses: one with respect to color and one with respect to shape. Converging GRT analyses on the accuracy confusion matrices provided substantial evidence for binding in the form of violations of perceptual independence at the level of the individual stimulus, such that positive correlations were obtained when both features repeated or alternated together, while negative correlations were obtained when one feature repeated and the other alternated. This “cloverleaf” GRT pattern of binding was similar whether the spatial location of the preview and target repeated or altered. The current results are consistent with: (a) the discrete memory “slots” model of VSTM, and (b) the notion that spatial location is not necessary for the formation of “object files.” The GRT approach presented here offers a viable quantitative model for testing various questions regarding feature binding in VSTM.     

Splitting attention across the two visual fields in visual short-term memory

Humans have the ability to attentionally select the most relevant visual information from their extrapersonal world and to retain it in a temporary buffer, known as visual short-term memory (VSTM). Research suggests that at least two non-contiguous items can be selected simultaneously when they are distributed across the two visual hemifields. In two experiments, we show that attention can also be split between the left and right sides of internal representations held in VSTM. Participants were asked to remember several colors, while cues presented during the delay instructed them to orient their attention to a subset of memorized colors. Experiment 1 revealed that orienting attention to one or two colors strengthened equally participants' memory for those colors, but only when they were from separate hemifields. Experiment 2 showed that in the absence of attentional cues the distribution of the items in the visual field per se had no effect on memory. These findings strongly suggest the existence of independent attentional resources in the two hemifields for selecting and/or consolidating information in VSTM.     

Mechanisms of priming of pop-out: Stored representations or feature-gain modulations?

Previous research has shown that repetition of a task-relevant attention-capturing feature facilitates popout search. This priming of pop-out effect is due to some residual memory from recent trials. We explore two possible mechanisms of priming of pop-out: a top-down attentional benefit from a memory of the previous target representation that is stored in visual short-term memory (VSTM) and a bottom-up change of attentional gains from perceptual features of the previously attended target. We manipulated participants’ ability to form a memory trace in VSTM by occupying it with a distractor task and found that occupying VSTM did not interfere with priming of pop-out. We next manipulated attentional gains associated with feature values by inserting an irrelevant task between pop-out searches. We found that the color of the target from the intervening perceptual task influenced pop-out search: The current pop-out search was facilitated when the intervening task’s target matched the target color of the pop-out search. These results suggest that priming of pop-out might not be due to a memory trace of the previous targets in VSTM but, rather, might be due to changes in attentional control based on priming from relatively low-level feature representations of previously attended objects.     

Understanding the object benefit in visual short-term memory: the roles of feature proximity and connectedness

Past research has identified visual objects as the units of information processing in visual short-term memory (VSTM) and has shown that two features from the same object can be remembered in VSTM as well (or almost as well) as one feature of that object and are much better remembered than the same two features from two spatially separated objects. It is not clear, however, what drives this object benefit in VSTM. Is it the shared spatial location (proximity), the connectedness among features of an object, or both? In six change detection experiments, both location/proximity and connectedness were found to be crucial in determining the magnitude of the object benefit in VSTM. Together, these results indicate that location/proximity and connectedness are essential elements in defining a coherent visual object representation in VSTM.     

What kind of memory supports visual marking?

In visual search tasks, if a set of items is presented for 1 s before another set of new items (containing the target) is added, search can be restricted to the new set. The process that eliminates old items from search is visual marking. This study investigates the kind of memory that distinguishes the old items from the new items during search. Using an accuracy paradigm in which perfect marking results in 100% accuracy and lack of marking results in near chance performance, the authors show that search can be restricted to new items not by visual short-term memory (VSTM) of old locations but by a limited capacity and slow-decaying VSTM of new locations and a high capacity and fast-decaying memory for asynchrony.