Visual short-term memory benefit for objects on different 3-D surfaces

Visual short-term memory (VSTM) plays an important role in visual cognition. Although objects are located on different 3-dimensional (3-D) surfaces in the real world, how VSTM capacity may be influenced by the presence of multiple 3-D surfaces has never been examined. By manipulating binocular disparities of visual displays, the authors found that more colored objects could be held in VSTM when they were placed on 2 rather than on 1 planar 3-D surfaces. This between-surface benefit in VSTM was present only when binding of objects' colors to their 3-D locations was required (i.e., when observers needed to remember which color appeared where). When binding was not required, no between-surface benefit in VSTM was observed. This benefit in VSTM could not be attributed to the number of spatial locations attended within a given surface. It was not due to a general perceptual grouping effect either, because grouping by motion and grouping by different regions of the same surface did not yield the same benefit. This increment in capacity indicates that VSTM benefits from the placement of objects in a 3-D scene.     

Toward ecologically realistic theories in visual short-term memory research

Recent evidence from neuroimaging and psychophysics suggests common neural and representational substrates for visual perception and visual short-term memory (VSTM). Visual perception is adapted to a rich set of statistical regularities present in the natural visual environment. Common neural and representational substrates for visual perception and VSTM suggest that VSTM is adapted to these same statistical regularities too. This article discusses how the study of VSTM can be extended to stimuli that are ecologically more realistic than those commonly used in standard VSTM experiments and what the implications of such an extension could be for our current view of VSTM. We advocate for the development of unified models of visual perception and VSTM—probabilistic and hierarchical in nature—incorporating prior knowledge of natural scene statistics.     

Coarse-to-fine encoding of spatial frequency information into visual short-term memory for faces but impartial decay

Face perception studies investigated how spatial frequencies (SF) are extracted from retinal display while forming a perceptual representation, or their selective use during task-imposed categorization. Here we focused on the order of encoding low-spatial frequencies (LSF) and high-spatial frequencies (HSF) from perceptual representations into visual short-term memory (VSTM). We also investigated whether different SF-ranges decay from VSTM at different rates during a study-test stimulus-onset asynchrony. An old/new VSTM paradigm was used in which two broadband faces formed the positive set and the probes preserved either low or high SF ranges. Exposure time of 500 ms was sufficient to encode both HSF and LSF in the perceptual representation (experiment 1). Nevertheless, when the positive-set was exposed for 500 ms, LSF-probes were better recognized in VSTM compared with HSF-probes; this effect vanished at 800-ms exposure time (experiment 2). Backward masking the positive set exposed for 800 ms re-established the LSF-probes advantage (experiment 3). The speed of decay up to 10 seconds was similar for LSF- and HSF-probes (experiment 4). These results indicate that LSF are extracted and consolidated into VSTM faster than HSF, supporting a coarse-to-fine order, while the decay from VSTM is not governed by SF.     

Are multiple visual short-term memory storages necessary to explain the retro-cue effect?

Recent research has shown that change detection performance is enhanced when, during the retention interval, attention is cued to the location of the upcoming test item. This retro-cue advantage has led some researchers to suggest that visual short-term memory (VSTM) is divided into a durable, limited-capacity storage and a more fragile, high-capacity storage. Consequently, performance is poor on the no-cue trials because fragile VSTM is overwritten by the test display and only durable VSTM is accessible under these conditions. In contrast, performance is improved in the retro-cue condition because attention keeps fragile VSTM accessible. The aim of the present study was to test the assumptions underlying this two-storage account. Participants were asked to encode an array of colors for a change detection task involving no-cue and retro-cue trials. A retro-cue advantage was found even when the cue was presented after a visual (Experiment 1) or a central (Experiment 2) interference. Furthermore, the magnitude of the interference was comparable between the no-cue and retro-cue trials. These data undermine the main empirical support for the two-storage account and suggest that the presence of a retro-cue benefit cannot be used to differentiate between different VSTM storages.     

Distributing versus focusing attention in visual short-term memory

Visual short-term memory (VSTM) is traditionally considered a robust form of visual memory resistant to interference from subsequent visual input. This study shows that the robustness of VSTM depends on the way attention is allocated in VSTM. When attention is distributed across multiple memory items, VSTM for these items is vulnerable to interference from subsequent input, including passively viewed images and the postchange testing displays. Yet attention can readjust its focus in VSTM on the basis of an attentional orienting cue presented long after encoding. When attention is oriented to a particular memorized item, the memory is resistant to subsequent interference. This effect, however, is eliminated when the subset of items demanding focal attention exceeds one, suggesting that orienting attention in VSTM is less flexible than orienting attention in perception. We propose that the robustness of VSTM is influenced by whether attention is focused or distributed in VSTM.     

Visual cognition influences early vision: the role of visual short-term memory in amodal completion

A partly occluded visual object is perceptually filled in behind the occluding surface, a process known as amodal completion or visual interpolation. Previous research focused on the image-based properties that lead to amodal completion. In the present experiments, we examined the role of a higher-level visual process-visual short-term memory (VSTM)-in amodal completion. We measured the degree of amodal completion by asking participants to perform an object-based attention task on occluded objects while maintaining either zero or four items in visual working memory. When no items were stored in VSTM, participants completed the occluded objects; when four items were stored in VSTM, amodal completion was halted (Experiment 1). These results were not caused by the influence of VSTM on object-based attention per se (Experiment 2) or by the specific location of to-be-remembered items (Experiment 3). Items held in VSTM interfere with amodal completion, which suggests that amodal completion may not be an informationally encapsulated process, but rather can be affected by high-level visual processes.     

Understanding the object benefit in visual short-term memory: the roles of feature proximity and connectedness

Past research has identified visual objects as the units of information processing in visual short-term memory (VSTM) and has shown that two features from the same object can be remembered in VSTM as well (or almost as well) as one feature of that object and are much better remembered than the same two features from two spatially separated objects. It is not clear, however, what drives this object benefit in VSTM. Is it the shared spatial location (proximity), the connectedness among features of an object, or both? In six change detection experiments, both location/proximity and connectedness were found to be crucial in determining the magnitude of the object benefit in VSTM. Together, these results indicate that location/proximity and connectedness are essential elements in defining a coherent visual object representation in VSTM.     

Visual short-term memory load strengthens selective attention

Perceptual load theory accounts for many attentional phenomena; however, its mechanism remains elusive because it invokes underspecified attentional resources. Recent dual-task evidence has revealed that a concurrent visual short-term memory (VSTM) load slows visual search and reduces contrast sensitivity, but it is unknown whether a VSTM load also constricts attention in a canonical perceptual load task. If attentional selection draws upon VSTM resources, then distraction effects—which measure attentional “spill-over”—will be reduced as competition for resources increases. Observers performed a low perceptual load flanker task during the delay period of a VSTM change detection task. We observed a reduction of the flanker effect in the perceptual load task as a function of increasing concurrent VSTM load. These findings were not due to perceptual-level interactions between the physical displays of the two tasks. Our findings suggest that perceptual representations of distractor stimuli compete with the maintenance of visual representations held in memory. We conclude that access to VSTM determines the degree of attentional selectivity; when VSTM is not completely taxed, it is more likely for task-irrelevant items to be consolidated and, consequently, affect responses. The “resources” hypothesized by load theory are at least partly mnemonic in nature, due to the strong correspondence they share with VSTM capacity.     

Feature binding in visual short term memory: A General Recognition Theory analysis

Creating and maintaining accurate bindings of elementary features (e.g., color and shape) in visual short-term memory (VSTM) is fundamental for veridical perception. How are low-level features bound in memory? The present work harnessed a multivariate model of perception – the General Recognition Theory (GRT) – to unravel the internal representations underlying feature binding in VSTM. On each trial, preview and target colored shapes were presented in succession, appearing in either repeated or altered spatial locations. Participants gave two same/different responses: one with respect to color and one with respect to shape. Converging GRT analyses on the accuracy confusion matrices provided substantial evidence for binding in the form of violations of perceptual independence at the level of the individual stimulus, such that positive correlations were obtained when both features repeated or alternated together, while negative correlations were obtained when one feature repeated and the other alternated. This “cloverleaf” GRT pattern of binding was similar whether the spatial location of the preview and target repeated or altered. The current results are consistent with: (a) the discrete memory “slots” model of VSTM, and (b) the notion that spatial location is not necessary for the formation of “object files.” The GRT approach presented here offers a viable quantitative model for testing various questions regarding feature binding in VSTM.     

Eye movements and the integration of visual memory and visual perception

Because visual perception has temporal extent, temporally discontinuous input must be linked in memory. Recent research has suggested that this may be accomplished by integrating the active contents of visual short-term memory (VSTM) with subsequently perceived information. In the present experiments, we explored the relationship between VSTM consolidation and maintenance and eye movements, in order to discover how attention selects the information that is to be integrated. Specifically, we addressed whether stimuli needed to be overtly attended in order to be included in the memory representation or whether covert attention was sufficient. Results demonstrated that in static displays in which the to-be-integrated information was presented in the same spatial location, VSTM consolidation proceeded independently of the eyes, since subjects made few eye movements. In dynamic displays, however, in which the to-be-integrated information was presented in different spatial locations, eye movements were directly related to task performance. We conclude that these differences are related to different encoding strategies. In the static display case, VSTM was maintained in the same spatial location as that in which it was generated. This could apparently be accomplished with covert deployments of attention. In the dynamic case, however, VSTM was generated in a location that did not overlap with one of the to-be-integrated percepts. In order to "move" the memory trace, overt shifts of attention were required.     

Predominance of ground over ceiling surfaces in binocular rivalry

The superiority of ground surfaces over ceiling surfaces in determining the representation of the visual world, demonstrated in several studies of visual perception and visual search, has been attributed to a preference for top-away projections resulting from ecological constraints. Recent research on binocular rivalry indicates that ecological constraints affect predominance relations. The present study considered whether there is a difference in predominance between ground and ceiling surfaces. In Experiment 1, we examined whether a ground surface would dominate a ceiling surface when one surface was presented to each eye. In Experiment 2, we used an eye-swapping paradigm to determine whether a ground surface would come to dominance faster than a ceiling surface when presented to the suppressed eye. The eye-swapping paradigm was used again in Experiment 3, but the ground and ceiling planes were replaced with frontal planes with similar variations in texture density. The results of these experiments indicate that ground surfaces are predominant over ceiling surfaces, with this predominance affecting both the dominance and suppression phases of binocular rivalry. This superiority of ground planes is independent of image properties such as the increase or decrease in texture density from the lower half to the upper half of the images.     

Manual tapping enhances visual short‐term memory performance where visual and motor coordinates correspond

Visuo‐manual interaction in visual short‐term memory (VSTM) has been investigated little, despite its importance in everyday tasks requiring the coordination of visual perception and manual action. This study examines the influence of a manual action performed during stimulus learning on a subsequent VSTM test for object appearance. The memory display comprised a sequence of briefly presented 1/f noise discs (i.e., possessing spectral properties akin to natural images), wherein each new stimulus was presented at a unique screen location. Participants either did (or did not) perform a concurrent manual action (spatial tapping) task requiring that a hand‐held stylus be moved to a position on a touch tablet that corresponded (or did not correspond) to the screen position of each new stimulus as it appeared. At test, a single stimulus was presented, either at one of the original screen positions, or at a new position. Two factors were examined: the execution (or otherwise) of spatial tapping at a corresponding or non‐corresponding position, and the presentation of test stimuli either at their original spatial positions, or at new positions. We find that spatial tapping at corresponding positions elevates VSTM performance by more than 15%, but this occurs only when stimulus positions are matched from memory to test display. Our findings suggest that multimodal attentional focus during stimulus encoding (incorporating visual, spatial, and manual components) leads to stronger, more robust memory representations. We posit several possible explanations for this effect.     

Testing effects in visual short-term memory: The case of an object's size

In many daily activities, we need to form and retain temporary representations of an object’s size. Typically, such visual short-term memory (VSTM) representations follow perception and are considered reliable. Here, participants were asked to hold in mind a single simple object for a short duration and to reproduce its size by adjusting the length and width of a test probe. Experiment 1 revealed two powerful findings: First, similar to a recently reported perceptual illusion, participants greatly overestimated the size of open objects – ones with missing boundaries – relative to the same-size fully closed objects. This finding confirms that object boundaries are critical for size perception and memory. Second, and in contrast to perception, even the size of the closed objects was largely overestimated. Both inflation effects were substantial and were replicated and extended in Experiments 2–5. Experiments 6–8 used a different testing procedure to examine whether the overestimation effects are due to inflation of size in VSTM representations or to biases introduced during the reproduction phase. These data showed that while the overestimation of the open objects was repeated, the overestimation of the closed objects was not. Taken together, these findings suggest that similar to perception, only the size representation of open objects is inflated in VSTM. Importantly, they demonstrate the considerable impact of the testing procedure on VSTM tasks and further question the use of reproduction procedures for measuring VSTM.     

The capacity of visual short-term memory within and between hemifields

Visual short-term memory (VSTM) and attention are both thought to have a capacity limit of four items [e.g. Luck, S. J., & Vogel, E. K. (1997). The capacity of visual working memory for features and conjunctions. Nature, 309, 279-281; Pylyshyn, Z. W., & Storm, R. W. (1988). Tracking multiple independent targets: evidence for a parallel tracking mechanism. Spatial Vision, 3, 179-197.]. Using the multiple object visual tracking paradigm (MOT), it has recently been shown that twice as many items can be simultaneously attended when they are separated between two visual fields compared to when they are all presented within the same hemifield [Alvarez, G. A., & Cavanagh, P. (2004). Independent attention resources for the left and right visual hemifields (Abstract). Journal of Vision, 4(8), 29a.]. Does VSTM capacity also increase when the items to be remembered are distributed between the two visual fields? The current paper investigated this central issue in two different tasks, namely a color and spatial location change detection task, in which the items were displayed either in the two visual fields or in the same hemifield. The data revealed that only memory capacity for spatial locations and not colors increased when the items were separated between the two visual fields. These findings support the view of VSTM as a chain of capacity limited operations where the spatial selection of stimuli, which dominates in both spatial location VSTM and MOT, occupies the first place and shows independence between the two fields.     

Feature binding in visual short-term memory is unaffected by task-irrelevant changes of location,shape, and color

Three experiments used a change detection paradigm across a range of study–test intervals to address the respective contributions of location, shape, and color to the formation of bindings of features in sensory memory and visual short-term memory (VSTM). In Experiment 1, location was designated task irrelevant and was randomized between study and test displays. The task was to detect changes in the bindings between shape and color. In Experiments 2 and 3, shape and color, respectively, were task irrelevant and randomized, with bindings tested between location and color (Experiment 2) and location and shape (Experiment 3). At shorter study–test intervals, randomizing location was most disruptive, followed by shape and then color. At longer intervals, randomizing any task-irrelevant feature had no impact on change detection for bindings between features, and location had no special role. Results suggest that location is crucial for initial perceptual binding but loses that special status once representations are formed in VSTM, which operates according to different principles, than do visual attention and perception.     

Visual short-term memory is not improved by training

A critical question in visual working or short-term memory (VSTM) research is whether the ability to remember briefly presented visual stimuli can be increased. Here we test whether VSTM for locations and shapes is improved by training that allows one to utilize another memory system, visual longterm memory (VLTM). Training was done by repeatedly presenting a subset of memory displays, creating long-term memory traces for these displays. Surprisingly, VSTM performance for repeated displays was not higher than for nonrepeated ones, even though participants recognized repeated displays on a forced-choice test given at the end of the experiment. We suggest that the fidelity of information held by VLTM is inferior to that of information held by VSTM and thus provides no additional benefit over what is extracted on the fly by VSTM.     

Is visual short-term memory object based? Rejection of the “strong-object” hypothesis

Is the capacity of visual short-term memory (VSTM) limited by the number of objects or by the number of features? VSTM for objects with either one feature or two color features was tested. Results show that capacity was limited primarily by the number of colors to be memorized, not by the number of objects. This result held up with variations in color saturation, blocked or mixed conditions, duration of memory image, and absence or presence of verbal load. However, conjoining features into objects improved VSTM capacity when size-orientation and color-orientation conjunctions were tested. Nevertheless, the number of features still mattered. When feature heterogeneity was controlled, VSTM for conjoined objects was worse than VSTM for objects made of single features. Our results support a weak-object hypothesis of VSTM capacity that suggests that VSTM is limited by both the number of objects and the feature composition of those objects.     

Is visual short-term memory object based? Rejection of the "strong-object" hypothesis

Is the capacity of visual short-term memory (VSTM) limited by the number of objects or by the number of features? VSTM for objects with either one feature or two color features was tested. Results show that capacity was limited primarily by the number of colors to be memorized, not by the number of objects. This result held up with variations in color saturation, blocked or mixed conditions, duration of memory image, and absence or presence of verbal load. However, conjoining features into objects improved VSTM capacity when size-orientation and color-orientation conjunctions were tested. Nevertheless, the number of features still mattered. When feature heterogeneity was controlled, VSTM for conjoined objects was worse than VSTM for objects made of single features. Our results support a weak-object hypothesis of VSTM capacity that suggests that VSTM is limited by both the number of objects and the feature composition of those objects.     

Texture perception in sighted and blind observers

The purpose of the present study was to evaluate the utility of visual imagery for texture perception. In Experiment 1, sighted, early-blind, and late-blind observers made relative smoothness judgments of abrasive surfaces using active or passive tough. In Experiment 2, subjects compared vision and touch in the accuracy of smoothness detection, using a broad range of textures, including very fine surfaces. No differences appeared between the sighted and the blind, and it did not matter if touch were active or passive. Vision and touch showed similar performance with relatively coarse textures, but touch was superior to vision for much finer surface textures. The results were consistent with the notion that visual coding of tactual stimuli is not advantageous (or necessary) for texture perception, since touch may hold advantages for the detection of the smoothness of surfaces.     

视觉长时记忆激活度对促进视觉短时记忆的影响

短时记忆与长时记忆的关系是记忆领域研究的重要内容。基于此, 研究者们就视觉长时记忆是否能促进视觉短时记忆的问题展开了大量研究, 但所得出的结论并不一致。通过不同程度的学习形成不同激活度的视觉长时记忆, 考察对几何图形的视觉长时记忆是否能对变化检测中的短时记忆起到促进作用。结果发现, 低激活水平的视觉长时记忆不能促进视觉短时记忆, 而事先存在且高度激活的视觉长时记忆对视觉短时记忆却具有促进作用; 同时, 随着视觉长时记忆激活水平的提高, 刺激间间隔的效应值也逐渐减小。本研究说明, 视觉长时记忆能否促进视觉短时记忆取决于视觉长时记忆的激活水平, 高激活的视觉长时记忆对阻止视觉短时记忆痕迹的迅速消退具有重要意义。