Understanding the function of visual short-term memory: transsaccadic memory, object correspondence, and gaze correction

Visual short-term memory (VSTM) has received intensive study over the past decade, with research focused on VSTM capacity and representational format. Yet, the function of VSTM in human cognition is not well understood. Here, the authors demonstrate that VSTM plays an important role in the control of saccadic eye movements. Intelligent human behavior depends on directing the eyes to goal-relevant objects in the world, yet saccades are very often inaccurate and require correction. The authors hypothesized that VSTM is used to remember the features of the current saccade target so that it can be rapidly reacquired after an errant saccade, a task faced by the visual system thousands of times each day. In 4 experiments, memory-based gaze correction was accurate, fast, automatic, and largely unconscious. In addition, a concurrent VSTM load interfered with memory-based gaze correction, but a verbal short-term memory load did not. These findings demonstrate that VSTM plays a direct role in a fundamentally important aspect of visually guided behavior, and they suggest the existence of previously unknown links between VSTM representations and the occulomotor system.     

Understanding the object benefit in visual short-term memory: the roles of feature proximity and connectedness

Past research has identified visual objects as the units of information processing in visual short-term memory (VSTM) and has shown that two features from the same object can be remembered in VSTM as well (or almost as well) as one feature of that object and are much better remembered than the same two features from two spatially separated objects. It is not clear, however, what drives this object benefit in VSTM. Is it the shared spatial location (proximity), the connectedness among features of an object, or both? In six change detection experiments, both location/proximity and connectedness were found to be crucial in determining the magnitude of the object benefit in VSTM. Together, these results indicate that location/proximity and connectedness are essential elements in defining a coherent visual object representation in VSTM.     

Multidimensional shape similarity in the development of visual object classification

The current work examined age differences in the classification of novel object images that vary in continuous dimensions of structural shape. The structural dimensions employed are two that share a privileged status in the visual analysis and representation of objects: the shape of discrete prominent parts and the attachment positions of those parts. Experiment 1 involved a triad classification task in which participants at each of three different ages (5 years, 8 years, and adult) classified object images from two distinct stimulus sets. Across both sets, the youngest children demonstrated a systematic bias toward the shape of discrete parts during their judgments. With increasing age, participants increasingly came to select both the shape and the position of parts when classifying the images. The findings from Experiment 2 indicate that the local shape bias observed in young children's classifications is not merely a consequence of a discrimination advantage for that dimension. Results are discussed in relation to corresponding age-related changes in other functional contexts of visual processing.     

Internal attention to features in visual short-term memory guides object learning

Attending to objects in the world affects how we perceive and remember them. What are the consequences of attending to an object in mind? In particular, how does reporting the features of a recently seen object guide visual learning? In three experiments, observers were presented with abstract shapes in a particular color, orientation, and location. After viewing each object, observers were cued to report one feature from visual short-term memory (VSTM). In a subsequent test, observers were cued to report features of the same objects from visual long-term memory (VLTM). We tested whether reporting a feature from VSTM: (1) enhances VLTM for just that feature (practice-benefit hypothesis), (2) enhances VLTM for all features (object-based hypothesis), or (3) simultaneously enhances VLTM for that feature and suppresses VLTM for unreported features (feature-competition hypothesis). The results provided support for the feature-competition hypothesis, whereby the representation of an object in VLTM was biased towards features reported from VSTM and away from unreported features (Experiment 1). This bias could not be explained by the amount of sensory exposure or response learning (Experiment 2) and was amplified by the reporting of multiple features (Experiment 3). Taken together, these results suggest that selective internal attention induces competitive dynamics among features during visual learning, flexibly tuning object representations to align with prior mnemonic goals.     

Encoding color and shape from different parts of an object in visual short-term memory

Can we find an object-based encoding benefit in visual short-term memory (VSTM) when the features to be remembered are from different parts of an object? Using object parts defined by either figure-ground separation or negative minima of curvature, results from five experiments in which the visual change detection paradigm was used showed that the object-based encoding benefit in VSTM is modulated by how features are assigned to parts of an object: Features are best retained when the color and shape features to be remembered belong to the same part of an object. Although less well retained in comparison, features from different parts of an object are still better remembered than features from spatially separated objects. An object-based feature binding therefore exists even when the color and shape features to be remembered are from different parts of an object.     

Homogeneity computation: How interitem similarity in visual short-term memory alters recognition

Visual short-term recognition memory for multiple stimuli is strongly influenced by the study items’ similarity to one another—that is, by their homogeneity. However, the mechanism responsible for this homogeneity effect has remained unclear. We evaluated competing explanations of this effect, using controlled sets of Gabor patches as study items and probe stimuli. Our results, based on recognition memory for spatial frequency, rule out the possibility that the homogeneity effect arises because similar study items are encoded and/or maintained with higher fidelity in memory than dissimilar study items are. Instead, our results support the hypothesis that the homogeneity effect reflects trial-by-trial comparisons of study items, which generate a homogeneity signal. This homogeneity signal modulates recognition performance through an adjustment of the subject’s decision criterion. Additionally, it seems the homogeneity signal is computed prior to the presentation of the probe stimulus, by evaluating the familiarity of each new stimulus with respect to the items already in memory. This suggests that recognition-like processes operate not only on the probe stimulus, but on study items as well.     

Working memory binding of visual object features in older adults

Accurate mental representation of visual stimuli requires retaining not only the individual features but also the correct relationship between them. This associative process of binding is mediated by working memory (WM) mechanisms. The present study re-examined reports of WM-related binding deficits with aging. In Experiment 1, 31 older and 31 younger adults completed a visual change detection task with feature–location relations presented either simultaneously or sequentially; the paradigm was also designed specifically to minimize the impact of lengthy retention intervals, elaborative rehearsal, and processing demands of multi-stimulus probes. In Experiment 2, 38 older and 42 younger adults completed a modified task containing both feature–location relations and feature–feature conjunctions. In Experiment 1 although feature–location binding was more difficult with sequential compared with simultaneous presentation, the effect was independent of age. In Experiment 2 while older adults were overall slower and less accurate than young adults, there were no age-specific deficits in WM binding. Overall, after controlling for methodological factors, there was no evidence of an age-related visual WM binding deficit for surface or location features. However, unlike younger adults, older adults appeared less able to restrict processing of irrelevant features, consistent with reported declines with age in strategic capacities of WM.     

A configural effect in visual short-term memory for features from different parts of an object

Previous studies have shown that change detection performance is improved when the visual display holds features (e.g., a colour and an orientation) that are grouped into different parts of the same object compared to when they are all spatially separated (Xu, 2002a, 2002b). These findings indicate that visual short-term memory (VSTM) encoding can be "object based". Recently, however, it has been demonstrated that changing the orientation of an item could affect the spatial configuration of the display (Jiang, Chun, & Olson, 2004), which may have an important influence on change detection. The perceptual grouping of features into an object obviously reduces the amount of distinct spatial relations in a display and hence the complexity of the spatial configuration. In the present study, we ask whether the object-based encoding benefit observed in previous studies may reflect the use of configural coding rather than the outcome of a true object-based effect. The results show that when configural cues are removed, the object-based encoding benefit remains for features (i.e., colour and orientation) from different parts of an object, but is significantly reduced. These findings support the view that memory for features from different parts of an object can benefit from object-based encoding, but the use of configural coding significantly helps enlarge this effect.     

A configural effect in visual short-term memory for features from different parts of an object

Previous studies have shown that change detection performance is improved when the visual display holds features (e.g., a colour and an orientation) that are grouped into different parts of the same object compared to when they are all spatially separated (Xu, 2002a Xu, Y. 2002a. Encoding colour and shape from different parts of an object in visual short-term memory. Perception and Psychophysics, 64: 1260–1280. [Crossref], [PubMed], [Web of Science ®] , [Google Scholar], 2002b Xu, Y. 2002b. Limitations of object-based feature encoding in visual short-term memory. Journal of Experimental Psychology: Human Perception and Performance, 28: 458–468. [Crossref], [PubMed], [Web of Science ®] , [Google Scholar]). These findings indicate that visual short-term memory (VSTM) encoding can be “object based”. Recently, however, it has been demonstrated that changing the orientation of an item could affect the spatial configuration of the display (Jiang, Chun, & Olson, 2004 Jiang, Y., Chun, M. M. and Olson, I. R. 2004. Perceptual grouping in change detection. Perception & Psychophysics, 66: 446–453.  [Google Scholar]), which may have an important influence on change detection. The perceptual grouping of features into an object obviously reduces the amount of distinct spatial relations in a display and hence the complexity of the spatial configuration. In the present study, we ask whether the object-based encoding benefit observed in previous studies may reflect the use of configural coding rather than the outcome of a true object-based effect. The results show that when configural cues are removed, the object-based encoding benefit remains for features (i.e., colour and orientation) from different parts of an object, but is significantly reduced. These findings support the view that memory for features from different parts of an object can benefit from object-based encoding, but the use of configural coding significantly helps enlarge this effect.     

Pictorial similarity judgments and the organization of visual memory in the rhesus monkey

The organization of visual memory for pictures was studied in the rhesus monkey. Two monkeys were tested in a same/different task in which sequentially presented pictures were compared to each other in a pair-wise fashion. The resulting confusion matrixes were analyzed using a multidimensional scaling procedure to obtain two- and three-dimensional graphic representations of the stimulus space. In Experiment 1, the monkeys' confusion errors caused pictures of human and rhesus monkey faces to fall in the same region of multidimensional space, which suggested that the monkeys categorized facial stimuli. A similar effect was found for pictures of different types of fruit. Experiment 2 replicated the categorization of faces with a more diverse collection of human and nonhuman primate faces. Experiment 3 explored the fruit category by varying stimulus attributes orthogonally. The results from this experiment showed that both monkeys encoded the pictures in this category by type of fruit (apples or grapes) and color (red or yellow). Taken together, these studies indicate that rhesus monkeys will treat some classes of pictorial stimuli categorically in visual memory.     

Eccentricity biases of object categories are evident in visual working memory

In high-order object areas, face-selective areas prefer centrally presented stimuli, whereas building-selective areas prefer peripherally presented stimuli (Levy et al., 2001). We investigated whether this eccentricity bias was also evident in visual working memory. In Experiment 1, we found that working memory performance for faces decreased towards the periphery while the performance for buildings remained unchanged across different eccentricities. To rule out the possibility that lower level features influence these results, we manipulated the spatial frequency of faces and buildings (Experiment 2) and the spatial layout information of the buildings (Experiment 3). In both of the experiments, we replicated the results of Experiment 1, even when these lower level features of stimuli were controlled. Consistent with previous findings, the current results suggest that each object category is processed in a different manner depending on the eccentricity. This eccentricity bias is likely the result of how the high-order object areas represent different object categories.     

The assessment of subjective visual vertical: comparison of two psychophysical paradigms and age-related performance

Perception of the subjective visual vertical (SVV) is usually assessed by asking to subjects, in complete darkness, to adjust the position of a luminous rod that is variably tilted (i.e., by the method of adjustment [ADJ]). Conversely, the two-alternative forced choice (2AFC) method requires subjects to categorize, as tilted either clockwise (CW) or counterclockwise (CCW), stimuli that are presented on a computer screen and are variably tilted from vertical. In this study, we aimed to compare the results of these two methods and investigate age-related effects on the SVV. SVV was assessed in 102 healthy individuals, 50 women and 52 men, with a mean age of 45.7 (range 20–91), using both ADJ (ten trials, initial 1°, 2°, 4°, 8°, or 12° bar tilts both CW and CCW) and 2AFC (120 stimuli with a 1°–32° variable tilt). Also, 50 of the subjects performed the ADJ test twice, with different bar lengths. We estimated bias and threshold for the two methods, and found that neither measure differed across the methods. Age was a significant predictor of threshold (2AFC, R ² = .141; ADJ, R ² = .190; p < .001), implying lower sensitivity with increasing age. Moreover, the ADJ method showed a significant increase of bias when the initial tilt was farthest from vertical, whereas the rod length was irrelevant. SVV measures obtained with the ADJ and 2AFC methods were comparable, but the latter measures were more resistant to artifacts that might affect the measurement. The lower sensitivity found in older persons may have an influence on their ability to interact with the environment and may contribute to impairment of postural control.     

The effects of direction similarity in visual working memory: Behavioural and event-related potential studies

Object similarity can improve visual working memory (VWM) performance in the change-detection task, but impair the recognition performance when it occurs at retrieval of VWM in the recognition task. The effect of direction similarity is an issue that has not been well resolved. Furthermore, electrophysiological evidence in support of the mechanisms that underlie the effects of similarity is still scarce. In the current study, we conducted three behavioural experiments to examine the effects of direction similarity on memory performance with regard to both the encoding and retrieval phases of VWM and one event-related potential (ERP) experiment to explore the neural signatures of direction similarity in VWM. Our behavioural studies indicated that direction similarity improved performance when it occurred at the encoding phase but impaired performance when it occurred at the retrieval phase. Moreover, the ERP experiment showed that the amplitude of the contralateral delay activity (CDA) increased with the increasing set size for similar but not dissimilar directions. In addition, the CDA amplitude for similar directions was lower than that for dissimilar directions at set size 2. Taken together, these findings suggest that direction similarity at encoding has a positive effect on VWM performance and at retrieval has a negative effect. Given that VWM capacity depends on information load and the number of objects, the positive effect of similarity may be attributed to reduced information load of memory objects.     

Understanding age-related reductions in visual working memory capacity: Examining the stages of change detection

Visual working memory (VWM) capacity is reduced in older adults. Research has shown age-related impairments to VWM encoding, but aging is likely to affect multiple stages of VWM. In the present study, we recorded the event-related potentials (ERPs) of younger and older adults during VWM maintenance and retrieval. We measured encoding-stage processing with the P1 component, maintenance-stage processing with the contralateral delay activity (CDA), and retrieval-stage processing by comparing the activity for old and new items (old–new effect). Older adults showed lower behavioral capacity estimates (K) than did younger adults, but surprisingly, their P1 components and CDAs were comparable to those of younger adults. This remarkable dissociation between neural activity and behavior in the older adults indicated that the P1 and CDA did not accurately assess their VWM capacity. However, the neural activity evoked during VWM retrieval yielded results that helped clarify the age-related differences. During retrieval, younger adults showed early old–new effects in frontal and occipital areas and a late central–parietal old–new effect, whereas older adults showed a late right-lateralized parietal old–new effect. The younger adults’ early old–new effects strongly resembled an index of perceptual fluency, suggesting that perceptual implicit memory was activated. The activation of implicit memory could have facilitated the younger adults’ behavior, and the lack of these early effects in older adults may suggest that they have much lower-resolution memory than do younger adults. From these data, we speculated that younger and older adults store the same number of items in VWM, but that younger adults store a higher-resolution representation than do older adults.     

Testing sequence effects in visual memory: clues for a structural model

In order to probe the internal organization of visual temporary memory, systematic experiments were performed in which the subjects had to memorize a series of 2-5 images then pass recognition tests, either in all possible testing permutations, (in the case of 2-4 images) or in 20 selected permutations (in the case of 5 images). Over 300,000 tests were performed, generating more than 40,000 errors. The error-rates were found to follow simple rules. Both the 3 and the 4 images results are compatible with the presence of only four typical accuracy levels. On the other hand, the reaction time (RT) results revealed surprisingly rich patterns. The RT for recognizing image i at testing stage t > 1 depends upon which image (j) was tested just before. The ranking of the RTs for (j, i) couples evolves from one testing stage to the next. It is proposed that these RTs reflect, in part, the time needed to localize the trace in memory of a given image, starting from the position at which the previous test occurred. If this assumption is correct, the results of this study are providing a picture of the configurations formed by memorized items in the visual temporary store. A hypothetical minimal model, involving several rows of slots on a triangular mesh is proposed to account for the structures in both the error-rates and the RT results.     

Forgetting the once-seen face: estimating the strength of an eyewitness's memory representation

The fidelity of an eyewitness's memory representation is an issue of paramount forensic concern. Psychological science has been unable to offer more than vague generalities concerning the relation of retention interval to memory trace strength for the once-seen face. A meta-analysis of 53 facial memory studies produced a highly reliable association (r=.18, d=0.37) between longer retention intervals and positive forgetting of once-seen faces, an effect equally strong for both face recognition and eyewitness identification studies. W. A. Wickelgren's (1974, 1975, 1977) theory of recognition memory provided statistically satisfactory fits to 11 different empirical forgetting functions. Applied to the results of field studies of eyewitness memory, the theory yields predictions relevant to fact finders' evaluations of eyewitness credibility. A plausible upper limit for witness initial memory strength corresponds to a probability of .67 of being correct on a fair six-person lineup. Furthermore, not only can the percentage of remaining memory strength be determined for any retention interval, but this strength estimate can be translated into an estimated probability of being correct on a fair lineup of a specified size.     

The potentiation of two components of the reach-to-grasp action during object categorisation in visual memory

Stimulus-Response Compatibility Effects have been reported for several components of the reach-to-grasp action during visual object recognition [Tucker, M., & Ellis, R. (1998). On the relations between seen objects and components of potential actions. Journal of Experimental Psychology: Human Perception and Performance, 24, 830-846; Ellis, R., & Tucker, M. (2000). Micro-affordance: The potentiation of actions by seen objects. British Journal of Psychology, 91, 451-471; Tucker, M., & Ellis, R. (2001). The potentiation of grasp types during visual object categorization. Visual Cognition, 8, 769-800; Creem, S. H., & Proffitt, D. R. (2001). Grasping objects by their handles: A necessary interaction between cognition and action. Journal of Experimental Psychology: Human Perception and Performance, 27, 218-228; Craighero, L. Bello, A. Fadiga, L., & Rizzolatti, G. (2002). Hand action preparation influences the responses to hand pictures. Neuropsychologia, 40, 492-502]. The present study investigates compatibility effects for two elements of reach-to-grasp action during the visual mental imagery of objects-the compatibility of an object for grasping with a power and precision grasp, and the orientation of an object (left/right) for grasping by a particular hand (left/right). Experiment 1 provides further evidence for compatibility effects of a 'seen' object for grasping with a power and precision grasp. The experiment shows that compatibility effects are obtainable when an object is presented in an array of four objects and not just on its own. Experiment 2 provides evidence that compatibility effects of an object for grasping with a power and precision grasp can also be observed when participants make an action response to an object 700 ms after it has been removed from view. Experiment 3 investigates compatibility effects for the orientation of an object for grasping by a particular hand during visual mental imagery, but finds no evidence for such effects. The findings are discussed in relation to two arguments put forward to reconcile ecological and representational theories of visual object recognition.     

Similarity on the rebound: Inhibition of similarity assessment leads to an ironic postsuppressional rebound

A widely held but rarely tested assumption among cognitive scientists is that different cognitive tasks may rely upon a single basic cognitive process. Using an established methodology to examine the suppression and subsequent rebound of mental operations, the present research indicates that suppressing use of similarity in one domain results in the subsequent rebound of similarity assessment in a different domain, suggesting that both domains rely on the same underlying cognitive process. In two studies, we demonstrate that leading people to suppress natural similarity assessment in one task produces increased reliance on similarity in subsequent, different, and apparently unrelated tasks. In Experiment 1, participants led to suppress similarity in a concentration task subsequently made more errors in a false-memory paradigm than did control participants. In Experiment 2, participants suppressing similarity in a categorization task made more false-memory errors and perceived more similarity between word pairs than participants who did not suppress. The findings suggest that the cognitive process of similarity assessment may be a domain-general process, such that it is widespread across a number of different mental tasks.     

Similarity on the rebound: inhibition of similarity assessment leads to an ironic postsuppressional rebound

A widely held but rarely tested assumption among cognitive scientists is that different cognitive tasks may rely upon a single basic cognitive process. Using an established methodology to examine the suppression and subsequent rebound of mental operations, the present research indicates that suppressing use of similarity in one domain results in the subsequent rebound of similarity assessment in a different domain, suggesting that both domains rely on the same underlying cognitive process. In two studies, we demonstrate that leading people to suppress natural similarity assessment in one task produces increased reliance on similarity in subsequent, different, and apparently unrelated tasks. In Experiment 1, participants led to suppress similarity in a concentration task subsequently made more errors in a false-memory paradigm than did control participants. In Experiment 2, participants suppressing similarity in a categorization task made more false-memory errors and perceived more similarity between word pairs than participants who did not suppress. The findings suggest that the cognitive process of similarity assessment may be a domain-general process, such that it is widespread across a number of different mental tasks.     

Object memory and change detection: dissociation as a function of visual and conceptual similarity

People often fail to detect a change between two visual scenes, a phenomenon referred to as change blindness. This study investigates how a post-change object's similarity to the pre-change object influences memory of the pre-change object and affects change detection. The results of Experiment 1 showed that similarity lowered detection sensitivity but did not affect the speed of identifying the pre-change object, suggesting that similarity between the pre- and post-change objects does not degrade the pre-change representation. Identification speed for the pre-change object was faster than naming the new object regardless of detection accuracy. Similarity also decreased detection sensitivity in Experiment 2 but improved the recognition of the pre-change object under both correct detection and detection failure. The similarity effect on recognition was greatly reduced when 20% of each pre-change stimulus was masked by random dots in Experiment 3. Together the results suggest that the level of pre-change representation under detection failure is equivalent to the level under correct detection and that the pre-change representation is almost complete. Similarity lowers detection sensitivity but improves explicit access in recognition. Dissociation arises between recognition and change detection as the two judgments rely on the match-to-mismatch signal and mismatch-to-match signal, respectively.