Memory for where, but not what, is used during visual search

Although the role of memory in visual search is debatable, most researchers agree with a limited-capacity model of memory in visual search. The authors demonstrate the role of memory by replicating previous findings showing that visual search is biased away from old items (previously examined items) and toward new items (nonexamined items). Furthermore, the authors examined the type of memory representations used to bias search by changing an item's individuating feature or location during search. Changing the individuating feature of an item did not disrupt normal search biases. However, when the location of an item changed, normal search biases were disrupted. These results suggest that memory used in visual search is based on items' locations rather than their identity.     

Proactive interference from items previously stored in visual working memory

This study investigates the fate of information that was previously stored in visual working memory but that is no longer needed. Previous research has found inconsistent results, with some showing effective release of irrelevant information and others showing proactive interference. Using change detection tasks of colors or shapes, we show that participants tend to falsely classify a changed item as "no change" if it matches one of the memory items on the preceding trial. The interference is spatially specific: Memory for the preceding trial interferes more if it matches the feature value and the location of a test item than if it does not. Interference results from retaining information in visual working memory, since it is absent when items on the preceding trials are passively viewed, or are attended but not memorized. We conclude that people cannot fully eliminate unwanted visual information from current working memory tasks.     

Visual memory for feature bindings: The disruptive effect of responding to new perceptual input

In the current study, we examined how short-term memory for location–identity feature bindings is influenced by subsequent cognitive and perceptual processing demands. Previous work has shown that memory performance for feature bindings can be disrupted by the presentation of subsequent visual information, particularly when this information is similar to that held in memory. The present study demonstrates that memory performance for feature bindings can be profoundly disrupted by also requiring a response to visual information presented subsequent to the visual memory array. Across five experiments, memory for a location–identity binding was substantially impaired following a localization response to a following item that matched the location but mismatched the identity of the memory target. The results point to an important role for action in the episodic integration processes that control short-term visual memory performance.     

Flexible working memory representation of the relationship between an object and its location as revealed by interactions with attention

Working memory (WM) selectively maintains a limited amount of currently relevant information in an active state to influence future perceptual processing, thought, and behavior. The representation of the information held in WM is unknown, particularly the degree of separation between the representation of an object’s identity and its location. The present experiments examined the flexibility of object and location WM representations by measuring reaction times on a visual discrimination task during the delay period of a WM recognition task for object identities, locations, or both. The results demonstrate that during WM delay periods, attention is biased toward information that matches the current contents of WM. Attention is not biased toward information that was present in the encoded memory sample but not relevant for the memory recognition test. This specificity of the interaction between WM and attention applies to both the identity and the location of the remembered stimulus and to the relationship between an object and its location. The results suggest that when this relationship is necessary for task performance, WM represents an object and its identity in an integrated manner. However, if this relationship is not task relevant, the object and location information are represented in WM separately.     

The aftermath of memory retrieval for recycling visual working memory representations

We examined the aftermath of accessing and retrieving a subset of information stored in visual working memory (VWM)—namely, whether detection of a mismatch between memory and perception can impair the original memory of an item while triggering recognition-induced forgetting for the remaining, untested items. For this purpose, we devised a consecutive-change detection task wherein two successive testing probes were displayed after a single set of memory items. Across two experiments utilizing different memory-testing methods (whole vs. single probe), we observed a reliable pattern of poor performance in change detection for the second test when the first test had exhibited a color change. The impairment after a color change was evident even when the same memory item was repeatedly probed; this suggests that an attention-driven, salient visual change made it difficult to reinstate the previously remembered item. The second change detection, for memory items untested during the first change detection, was also found to be inaccurate, indicating that recognition-induced forgetting had occurred for the unprobed items in VWM. In a third experiment, we conducted a task that involved change detection plus continuous recall, wherein a memory recall task was presented after the change detection task. The analyses of the distributions of recall errors with a probabilistic mixture model revealed that the memory impairments from both visual changes and recognition-induced forgetting are explained better by the stochastic loss of memory items than by their degraded resolution. These results indicate that attention-driven visual change and recognition-induced forgetting jointly influence the “recycling” of VWM representations.     

Learning in repeated visual search

Visual search (e.g., finding a specific object in an array of other objects) is performed most effectively when people are able to ignore distracting nontargets. In repeated search, however, incidental learning of object identities may facilitate performance. In three experiments, with over 1,100 participants, we examined the extent to which search could be facilitated by object memory and by memory for spatial layouts. Participants searched for new targets (real-world, nameable objects) embedded among repeated distractors. To make the task more challenging, some participants performed search for multiple targets, increasing demands on visual working memory (WM). Following search, memory for search distractors was assessed using a surprise two-alternative forced choice recognition memory test with semantically matched foils. Search performance was facilitated by distractor object learning and by spatial memory; it was most robust when object identity was consistently tied to spatial locations and weakest (or absent) when object identities were inconsistent across trials. Incidental memory for distractors was better among participants who searched under high WM load, relative to low WM load. These results were observed when visual search included exhaustive-search trials (Experiment 1) or when all trials were self-terminating (Experiment 2). In Experiment 3, stimulus exposure was equated across WM load groups by presenting objects in a single-object stream; recognition accuracy was similar to that in Experiments 1 and 2. Together, the results suggest that people incidentally generate memory for nontarget objects encountered during search and that such memory can facilitate search performance.     

Cognitive control over working memory biases of selection

Across many studies, researchers have found that representations in working memory (WM) can guide visual attention toward items that match the features of the WM contents. While some researchers have contended that this occurs involuntarily, others have suggested that the impact of WM contents on attention can be strategically controlled. Here, we varied the probability that WM items would coincide with either targets or distractors in a visual search task to examine (1) whether participants could intentionally enhance or inhibit the influence of WM items on attention and (2) whether cognitive control over WM biases would also affect access to the memory contents in a surprise recognition test. We found visual search to be faster when the WM item coincided with the search target, and this effect was enhanced when the memory item reliably predicted the location of the target. Conversely, visual search was slowed when the memory item coincided with a search distractor, and this effect was diminished, but not abolished, when the memory item was reliably associated with distractors. This strategic dampening of the influence of WM items on attention came at a price to memory, however, as participants were slowest to perform WM recognition tests on blocks in which the WM contents were consistently invalid. These results document that attentional capture by WM contents is partly, but not fully, malleable by top-down control, which appears to adjust the state of the WM contents to optimize search behavior. These data illustrate the role of cognitive control in modulating the strength of WM biases of selection, and they support a tight coupling between WM and attention.     

Spatial attention based on 2D location and relative depth order modulates visual working memory in a 3D environment

The attentional effect on visual working memory (VWM) has been a heated research topic in the past two decades. Studies show that VWM performance for an attended memory item can be improved by cueing its two-dimensional (2D) spatial location during retention. However, few studies have investigated the effect of attentional selection on VWM in a three-dimensional setting, and it remains unknown whether depth information can produce beneficial attentional effects on 2D visual representations similar to 2D spatial information. Here we conducted four experiments, displaying memory items at various stereoscopic depth planes, and examined the retro-cue effects of four types of cues – a cue would either indicate the 2D or depth location of a memory item, and either in the form of physical (directly pointing to a location) or symbolic (numerically mapping onto a location) cues. We found that retro-cue benefits were only observed for cues directly pointing to a 2D location, whereas a null effect was observed for cues directly pointing to a depth location. However, there was a retro-cue effect when cueing the relative depth order, though the effect was weaker than that for cueing the 2D location. The selective effect on VWM based on 2D spatial attention is different from depth-based attention, and the divergence suggests that an object representation is primarily bound with its 2D spatial location, weakly bound with its depth order but not with its metric depth location. This indicates that attentional selection based on memory for depth, particularly metric depth, is ineffective.     

Differential central resource demands of memory scanning and visual search: The role of consistent and varied mapping

In this paper we compare the effect of mapping on the central resource requirements of two search processes. With consistent mapping of items to the role of either target or distractor, search becomes very efficient; with varied mapping (each item serves as both a target and a distractor in the same experiment), search is less efficient. One interpretation of this effect of manipulating the consistency of mapping is that search under varied mapping requires limited capacity central resources but the requirement for these resources is reduced with consistent mapping. We tested this interpretation by varying the consistency of mapping in memory scanning (Experiments 1 and 2) and in visual search (Experiments 3 and 4) as the second of two tasks in the psychological refractory period (PRP) paradigm. Responses were location based, rather than present/absent judgements, and a target was present on every trial. The results indicated that there was less of a requirement for central resources with consistent mapping than with varied mapping in both memory scanning and in visual search; however, the effect of the mapping manipulation on central resource requirements was different for memory scanning than for visual search. Memory scanning was not done in parallel with a resource-demanding stage of a second task under varied mapping, but was done in parallel with this stage under consistent mapping. Part of visual search under both types of mapping was done in parallel with a resource-demanding stage of a second task and part was not; consistent mapping reduced the duration of the stage of processing that could not be carried out in parallel. We conclude that under consistent mapping, switching among items in memory becomes less controlled and therefore has less of a requirement for central resources than under varied mapping. In visual search, however, consistent mapping improves performance by facilitating the comparison of the target item to items in the search array.     

Exogenous attention influences visual short-term memory in infants

Two experiments examined the hypothesis that developing visual attentional mechanisms influence infants' Visual Short-Term Memory (VSTM) in the context of multiple items. Five- and 10-month-old infants (N = 76) received a change detection task in which arrays of three differently colored squares appeared and disappeared. On each trial one square changed color and one square was cued; sometimes the cued item was the changing item, and sometimes the changing item was not the cued item. Ten-month-old infants exhibited enhanced memory for the cued item when the cue was a spatial pre-cue (Experiment 1) and 5-month-old infants exhibited enhanced memory for the cued item when the cue was relative motion (Experiment 2). These results demonstrate for the first time that infants younger than 6 months can encode information in VSTM about individual items in multiple-object arrays, and that attention-directing cues influence both perceptual and VSTM encoding of stimuli in infants as in adults.     

A developmental comparison of the processing of two types of visual information

Simple geometric forms and spatial position markers were tachistoscopically presented to Ss from four age groups (ranging from five years to adult) who responded by recreating the visual array with magnetic tokens.The basic findings were: (a) Ss remembered item positions better than identities; (b) no asymptote of position recall was reached by any age group when up to five position markers were presented; (c) when required to process identity information, position memory suffered, younger Ss showing the greatest decrement; (d) with unlimited viewing time, the pattern of results relating identity and spatial recall was not altered; (e) recall was better for the locations of position markers than for locations of geometric items.The results supported an hypothesis of separate processing mechanisms for spatial location and identity information. Improvement with age seems to result from an increasing ability to integrate the functioning of these separate mechanisms.     

Dissociating effects of stimulus identity and load on working memory attentional guidance: Lengthening encoding time eliminates the effect of load but not identity

Effects of the identity and load of items in working memory (WM) on visual attention were examined. With a short interval between the WM item and a subsequent search task, there were effects of both load (slowed overall reaction times, RTs, in a WM condition relative to a mere repetition baseline) and identity (search RTs were affected by re-presentation of the item in WM in the search display). As the time to encode the initial display increased, the effects of load decreased while the effect of identity remained. The data indicate that the identity of stimuli in WM can affect the subsequent deployment of attention even when time is allowed for consolidation of the stimuli in WM, and that the WM effects are not causally related to the presence of cognitive load. The results are consistent with the identity of stimuli in WM modulating attention post the memory consolidation stage.     

A two-stage search of visual working memory: investigating speed in the change-detection paradigm

A popular procedure for investigating working memory processes has been the visual change-detection procedure. Models of performance based on that procedure, however, tend to be based on performance accuracy and treat working memory search as a one-step process, in which memory representations are compared to a test probe to determine if a match is present. To gain a clearer understanding of how search of these representations operate in the change-detection task, we examined reaction time in two experiments, with a single-item probe either located centrally or at the location of an array item. Contrary to current models of visual working memory capacity, our data point to a two-stage search process: a fast first step to check for the novelty of the probe and, in the absence of such novelty, a second, slower step to search exhaustively for a match between the test probe and a memory representation. In addition to these results, we found that participants tended not to use location information provided by the probe that theoretically could have abbreviated the search process. We suggest some basic revisions of current models of processing in this type of visual working memory task.     

Dissociating effects of stimulus identity and load on working memory attentional guidance: Lengthening encoding time eliminates the effect of load but not identity

Effects of the identity and load of items in working memory (WM) on visual attention were examined. With a short interval between the WM item and a subsequent search task, there were effects of both load (slowed overall reaction times, RTs, in a WM condition relative to a mere repetition baseline) and identity (search RTs were affected by re-presentation of the item in WM in the search display). As the time to encode the initial display increased, the effects of load decreased while the effect of identity remained. The data indicate that the identity of stimuli in WM can affect the subsequent deployment of attention even when time is allowed for consolidation of the stimuli in WM, and that the WM effects are not causally related to the presence of cognitive load. The results are consistent with the identity of stimuli in WM modulating attention post the memory consolidation stage.     

Constructing visual representations of natural scenes: the roles of short- and long-term visual memory

A "follow-the-dot" method was used to investigate the visual memory systems supporting accumulation of object information in natural scenes. Participants fixated a series of objects in each scene, following a dot cue from object to object. Memory for the visual form of a target object was then tested. Object memory was consistently superior for the two most recently fixated objects, a recency advantage indicating a visual short-term memory component to scene representation. In addition, objects examined earlier were remembered at rates well above chance, with no evidence of further forgetting when 10 objects intervened between target examination and test and only modest forgetting with 402 intervening objects. This robust prerecency performance indicates a visual long-term memory component to scene representation.     

Reduced effects of pictorial distinctiveness on false memory following dynamic visual noise

High levels of false recognition for non-presented items typically occur following exposure to lists of associated words. These false recognition effects can be reduced by making the studied items more distinctive by the presentation of pictures during encoding. One explanation of this is that during recognition, participants expect or attempt to retrieve distinctive pictorial information in order to evaluate the study status of the test item. If this involves the retrieval and use of visual imagery, then interfering with imagery processing should reduce the effectiveness of pictorial information in false memory reduction. In the current experiment, visual-imagery processing was disrupted at retrieval by the use of dynamic visual noise (DVN). It was found that effects of DVN dissociated true from false memory. Memory for studied words was not influenced by the presence of an interfering noise field. However, false memory was increased and the effects of picture-induced distinctiveness was eliminated. DVN also increased false recollection and remember responses to unstudied items.     

Effects of distinctive context on memory for objects and their locations in young and elderly adults

Sharps and Gollin (1987, 1988) reported that older adults' memory for both objects and their spatial location is more facilitated than that of young adults when items are studied in a distinctive visual context. They also reported that age differences in spatial memory and item memory can be ameliorated if the items are studied in a distinctive context. The goal of this study was to verify and extend these findings by examining memory for everyday objects when the objects were studied on a (a) plain surface, (b) black-and-white schematic map, or (c) colored model. In Experiment 1, item memory was examined, and in Experiment 2, spatial memory was studied. The results indicated that spatial memory improved with the addition of distinctive context but item memory did not. Moreover, there was no evidence that elderly adults were more facilitated by the distinctive context than were the young.     

Exogenous temporal cues enhance recognition memory in an object-based manner

Exogenous attention enhances the perception of attended items in both a space-based and an object-based manner. Exogenous attention also improves recognition memory for attended items in the space-based mode. However, it has not been examined whether object-based exogenous attention enhances recognition memory. To address this issue, we examined whether a sudden visual change in a task-irrelevant stimulus (an exogenous cue) would affect participants' recognition memory for items that were serially presented around a cued time. The results showed that recognition accuracy for an item was strongly enhanced when the visual cue occurred at the same location and time as the item (Experiments 1 and 2). The memory enhancement effect occurred when the exogenous visual cue and an item belonged to the same object (Experiments 3 and 4) and even when the cue was counterpredictive of the timing of an item to be asked about (Experiment 5). The present study suggests that an exogenous temporal cue automatically enhances the recognition accuracy for an item that is presented at close temporal proximity to the cue and that recognition memory enhancement occurs in an object-based manner.     

Action relevance induces an attentional weighting of representations in visual working memory

Information maintained in visual working memory (VWM) can be strategically weighted according to its task-relevance. This is typically studied by presenting cues during the maintenance interval, but under natural conditions, the importance of certain aspects of our visual environment is mostly determined by intended actions. We investigated whether representations in VWM are also weighted with respect to their potential action relevance. In a combined memory and movement task, participants memorized a number of items and performed a pointing movement during the maintenance interval. The test item in the memory task was subsequently presented either at the movement goal or at another location. We found that performance was better for test items presented at a location that corresponded to the movement goal than for test items presented at action-irrelevant locations. This effect was sensitive to the number of maintained items, suggesting that preferential maintenance of action relevant information becomes particularly important when the demand on VWM is high. We argue that weighting according to action relevance is mediated by the deployment of spatial attention to action goals, with representations spatially corresponding to the action goal benefitting from this attentional engagement. Performance was also better at locations next to the action goal than at locations farther away, indicating an attentional gradient spreading out from the action goal. We conclude that our actions continue to influence visual processing at the mnemonic level, ensuring preferential maintenance of information that is relevant for current behavioral goals.     

Strategic and automatic effects of visual working memory on attention in visual search

Previous research indicates that visual attention can be automatically captured by sensory inputs that match the contents of visual working memory. However, Woodman and Luck (2007) showed that information in working memory can be used flexibly as a template for either selection or rejection according to task demands. We report two experiments that extend their work. Participants performed a visual search task while maintaining items in visual working memory. Memory items were presented for either a short or long exposure duration immediately prior to the search task. Memory was tested by a change-detection task immediately afterwards. On a random half of trials items in memory matched either one distractor in the search task (Experiment 1) or three (Experiment 2). The main result was that matching distractors speeded or slowed target detection depending on whether memory items were presented for a long or short duration. These effects were more in evidence with three matching distractors than one. We conclude that the influence of visual working memory on visual search is indeed flexible but is not solely a function of task demands. Our results suggest that attentional capture by perceptual inputs matching information in visual working memory involves a fast automatic process that can be overridden by a slower top-down process of attentional avoidance.