Visual search is guided by prospective and retrospective memory

Although there has been some controversy as to whether attention is guided by memory during visual search, recent findings have suggested that memory helps to prevent attention from needlessly reinspecting examined items. Until now, it has been assumed that some form of retrospective memory is responsible for keeping track of examined items and preventing revisitations. Alternatively, some form of prospective memory, such as strategic scanpath planning, could be responsible for guiding attention away from examined items. We used a new technique that allowed us to selectively prevent retrospective or prospective memory from contributing to search. We demonstrated that both retrospective and prospective memory guide attention during visual search.     

Memory for where, but not what, is used during visual search

Although the role of memory in visual search is debatable, most researchers agree with a limited-capacity model of memory in visual search. The authors demonstrate the role of memory by replicating previous findings showing that visual search is biased away from old items (previously examined items) and toward new items (nonexamined items). Furthermore, the authors examined the type of memory representations used to bias search by changing an item's individuating feature or location during search. Changing the individuating feature of an item did not disrupt normal search biases. However, when the location of an item changed, normal search biases were disrupted. These results suggest that memory used in visual search is based on items' locations rather than their identity.     

Explicit memory for rejected distractors during visual search

Although memory for the identities of examined items is not used to guide visual search, identity memory may be acquired during visual search. In all experiments reported here, search was occasionally terminated and a memory test was presented for the identity of a previously examined item. Participants demonstrated memory for the locations of the examined items by avoiding revisits to these items and memory performance for the items’ identities was above chance but lower than expected based on performance in intentional memory tests. Memory performance improved when the foil was not from the search set, suggesting that explicit identity memory is not bound to memory for location. Providing context information during test improved memory for the most recently examined item. Memory for the identities of previously examined items was best when the most recently examined item was tested, contextual information was provided, and location memory was not required.     

Visual Search Remains Efficient When Visual Working Memory is Full

Many theories of attention have proposed that visual working memory plays an important role in visual search tasks. The present study examined the involvement of visual working memory in search using a dual-task paradigm in which participants performed a visual search task while maintaining no, two, or four objects in visual working memory. The presence of a working memory load added a constant delay to the visual search reaction times, irrespective of the number of items in the visual search array. That is, there was no change in the slope of the function relating reaction time to the number of items in the search array, indicating that the search process itself was not slowed by the memory load. Moreover, the search task did not substantially impair the maintenance of information in visual working memory. These results suggest that visual search requires minimal visual working memory resources, a conclusion that is inconsistent with theories that propose a close link between attention and working memory.     

Implications of search accuracy for serial self-terminating models of search

In two experiments, the data from a total of 35 participants was compared to predictions from two competing serial search models, one with perfect memory and one with no memory. Reaction times were collected from an unlimited exposure duration, 4-alternative, target identification search task with array sizes of 8, 12, or 16 items. From the linear function relating RT to number of distractors a hypothetical “inspection time/item” was computed under the assumption of SSTS. Target identification accuracy was obtained from the same participants using the same displays, but with pre- and post-masks used to limit total inspection time (to 120, 180, 300, and 540 ms). Neither the memoryful nor memoryless model provided a good enough fit to the data to support the assumption that serial inspection is the primary process determining search time. Despite direct evidence for serial allocation of attention in visual search, these results suggest that scholars should seriously consider the possibility of hybrid search models in which serial and parallel strategies operate jointly.     

Why do we move our eyes while trying to remember? The relationship between non-visual gaze patterns and memory

Non-visual gaze patterns (NVGPs) involve saccades and fixations that spontaneously occur in cognitive activities that are not ostensibly visual. While reasons for their appearance remain obscure, convergent empirical evidence suggests that NVGPs change according to processing requirements of tasks. We examined NVPGs in tasks with long-term memory (LTM) and working memory (WM) requirements. Experiment 1 yielded significantly higher eye movement rate (EMR) in tasks requiring LTM search than in a WM task requiring maintenance of information. Experiment 2 manipulated accessibility of items in study-test episodic tasks using the levels of processing paradigm. EMR was high in episodic recall irrespective of item accessibility. Experiment 3 examined functional significance of saccades in LTM tasks. Voluntary saccadic suppression produced no evidence that saccades contribute to task performance. We discuss the apparent epiphenomenal nature of spontaneous saccades from an evolutionary perspective and outline a neuroanatomical model of the link between the saccadic and memory system.     

Incidental memory for parts of scenes from eye movements

Incidental memory for parts of scenes was examined in two search experiments and one memory control experiment. Eye movements were recorded during the search experiments and used to select gaze-contingent sections from search scenes for a surprise memory recognition task. Results from the recognition task showed incidental memory was better for sections viewed longer and with multiple fixations. Sections not fixated during search were still recognized above chance as well. Differences in sections did not affect memory performance in a control experiment when viewing time was held constant. These results show that memory for parts of scenes can occur incidentally during search and encoding of tested sections is better with longer viewing time and with multiple fixations.     

Eye movements and time-based selection: Where do the eyes go in preview search?

In visual search tasks, presenting one set of distractors (previewing them) before a second set which contains the target, improves search efficiency compared to when all items appear simultaneously. It has been proposed that this preview benefit reflects an attentional bias against old information and toward new information. Here we tested directly whether there was such a bias by measuring eye movement behavior. The main findings were that fixations were biased against, and overall dwell times were shorter on, old stimuli during search in the preview condition. In addition, the initial onset of search was delayed in the preview condition and saccades made during the preview period did not disrupt the ability to prioritize new items. The data demonstrate directly that preview search results in an attentional bias toward new items and against old items.     

Out with the old: Inhibition of old items in a preview search is limited

If some of the distractors in a visual search task are previewed prior to the presentation of the remaining distractors and the target, search time is reduced relative to when all of the items are displayed simultaneously. Here, we tested whether the ability to preferentially search new items during such a preview search is limited. We confirmed previous studies: The proportion of fixations on old items was significantly less than chance. However, the probability of fixating old locations was negatively affected by increasing the number of previewed distractors, suggesting that inhibition is limited to a small number of old items. Furthermore, the ability to inhibit old locations was limited to the first four fixations, indicating that by the fifth fixation, the resources required to sustain inhibition had been depleted. Together, these findings suggest that inhibition of old items in a preview search is a top-down mediated process dependent on capacity-limited cognitive resources.     

Selective scanpath repetition during memory-guided visual search

Visual search efficiency improves with repetition of a search display, yet the mechanisms behind these processing gains remain unclear. According to Scanpath Theory, memory retrieval is mediated by repetition of the pattern of eye movements or “scanpath” elicited during stimulus encoding. Using this framework, we tested the prediction that scanpath recapitulation reflects relational memory guidance during repeated search events. Younger and older subjects were instructed to find changing targets within flickering naturalistic scenes. Search efficiency (search time, number of fixations, fixation duration) and scanpath similarity (repetition) were compared across age groups for novel (V1) and repeated (V2) search events. Younger adults outperformed older adults on all efficiency measures at both V1 and V2, while the search time benefit for repeated viewing (V1–V2) did not differ by age. Fixation-binned scanpath similarity analyses revealed repetition of initial and final (but not middle) V1 fixations at V2, with older adults repeating more initial V1 fixations than young adults. In young adults only, early scanpath similarity correlated negatively with search time at test, indicating increased efficiency, whereas the similarity of V2 fixations to middle V1 fixations predicted poor search performance. We conclude that scanpath compression mediates increased search efficiency by selectively recapitulating encoding fixations that provide goal-relevant input. Extending Scanpath Theory, results suggest that scanpath repetition varies as a function of time and memory integrity.     

The boundary conditions of priming of visual search: From passive viewing through task-relevant working memory load

Priming of visual search has a dominating effect upon attentional shifts and is thought to play a decisive role in visual stability. Despite this importance, the nature of the memory underlying priming remains controversial. To understand more fully the necessary conditions for priming, we contrasted passive versus active viewing of visual search arrays. There was no priming from passive viewing of search arrays, while it was strong for active search of the same displays. Displays requiring no search resulted in no priming, again showing that search is needed for priming to occur. Finally, we introduced working memory load during visual search in an effort to disrupt priming. The memorized items had either the same colors as or different colors from the visual search items. Retaining items in working memory inhibited priming of the working memory task-relevant colors, while little interference was observed for unrelated colors. The picture that emerges of priming is that it requires active attentional processing of the search items in addition to the operation of visual working memory, where the task relevance of the working memory load plays a key role.     

Eye movements during search and detection in comparative visual search

Motivated by the fact that previous visual memory paradigms have imposed encoding and retrieval constraints, the present article presents two experiments that address how observers allocate eye movements in memory and comparison processes in the absence of constraints. A comparative visual search design (Pomplun, Sichelschmidt, et al., 2001) was utilized in which observers searched for a difference between two images presented side by side. Robust time course effects were obtained, whereby search was characterized by brief fixations and a high proportion of comparative saccades. Then, upon target detection, fixations were extended, more comparative saccades were elicited, and the search focus was narrowed. The saliency and presence of differences did not guide attention, and detection was contingent upon direct fixation of the targets. The results indicate that, when full control is given, observers adopt a strategy that cuts down on memory usage in favor of restricted encoding and active scanning.     

Covert visual attention and extrafoveal information use during object identification

Three experiments are reported that examined the relationship between covert visual attention and a viewer's ability to use extrafoveal visual information during object identification. Subjects looked at arrays of four objects while their eye movements were recorded. Their task was to identify the objects in the array for an immediate probe memory test. During viewing, the number and location of objects visible during given fixations were manipulated. In Experiments 1 and 2, we found that multiple extrafoveal previews of an object did not afford any more benefit than a single extrafoveal preview, as assessed by means of time of fixation on the objects. In Experiment 3, we found evidence for a model in which extrafoveal information acquired during a fixation derives primarily from the location toward which the eyes will move next. The results are discussed in terms of their implications for the relationship between covert visual attention and extrafoveal information use, and a sequential attention model is proposed.     

Scrambled eyes? Disrupting scene structure impedes focal processing and increases bottom-up guidance

Previous research has demonstrated that search and memory for items within natural scenes can be disrupted by "scrambling" the images. In the present study, we asked how disrupting the structure of a scene through scrambling might affect the control of eye fixations in either a search task (Experiment 1) or a memory task (Experiment 2). We found that the search decrement in scrambled scenes was associated with poorer guidance of the eyes to the target. Across both tasks, scrambling led to shorter fixations and longer saccades, and more distributed, less selective overt attention, perhaps corresponding to an ambient mode of processing. These results confirm that scene structure has widespread effects on the guidance of eye movements in scenes. Furthermore, the results demonstrate the trade-off between scene structure and visual saliency, with saliency having more of an effect on eye guidance in scrambled scenes.     

Focal lung pathology detection in radiology: Is there an effect of experience on visual search behavior?

In radiology, 60% to 80% of diagnostic errors are perceptual. The use of more efficient visual search behaviors is expected to reduce these errors. We collected eye-tracking data from participants with different levels of experience when interpreting chest X-rays during the completion of a pathology-detection task. Eye-tracking measures were assessed in the context of three existing visual search theories from the literature to understand the association between visual search behavior and underlying processes: the long-term working memory theory, the information-reduction hypothesis, and the holistic model of image perception. The most experienced participants (radiology residents) showed the highest level of performance, albeit their visual search behaviors did not differ from the intermediate group. This suggests that radiology residents better processed the represented information on the X-ray, using a visual search strategy similar to the intermediate group. Since similar visual search resulted in more information extraction in the radiology residents compared with the intermediates, we suggest that this result might support the long-term working memory theory. Furthermore, compared with novices, intermediates and radiology residents fixated longer on areas that were more important to avoid missing any pathology, which possibly confirms the information-reduction hypothesis. Finally, the larger distances between fixations observed in more experienced participants could support the holistic model of image perception. In addition, measures of generic skills were related to a lower time cost for switching between global and local information processing. Our findings suggest that the three theories may be complementary in chest X-ray interpretation. Therefore, a unified theory explaining perceptual-cognitive superiority in radiology is considered.

     

Attentional set protects visual marking from visual transients

When some distractors (old items) appear before others (new items) in an inefficient visual search task, the old items are excluded from the search (visual marking). Previous studies have shown that changing the shape of old items eliminates this effect, suggesting that shape identity must be maintained for successful visual marking. However, the contribution of top-down target knowledge to the maintenance of visual marking under shape change conditions has not been systematically examined. The present study tested whether the vulnerability of visual marking to shape change is contingent on observers' attentional set, by manipulating compatibility of the set and the domains in which the change occurs. The results indicated that visual marking survived shape changes when the observer's attentional set was consistent with critical features between the old and new items. This protection was observed when the set was based on explicit instructions at the beginning of the experiment, and when the task set was implicitly carried over from the previous task. These results suggest that top-down processes play a role in maintaining memory templates by enhancing the grouping and suppression processes during visual search, despite disruptive bottom-up signals.     

Visual marking survives graphical change if meaning is retained

When some distractors (old items) appear before others (new items) in an inefficient visual search task, the old items are excluded from the search (visual marking). Previous studies have shown that shape changes of static old items are sufficient to eliminate this effect when global luminance is maintained, suggesting that shape identity must be maintained for successful visual marking. It was unclear whether the change in meaning or shape was critical, because these changes were confounded in previous studies. The present study examined whether consistency in the semantic or the graphical identity of old items is critical for visual marking by introducing shape change in the absence of meaning change. The results indicated that visual marking survived graphical changes in old items as long as their meaning was maintained, suggesting that the memory template underlying visual marking represents the semantic identity of old items.     

Searching while loaded: Visual working memory does not interfere with hybrid search efficiency but hybrid search uses working memory capacity

In “hybrid search” tasks, such as finding items on a grocery list, one must search the scene for targets while also searching the list in memory. How is the representation of a visual item compared with the representations of items in the memory set? Predominant theories would propose a role for visual working memory (VWM) either as the site of the comparison or as a conduit between visual and memory systems. In seven experiments, we loaded VWM in different ways and found little or no effect on hybrid search performance. However, the presence of a hybrid search task did reduce the measured capacity of VWM by a constant amount regardless of the size of the memory or visual sets. These data are broadly consistent with an account in which VWM must dedicate a fixed amount of its capacity to passing visual representations to long-term memory for comparison to the items in the memory set. The data cast doubt on models in which the search template resides in VWM or where memory set item representations are moved from LTM through VWM to earlier areas for comparison to visual items.     

Aging, memory and visual search

Potential age-related differences in the memory processes that underlie visual search are examined in the present study. Using a dynamic, gaze-contingent search paradigm developed to assess memory for previously examined distractors, older adults demonstrated no memory deficit. Surprisingly, older adults made fewer refixations compared to their younger counterparts, indicating better memory for previously inspected objects. This improved memory was not the result of a speed-accuracy trade-off or larger Inhibition-of-Return effects for older than for younger adults. Additional analyses suggested that older adults may derive their benefit from finer spatial encoding of search items. These findings suggest that some of the memory processes that support visual search are relatively age invariant.     

The influence of action video game playing on eye movement behaviour during visual search in abstract,in-game and natural scenes

Action game playing has been associated with several improvements in visual attention tasks. However, it is not clear how such changes might influence the way we overtly select information from our visual world (i.e. eye movements). We examined whether action-video-game training changed eye movement behaviour in a series of visual search tasks including conjunctive search (relatively abstracted from natural behaviour), game-related search, and more naturalistic scene search. Forty nongamers were trained in either an action first-person shooter game or a card game (control) for 10 hours. As a further control, we recorded eye movements of 20 experienced action gamers on the same tasks. The results did not show any change in duration of fixations or saccade amplitude either from before to after the training or between all nongamers (pretraining) and experienced action gamers. However, we observed a change in search strategy, reflected by a reduction in the vertical distribution of fixations for the game-related search task in the action-game-trained group. This might suggest learning the likely distribution of targets. In other words, game training only skilled participants to search game images for targets important to the game, with no indication of transfer to the more natural scene search. Taken together, these results suggest no modification in overt allocation of attention. Either the skills that can be trained with action gaming are not powerful enough to influence information selection through eye movements, or action-game-learned skills are not used when deciding where to move the eyes.