Introspection during visual search

Recent advances in the field of metacognition have shown that human participants are introspectively aware of many different cognitive states, such as confidence in a decision. Here we set out to expand the range of experimental introspection by asking whether participants could access, through pure mental monitoring, the nature of the cognitive processes that underlie two visual search tasks: an effortless “pop-out” search, and a difficult, effortful, conjunction search. To this aim, in addition to traditional first order performance measures, we instructed participants to give, on a trial-by-trial basis, an estimate of the number of items scanned before a decision was reached. By controlling response times and eye movements, we assessed the contribution of self-observation of behavior in these subjective estimates. Results showed that introspection is a flexible mechanism and that pure mental monitoring of cognitive processes is possible in elementary tasks.     

Eye movements during search and detection in comparative visual search

Motivated by the fact that previous visual memory paradigms have imposed encoding and retrieval constraints, the present article presents two experiments that address how observers allocate eye movements in memory and comparison processes in the absence of constraints. A comparative visual search design (Pomplun, Sichelschmidt, et al., 2001) was utilized in which observers searched for a difference between two images presented side by side. Robust time course effects were obtained, whereby search was characterized by brief fixations and a high proportion of comparative saccades. Then, upon target detection, fixations were extended, more comparative saccades were elicited, and the search focus was narrowed. The saliency and presence of differences did not guide attention, and detection was contingent upon direct fixation of the targets. The results indicate that, when full control is given, observers adopt a strategy that cuts down on memory usage in favor of restricted encoding and active scanning.     

Serial deployment of attention during visual search

This study examined whether objects are attended in serial or in parallel during a demanding visual search task. A component of the event-related potential waveform, the N2pc wave, was used as a continuous measure of the allocation of attention to possible targets in the search arrays. Experiment 1 demonstrated that the relative allocation of attention shifts rapidly, favoring one item and then another. In Experiment 2, a paradigm was used that made it possible to track the absolute allocation of attention to individual items. This experiment showed that attention was allocated to one object for 100-150 ms before attention began to be allocated to the next object. These findings support models of attention that posit serial processing in demanding visual search tasks.     

Selective scanpath repetition during memory-guided visual search

Visual search efficiency improves with repetition of a search display, yet the mechanisms behind these processing gains remain unclear. According to Scanpath Theory, memory retrieval is mediated by repetition of the pattern of eye movements or “scanpath” elicited during stimulus encoding. Using this framework, we tested the prediction that scanpath recapitulation reflects relational memory guidance during repeated search events. Younger and older subjects were instructed to find changing targets within flickering naturalistic scenes. Search efficiency (search time, number of fixations, fixation duration) and scanpath similarity (repetition) were compared across age groups for novel (V1) and repeated (V2) search events. Younger adults outperformed older adults on all efficiency measures at both V1 and V2, while the search time benefit for repeated viewing (V1–V2) did not differ by age. Fixation-binned scanpath similarity analyses revealed repetition of initial and final (but not middle) V1 fixations at V2, with older adults repeating more initial V1 fixations than young adults. In young adults only, early scanpath similarity correlated negatively with search time at test, indicating increased efficiency, whereas the similarity of V2 fixations to middle V1 fixations predicted poor search performance. We conclude that scanpath compression mediates increased search efficiency by selectively recapitulating encoding fixations that provide goal-relevant input. Extending Scanpath Theory, results suggest that scanpath repetition varies as a function of time and memory integrity.     

Coarse-to-fine eye movement behavior during visual search

It has previously been argued that, during visual search, eye movement behavior is indicative of an underlying scanning “strategy” that starts on a global, or “coarse,” scale but then progressively focuses to a more local, or “fine,” scale. This conclusion is motivated by the finding that, as a trial progresses, fixation durations tend to increase and saccade amplitudes tend to decrease. In the present study, we replicate these effects but offer an alternative explanation for them—that they emerge from a few stochastic factors that control eye movement behavior. We report the results of a simulation supporting this hypothesis and discuss implications for future models of visual search.     

The contents of visual working memory reduce uncertainty during visual search

Information held in visual working memory (VWM) influences the allocation of attention during visual search, with targets matching the contents of VWM receiving processing benefits over those that do not. Such an effect could arise from multiple mechanisms: First, it is possible that the contents of working memory enhance the perceptual representation of the target. Alternatively, it is possible that when a target is presented among distractor items, the contents of working memory operate postperceptually to reduce uncertainty about the location of the target. In both cases, a match between the contents of VWM and the target should lead to facilitated processing. However, each effect makes distinct predictions regarding set-size manipulations; whereas perceptual enhancement accounts predict processing benefits regardless of set size, uncertainty reduction accounts predict benefits only with set sizes larger than 1, when there is uncertainty regarding the target location. In the present study, in which briefly presented, masked targets were presented in isolation, there was a negligible effect of the information held in VWM on target discrimination. However, in displays containing multiple masked items, information held in VWM strongly affected target discrimination. These results argue that working memory representations act at a postperceptual level to reduce uncertainty during visual search.     

Explicit memory for rejected distractors during visual search

Although memory for the identities of examined items is not used to guide visual search, identity memory may be acquired during visual search. In all experiments reported here, search was occasionally terminated and a memory test was presented for the identity of a previously examined item. Participants demonstrated memory for the locations of the examined items by avoiding revisits to these items and memory performance for the items’ identities was above chance but lower than expected based on performance in intentional memory tests. Memory performance improved when the foil was not from the search set, suggesting that explicit identity memory is not bound to memory for location. Providing context information during test improved memory for the most recently examined item. Memory for the identities of previously examined items was best when the most recently examined item was tested, contextual information was provided, and location memory was not required.     

Memory benefits during visual search depend on difficulty

In three experiments we explored whether memory for previous locations of search items influences search efficiency more as the difficulty of exhaustive search increases. Difficulty was manipulated by varying item eccentricity and item similarity (discriminability). Participants searched through items placed at three levels of eccentricity. The search displays were either identical on every trial (repeated condition) or the items were randomly reorganised from trial to trial (random condition), and search items were either relatively easy or difficult to discriminate from each other. Search was both faster and more efficient (i.e., search slopes were shallower) in the repeated condition than in the random condition. More importantly, this advantage for repeated displays was greater (1) for items that were more difficult to discriminate and (2) for eccentric targets when items were easily discriminable. Thus, increasing target eccentricity and reducing item discriminability both increase the influence of memory during search.     

Measuring the time course of selection during visual search

In visual search tasks, observers can guide their attention towards items in the visual field that share features with the target item. In this series of studies, we examined the time course of guidance toward a subset of items that have the same color as the target item. Landolt Cs were placed on 16 colored disks. Fifteen distractor Cs had gaps facing up or down while one target C had a gap facing left or right. Observers searched for the target C and reported which side contained the gap as quickly as possible. In the absence of other information, observers must search at random through the Cs. However, during the trial, the disks changed colors. Twelve disks were now of one color and four disks were of another color. Observers knew that the target C would always be in the smaller color set. The experimental question was how quickly observers could guide their attention to the smaller color set. Results indicate that observers could not make instantaneous use of color information to guide the search, even when they knew which two colors would be appearing on every trial. In each study, it took participants 200–300 ms to fully utilize the color information once presented. Control studies replicated the finding with more saturated colors and with colored C stimuli (rather than Cs on colored disks). We conclude that segregation of a display by color for the purposes of guidance takes 200–300 ms to fully develop.     

Age-related preservation of top-down attentional guidance during visual search

Younger (19-27 years of age) and older (60-82 years of age) adults performed a letter search task in which a color singleton was either noninformative (baseline condition) or highly informative (guided condition) regarding target location. In the guided condition, both age groups exhibited a substantial decrease in response time (RT) to singleton targets, relative to the baseline condition, as well as an increase in RT to nonsingleton targets. The authors conclude that under conditions that equate the physical structure of individual displays, top-down attentional guidance can be at least as effective for older adults as for younger adults.     

Information extraction during visual search: A developmental progression

Five developmental experiments examine the role of context in visual search. Experiments I and II presented either lists or paragraphs that were either in meaningful context or scrambled. Results showed a significant interaction of these variables. Experiment II examined within-item context or predictability and replicated and extended the work of Brand (1971) and Ingling (1972). Experiments IV and V examined search through lists of various widths; results showed a significant interaction of Grade × Position × Width reflecting the different perceptual spans of children and adults. Although these were search tasks, appropriate comparisons between list and paragraph performance were made allowing interpretations of the results to be supportive of a model of reading in which meaningfulness, predictability, and peripheral vision play critical roles.     

Amount and duration of attentional demands during visual search

Previous research has suggested that, in visual-search tasks, the comparison between target and display items does not require attentional capacity. In the present experiment we used a secondary-task paradigm to distinguish the amount and duration of the attentional demands of visual search. The subjects performed visual search (the primary task) and tone detection (the secondary task) concurrently over the course of five experimental sessions (1,440 trials). For each subject, target-response mapping was either consistent or varied for Days 1-5. The results indicate that the amount of attentional demand, as reflected in secondary-task performance, increased as a function of display size in the search task. Switching from consistent to varied mapping in a sixth experimental session increased both the amount and the duration of the attentional demands of the search. The present results support models of visual-search performance in which the comparison of target and display items requires attentional capacity.     

Person perception informs understanding of cognition during visual search

Does person perception—the impressions we form from watching others—hold clues to the mental states of people engaged in cognitive tasks? We investigated this with a two-phase method: In Phase 1, participants searched on a computer screen (Experiment 1) or in an office (Experiment 2); in Phase 2, other participants rated the searchers’ video-recorded behavior. The results showed that blind raters are sensitive to individual differences in search proficiency and search strategy, as well as to environmental factors affecting search difficulty. Also, different behaviors were linked to search success in each setting: Eye movement frequency predicted successful search on a computer screen; head movement frequency predicted search success in an office. In both settings, an active search strategy and positive emotional expressions were linked to search success. These data indicate that person perception informs cognition beyond the scope of performance measures, offering the potential for new measurements of cognition that are both rich and unobtrusive.     

The nature of the visual environment induces implicit biases during language-mediated visual search

Four eyetracking experiments examined whether semantic and visual-shape representations are routinely retrieved from printed word displays and used during language-mediated visual search. Participants listened to sentences containing target words that were similar semantically or in shape to concepts invoked by concurrently displayed printed words. In Experiment 1, the displays contained semantic and shape competitors of the targets along with two unrelated words. There were significant shifts in eye gaze as targets were heard toward semantic but not toward shape competitors. In Experiments 2–4, semantic competitors were replaced with unrelated words, semantically richer sentences were presented to encourage visual imagery, or participants rated the shape similarity of the stimuli before doing the eyetracking task. In all cases, there were no immediate shifts in eye gaze to shape competitors, even though, in response to the Experiment 1 spoken materials, participants looked to these competitors when they were presented as pictures (Huettig & McQueen, 2007). There was a late shape-competitor bias (more than 2,500 ms after target onset) in all experiments. These data show that shape information is not used in online search of printed word displays (whereas it is used with picture displays). The nature of the visual environment appears to induce implicit biases toward particular modes of processing during language-mediated visual search.     

The visual hemifield asymmetry in the spatial blink during singleton search and feature search

The present study examined a visual field asymmetry in the contingent capture of attention that was previously observed by Du and Abrams (2010). In our first experiment, color singleton distractors that matched the color of a to-be-detected target produced a stronger capture of attention when they appeared in the left visual hemifield than in the right visual hemifield. This replicated Du and Abrams and also revealed a difference between hemifields in the time course of this effect. Our second experiment suggested that this asymmetry is moderated by the tuning of attentional control settings: when the target was easier to detect the asymmetry was attenuated. Our third experiment showed that this asymmetry is also present during singleton detection: a color singleton distractor produced a larger capture effect in the left hemifield than in the right hemifield. Finally, our fourth experiment suggested that this asymmetry is moderated by the salience of the attention-capturing distractor: when the distractor was not salient, the asymmetry was attenuated. These results suggest that there are boundary conditions in the observed hemifield asymmetry in the contingent capture of attention and several underlying brain systems might be involved.