Introspection during visual search

Recent advances in the field of metacognition have shown that human participants are introspectively aware of many different cognitive states, such as confidence in a decision. Here we set out to expand the range of experimental introspection by asking whether participants could access, through pure mental monitoring, the nature of the cognitive processes that underlie two visual search tasks: an effortless “pop-out” search, and a difficult, effortful, conjunction search. To this aim, in addition to traditional first order performance measures, we instructed participants to give, on a trial-by-trial basis, an estimate of the number of items scanned before a decision was reached. By controlling response times and eye movements, we assessed the contribution of self-observation of behavior in these subjective estimates. Results showed that introspection is a flexible mechanism and that pure mental monitoring of cognitive processes is possible in elementary tasks.     

Eye movements during search and detection in comparative visual search

Motivated by the fact that previous visual memory paradigms have imposed encoding and retrieval constraints, the present article presents two experiments that address how observers allocate eye movements in memory and comparison processes in the absence of constraints. A comparative visual search design (Pomplun, Sichelschmidt, et al., 2001) was utilized in which observers searched for a difference between two images presented side by side. Robust time course effects were obtained, whereby search was characterized by brief fixations and a high proportion of comparative saccades. Then, upon target detection, fixations were extended, more comparative saccades were elicited, and the search focus was narrowed. The saliency and presence of differences did not guide attention, and detection was contingent upon direct fixation of the targets. The results indicate that, when full control is given, observers adopt a strategy that cuts down on memory usage in favor of restricted encoding and active scanning.     

Serial deployment of attention during visual search

This study examined whether objects are attended in serial or in parallel during a demanding visual search task. A component of the event-related potential waveform, the N2pc wave, was used as a continuous measure of the allocation of attention to possible targets in the search arrays. Experiment 1 demonstrated that the relative allocation of attention shifts rapidly, favoring one item and then another. In Experiment 2, a paradigm was used that made it possible to track the absolute allocation of attention to individual items. This experiment showed that attention was allocated to one object for 100-150 ms before attention began to be allocated to the next object. These findings support models of attention that posit serial processing in demanding visual search tasks.     

Coarse-to-fine eye movement behavior during visual search

It has previously been argued that, during visual search, eye movement behavior is indicative of an underlying scanning “strategy” that starts on a global, or “coarse,” scale but then progressively focuses to a more local, or “fine,” scale. This conclusion is motivated by the finding that, as a trial progresses, fixation durations tend to increase and saccade amplitudes tend to decrease. In the present study, we replicate these effects but offer an alternative explanation for them—that they emerge from a few stochastic factors that control eye movement behavior. We report the results of a simulation supporting this hypothesis and discuss implications for future models of visual search.     

The contents of visual working memory reduce uncertainty during visual search

Information held in visual working memory (VWM) influences the allocation of attention during visual search, with targets matching the contents of VWM receiving processing benefits over those that do not. Such an effect could arise from multiple mechanisms: First, it is possible that the contents of working memory enhance the perceptual representation of the target. Alternatively, it is possible that when a target is presented among distractor items, the contents of working memory operate postperceptually to reduce uncertainty about the location of the target. In both cases, a match between the contents of VWM and the target should lead to facilitated processing. However, each effect makes distinct predictions regarding set-size manipulations; whereas perceptual enhancement accounts predict processing benefits regardless of set size, uncertainty reduction accounts predict benefits only with set sizes larger than 1, when there is uncertainty regarding the target location. In the present study, in which briefly presented, masked targets were presented in isolation, there was a negligible effect of the information held in VWM on target discrimination. However, in displays containing multiple masked items, information held in VWM strongly affected target discrimination. These results argue that working memory representations act at a postperceptual level to reduce uncertainty during visual search.     

Memory benefits during visual search depend on difficulty

In three experiments we explored whether memory for previous locations of search items influences search efficiency more as the difficulty of exhaustive search increases. Difficulty was manipulated by varying item eccentricity and item similarity (discriminability). Participants searched through items placed at three levels of eccentricity. The search displays were either identical on every trial (repeated condition) or the items were randomly reorganised from trial to trial (random condition), and search items were either relatively easy or difficult to discriminate from each other. Search was both faster and more efficient (i.e., search slopes were shallower) in the repeated condition than in the random condition. More importantly, this advantage for repeated displays was greater (1) for items that were more difficult to discriminate and (2) for eccentric targets when items were easily discriminable. Thus, increasing target eccentricity and reducing item discriminability both increase the influence of memory during search.     

Measuring the time course of selection during visual search

In visual search tasks, observers can guide their attention towards items in the visual field that share features with the target item. In this series of studies, we examined the time course of guidance toward a subset of items that have the same color as the target item. Landolt Cs were placed on 16 colored disks. Fifteen distractor Cs had gaps facing up or down while one target C had a gap facing left or right. Observers searched for the target C and reported which side contained the gap as quickly as possible. In the absence of other information, observers must search at random through the Cs. However, during the trial, the disks changed colors. Twelve disks were now of one color and four disks were of another color. Observers knew that the target C would always be in the smaller color set. The experimental question was how quickly observers could guide their attention to the smaller color set. Results indicate that observers could not make instantaneous use of color information to guide the search, even when they knew which two colors would be appearing on every trial. In each study, it took participants 200–300 ms to fully utilize the color information once presented. Control studies replicated the finding with more saturated colors and with colored C stimuli (rather than Cs on colored disks). We conclude that segregation of a display by color for the purposes of guidance takes 200–300 ms to fully develop.     

Age-related preservation of top-down attentional guidance during visual search

Younger (19-27 years of age) and older (60-82 years of age) adults performed a letter search task in which a color singleton was either noninformative (baseline condition) or highly informative (guided condition) regarding target location. In the guided condition, both age groups exhibited a substantial decrease in response time (RT) to singleton targets, relative to the baseline condition, as well as an increase in RT to nonsingleton targets. The authors conclude that under conditions that equate the physical structure of individual displays, top-down attentional guidance can be at least as effective for older adults as for younger adults.     

Amount and duration of attentional demands during visual search

Previous research has suggested that, in visual-search tasks, the comparison between target and display items does not require attentional capacity. In the present experiment we used a secondary-task paradigm to distinguish the amount and duration of the attentional demands of visual search. The subjects performed visual search (the primary task) and tone detection (the secondary task) concurrently over the course of five experimental sessions (1,440 trials). For each subject, target-response mapping was either consistent or varied for Days 1-5. The results indicate that the amount of attentional demand, as reflected in secondary-task performance, increased as a function of display size in the search task. Switching from consistent to varied mapping in a sixth experimental session increased both the amount and the duration of the attentional demands of the search. The present results support models of visual-search performance in which the comparison of target and display items requires attentional capacity.     

Person perception informs understanding of cognition during visual search

Does person perception—the impressions we form from watching others—hold clues to the mental states of people engaged in cognitive tasks? We investigated this with a two-phase method: In Phase 1, participants searched on a computer screen (Experiment 1) or in an office (Experiment 2); in Phase 2, other participants rated the searchers’ video-recorded behavior. The results showed that blind raters are sensitive to individual differences in search proficiency and search strategy, as well as to environmental factors affecting search difficulty. Also, different behaviors were linked to search success in each setting: Eye movement frequency predicted successful search on a computer screen; head movement frequency predicted search success in an office. In both settings, an active search strategy and positive emotional expressions were linked to search success. These data indicate that person perception informs cognition beyond the scope of performance measures, offering the potential for new measurements of cognition that are both rich and unobtrusive.     

Information extraction during visual search: A developmental progression

Five developmental experiments examine the role of context in visual search. Experiments I and II presented either lists or paragraphs that were either in meaningful context or scrambled. Results showed a significant interaction of these variables. Experiment II examined within-item context or predictability and replicated and extended the work of Brand (1971) and Ingling (1972). Experiments IV and V examined search through lists of various widths; results showed a significant interaction of Grade × Position × Width reflecting the different perceptual spans of children and adults. Although these were search tasks, appropriate comparisons between list and paragraph performance were made allowing interpretations of the results to be supportive of a model of reading in which meaningfulness, predictability, and peripheral vision play critical roles.     

The nature of the visual environment induces implicit biases during language-mediated visual search

Four eyetracking experiments examined whether semantic and visual-shape representations are routinely retrieved from printed word displays and used during language-mediated visual search. Participants listened to sentences containing target words that were similar semantically or in shape to concepts invoked by concurrently displayed printed words. In Experiment 1, the displays contained semantic and shape competitors of the targets along with two unrelated words. There were significant shifts in eye gaze as targets were heard toward semantic but not toward shape competitors. In Experiments 2–4, semantic competitors were replaced with unrelated words, semantically richer sentences were presented to encourage visual imagery, or participants rated the shape similarity of the stimuli before doing the eyetracking task. In all cases, there were no immediate shifts in eye gaze to shape competitors, even though, in response to the Experiment 1 spoken materials, participants looked to these competitors when they were presented as pictures (Huettig & McQueen, 2007). There was a late shape-competitor bias (more than 2,500 ms after target onset) in all experiments. These data show that shape information is not used in online search of printed word displays (whereas it is used with picture displays). The nature of the visual environment appears to induce implicit biases toward particular modes of processing during language-mediated visual search.     

The visual hemifield asymmetry in the spatial blink during singleton search and feature search

The present study examined a visual field asymmetry in the contingent capture of attention that was previously observed by Du and Abrams (2010). In our first experiment, color singleton distractors that matched the color of a to-be-detected target produced a stronger capture of attention when they appeared in the left visual hemifield than in the right visual hemifield. This replicated Du and Abrams and also revealed a difference between hemifields in the time course of this effect. Our second experiment suggested that this asymmetry is moderated by the tuning of attentional control settings: when the target was easier to detect the asymmetry was attenuated. Our third experiment showed that this asymmetry is also present during singleton detection: a color singleton distractor produced a larger capture effect in the left hemifield than in the right hemifield. Finally, our fourth experiment suggested that this asymmetry is moderated by the salience of the attention-capturing distractor: when the distractor was not salient, the asymmetry was attenuated. These results suggest that there are boundary conditions in the observed hemifield asymmetry in the contingent capture of attention and several underlying brain systems might be involved.     

Electrophysiological evidence for parallel and serial processing during visual search

Event-related potentials were recorded from young adults during a visual search task in order to evaluate parallel and serial models of visual processing in the context of Treisman's feature integration theory. Parallel and serial search strategies were produced by the use of feature-present and feature-absent targets, respectively. In the feature-absent condition, the slopes of the functions relating reaction time and latency of the P3 component to set size were essentially identical, indicating that the longer reaction times observed for larger set sizes can be accounted for solely by changes in stimulus identification and classification time, rather than changes in post-perceptual processing stages. In addition, the amplitude of the P3 wave on target-present trials in this condition increased with set size and was greater when the preceding trial contained a target, whereas P3 activity was minimal on target-absent trials. These effects are consistent with the serial self-terminating search model and appear to contradict parallel processing accounts of attention-demanding visual search performance, at least for a subset of search paradigms. Differences in ERP scalp distributions further suggested that different physiological processes are utilized for the detection of feature presence and absence.     

Effort during visual search and counting: insights from pupillometry

We investigated the processing effort during visual search and counting tasks using a pupil dilation measure. Search difficulty was manipulated by varying the number of distractors as well as the heterogeneity of the distractors. More difficult visual search resulted in more pupil dilation than did less difficult search. These results confirm a link between effort and increased pupil dilation. The pupil dilated more during the counting task than during target-absent search, even though the displays were identical, and the two tasks were matched for reaction time. The moment-to-moment dilation pattern during search suggests little effort in the early stages, but increasingly more effort towards response, whereas the counting task involved an increased initial effort, which was sustained throughout the trial. These patterns can be interpreted in terms of the differential memory load for item locations in each task. In an additional experiment, increasing the spatial memory requirements of the search evoked a corresponding increase in pupil dilation.These results support the view that search tasks involve some, but limited, memory for item locations, and the effort associated with this memory load increases during the trials. In contrast, counting involves a heavy locational memory component from the start.     

Distractor ratio influences patterns of eye movements during visual search

We examined the flexibility of guidance in a conjunctive search task by manipulating the ratios between different types of distractors. Participants were asked to decide whether a target was present or absent among distractors sharing either colour or shape. Results indicated a strong effect of distractor ratio on search performance. Shorter latency to move, faster manual response, and fewer fixations per trial were observed at extreme distractor ratios. The distribution of saccadic endpoints also varied flexibly as a function of distractor ratio. When there were very few same-colour distractors, the saccadic selectivity was biased towards the colour dimension. In contrast, when most of the distractors shared colour with the target, the saccadic selectivity was biased towards the shape dimension. Results are discussed within the framework of the guided search model.     

Autonomic responsivity during visual search of hyperactive and reading-disabled children

Heart rate and skin conductance measures, recorded during a visual search task, were compared for hyperactive, reading-disabled, hyperactive reading-disabled, and control elementary school boys. As shown in past work, basal autonomic levels did not statistically differentiate groups. In all groups, heart rate levels increased with task complexity, but more so on reward than on nonreward trials. In the intertriai interval, heart rate decelerated consistently when subjects were anticipating stimuli associated with reward, but when reward was uncertain, heart rate accelerated slightly or stayed the same. Control subjects exhibited anticipatory heart rate deceleration more consistently than did clinical subjects, especially the solely hyperactive ones. Skin conductance levels first decreased, then rose as task complexity increased, but the groups did not differ on this measure. A conditioning model is outlined to explain directional changes in heart rate.     

Effort during visual search and counting: Insights from pupillometry

We investigated the processing effort during visual search and counting tasks using a pupil dilation measure. Search difficulty was manipulated by varying the number of distractors as well as the heterogeneity of the distractors. More difficult visual search resulted in more pupil dilation than did less difficult search. These results confirm a link between effort and increased pupil dilation. The pupil dilated more during the counting task than during target-absent search, even though the displays were identical, and the two tasks were matched for reaction time. The moment-to-moment dilation pattern during search suggests little effort in the early stages, but increasingly more effort towards response, whereas the counting task involved an increased initial effort, which was sustained throughout the trial. These patterns can be interpreted in terms of the differential memory load for item locations in each task. In an additional experiment, increasing the spatial memory requirements of the search evoked a corresponding increase in pupil dilation.These results support the view that search tasks involve some, but limited, memory for item locations, and the effort associated with this memory load increases during the trials. In contrast, counting involves a heavy locational memory component from the start.