Eye movements and attention in reading,scene perception,and visual search

Eye movements are now widely used to investigate cognitive processes during reading, scene perception, and visual search. In this article, research on the following topics is reviewed with respect to reading: (a) the perceptual span (or span of effective vision), (b) preview benefit, (c) eye movement control, and (d) models of eye movements. Related issues with respect to eye movements during scene perception and visual search are also reviewed. It is argued that research on eye movements during reading has been somewhat advanced over research on eye movements in scene perception and visual search and that some of the paradigms developed to study reading should be more widely adopted in the study of scene perception and visual search. Research dealing with “real-world” tasks and research utilizing the visual-world paradigm are also briefly discussed.     

You detect while I search: examining visual search efficiency in a joint search task

Numerous factors impact attentional allocation, with behaviour being strongly influenced by the interaction between individual intent and our visual environment. Traditionally, visual search efficiency has been studied under solo search conditions. Here, we propose a novel joint search paradigm where one individual controls the visual input available to another individual via a gaze contingent window (e.g., Participant 1 controls the window with their eye movements and Participant 2 – in an adjoining room – sees only stimuli that Participant 1 is fixating and responds to the target accordingly). Pairs of participants completed three blocks of a detection task that required them to: (1) search and detect the target individually, (2) search the display while their partner performed the detection task, or (3) detect while their partner searched. Search was most accurate when the person detecting was doing so for the second time while the person controlling the visual input was doing so for the first time, even when compared to participants with advanced solo or joint task experience (Experiments 2 and 3). Through surrendering control of one’s search strategy, we posit that there is a benefit of a reduced working memory load for the detector resulting in more accurate search. This paradigm creates a counterintuitive speed/accuracy trade-off which combines the heightened ability that comes from task experience (discrimination task) with the slower performance times associated with a novel task (the initial search) to create a potentially more efficient method of visual search.     

Area activation: a computational model of saccadic selectivity in visual search

The Area Activation Model (Pomplun, Reingold, Shen, & Williams, 2000) is a computational model predicting the statistical distribution of saccadic endpoints in visual search tasks. Its basic assumption is that saccades in visual search tend to foveate display areas that provide a maximum amount of task‐relevant information for processing during the subsequent fixation. In the present study, a counterintuitive prediction by the model is empirically tested, namely that saccadic selectivity towards stimulus features depends on the spatial arrangement of search items. We find good correspondence between simulated and empirically observed selectivity patterns, providing strong support for the Area Activation Model.     

Spontaneous visual search during the first two years: Improvement with age but no evidence of efficient search

Efficient visual search, wherein reaction times to acquire targets are largely independent of array size, is commonly observed in adults. Evidence for efficient search in infants may imply that selective attention to visual features is similar across development. In the current cross-sectional eye-tracking study, we examined spontaneous visual search at 6, 12, 18, and 24 months. Infants were presented with Random arrays (one target among 7, 13, or 26 pseudorandomly distributed elements) and Circle arrays (one target among 4, 7, or 13 elements arranged in a circle). Contrary to predictions, we did not find evidence of efficient search among infants. With increasing array size, time-to-target increased, the proportion of targets fixated (analogous to accuracy) decreased, and the proportion of first looks to the target decreased for both types of array (ps < .001). For Random arrays, the proportion of first looks to the target was similar to chance for all ages and array sizes; for Circle arrays, it exceeded chance for some ages and array sizes. The proportion of targets fixated and first looks to target increased with age across display types (ps < .05). We also tested adults with the same stimuli under similar conditions; the adults showed evidence of efficient visual search. Possible explanations and implications are discussed.     

Looking away from death: Defensive attention as a form of terror management

Previous research has suggested that the physical aspects of human nature in general, and physical human frailties in particular become disagreeable and repugnant following death primes. The current research tested this hypothesis in two studies using an eye-tracking methodology. Participants were subliminally primed with death or with a control word and then viewed a series of arrays containing four pictures each, during which their eye-movements were monitored. In Study 1, the arrays included pictures of physical injury or neutral objects, and in Study 2 pictures of physical injury, threatening images, and neutral objects. The results indicated that in both studies death primes significantly decreased gaze duration towards pictures of physical injury, and did not have a significant effect on gaze duration towards neutral images. However, in Study 2 death primes increased gaze duration towards threatening images. The discussion examines the role of motivated unconscious attention in terror management processes.     

Exploring conflict- and target-related movement of visual attention

Intermixing trials of a visual search task with trials of a modified flanker task, the authors investigated whether the presentation of conflicting distractors at only one side (left or right) of a target stimulus triggers shifts of visual attention towards the contralateral side. Search time patterns provided evidence for lateral attention shifts only when participants performed the flanker task under an instruction assumed to widen the focus of attention, demonstrating that instruction-based control settings of an otherwise identical task can impact performance in an unrelated task. Contrasting conditions with response-related and response-unrelated distractors showed that shifting attention does not depend on response conflict and may be explained as stimulus-conflict-related withdrawal or target-related deployment of attention.     

Not all visual memories are created equal

Two experiments examined how well the long-term visual memories of objects that are encountered multiple times during visual search are updated. Participants searched for a target two or four times (e.g., white cat) among distractors that shared the target's colour, category, or were unrelated while their eye movements were recorded. Following the search, a surprise visual memory test was given. With additional object presentations, only target memory reliably improved; distractor memory was unaffected by the number of object presentations. Regression analyses using the eye movement variables as predictors indicated that number of object presentations predicted target memory with no additional impact of other viewing measures. In contrast, distractor memory was best predicted by the viewing pattern on the distractor objects. Finally, Experiment 2 showed that target memory was influenced by number of target object presentations, not number of searches for the target. Each of these experiments demonstrates visual memory differences between target and distractor objects and may provide insight into representational differences in visual memory.     

On the visual span during object search in real-world scenes

The current study investigated from how large a region around their current point of gaze viewers can take in information when searching for objects in real-world scenes. Visual span size was estimated using the gaze-contingent moving window paradigm. Experiment 1 featured window radii measuring 1, 3, 4, 4.7, 5.4, and 6.1°. Experiment 2 featured six window radii measuring between 5 and 10°. Each scene occupied a 24.8 × 18.6° field of view. Inside the moving window, the scene was presented in high resolution. Outside the window, the scene image was low-pass filtered to impede the parsing of the scene into constituent objects. Visual span was defined as the window size at which object search times became indistinguishable from search times in the no-window control condition; this occurred with windows measuring 8° and larger. Notably, as long as central vision was fully available (window radii ≥ 5°), the distance traversed by the eyes through the scene to the search target was comparable to baseline performance. However, to move their eyes to the target, viewers made shorter saccades, requiring more fixations to cover the same image space, and thus more time. Moreover, a gaze-data based decomposition of search time revealed disruptions in specific subprocesses of search. In addition, nonlinear mixed models analyses demonstrated reliable individual differences in visual span size and parameters of the search time function.     

The role of priming in conjunctive visual search

To assess the role of priming in conjunctive visual search tasks, we systematically varied the consistency of the target and distractor identity between different conditions. Search was fastest in the standard conjunctive search paradigm where identities remained constant. Search was slowest when potential target identity varied predictably for each successive trial (the 'switch' condition). The role of priming was also demonstrated on a trial-by-trial basis in a 'streak' condition where target and distractor identity was unpredictable yet was consistent within streaks. When the target to be found was the same for a few trials in a row, search performance became similar to that when the potential target was the same on all trials. A similar pattern was found for the target absent trials, suggesting that priming is based on the whole search array rather than just the target in each case. Further analysis indicated that the effects of priming are sufficiently strong to account for the advantage seen for the conjunctive search task. We conclude that the role of priming in visual search is underestimated in current theories of visual search and that differences in search times often attributed to top-down guidance may instead reflect the benefits of priming.     

A taxonomy of errors in multiple-target visual search

Multiple-target visual searches are especially error prone; once one target is found, additional targets are likely to be missed. This phenomenon, often called satisfaction of search (which we refer to here as subsequent search misses; SSMs), is well known in radiology, despite no existing consensus about the underlying cause(s). Taking a cognitive laboratory approach, we propose that there are multiple causes of SSMs and present a taxonomy of SSMs based on searchers' eye movements during a multiple-target search task, including both previously identified and novel sources of SSMs. The types and distributions of SSMs revealed effects of working memory load, search strategy, and additional causal factors, suggesting that there is no single cause of SSMs. A multifaceted approach is likely needed to understand the psychological causes of SSMs and then to mitigate them in applied settings such as radiology and baggage screening.     

The target effect: Visual memory for unnamed search targets

Search targets are typically remembered much better than other objects even when they are viewed for less time. However, targets have two advantages that other objects in search displays do not have: They are identified categorically before the search, and finding them represents the goal of the search task. The current research investigated the contributions of both of these types of information to the long-term visual memory representations of search targets. Participants completed either a predefined search or a unique-object search in which targets were not defined with specific categorical labels before searching. Subsequent memory results indicated that search target memory was better than distractor memory even following ambiguously defined searches and when the distractors were viewed significantly longer. Superior target memory appears to result from a qualitatively different representation from those of distractor objects, indicating that decision processes influence visual memory.     

Cognitive control of fixation duration in visual search: The role of extrafoveal processing

Participants' eye movements were monitored in two visual search experiments that manipulated target-distractor similarity (high vs. low) as well as the availability of distractors for extrafoveal processing (Free-Viewing vs. No-Preview). The influence of the target-distractor similarity by preview manipulation on the distributions of first fixation and second fixation duration was examined by using a survival analysis technique which provided precise estimates of the timing of the first discernible influence of target-distractor similarity on fixation duration. We found a significant influence of target-distractor similarity on first fixation duration in normal visual search (Free Viewing) as early as 26–28 ms from the start of fixation. In contrast, the influence of target-distractor similarity occurred much later (199–233 ms) in the No-Preview condition. The present study also documented robust and fast acting extrafoveal and foveal preview effects. Implications for models of eye-movement control and visual search are discussed.     

Emotional attention: effects of emotion and gaze direction on overt orienting of visual attention

In the present study we considered the two factors that have been advocated for playing a role in emotional attention: perception of gaze direction and facial expression of emotions. Participants performed an oculomotor task in which they had to make a saccade towards one of the two lateral targets, depending on the colour of the fixation dot which appeared at the centre of the computer screen. At different time intervals (stimulus onset asynchronies, SOAs: 50,100,150 ms) following the onset of the dot, a picture of a human face (gazing either to the right or to the left) was presented at the centre of the screen. The gaze direction of the face could be congruent or incongruent with respect to the location of the target, and the expression could be neutral or angry. In Experiment 1 the facial expressions were presented randomly in a single block, whereas in Experiment 2 they were shown in separate blocks. Latencies for correct saccades and percentage of errors (saccade direction errors) were considered in the analyses. Results showed that incongruent trials determined a significantly higher percentage of saccade direction errors with respect to congruent trials, thus confirming that gaze direction, even when task-irrelevant, interferes with the accuracy of the observer’s oculomotor behaviour. The angry expression was found to hold attention for a longer time with respect to the neutral one, producing delayed saccade latencies. This was particularly evident at 100 ms SOA and for incongruent trials. Emotional faces may then exert a modulatory effect on overt attention mechanisms.     

Attention capacity and task difficulty in visual search

When a visual search task is very difficult (as when a small feature difference defines the target), even detection of a unique element may be substantially slowed by increases in display set size. This has been attributed to the influence of attentional capacity limits. We examined the influence of attentional capacity limits on three kinds of search task: difficult feature search (with a subtle featural difference), difficult conjunction search, and spatial-configuration search. In all 3 tasks, each trial contained sixteen items, divided into two eight-item sets. The two sets were presented either successively or simultaneously. Comparison of accuracy in successive versus simultaneous presentations revealed that attentional capacity limitations are present only in the case of spatial-configuration search. While the other two types of task were inefficient (as reflected in steep search slopes), no capacity limitations were evident. We conclude that the difficulty of a visual search task affects search efficiency but does not necessarily introduce attentional capacity limits.     

The consequence of a limited-capacity short-term memory on repeated visual search

ABSTRACT

When participants search the same letter display repeatedly for different targets we might expect performance to improve on each subsequent search as they memorize characteristics of the display. However, here we find that search performance improved from a first search to a second search but not for a third search of the same display. This is predicted by a simple model that supports search with only a limited capacity short-term memory for items in the display. To support this model we show that a short-term memory recency effect is present in both the second and the third search. The magnitude of these effects is the same in both searches and as a result there is no additional benefit from the second to the third search.     

Strategic and automatic effects of visual working memory on attention in visual search

Previous research indicates that visual attention can be automatically captured by sensory inputs that match the contents of visual working memory. However, Woodman and Luck (2007) showed that information in working memory can be used flexibly as a template for either selection or rejection according to task demands. We report two experiments that extend their work. Participants performed a visual search task while maintaining items in visual working memory. Memory items were presented for either a short or long exposure duration immediately prior to the search task. Memory was tested by a change-detection task immediately afterwards. On a random half of trials items in memory matched either one distractor in the search task (Experiment 1) or three (Experiment 2). The main result was that matching distractors speeded or slowed target detection depending on whether memory items were presented for a long or short duration. These effects were more in evidence with three matching distractors than one. We conclude that the influence of visual working memory on visual search is indeed flexible but is not solely a function of task demands. Our results suggest that attentional capture by perceptual inputs matching information in visual working memory involves a fast automatic process that can be overridden by a slower top-down process of attentional avoidance.     

Concurrent deployment of visual attention and response selection bottleneck in a dual-task: Electrophysiological and behavioural evidence

Visual attention and response selection are limited in capacity. Here, we investigated whether visual attention requires the same bottleneck mechanism as response selection in a dual-task of the psychological refractory period (PRP) paradigm. The dual-task consisted of an auditory two-choice discrimination Task 1 and a conjunction search Task 2, which were presented at variable temporal intervals (stimulus onset asynchrony, SOA). In conjunction search, visual attention is required to select items and to bind their features resulting in a serial search process around the items in the search display (i.e., set size). We measured the reaction time of the visual search task (RT2) and the N2pc, an event-related potential (ERP), which reflects lateralized visual attention processes. If the response selection processes in Task 1 influence the visual attention processes in Task 2, N2pc latency and amplitude would be delayed and attenuated at short SOA compared to long SOA. The results, however, showed that latency and amplitude were independent of SOA, indicating that visual attention was concurrently deployed to response selection. Moreover, the RT2 analysis revealed an underadditive interaction of SOA and set size. We concluded that visual attention does not require the same bottleneck mechanism as response selection in dual-tasks.     

The transference of visual search between two unrelated tasks: Measuring the temporal characteristics of carry-over

Investigations into the persistence of top-down control settings do not accurately reflect the nature of dynamic tasks. They typically involve extended practice with an initial task, and this initial task usually shares similar stimuli with a second task. Recent work shows that visual attention and search can be affected by limited exposure to a preceding, unrelated task, and the current study explored the temporal characteristics of this “carry-over” effect. Thirty-four participants completed one, four, or eight simple letter searches and then searched a natural scene. The spatial layout of letters influenced spread of search in the pictures, and this was further impacted by the time spent in the initial task, yet the carry-over effect diminished quickly. The results have implications for theories of top-down control and models that attempt to predict search in natural scenes. They are also relevant to real-world tasks in which performance is closely related to visual attention and search.     

Incidental memory for parts of scenes from eye movements

Incidental memory for parts of scenes was examined in two search experiments and one memory control experiment. Eye movements were recorded during the search experiments and used to select gaze-contingent sections from search scenes for a surprise memory recognition task. Results from the recognition task showed incidental memory was better for sections viewed longer and with multiple fixations. Sections not fixated during search were still recognized above chance as well. Differences in sections did not affect memory performance in a control experiment when viewing time was held constant. These results show that memory for parts of scenes can occur incidentally during search and encoding of tested sections is better with longer viewing time and with multiple fixations.     

Proneness to psychosis and selection of objects of visual attention: individual differences in visual marking

Deficits in early visual attention and perceptual organisation have frequently been shown to associate with both poor pre-morbid schizophrenia and those at a greater putative risk of psychosis. The nature of the deficit is unclear. The present study investigated the relationship between speed of visual marking and proneness to psychosis. 20 males and 20 females completed several tasks assessing speed of selection and de-selection of visual objects. As predicted, negative schizotypy was associated with poorer marking in males, but socially desirable responding potentially confounded this result. In addition, impulsive non-conformity swas associated with poorer visual marking, more prominently in females. These results are discussed in relation to possible mechanisms by which psychosis-proneness and impulsivity may restrict the top–down influences operating on early visual attention.