Visual search for faces with emotional expressions

The goal of this review is to critically examine contradictory findings in the study of visual search for emotionally expressive faces. Several key issues are addressed: Can emotional faces be processed preattentively and guide attention? What properties of these faces influence search efficiency? Is search moderated by the emotional state of the observer? The authors argue that the evidence is consistent with claims that (a) preattentive search processes are sensitive to and influenced by facial expressions of emotion, (b) attention guidance is influenced by a dynamic interplay of emotional and perceptual factors, and (c) visual search for emotional faces is influenced by the emotional state of the observer to some extent. The authors also argue that the way in which contextual factors interact to determine search performance needs to be explored further to draw sound conclusions about the precise influence of emotional expressions on search efficiency. Methodological considerations (e.g., set size, distractor background, task set) and ecological limitations of the visual search task are discussed. Finally, specific recommendations are made for future research directions.     

Monitoring eye movements while searching for affective faces

A single experiment is reported in which we provide a novel analysis of eye movements during visual search to disentangle the contributions of unattended guidance and focal target processing to visual search performance. This technique is used to examine the controversial claim that unattended affective faces can guide attention during search. Results indicated that facial expression influences how efficiently the target was fixated for the first time as a function of set size. However, affective faces did not influence how efficiently the target was identified as a function of set size after it was first fixated. These findings suggest that, in the present context, facial expression can influence search before the target is attended and that the present measures are able to distinguish between the guidance of attention by targets and the processing of targets within the focus of attention.     

Increased gaze following for fearful faces. It depends on what you’re looking for!

An oculomotor visual search task was used to investigate how participants follow the gaze of a non-predictive and task irrelevant distractor gaze, and the way in which this gaze following is influenced by the emotional expression (fearful vs. happy) as well as participants' goal. Previous research has suggested that fearful emotions should result in stronger cueing effects than happy faces. Our results demonstrated that the degree to which the emotional expression influenced this gaze following varied as a function of the search target. When searching for a threatening target, participants were more likely to look in the direction of eye gaze on a fearful compared to a happy face. However, when searching for a pleasant target, this stronger cueing effect for fearful faces disappeared. Therefore, gaze following is influenced by contextual factors such as the emotional expression, as well as the participant's goal.     

Visual marking: dissociating effects of new and old set size

Visual marking makes it possible to ignore old items during search. In a typical study, old items are previewed 1 s before adding an equal number of new items, one of which is the target. Previewing half of the items reduces the search slope relating response time (RT) to overall set size by half. However, this manipulation sometimes only reduces overall RT but not search slope (Experiment 1). By orthogonally varying the numbers of old and new items, Experiment 2 shows that old and new set sizes interactively affect visual marking. Given a constant new set size, the size of the old set has negligible effect on RT. However, increasing the new set size reduces the preview benefit in overall RT. Experiment 3 shows that this reduction may be restricted to paradigms that use temporal segregation cues. Studies should vary old and new set size orthogonally to avoid missing a visual marking effect where one may be present.     

Fixed-target efficient search has logarithmic efficiency with and without eye movements

Stage 1 processing in visual search (e.g., efficient search) has long been thought to be unaffected by factors such as set size or lure–distractor similarity (or at least to be only minimally affected). Recent research from Buetti, Cronin, Madison, Wang, and Lleras (Journal of Experimental Psychology: General, 145, 672–707, 2016) showed that in efficient visual search with a fixed target, reaction times increase logarithmically as a function of set size and, further, that the slope of these logarithmic functions is modulated by target–distractor similarity. This has led to the proposal that the cognitive architecture of Stage 1 processing is parallel, of unlimited capacity, and exhaustive in nature. Such an architecture produces reaction time functions that increase logarithmically with set size (as opposed to being unaffected by it). However, in the previous studies, eye movements were not monitored. It is thus possible that the logarithmicity of the reaction time functions emerged simply as an artifact of eye movements rather than as a reflection of the underlying cognitive architecture. Here we ruled out the possibility that eye movements resulted in the observed logarithmic functions, by asking participants to keep their eyes at fixation while completing fixed-target efficient visual search tasks. The logarithmic RT functions still emerged even when participants were not allowed to make eye movements, thus providing further support for our proposal. Additionally, we found that search efficiency is slightly improved when eye movements are restricted and lure–target similarity is relatively high.     

Visual similarity is stronger than semantic similarity in guiding visual search for numbers

Using a visual search task, we explored how behavior is influenced by both visual and semantic information. We recorded participants’ eye movements as they searched for a single target number in a search array of single-digit numbers (0–9). We examined the probability of fixating the various distractors as a function of two key dimensions: the visual similarity between the target and each distractor, and the semantic similarity (i.e., the numerical distance) between the target and each distractor. Visual similarity estimates were obtained using multidimensional scaling based on the independent observer similarity ratings. A linear mixed-effects model demonstrated that both visual and semantic similarity influenced the probability that distractors would be fixated. However, the visual similarity effect was substantially larger than the semantic similarity effect. We close by discussing the potential value of using this novel methodological approach and the implications for both simple and complex visual search displays.     

Hybrid search in the temporal domain: Evidence for rapid,serial logarithmic search through memory

In hybrid search, observers memorize a number of possible targets and then search for any of these in visual arrays of items. Wolfe (2012) has previously shown that the response times in hybrid search increase with the log of the memory set size. What enables this logarithmic search of memory? One possibility is a series of steps in which subsets of the memory set are compared to all items in the visual set simultaneously. In the present experiments, we presented single visual items sequentially in a rapid serial visual presentation (RSVP) display, eliminating the possibility of simultaneous testing of all items. We used a staircasing procedure to estimate the time necessary to effectively detect the target in the RSVP stream. Processing time increased in a log–linear fashion with the number of potential targets. This finding eliminates the class of models that require simultaneous comparison of some memory items to all (or many) items in the visual display. Experiment 3 showed that, similar to visual search, memory search efficiency in this paradigm is influenced by the similarity between the target set and the distractors. These results indicate that observers perform separate memory searches on each eligible item in the visual display. Moreover, it appears that memory search for one item can proceed while other items are being categorized as “eligible” or “not eligible.”     

Why are there eccentricity effects in visual search? Visual and attentional hypotheses

In standard visual search experiments, observers search for a target item among distracting items. The locations of target items are generally random within the display and ignored as a factor in data analysis. Previous work has shown that targets presented near fixation are, in fact, found more efficiently than are targets presented at more peripheral locations. This paper proposes that the primary cause of this “eccentricity effect” (Carrasco, Evert, Chang, & Katz, 1995) is an attentional bias that allocates attention preferentially to central items. The first four experiments dealt with the possibility that visual, and not attentional, factors underlie the eccentricity effect. They showed that the eccentricity effect cannot be accounted for by the peripheral reduction in visual sensitivity, peripheral crowding, or cortical magnification. Experiment 5 tested the attention allocation model and also showed that RT X set size effects can be independent of eccentricity effects. Experiment 6 showed that the effective set size in a search task depends, in part, on the eccentricity of the target because observers search from fixation outward.     

Visual marking and the perception of salience in visual search

In the present study, the gap paradigm originally developed by Watson and Humphreys (1997) was used to investigate whether the process of visual marking can influence the perceptual salience of a target in visual search. Consistent with previous studies (Watson & Humphreys, 1997), the results showed that search was not affected by the presence of the preceding distractors when the target was relatively low in salience. This finding suggests that visual marking can increase the efficiency of visual search by decreasing the size of the search set. However, more important, the results also showed that search was affected by the presence of the preceding distractors when the target was relatively high in salience. This finding suggests that visual marking may be limited in its ability to increase the perceptual salience of the target. Together, the results of the present study suggest that the effectiveness of visual marking may vary as a function of search context.     

Improvement, learning and retention of skill at visual search

Three experiments examined factors responsible for improvement in visual search. In Experiment I three groups of subjects were each trained for 3000 trials to search for a particular set of target letters among a particular set of background letters. After intervals of 2, 4, or 6 weeks without further practice they were re-tested, either with the same displays or on new displays which they searched for the same target items among new background items. Negative transfer suggested that memory for specific cue-systems distinguishing target from background letters is retained for as long as 4 weeks.

Experiment II examined performance of three very highly practised subjects. After 25 days practice, variations in the size of the target set no longer affected search time. This could not be explained by learning of specific cue systems, since after this amount of practice subjects showed perfect transfer to displays which they searched for the same target items among new background items. Experiment III suggested that practised subjects achieve this high level of competence by learning to make two independent successive decisions, first to locate any member of the target set on the display and next to identify which particular member of the target set has been located.     

视觉负后像对视觉搜索效率的干扰效应

为了探究视觉负后像对视觉搜索效率是否会通过颜色这一特征产生干扰,结合Van Lier后像和视觉搜索范式,通过两个实验考察了负后像对视觉搜索效率的干扰效应,并对负后像填充区域和原始发生区域进行了对比。实验结果表明,在有负后像的情况下,视觉搜索效率明显降低,但是不同颜色之间的差异并不显著;而负后像填充区域的干扰效应要弱于原始发生区域。研究结论对于进一步提高视觉搜索效率的颜色设计以及视觉负后像的新研究方法具有重要的参考意义。     

Priming of luminance-defined motion direction in visual search

Features that we have recently attended to strongly influence how we allocate visual attention across a subsequently viewed visual scene. Here, we investigate the characteristics of any such repetition effects during visual search for Gabor patch targets drifting in the odd direction relative to a set of distractors. The results indicate that repetition of motion direction has a strong effect upon subsequent allocation of attention. This was the case for judgments of a target’s presence or absence, of a target’s location, and of the color of a target drifting in the odd direction. Furthermore, distractor repetition on its own can facilitate search performance on subsequent trials, indicating that the benefits of repetition of motion direction are not confined to repetition of target features. We also show that motion direction need not be the target-defining dimension throughout a trial block for motion priming to occur, but that priming can build up with only one presentation of a given target direction, even within blocks of trials where the target may, unpredictably, be defined by a different feature (color, in this case), showing that dimensional-weighting accounts cannot, on their own, account for motion direction priming patterns. Finally, we show by randomizing the set size between trials that priming of motion direction can decrease search rates in visual search.     

Attention capacity and task difficulty in visual search

When a visual search task is very difficult (as when a small feature difference defines the target), even detection of a unique element may be substantially slowed by increases in display set size. This has been attributed to the influence of attentional capacity limits. We examined the influence of attentional capacity limits on three kinds of search task: difficult feature search (with a subtle featural difference), difficult conjunction search, and spatial-configuration search. In all 3 tasks, each trial contained sixteen items, divided into two eight-item sets. The two sets were presented either successively or simultaneously. Comparison of accuracy in successive versus simultaneous presentations revealed that attentional capacity limitations are present only in the case of spatial-configuration search. While the other two types of task were inefficient (as reflected in steep search slopes), no capacity limitations were evident. We conclude that the difficulty of a visual search task affects search efficiency but does not necessarily introduce attentional capacity limits.     

Allocation of attention in texture segregation, visual search, and location-precuing paradigms

Is there a common mechanism (or set of mechanisms) that controls the allocation of spatial attention in texture segregation, visual search, and location-precuing tasks? This question was addressed by comparing data from our previous research on location-cuing and search tasks with data from a new set of experiments on texture segregation, in which a common set of stimuli was used across the three paradigms. However, when we ranked the scores for the targets on texture segregation speed, search rate, and improvement due to precuing effects, agreement between these measures was less than perfect. Further analyses of these results led to the following conclusions: (a) speed of texture segregation is affected by perceptual aspects of the display other than the attentional salience of a particular target; (b) visual search rate and the size of precuing effects are strongly related over most of the set of targets we used; and (c) some cases in which search rate was not consistent with size of precue effects may be related to the presence of nearby distractors in the search task.     

Concurrent visual search and time reproduction with cross-talk

Previous experiments indicate that humans can reproduce a time interval while concurrently searching a visual display presented near the beginning of the reproduced interval (Fortin, C., Rousseau, R., Bourque, P. & Kirouac, E. (1993). Time estimation and concurrent nontemporal processing: Specific interference from short-term-memory demands. Perception & Psychophysics, 53, 536-548.). In the earlier experiments, reproduced intervals were either insensitive to or shortened with increasing duration of visual search, suggesting that reproduction and search proceeded concurrently but sometimes with cross-talk. Here we test concurrency of time reproduction and visual search using a different methodology than before. Assuming the underlying mental processes are in a critical path network, factors selectively influencing concurrent processes are predicted to produce certain patterns in response time means and, through recent developments, certain patterns in cumulative distribution functions. Processes in an experiment were influenced by manipulating four factors, the time interval to be produced, stimulus onset asynchrony, display size, and target presence/absence. Patterns in response times provide evidence for concurrent time reproduction and visual search. However, a prediction based on selective influence failed in a manner suggesting the presence of cross-talk produced by a filled interval illusion.     

视觉工作记忆对前注意阶段注意定向的调节

在注意阶段,视觉工作记忆与注意共享某些特征时,视觉工作记忆内容会引导注意定向。在前注意阶段的影响又是如何？对此问题的研究却很少见,本研究采用工作记忆与基于特征的视觉搜索相结合的任务深入探讨了此问题。研究结果发现：即使在前注意阶段,注意定向也会受到视觉工作记忆自上而下的调节。然而与注意阶段不同的是：当视觉工作记忆内容与随后视觉搜索任务的目标相关时,与记忆内容匹配的搜索项目会引导注意定向到该项目所在的位置,但只有此项目作为分心物时,才会明显地延迟对靶子的加工,而此项目作为靶子时,这种影响并不明显;当视觉工作记忆中保持的项目与随后视觉搜索任务的目标无关时,注意会忽略视觉工作记忆中保持的项目,有效地检测靶子     

Inefficient conjunction search made efficient by concurrent spoken delivery of target identity

Visual search based on a conjunction of two features typically elicits reaction times that increase linearly as a function of the number of distractors, whereas search based on a single feature is essentially unaffected by set size. These and related findings have often been interpreted as evidence of a serial search stage that follows a parallel search stage. However, a wide range of studies has been showing a form of blending of these two processes. For example, when a spoken instruction identifies the conjunction target concurrently with the visual display, the effect of set size is significantly reduced, suggesting that incremental linguistic processing of the first feature adjective and then the second feature adjective may facilitate something approximating a parallel extraction of objects during search for the target. Here, we extend these results to a variety of experimental designs. First, we replicate the result with a mixed-trials design (ruling out potential strategies associated with the blocked design of the original study). Second, in a mixed-trials experiment, the order of adjective types in the spoken query varies randomly across conditions. In a third experiment, we extend the effect to a triple-conjunction search task. A fourth (control) experiment demonstrates that these effects are not due to an efficient odd-one-out search that ignores the linguistic input. This series of experiments, along with attractor-network simulations of the phenomena, provide further evidence toward understanding linguistically mediated influences in real-time visual search processing.     

When visual search functions look like item recognition functions

Visual search data were collected from six Ss on three target set sizes on each of 30 days. Error level was low, and items assigned to memory sets were nonnested and changed from session tosession. For each S. the same item sometimes required a positive and sometimes a negative response (response inconsistency). Combining data over Ss and over successive 6-day blocks, visual search rates as a function of target set size were found to be linear for each of the five 6-day blocks. The slopes of the above functions (memory search time) did not differ significantly over the final four 6-day blocks, and averaged approximately.500 sec per six-character item. These results are qualitatively very similar to results obtained from item recognition studies when error level, memory set structure, degree of response’_ consistency, and practice are handled in the same way in that task. The significantly lower slope obtained on the first 6-day block is shown to be consistent with a speed-accuracy trade off interpretation when error rate is expressed per unit of processing time (percent errors/set size). Over the final three 6-day blocks, where all important parameters ofthe data were highly stable, the intercepts of the memory search functions were found to closely approximate zero, averaging .0068 sec. From this finding, along with the finding that the memory search functions are linear, it is inferred that visual search time is determined entirely by memory search time, or by memory search time and other processes which increase linearly with set size, under the conditions of this experiment. The estimate of memory search time (approximately 83 msec/character) obtained using this visual search procedure is much slower than that obtained using the item recognition procedure (approximately 35\2-40 msec/character). An explanation for this difference is proposed.     

The reconfiguration of task set has no effect on the efficiency of feature search

In this study, we investigated whether or not the reconfiguration of a set task could modulate the efficiency of feature search. In Experiment 1, target set size was manipulated. The results revealed that enumerating targets defined by color yields a much steeper slope when executed simultaneously with a discrimination task than when done singly. However, when distractor set size was manipulated, there was no significant difference in the efficiency of individualizing targets from distractors between the two task conditions, no matter whetherthe search was guided by bottom-up information (Experiment 2) or top-down information (Experiment 3). Drawing upon these results, the authors propose that (1) an additional task can impair the efficiency of enumeration, resulting in a steep slope of counting even when target set size is quite small, and (2) in feature search, the visual system first reconfigures the task set to match the input feature information and then detects the targets with an optimal efficiency, which does not support the theory proposed by Di Lollo, Kawahara, Zuvic, and Visser (2001; Di Lollo, Smilek, Kawahara, & Ghorashi, 2005).     

The role of categorization in visual search for orientation.

Visual search for 1 target orientation is fast and virtually independent of set size if all of the distractors are of a single, different orientation. However, in the presence of distractors of several orientations, search can become inefficient and strongly dependent on set size (Exp. 1). Search can be inefficient even if only 2 distractor orientations are used and even if those orientations are quite remote from the target orientation (e.g. 20 degrees or even 40 degrees away, Exp. 2). Search for 1 orientation among heterogeneous distractor orientations becomes more efficient if the target orientation is the only item possessing a categorical attribute such as steep, shallow (Exp. 3), tilted left or tilted right (Exp. 4), or simply tilted (Exps. 5 and 6). Orientation categories appear to be 1 of several strategies used in visual search for orientation. These serve as a compromise between the limits on parallel visual processing and the demands of a complex visual world.