Effort during visual search and counting: Insights from pupillometry

We investigated the processing effort during visual search and counting tasks using a pupil dilation measure. Search difficulty was manipulated by varying the number of distractors as well as the heterogeneity of the distractors. More difficult visual search resulted in more pupil dilation than did less difficult search. These results confirm a link between effort and increased pupil dilation. The pupil dilated more during the counting task than during target-absent search, even though the displays were identical, and the two tasks were matched for reaction time. The moment-to-moment dilation pattern during search suggests little effort in the early stages, but increasingly more effort towards response, whereas the counting task involved an increased initial effort, which was sustained throughout the trial. These patterns can be interpreted in terms of the differential memory load for item locations in each task. In an additional experiment, increasing the spatial memory requirements of the search evoked a corresponding increase in pupil dilation.These results support the view that search tasks involve some, but limited, memory for item locations, and the effort associated with this memory load increases during the trials. In contrast, counting involves a heavy locational memory component from the start.     

Visual significance of pupillary response to an auditory task.

Subjects simultaneously performed visual and auditory detection tasks. Pupillary dilation accompanies increased cognitive load such as that caused by the auditory tasks. Errors in the visual task increased when the auditory task became more difficult. The increase was greater when the effects of pupillary dilation were blocked by an artificial pupil.     

Distractor devaluation requires visual working memory

Visual stimuli seen previously as distractors in a visual search task are subsequently evaluated more negatively than those seen as targets. An attentional inhibition account for this distractor-devaluation effect posits that associative links between attentional inhibition and to-be-ignored stimuli are established during search, stored, and then later reinstantiated, implying that distractor devaluation may require visual working memory (WM) resources. To assess this, we measured distractor devaluation with and without a concurrent visual WM load. Participants viewed a memory array, performed a simple search task, evaluated one of the search items (or a novel item), and then viewed a memory test array. Although distractor devaluation was observed with low (and no) WM load, it was absent when WM load was increased. This result supports the notions that active association of current attentional states with stimuli requires WM and that memory for these associations plays a role in affective response.     

瞳孔变化在记忆加工中的生物标记作用

瞳孔在不同记忆任务的同一加工阶段内具有变化模式的一致性, 瞳孔的缩放变化反映了信息加工过程中神经系统的激活状态。在刺激呈现前的预编码阶段, 瞳孔随个体注意集中水平的提高而收缩。在信息编码阶段, 瞳孔扩张程度作为相继记忆效应的生物标记, 可预测信息未来提取阶段的回忆成绩。在信息提取阶段, 采用自由回忆任务时, 瞳孔随记忆负荷的释放而迅速收缩, 但每个项目的提取均会引发小幅度瞳孔扩张; 而在再认提取任务中, 呈现旧刺激比呈现新刺激时瞳孔扩张更明显, 产生瞳孔新旧效应。瞳孔是记忆加工中敏感而稳定的生物标记, 而瞳孔测量法是探究记忆加工生理机制的有效手段。未来可以从瞳孔追踪术的角度, 采用多种研究手段进一步探究记忆的生理机制。     

Visual search is slowed when visuospatial working memory is occupied

Visual working memory plays a central role in most models of visual search. However, a recent study showed that search efficiency was not impaired when working memory was filled to capacity by a concurrent object memory task (Woodman, Vogel, & Luck, 2001). Objects and locations may be stored in separate working memory subsystems, and it is plausible that visual search relies on the spatial subsystem, but not on the object subsystem. In the present study, we sought to determine whether maintaining spatial information in visual working memory impairs the efficiency of a concurrent visual search task. Visual search efficiency and spatial memory accuracy were both impaired when the search and the memory tasks were performed concurrently, as compared with when the tasks were performed separately. These findings suggest that common mechanisms are used to process information during difficult visual search tasks and to maintain spatial information in working memory.     

Stressing the mind: the effect of cognitive load and articulatory suppression on attentional guidance from working memory

Four experiments explored the effect of cognitive load on the time course of top-down guidance of attention from working memory (WM). Observers had to search for a target presented among several distractors, with the target and distractor stimuli embedded inside different objects. On half of the trials, one of the distractor objects was cued by a matching item held in WM. When a single item was maintained in memory, search performance was impaired relative to a neutral baseline, where the memory and search displays did not match. These effects of WM on subsequent search were reduced by including a verbal suppression task during the WM and search displays, and by varying the WM load. The degree of competition for resources in WM is a key factor in determining the time course and magnitude of the interaction between WM and visual selection.     

The role of working memory in attentional capture

Much previous research has demonstrated that visual search is typically disrupted by the presence of a unique “singleton” distractor in the search display. Here we show that attentional capture by an irrelevant color singleton during shape search critically depends on availability of working memory to the search task: When working memory is loaded in a concurrent yet unrelated verbal short-term memory task, capture increases. These findings converge with previous demonstrations that increasing working memory load results in greater distractor interference in Stroop-like tasks (de Fockert, Rees, Frith, & Lavie, 2001; Lavie, Hirst, de Fockert, & Viding, 2004), which support the hypothesis that working memory provides goal-directed control of visual selective attention allowing to minimize interference by goal-irrelevant distractors.     

Learning in repeated visual search

Visual search (e.g., finding a specific object in an array of other objects) is performed most effectively when people are able to ignore distracting nontargets. In repeated search, however, incidental learning of object identities may facilitate performance. In three experiments, with over 1,100 participants, we examined the extent to which search could be facilitated by object memory and by memory for spatial layouts. Participants searched for new targets (real-world, nameable objects) embedded among repeated distractors. To make the task more challenging, some participants performed search for multiple targets, increasing demands on visual working memory (WM). Following search, memory for search distractors was assessed using a surprise two-alternative forced choice recognition memory test with semantically matched foils. Search performance was facilitated by distractor object learning and by spatial memory; it was most robust when object identity was consistently tied to spatial locations and weakest (or absent) when object identities were inconsistent across trials. Incidental memory for distractors was better among participants who searched under high WM load, relative to low WM load. These results were observed when visual search included exhaustive-search trials (Experiment 1) or when all trials were self-terminating (Experiment 2). In Experiment 3, stimulus exposure was equated across WM load groups by presenting objects in a single-object stream; recognition accuracy was similar to that in Experiments 1 and 2. Together, the results suggest that people incidentally generate memory for nontarget objects encountered during search and that such memory can facilitate search performance.     

视空工作记忆负载对真实场景搜索绩效的影响

采用视空工作记忆任务和真实场景搜索任务相结合的双任务范式, 结合眼动技术将搜索过程划分为起始阶段、扫描阶段和确认阶段, 探究视空工作记忆负载对真实场景搜索绩效的影响机制, 同时考查试次间搜索目标是否变化、目标模板的具体化程度以及搜索场景画面的视觉混乱度所起的调节作用。结果表明, 视空工作记忆负载会降低真实场景搜索的成绩, 在搜索过程中表现为视空负载条件下扫描阶段持续时间的延长、注视点数目的增加和空间负载条件下确认阶段持续时间的延长, 视空负载对搜索过程的影响与目标模板的具体化程度有关; 空间负载会降低真实场景搜索的效率, 且与搜索画面的视觉混乱度有关, 而客体负载则不会。由此可见, 视空工作记忆负载对真实场景搜索绩效的影响不同, 空间负载对搜索过程的影响比客体负载更长久, 二者均受到目标模板具体化程度的调节; 仅空间负载会降低真实场景的搜索效率, 且受到搜索场景画面视觉混乱度的调节。     

Pupil Dilation as a Measure of Processing Load in Simultaneous Interpretation and Other Language Tasks

The present study tested whether the pupillary response can be applied to study the variation in processing load during simultaneous interpretation. In Experiment 1, the global processing load in simultaneous interpretation as reflected in the average pupil size was compared to that in two other language tasks, listening to and repeating back an auditorily presented text. Experiment 1 showed clear differences between the experimental tasks. In Experiment 2, the task effect was replicated using single words as stimuli. Experiment 2 showed that momentary variations in processing load during a lexical translation task are reflected in pupil size. Words that were chosen to be more difficult to translate induced higher levels of pupil dilation than did easily translatable words. Moreover, repeating back words in a non-native language was accompanied by increased pupil dilations, in comparison to repetition in the subject's native language. In sum, the study lends good support to the use of the pupillary response as an indicator of processing load.     

Temporal order in memory and interval timing: an interference analysis

The effect of varying load in memory tasks performed during a time interval production was examined. In a first experiment, increasing load in memory search for temporal order affected concurrent time production more strongly than varying load in a spatial memory task of equivalent difficulty. This result suggests that timing uses some specific resources also required in processing temporal order in memory, resources that would not be used in the spatial memory task. A second experiment showed that the interference between time production and memory search involving temporal order was stronger when, during the timing task, a decision was made on the temporal position of a memory item, than when information on temporal order was retained throughout the interval to be produced. These results underscore the importance of considering the specific resources and processes involved when the interference between timing and concurrent non-temporal tasks is analyzed.     

The role of working memory in auditory selective attention

A growing body of research now demonstrates that working memory plays an important role in controlling the extent to which irrelevant visual distractors are processed during visual selective attention tasks (e.g., Lavie, Hirst, De Fockert, & Viding, 2004). Recently, it has been shown that the successful selection of tactile information also depends on the availability of working memory (Dalton, Lavie, & Spence, 2009). Here, we investigate whether working memory plays a role in auditory selective attention. Participants focused their attention on short continuous bursts of white noise (targets) while attempting to ignore pulsed bursts of noise (distractors). Distractor interference in this auditory task, as measured in terms of the difference in performance between congruent and incongruent distractor trials, increased significantly under high (vs. low) load in a concurrent working-memory task. These results provide the first evidence demonstrating a causal role for working memory in reducing interference by irrelevant auditory distractors.     

The influence of cognitive load on spatial search performance

During search, executive function enables individuals to direct attention to potential targets, remember locations visited, and inhibit distracting information. In the present study, we investigated these executive processes in large-scale search. In our tasks, participants searched a room containing an array of illuminated locations embedded in the floor. The participants’ task was to press the switches at the illuminated locations on the floor so as to locate a target that changed color when pressed. The perceptual salience of the search locations was manipulated by having some locations flashing and some static. Participants were more likely to search at flashing locations, even when they were explicitly informed that the target was equally likely to be at any location. In large-scale search, attention was captured by the perceptual salience of the flashing lights, leading to a bias to explore these targets. Despite this failure of inhibition, participants were able to restrict returns to previously visited locations, a measure of spatial memory performance. Participants were more able to inhibit exploration to flashing locations when they were not required to remember which locations had previously been visited. A concurrent digit-span memory task further disrupted inhibition during search, as did a concurrent auditory attention task. These experiments extend a load theory of attention to large-scale search, which relies on egocentric representations of space. High cognitive load on working memory leads to increased distractor interference, providing evidence for distinct roles for the executive subprocesses of memory and inhibition during large-scale search.     

Incidental and intentional visual memory: What memories are and are not affected by encoding tasks?

Prior research into the impact of encoding tasks on visual memory (Castelhano & Henderson, 2005) indicated that incidental and intentional encoding tasks led to similar memory performance. The current study investigated whether different encoding tasks impacted visual memories equally for all types of objects in a conjunction search (e.g., targets, colour distractors, object category distractors, or distractors unrelated to the target). In sequences of pictures, participants searched for prespecified targets (e.g., green apple; Experiment 1), memorized all objects (Experiment 2), searched for specified targets while memorizing all objects (Experiment 3), searched for postidentified targets (Experiment 4), or memorized all objects with one object prespecified (Experiment 5). Encoding task significantly improved visual memory for targets and led to worse memory for unrelated distractors, but did not influence visual memory of distractors that were related to the target's colour or object category. The differential influence of encoding task indicates that the relative importance of the object both positively and negatively influences the memory retained.     

Cognitive control over working memory biases of selection

Across many studies, researchers have found that representations in working memory (WM) can guide visual attention toward items that match the features of the WM contents. While some researchers have contended that this occurs involuntarily, others have suggested that the impact of WM contents on attention can be strategically controlled. Here, we varied the probability that WM items would coincide with either targets or distractors in a visual search task to examine (1) whether participants could intentionally enhance or inhibit the influence of WM items on attention and (2) whether cognitive control over WM biases would also affect access to the memory contents in a surprise recognition test. We found visual search to be faster when the WM item coincided with the search target, and this effect was enhanced when the memory item reliably predicted the location of the target. Conversely, visual search was slowed when the memory item coincided with a search distractor, and this effect was diminished, but not abolished, when the memory item was reliably associated with distractors. This strategic dampening of the influence of WM items on attention came at a price to memory, however, as participants were slowest to perform WM recognition tests on blocks in which the WM contents were consistently invalid. These results document that attentional capture by WM contents is partly, but not fully, malleable by top-down control, which appears to adjust the state of the WM contents to optimize search behavior. These data illustrate the role of cognitive control in modulating the strength of WM biases of selection, and they support a tight coupling between WM and attention.     

视觉工作记忆负载类型对注意选择的影响

通过操纵Flanker任务相对于视觉工作记忆任务的呈现位置, 探讨在视觉工作记忆编码和保持阶段, 精度负载和容量负载对注意选择的影响。行为结果发现, Flanker任务呈现位置和视觉工作记忆负载类型影响注意选择; ERP结果发现, 在保持阶段, 当搜索目标和干扰项不一致时, 负载类型影响N2成分。研究表明, 在编码阶段, 视觉工作记忆负载主要通过占用更多知觉资源降低干扰效应, 支持知觉负载理论; 而在保持阶段, 当Flanker任务位于记忆项内部时, 两类负载在工作记忆表征过程中不同的神经活动导致投入到注意选择的认知控制资源不同, 可能是两类负载影响保持阶段注意选择的机制。     

Mental workload and visual impairment: differences between pupil, blink, and subjective rating

This research has two aims: (a) To study the concurrent validity of three measures of mental workload, NASA TLX rating scale, pupil dilation and blink rate, testing the hypothesis that they will provide convergent results using a single-task, and dissociative results for dual-task; and (b) To analyse their capability to predict visual search impairment. These three measures were analyzed in the same cognitive tasks in single-task and dual-task (cognitive task and visual search) conditions in a within-subjects experiment with twenty-nine participants. Mental workload measures showed concurrent validity under single-task condition, but a complex pattern of results arose in the dual-task condition: it is suggested that NASA TLX would be a subjective addition of the rating of each task; pupil dilation would measure the average arousal underlying the cognitive tasks; and the blink rate would produce opposite effects: whereas mental workload of cognitive tasks would increase blink rate, visual demand would inhibit it. All three measures were good predictors of visual impairment. The soundness of these measures is discussed with regard to the applied field of driving and other activities.     

The role of spatial working memory in visual search efficiency

Many theories have proposed that visual working memory plays an important role in visual search. In contrast, by showing that a nonspatial working memory load did not interfere with search efficiency, Woodman, Vogel, and Luck (2001) recently proposed that the role of working memory in visual search is insignificant. However, the visual search process may interfere with spatial working memory. In the present study, a visual search task was performed concurrently with either a spatial working memory task (Experiment 1) or a nonspatial working memory task (Experiment 2). We found that the visual search process interfered with a spatial working memory load, but not with a nonspatial working memory load. These results suggest that there is a distinction between spatial and nonspatial working memory in terms of interactions with visual search tasks. These results imply that the visual search process and spatial working memory storage require the same limited-capacity mechanisms.     

空间工作记忆负载对真实场景情境提示效应的影响

重复的画面布局能够促进观察者对目标项的搜索 (情境提示效应)。本研究采用双任务范式,分别在视觉搜索任务的学习阶段 (实验2a) 和测验阶段 (实验2b) 加入空间工作记忆任务, 并与单任务基线 (实验1)进行比较, 考察空间工作记忆负载对真实场景搜索中情境线索学习和情境提示效应表达的影响。结果发现: 空间负载会增大学习阶段的情境提示效应量, 同时削弱测验阶段的情境提示效应量, 而不影响情境线索的外显提取。由此可见, 真实场景中情境线索的学习和提示效应的表达均会受到有限的工作记忆资源的影响, 但情境线索提取的外显性不变。     

Searching for one versus two identical targets: when visual search has a memory

A recent study has suggested that observers' visual explorations of the external world can proceed unimpaired when the visual environment precludes the operation of memory processes (as, for instance, when the display elements change locations every 100 ms). One theoretical limitation of this study was that distractors were the only elements that had the potential to be tagged during visual search. The present study sought to clarify the amnesic-search hypothesis by investigating whether memory processes can guide search in other contexts in which targets also have the potential to be tagged. Accordingly, the experimental conditions of the previous study were repeated using a different search task in which observers had to decide whether one target or two were present among a variable number of similar distractors. Under these search conditions, the present findings provided strong evidence that memory processes can guide visual search.