Visual memory and the perception of a stable visual environment

The visual world appears stable and continuous despite eye movements. One hypothesis about how this perception is achieved is that the contents of successive fixations are fused in memory according to environmental coordinates. Two experiments failed to support this hypothesis; they showed that one's ability to detect a grating presented after a saccade is unaffected by the presentation of a grating with the same spatial frequency in the same spatial location before the saccade. A third experiment tested an alternative explanation of perceptual stability that claims that the contents of successive fixations are compared, rather than fused, across saccades, allowing one to determine whether the world has remained stable. This hypothesis was supported: Experienced subjects could accurately determine whether two patterns viewed in successive fixations were identical or different, even when the two patterns appeared in different spatial positions across the saccade. Taken together, these results suggest that perceptual stability and information integration across saccades rely on memory for the relative positions of objects in the environment, rather than on the spatiotopic fusion of visual information from successive fixations.     

Integration of form across saccadic eye movements.

To perceive a stable world, one must somehow be able to relate visual information from successive fixations. Little is known, however, about the nature of the integrative process. By using a task which requires the integration of spatial position information from different fixations, it is demonstrated that visual information from previous fixations is preserved in a world-centered representation which is precise enough to support judgements of geometric shape. It is also shown that successive views are aligned with respect to common visual features, indicating that visual stability may be normally accomplished by a visual matching strategy in combination with cancellation by an eye-position signal.     

Trans-saccadic perception

A basic question in cognition is how visual information obtained in separate glances can produce a stable, continuous percept. Previous explanations have included theories such as integration in a trans-saccadic buffer or storage in visual memory, or even that perception begins anew with each fixation. Converging evidence from primate neurophysiology, human psychophysics and neuroimaging indicate an additional explanation: the intention to make a saccadic eye movement leads to a fundamental alteration in visual processing itself before and after the saccadic eye movement. We outline five principles of 'trans-saccadic perception' that could help to explain how it is possible - despite discrete sensory input and limited memory - that conscious perception across saccades seems smooth and predictable.     

Attention to repeated images on the World-Wide Web: Another look at scanpath theory

The scanpath theory of visual perception was tested using Web pages as visual stimuli.Scanpaths are repetitive sequences of fixations and saccades that occur upon reexposure to a visual stimulus. Since Internet users are exposed to repeated visual displays, the Web provides ideal stimuli to test this theory. Eye movement data were recorded for subjects’repeated viewings of three Web pages over three sessions. Resemblance of eye path sequences was measured with a string-edit method; multidimensional scaling and cluster analysis were used to group sequences. Support was found for the scanpath theory; some clusters included pairs of sequences from the same subject. A repeated measures analysis of variance revealeda statistically significant main effect for stimulus type, with a text-intensive news story page generating more similar sequences than a graphic-intensive advertising page. There was a statistically significant main effect for cross-viewing comparisons, reflecting a linear trend in which eye paths for the same subject became more alike over time.     

Conceptual processing of text during skimming and rapid sequential reading

Potential conceptual processing difficulties associated with abbreviating the duration of eye fixations during reading were explored using a rapid sequential visual presentation (RSVP) paradigm in which each word of a text was briefly presented (for as little as 86 msec) at a fixed location. Question answering and accuracy of summaries associated with RSVP were found to be inferior to performance obtained when subjects skimmed conventionally presented passages. RSVP performance was enhanced by inserting brief pauses between sentences. The pauses produced only minor changes in the perceptual characteristics of RSVP, but they probably allowed subjects to more readily process information that had been buffered in working memory. These results imply that eye fixation durations in conventional reading reflect the time constraints of conceptual processing operations (as well as constraints of planning and initiating eye movements), but also that these conceptual operations may be distributed over later fixations and even postponed until a syntactic boundary is reached.     

Word meaning and the control of eye fixation: semantic competitor effects and the visual world paradigm

When participants are presented simultaneously with spoken language and a visual display depicting objects to which that language refers, participants spontaneously fixate the visual referents of the words being heard [Cooper, R. M. (1974). The control of eye fixation by the meaning of spoken language: A new methodology for the real-time investigation of speech perception, memory, and language processing. Cognitive Psychology, 6(1), 84-107; Tanenhaus, M. K., Spivey-Knowlton, M. J., Eberhard, K. M., & Sedivy, J. C. (1995). Integration of visual and linguistic information in spoken language comprehension. Science, 268(5217), 1632-1634]. We demonstrate here that such spontaneous fixation can be driven by partial semantic overlap between a word and a visual object. Participants heard the word ‘piano’ when (a) a piano was depicted amongst unrelated distractors; (b) a trumpet was depicted amongst those same distractors; and (c), both the piano and trumpet were depicted. The probability of fixating the piano and the trumpet in the first two conditions rose as the word ‘piano’ unfolded. In the final condition, only fixations to the piano rose, although the trumpet was fixated more than the distractors. We conclude that eye movements are driven by the degree of match, along various dimensions that go beyond simple visual form, between a word and the mental representations of objects in the concurrent visual field.     

Eye fixations and memory for emotional events.

Subjects watched either an emotional, neutral, or unusual sequence of slides containing 1 critical slide in the middle. Experiments 1 and 2 allowed only a single eye fixation on the critical slide by presenting it for 180 ms (Experiment 1) or 150 ms (Experiment 2). Despite this constraint, memory for a central detail was better for the emotional condition. In Experiment 3, subjects were allowed 2.70 s to view the critical slide while their eye movements were monitored. When subjects who had devoted the same number of fixations were compared, memory for the central detail of the emotional slide was again better. The results suggest that enhanced memory for detail information of an emotional event does not occur solely because more attention is devoted to the emotional information.     

Feature-based memory-driven attentional capture: visual working memory content affects visual attention

In 7 experiments, the authors explored whether visual attention (the ability to select relevant visual information) and visual working memory (the ability to retain relevant visual information) share the same content representations. The presence of singleton distractors interfered more strongly with a visual search task when it was accompanied by an additional memory task. Singleton distractors interfered even more when they were identical or related to the object held in memory, but only when it was difficult to verbalize the memory content. Furthermore, this content-specific interaction occurred for features that were relevant to the memory task but not for irrelevant features of the same object or for once-remembered objects that could be forgotten. Finally, memory-related distractors attracted more eye movements but did not result in longer fixations. The results demonstrate memory-driven attentional capture on the basis of content-specific representations.     

任务信息通达对视觉表象眼动的影响

通过两个实验探讨了视觉表象任务信息的通达对表象加工眼动的影响。结果表明, 在低通达条件下, 表象任务的眼动复制了知觉任务的眼动; 随着表象任务信息通达水平的提高, 眼动的注视点平均持续时间、平均眼跳距离和平均眼跳时间会发生规律性变化; 眼动控制与任务信息通达水平对表象眼动的影响存在不同的机制。实验结果佐证了眼动在视觉表象中起机能性作用的观点。     

The representation of visual scenes

The visual world exists all around us, yet this information must be gleaned through a succession of eye fixations in which high visual acuity is limited to the small foveal region of each retina. In spite of these physiological constraints, we experience a richly detailed and continuous visual world. Research on transsaccadic memory, perception, picture memory and imagination of scenes will be reviewed. Converging evidence suggests that the representation of visual scenes is much more schematic and abstract than our immediate experience would indicate. The visual system may have evolved to maximize comprehension of discrete views at the expense of representing unnecessary detail, but through the action of attention it allows the viewer to access detail when the need arises. This capability helps to maintain the 'illusion' of seeing a rich and detailed visual world at every glance.     

视觉表象眼动的变化：知识还是技能表征的差异？

以表象看到一个运动员完成三级跳远项目为实验任务,对表象任务的信息通达水平、眼动注视点的活动位置和被试对三级跳远项目的知识水平和技能水平进行系统的操纵,通过2个实验探讨了视觉表象眼动的变化是基于知识学习表征差异还是技能训练表征差异的问题。实验1以没有三级跳远运动专业技能知识且对该运动的认知水平也较低的大学生为被试,结果表明,在完成高信息通达水平的表象任务时,注视点需要较短的持续时间,但眼跳距离会增大,眼跳频率会变低;实验2对表象任务的知识学习表征水平和技能训练表征水平进行操纵,分别以对实验任务进行过知识学习和专业技能训练的人为被试,结果表明,随着被试知识习得水平和技能水平表征能力的提高,不同表象任务信息通达水平间的眼动差异将消失,但知识学习和技能表征的差异在平均眼跳时间上有差异,技能训练型的被试其平均眼跳时间要短于知识学习型被试,达到临界水平显著,注视点平均持续时间和平均眼跳距离等均没有差异。     

Fixation-dependent memory for natural scenes: an experimental test of scanpath theory

Many modern theories propose that perceptual information is represented by the sensorimotor activity elicited by the original stimulus. Scanpath theory (Noton & Stark, 1971) predicts that reinstating a sequence of eye fixations will help an observer recognize a previously seen image. However, the only studies to investigate this are correlational ones based on calculating scanpath similarity. We therefore describe a series of 5 experiments that constrain the fixations during encoding or recognition of images in order to manipulate scanpath similarity. Participants encoded a set of images and later had to recognize those that they had seen. They spontaneously selected regions that they had fixated during encoding (Experiment 1), and this was a predictor of recognition accuracy. Yoking the parts of the image available at recognition to the encoded scanpath led to better memory performance than randomly selected image regions (Experiment 2), and this could not be explained by the spatial distribution of locations (Experiment 3). However, there was no recognition advantage for re-viewing one's own fixations versus someone else's (Experiment 4) or for retaining their serial order (Experiment 5). Therefore, although it is beneficial to look at encoded regions, there is no evidence that scanpaths are stored or that scanpath recapitulation is functional in scene memory. This paradigm provides a controlled way of studying the integration of scene content, spatial structure, and oculomotor signals, with consequences for the perception, representation, and retrieval of visual information.     

视觉短时记忆与视知觉的信息整合

与知觉整合和跨眼跳的信息整合一样,视觉短时记忆与视知觉的信息整合也是人们加工并保持整体性场景知觉的重要前提条件。近年来研究者们使用空白单元格定位任务进行了一些研究,试图寻求视觉短时记忆与视知觉信息整合的实验证据并探讨其整合机制。该文对已有的相关实验范式、实验证据和整合机制进行了介绍并加以分析,并指出未来研究可以在行为数据的基础上结合眼动和脑成像的数据寻求汇聚性证据     

Scene perception and memory revealed by eye movements and receiver-operating characteristic analyses: Does a cultural difference truly exist?

Cultural differences have been observed in scene perception and memory: Chinese participants purportedly attend to the background information more than did American participants. We investigated the influence of culture by recording eye movements during scene perception and while participants made recognition memory judgements. Real-world pictures with a focal object on a background were shown to both American and Chinese participants while their eye movements were recorded. Later, memory for the focal object in each scene was tested, and the relationship between the focal object (studied, new) and the background context (studied, new) was manipulated. Receiver-operating characteristic (ROC) curves show that both sensitivity and response bias were changed when objects were tested in new contexts. However, neither the decrease in accuracy nor the response bias shift differed with culture. The eye movement patterns were also similar across cultural groups. Both groups made longer and more fixations on the focal objects than on the contexts. The similarity of eye movement patterns and recognition memory behaviour suggests that both Americans and Chinese use the same strategies in scene perception and memory.     

Scene perception and memory revealed by eye movements and receiver-operating characteristic analyses: does a cultural difference truly exist?

Cultural differences have been observed in scene perception and memory: Chinese participants purportedly attend to the background information more than did American participants. We investigated the influence of culture by recording eye movements during scene perception and while participants made recognition memory judgements. Real-world pictures with a focal object on a background were shown to both American and Chinese participants while their eye movements were recorded. Later, memory for the focal object in each scene was tested, and the relationship between the focal object (studied, new) and the background context (studied, new) was manipulated. Receiver-operating characteristic (ROC) curves show that both sensitivity and response bias were changed when objects were tested in new contexts. However, neither the decrease in accuracy nor the response bias shift differed with culture. The eye movement patterns were also similar across cultural groups. Both groups made longer and more fixations on the focal objects than on the contexts. The similarity of eye movement patterns and recognition memory behaviour suggests that both Americans and Chinese use the same strategies in scene perception and memory.     

Scrutinizing visual images: The role of gaze in mental imagery and memory

Gaze was monitored by use of an infrared remote eye-tracker during perception and imagery of geometric forms and figures of animals. Based on the idea that gaze prioritizes locations where features with high information content are visible, we hypothesized that eye fixations should focus on regions that contain one or more local features that are relevant for object recognition. Most importantly, we predicted that when observers looked at an empty screen and at the same time generated a detailed visual image of what they had previously seen, their gaze would probabilistically dwell within regions corresponding to the original positions of salient features or parts. Correlation analyses showed positive relations between gaze’s dwell time within locations visited during perception and those in which gaze dwelled during the imagery generation task. Moreover, the more faithful an observer’s gaze enactment, the more accurate was the observer’s memory, in a separate test, of the dimension or size in which the forms had been perceived. In another experiment, observers saw a series of pictures of animals and were requested to memorize them. They were then asked later, in a recall phase, to answer a question about a property of one of the encoded forms; it was found that, when retrieving from long-term memory a previously seen picture, gaze returned to the location of the part probed by the question. In another experimental condition, the observers were asked to maintain fixation away from the original location of the shape while thinking about the answer, so as to interfere with the gaze enactment process; such a manipulation resulted in measurable costs in the quality of memory. We conclude that the generation of mental images relies upon a process of enactment of gaze that can be beneficial to visual memory.     

Assessing cue usage in performance contexts: A comparison between eye-movement and concurrent verbal report methods

The relationship between eye fixations and concurrent verbal reports as measures of selective attention was examined in two different performance contexts. Experienced and less experienced soccer players were presented with filmed action sequences. In Experiment 1, subjects reacted to 11-versus-11 soccer simulations; in Experiment 2, 3-versus-3 situations were presented. Each study involved two experimental conditions: (1) visual fixations were recorded using an eye-movement system and (2) subjects continuously verbalized the location of their visual attention. In 11-versus-11 situations, there were no differences in performance across the two conditions, indicating that verbalization had no effect on performance. Also, there were no differences between the two methods in identifying important information sources. In 3-versus-3 situations, concurrent verbalization resulted in reactivity. Moreover, there were differences between the two methods in identifying important display areas. Findings showed that the relationship between visual fixation and selective attention is dependent on the nature of the stimulus presented. When peripheral vision is employed to extract task-specific information, verbal reports provide a more veritable measure of selective attention; eye fixations may yield a more accurate measure when the task requires information to be extracted foveally. Findings are discussed with regard to the differences between “looking” and “seeing” as a methodological issue.     

Selective attention in peacocks during predator detection

Predation can exert strong selective pressure on the evolution of behavioral and morphological traits in birds. Because predator avoidance is key to survival and birds rely heavily on visual perception, predation may have shaped avian visual systems as well. To address this question, we examined the role of visual attention in antipredator behavior in peacocks (Pavo cristatus). Peacocks were exposed to a model predator while their gaze was continuously recorded with a telemetric eye-tracker. We found that peacocks spent more time looking at and made more fixations on the predator compared to the same spatial location before the predator was revealed. The duration of fixations they directed toward conspecifics and environmental features decreased after the predator was revealed, indicating that the peacocks were rapidly scanning their environment with their eyes. Maximum eye movement amplitudes and amplitudes of consecutive saccades were similar before and after the predator was revealed. In cases where conspecifics detected the predator first, peacocks appeared to learn that danger was present by observing conspecifics’ antipredator behavior. Peacocks were faster to detect the predator when they were fixating closer to the area where the predator would eventually appear. In addition, pupil size increased after predator exposure, consistent with increased physiological arousal. These findings demonstrate that peacocks selectively direct their attention toward predatory threats and suggest that predation has influenced the evolution of visual orienting systems.     

Measuring the effect of multiple eye fixations on memory for visual attributes.

Because of limited peripheral vision, many visual tasks depend on multiple eye fixations. Good performance in such tasks demonstrates that some memory must survive from one fixation to the next. One factor that must influence performance is the degree to which multiple eye fixations interfere with the critical memories. In the present study, the amount of interference was measured by comparing visual discriminations based on multiple fixations to visual discriminations based on a single fixation. The procedure resembled partial report, but used a discrimination measure. In the prototype study, two lines were presented, followed by a single line and a cue. The cue pointed toward one of the positions of the first two lines. Observers were required to judge if the single line in the second display was longer or shorter than the cued line of the first display. These judgments were used to estimate a length threshold. The critical manipulation was to instruct observers either to maintain fixation between the lines of the first display or to fixate each line in sequence. The results showed an advantage for multiple fixations despite the intervening eye movements. In fact, thresholds for the multiple-fixation condition were nearly as good as those in a control condition where the lines were foveally viewed without eye movements. Thus, eye movements had little or no interfering effect in this task. Additional studies generalized the procedure and the stimuli. In conclusion, information about a variety of size and shape attributes was remembered with essentially no interference across eye fixations.     

Measuring the effect of multiple eye fixations on memory for visual attributes

Because of limited peripheral vision, many visual tasks depend on multiple eye fixations. Good performance in such tasks demonstrates that some memory must survive from one fixation to the next. One factor that must influence performance is the degree to which multiple eye fixations interfere with the critical memories. In the present study, the amount of interference was measured by comparing visual discriminations based on multiple fixations to visual discriminations based on a single fixation. The procedure resembled partial report, but used a discrimination measure. In the prototype study, two lines were presented, followed by a single line and a cue. The cue pointed toward one of the positions of the first two lines. Observers were required to judge if the single line in the second display was longer or shorter than the cued line of the first display. These judgments were used to estimate a length threshold. The critical manipulation was to instruct observers either to maintain fixation between the lines of the first display or to fixate each line in sequence. The results showed an advantage for multiple fixations despite the intervening eye movements. In fact, thresholds for the multiple-fixation condition were nearly as good as those in a control condition where the lines were foveally viewed without eye movements. Thus, eye movements had little or no interfering effect in this task. Additional studies generalized the procedure and the stimuli. In conclusion, information about a variety of size and shape attributes was remembered with essentially no interference across eye fixations.