Information integration across saccadic eye movements

The visual world contains more information than can be perceived in a single glance. Consequently, one's perceptual representation of the environment is built up via the integration of information across saccadic eye movements. The properties of transsaccadic integration were investigated in six experiments. Subjects viewed a random-dot pattern in one fixation, then judged whether a second dot pattern viewed in a subsequent fixation was identical to or different from the first. Interpattern interval, pattern complexity, and pattern displacement were varied in order to determined the duration, capacity, and representational format of transsaccadic memory. The experimental results indicated that transsaccadic memory is an undetailed, limited-capacity, long-lasting memory that is not strictly tied to absolute spatial position. In all these respects it is similar to, and perhaps identical with, visual short-term memory. The implication of these results for theories of perceptual stability across saccades are discussed.     

Investigating the parameters of transsaccadic memory: inhibition of return impedes information acquisition near a saccade target

A limited amount of visual information is retained between saccades, which is subsequently stored into a memory system, such as transsaccadic memory. Since the capacity of transsaccadic memory is limited, selection of information is crucial. Selection of relevant information is modulated by attentional processes such as the presaccadic shift of attention. This involuntary shift of attention occurs prior to execution of the saccade and leads to information acquisition at an intended saccade target. The aim of the present study was to investigate the influence that another attentional effect, inhibition of return (IOR), has on the information that gets stored into transsaccadic memory. IOR is the phenomenon where participants are slower to respond to a cue at a previously attended location. To this end, we used a transsaccadic memory paradigm in which stimuli, oriented on a horizontal axis relative to saccade direction, are only visible to the participant before executing a saccade. Previous research showed that items in close proximity to a saccade target are likely to be reported more accurately. In our current study, participants were cued to fixate one of the stimulus locations and subsequently refixated the centre fixation point before executing the transsaccadic memory task. Results indicate that information at a location near a saccade landing point is less likely to be acquired into transsaccadic memory when this location was previously associated with IOR. Furthermore, we found evidence which implicates a reduction of the overall amount of elements retained in transsaccadic memory when a location near a saccade target is associated with IOR. These results suggest that the presaccadic shift of attention may be modulated by IOR and thereby reduces information acquisition by transsaccadic memory.     

TRANSSACCADIC MEMORY AND INTEGRATION DURING REAL-WORLD OBJECT PERCEPTION

Abstract— What is the nature of the information that is preserved and combined across saccadic eye movements during the visual analysis of real-world objects? The two experiments reported investigated transsaccadic memory and transsaccadic integration, respectively In the critical condition, participants were presented with one set of contours from an object during one fixation and with a complementary set of contours during the next fixation In Experiment 1, participants could at best inconsistently detect contour changes across the saccade In Experiment 2, a change in visible contour had no influence on object identification. These results suggest that a veridical representation of object contour is neither consistently preserved nor integrated across a saccade.     

Types and tokens in transsaccadic object identification: effects of spatial position and left-right orientation

What types of representations support our ability to integrate information acquired during one eye fixation with information acquired during the next fixation? In Experiment 1, transsaccadic integration was explored by manipulating whether or not the relative position of a picture of an object was maintained across a saccade. In Experiment 2, the degree to which visual details of a picture are coded in a position-specific representational system was explored by manipulating whether or not both the relative position and the left-right orientation of the picture were maintained across a saccade. Position-specific and nonspecific preview benefits were observed in both experiments. Only the position-specific benefits were influenced by the number of task-relevant pictures presented in the preview display (Experiment 1) and the left-right orientation of the picture presented in the preview display (Experiment 2). The results support a model of transsaccadic integration based on two independent representational systems. One system codes abstract, prestored object types, and the other codes episodic tokens consisting of stimulus properties linked to scene- or configuration-based position markers.     

Transsaccadic memory for position and orientation of saccade source and target

Participants made a saccade from one biological-motion figure to another and had to detect saccadecontingent changes in either the walker to which the eyes were sent (the target) or the walker that served as launch site (the source). Intrasaccadic displacements in both source and target were relatively hard to detect, whereas changes in the walkers' depth orientation were readily noticed, indicating that previous findings on within-object saccades generalize to between-objects saccades. Contrary to predictions derived from theories that assign a privileged status to the saccade target, transsaccadic memory for the target's position and orientation was not more accurate than memory for the source. Displacements or rotations of one object toward the other object were more detectable than the same changes away from each other, suggesting that relational coding plays a prominent role in the integration of information across saccades.     

Robot studies on saccade-triggered visual prediction

Three robot studies on visual prediction are presented. In all of them, a visual forward model is used, which predicts the visual consequences of saccade-like camera movements. This forward model works by remapping visual information between the pre- and postsaccadic retinal images; at an abstract modeling level, this process is closely related to neurons whose visual receptive fields shift in anticipation of saccades. In the robot studies, predictive remapping is used (1) in the context of saccade adaptation, to reidentify target objects after saccades are carried out; (2) for a model of grasping, in which both fixated and non-fixated target objects are processed by the same foveal mechanism; and (3) in a computational architecture for mental imagery, which generates “gripper appearances” internally without real sensory inflow. The robotic experiments and their underlying computational models are discussed with regard to predictive remapping in the brain, transsaccadic memory, and attention. The results confirm that visual prediction is a mechanism that has to be considered in the design of artificial cognitive agents and the modeling of information processing in the human visual system.     

Remapping versus short-term memory in visual stability across saccades

Saccadic eye movements cause displacements of the image of the visual world projected on the retina. Despite the ubiquitous nature of saccades, subjective experience of the world is continuous and stable. In five experiments, we addressed the mechanisms that may support visual stability: matching of pre- and postsaccadic locations of the target by an internal copy of the saccade, or retention of the visual attributes of the target in short-term memory across the saccade. Healthy human adults were instructed to make a saccade to a peripheral Gabor patch. While the saccade was in midflight, the patch could change location, orientation, or both. The change occurred either immediately or following a 250-ms blank during which no visual stimuli were available. In separate experiments, subjects had to report either whether the patch stepped to the left or right or whether the orientation rotated clockwise or counterclockwise. Consistent with previous findings, we found that transsaccadic displacement discrimination was enhanced by the addition of the blank. However, contrary to previous findings reported in the literature, the feature change did not improve performance. Transsaccadic orientation change discrimination did not depend on either an irrelevant temporal blank or a simultaneous irrelevant target displacement. Taken together, these findings suggest that orientation is not a relevant visual feature for transsaccadic correspondence.     

数量适应后效的皮层映射特征

本研究探讨了观察者与观察目标存在相对运动时视觉系统对目标数量特征的适应后效的皮层映射特征, 并与对比度适应后效的映射规律进行比较。包括两项实验。其中, 实验一要求被试在适应目标后转换注视点, 考察眼跳后相同和不同视网膜区域以及相同和不同空间区域的适应后效, 发现数量适应后效具有部分空间-皮层映射特性, 而对比度适应后效则表现出完全的视网膜-皮层映射特征。实验二采用固定的注视点, 考察目标运动后目标原位置和新位置区域的适应后效, 发现数量适应后效不完全依赖于视网膜-皮层映射, 它可以“追随”客体运动重新映射到新的位置, 表现出基于客体映射的特征, 而对比度适应后效则完全依赖于视网膜-皮层映射, 不能“追随”客体移动在目标新位置重新形成映射。两项实验结果提示, 相对于对比度等低级表面特征而言, 数量特征对目标的描述涉及更高的加工水平, 它可以与观察目标的相对运动信息进行整合, 且这种整合在眼跳和非眼跳的观察条件下都可发生。     

Perceiving stimulus displacements across saccades

The visual world appears stable despite frequent retinal image movements caused by saccades. Many theories of visual stability assume that extraretinal eye position information is used to spatially adjust perceived locations across saccades, whereas others have proposed that visual stability depends upon coding of the relative positions of objects. McConkie and Currie (1996) proposed a refined combination of these views (called the Saccade Target Object Theory) in which the perception of stability across saccades relies on a local evaluation process centred on the saccade target object rather than on a remapping of the entire scene, with some contribution from memory for the relative positions of objects as well. Three experiments investigated the saccade target object theory, along with an alternative hypothesis that proposes that multiple objects are updated across saccades, but with variable resolution, with the saccade target object (by virtue of being the focus of attention before the saccade and residing near the fovea after the saccade) having priority in the perception of displacement. Although support was found for the saccade target object theory in Experiment 1, the results of Experiments 2 and 3 found that multiple objects are updated across saccades and that their positions are evaluated to determine perceived stability. There is an advantage for detecting displacements of the saccade target, most likely because of visual acuity or attentional focus being better near the fovea, but it is not the saccade target alone that determines the perception of stability and of displacements across saccades. Rather, multiple sources of information appear to contribute.     

The effects of eye and limb movements on working memory

Three experiments examined the role of eye and limb movements in the maintenance of information in spatial working memory. In Experiment 1, reflexive saccades interfered with memory span for spatial locations but did not interfere with memory span for letters. In Experiment 2, three different types of eye movements (reflexive saccades, pro-saccades, and anti-saccades) interfered with working memory to the same extent. In all three cases, spatial working memory was much more affected than verbal working memory. The results of these two experiments suggest that eye movements interfere with spatial working memory primarily by disrupting processes localised in the visuospatial sketchpad. In Experiment 3, limb movements performed while maintaining fixation produced as much interference with spatial working memory as reflexive saccades. These results suggest that the interference produced by eye movements is not the result of their visual consequences. Rather, all spatially directed movements appear to have similar effects on visuospatial working memory.     

Global transsaccadic change blindness during scene perception

Each time the eyes are spatially reoriented via a saccadic eye movement, the image falling on the retina changes. How visually specific are the representations that are functional across saccades during active scene perception? This question was investigated with a saccade-contingent display-change paradigm in which pictures of complex real-world scenes were globally changed in real time during eye movements. The global changes were effected by presenting each scene as an alternating set of scene strips and occluding gray bars, and by reversing the strips and bars during specific saccades. The results from two experiments demonstrated a global transsaccadic change-blindness effect, suggesting that point-by-point visual representations are not functional across saccades during complex scene perception.     

Stroboscopic visual training improves information encoding in short-term memory

The visual system has developed to transform an undifferentiated and continuous flow of information into discrete and manageable representations, and this ability rests primarily on the uninterrupted nature of the input. Here we explore the impact of altering how visual information is accumulated over time by assessing how intermittent vision influences memory retention. Previous work has shown that intermittent, or stroboscopic, visual training (i.e., practicing while only experiencing snapshots of vision) can enhance visual?Cmotor control and visual cognition, yet many questions remain unanswered about the mechanisms that are altered. In the present study, we used a partial-report memory paradigm to assess the possible changes in visual memory following training under stroboscopic conditions. In Experiment 1, the memory task was completed before and immediately after a training phase, wherein participants engaged in physical activities (e.g., playing catch) while wearing either specialized stroboscopic eyewear or transparent control eyewear. In Experiment 2, an additional group of participants underwent the same stroboscopic protocol but were delayed 24?h between training and assessment, so as to measure retention. In comparison to the control group, both stroboscopic groups (immediate and delayed retest) revealed enhanced retention of information in short-term memory, leading to better recall at longer stimulus-to-cue delays (640?C2,560?ms). These results demonstrate that training under stroboscopic conditions has the capacity to enhance some aspects of visual memory, that these faculties generalize beyond the specific tasks that were trained, and that trained improvements can be maintained for at least a day.     

Why do we move our eyes while trying to remember? The relationship between non-visual gaze patterns and memory

Non-visual gaze patterns (NVGPs) involve saccades and fixations that spontaneously occur in cognitive activities that are not ostensibly visual. While reasons for their appearance remain obscure, convergent empirical evidence suggests that NVGPs change according to processing requirements of tasks. We examined NVPGs in tasks with long-term memory (LTM) and working memory (WM) requirements. Experiment 1 yielded significantly higher eye movement rate (EMR) in tasks requiring LTM search than in a WM task requiring maintenance of information. Experiment 2 manipulated accessibility of items in study-test episodic tasks using the levels of processing paradigm. EMR was high in episodic recall irrespective of item accessibility. Experiment 3 examined functional significance of saccades in LTM tasks. Voluntary saccadic suppression produced no evidence that saccades contribute to task performance. We discuss the apparent epiphenomenal nature of spontaneous saccades from an evolutionary perspective and outline a neuroanatomical model of the link between the saccadic and memory system.     

学习材料组块方式对相似词长时记忆的影响

已有关于材料相似性影响短时记忆的研究提示, 不相似材料组块相比于相似材料组块可能促进记忆。为验证该假设, 该研究采用学习-测查范式, 通过4个实验考察了学习材料组块方式对相似词长时记忆的影响及机制。结果发现：1)与相似词组块相比, 不相似词组块促进了相似词记忆; 2)不相似词组块的促进效应是通过增强相似词表共同词根的记忆而实现的; 3)不相似词组块的促进效应可能依赖于语音相似性。该结果说明不相似词组块可能是促进相似词汇记忆的有效途径之一。     

Visual mental image generation does not overlap with visual short-term memory: A dual-task interference study

Visual mental imagery and working memory are often assumed to play similar roles in high-order functions, but little is known of their functional relationship. In this study, we investigated whether similar cognitive processes are involved in the generation of visual mental images, in short-term retention of those mental images, and in short-term retention of visual information. Participants encoded and recalled visually or aurally presented sequences of letters under two interference conditions: spatial tapping or irrelevant visual input (IVI). In Experiment 1, spatial tapping selectively interfered with the retention of sequences of letters when participants generated visual mental images from aural presentation of the letter names and when the letters were presented visually. In Experiment 2, encoding of the sequences was disrupted by both interference tasks. However, in Experiment 3, IVI interfered with the generation of the mental images, but not with their retention, whereas spatial tapping was more disruptive during retention than during encoding. Results suggest that the temporary retention of visual mental images and of visual information may be supported by the same visual short-term memory store but that this store is not involved in image generation.     

Visual memory and the perception of a stable visual environment

The visual world appears stable and continuous despite eye movements. One hypothesis about how this perception is achieved is that the contents of successive fixations are fused in memory according to environmental coordinates. Two experiments failed to support this hypothesis; they showed that one's ability to detect a grating presented after a saccade is unaffected by the presentation of a grating with the same spatial frequency in the same spatial location before the saccade. A third experiment tested an alternative explanation of perceptual stability that claims that the contents of successive fixations are compared, rather than fused, across saccades, allowing one to determine whether the world has remained stable. This hypothesis was supported: Experienced subjects could accurately determine whether two patterns viewed in successive fixations were identical or different, even when the two patterns appeared in different spatial positions across the saccade. Taken together, these results suggest that perceptual stability and information integration across saccades rely on memory for the relative positions of objects in the environment, rather than on the spatiotopic fusion of visual information from successive fixations.