TRANSSACCADIC MEMORY AND INTEGRATION DURING REAL-WORLD OBJECT PERCEPTION

Abstract— What is the nature of the information that is preserved and combined across saccadic eye movements during the visual analysis of real-world objects? The two experiments reported investigated transsaccadic memory and transsaccadic integration, respectively In the critical condition, participants were presented with one set of contours from an object during one fixation and with a complementary set of contours during the next fixation In Experiment 1, participants could at best inconsistently detect contour changes across the saccade In Experiment 2, a change in visible contour had no influence on object identification. These results suggest that a veridical representation of object contour is neither consistently preserved nor integrated across a saccade.     

视觉短时记忆与视知觉的信息整合

与知觉整合和跨眼跳的信息整合一样,视觉短时记忆与视知觉的信息整合也是人们加工并保持整体性场景知觉的重要前提条件。近年来研究者们使用空白单元格定位任务进行了一些研究,试图寻求视觉短时记忆与视知觉信息整合的实验证据并探讨其整合机制。该文对已有的相关实验范式、实验证据和整合机制进行了介绍并加以分析,并指出未来研究可以在行为数据的基础上结合眼动和脑成像的数据寻求汇聚性证据     

Visual memory and the perception of a stable visual environment

The visual world appears stable and continuous despite eye movements. One hypothesis about how this perception is achieved is that the contents of successive fixations are fused in memory according to environmental coordinates. Two experiments failed to support this hypothesis; they showed that one's ability to detect a grating presented after a saccade is unaffected by the presentation of a grating with the same spatial frequency in the same spatial location before the saccade. A third experiment tested an alternative explanation of perceptual stability that claims that the contents of successive fixations are compared, rather than fused, across saccades, allowing one to determine whether the world has remained stable. This hypothesis was supported: Experienced subjects could accurately determine whether two patterns viewed in successive fixations were identical or different, even when the two patterns appeared in different spatial positions across the saccade. Taken together, these results suggest that perceptual stability and information integration across saccades rely on memory for the relative positions of objects in the environment, rather than on the spatiotopic fusion of visual information from successive fixations.     

Investigating the parameters of transsaccadic memory: inhibition of return impedes information acquisition near a saccade target

A limited amount of visual information is retained between saccades, which is subsequently stored into a memory system, such as transsaccadic memory. Since the capacity of transsaccadic memory is limited, selection of information is crucial. Selection of relevant information is modulated by attentional processes such as the presaccadic shift of attention. This involuntary shift of attention occurs prior to execution of the saccade and leads to information acquisition at an intended saccade target. The aim of the present study was to investigate the influence that another attentional effect, inhibition of return (IOR), has on the information that gets stored into transsaccadic memory. IOR is the phenomenon where participants are slower to respond to a cue at a previously attended location. To this end, we used a transsaccadic memory paradigm in which stimuli, oriented on a horizontal axis relative to saccade direction, are only visible to the participant before executing a saccade. Previous research showed that items in close proximity to a saccade target are likely to be reported more accurately. In our current study, participants were cued to fixate one of the stimulus locations and subsequently refixated the centre fixation point before executing the transsaccadic memory task. Results indicate that information at a location near a saccade landing point is less likely to be acquired into transsaccadic memory when this location was previously associated with IOR. Furthermore, we found evidence which implicates a reduction of the overall amount of elements retained in transsaccadic memory when a location near a saccade target is associated with IOR. These results suggest that the presaccadic shift of attention may be modulated by IOR and thereby reduces information acquisition by transsaccadic memory.     

Types and tokens in transsaccadic object identification: effects of spatial position and left-right orientation

What types of representations support our ability to integrate information acquired during one eye fixation with information acquired during the next fixation? In Experiment 1, transsaccadic integration was explored by manipulating whether or not the relative position of a picture of an object was maintained across a saccade. In Experiment 2, the degree to which visual details of a picture are coded in a position-specific representational system was explored by manipulating whether or not both the relative position and the left-right orientation of the picture were maintained across a saccade. Position-specific and nonspecific preview benefits were observed in both experiments. Only the position-specific benefits were influenced by the number of task-relevant pictures presented in the preview display (Experiment 1) and the left-right orientation of the picture presented in the preview display (Experiment 2). The results support a model of transsaccadic integration based on two independent representational systems. One system codes abstract, prestored object types, and the other codes episodic tokens consisting of stimulus properties linked to scene- or configuration-based position markers.     

David Melcher: Award for Distinguished Scientific Early Career Contributions to Psychology

Presents David Melcher, the 2011 winner of the American Psychological Association Award for Distinguished Scientific Early Career Contributions to Psychology. "For his elegant and groundbreaking work on one of the most important problems in perceptual psychology, the transfer of perceptual representations across eye movements. David Melcher's innovative experiments used perceptual aftereffects to show how remapping of visual locations underlies the creation of the percept of a clear and stable world. His work on the accumulation of memory contributed importantly to the understanding of natural perceptual representations and their neural underpinnings. His elegant reviews of transsaccadic perception communicated to a broad audience the remarkable capacity of the brain to create seamless perceptual representations despite the disruptions produced by eye movements." (PsycINFO Database Record (c) 2011 APA, all rights reserved).     

Effects of perceptual context on transsaccadic visual matching

Transsaccadic visual matching refers to the phenomenon in which presaccadic signals at the location of the saccade goal influence the visibility of postsaccadic test signals presented at the fovea (Jüttner & Röhler, 1993). The present study uses variations of the perceptual context, provided by the structure of the presaccadic stimulus display, to investigate the influence of spatial information, decision factors, and visual attention on this form of transsaccadic information processing. The experiments yielded the following results: First, analysis of the data in terms of signal detection theory revealed that transsaccadic visual matching manifests in a shift of decision criterion (ß) rather than in a change of sensory sensitivity (d′). The criterion shift leads to a suppression of postsaccadic stimulus information that is incompatible with presaccadic processed information. Second, the matching effect strongly depends on the structure of the presaccadic display, which suggests that mechanisms of visual attention provide the spatial binding of pre- and postsaccadic stimulus information. Third, transsaccadic matching is phenomenologically tied to the execution of the saccade and effective during a postsaccadic period extending up to 160 msec after termination of the eye movement. These findings indicate a form of context-sensitive evaluation process that could subsidize the maintenance of visual stability.     

Temporal integration between visual images and visual percepts

Using a temporal integration task, subjects in 5 experiments were expected to combine information from temporally separated visual presentations. Evidence from these experiments indicated that perceptual information can be integrated with previously generated and currently maintained visual images to form a representation that contains information from each source. Properties and limitations of this integration process were also explored, including the time required to generated the image, the speed at which percepts were integrated with images, and the capacity of the representation. Implications for theories of visual processing and memory are discussed.     

Spatial frequency filtering and the direct control of fixation durations during scene viewing

The present study employed a saccade-contingent change paradigm to investigate the effect of spatial frequency filtering on fixation durations during scene viewing. Subjects viewed grayscale scenes while encoding them for a later memory test. During randomly chosen saccades, the scene was replaced with an alternate version that remained throughout the critical fixation that followed. In Experiment 1, during the critical fixation, the scene could be changed to high-pass and low-pass spatial frequency filtered versions. Under both conditions, fixation durations increased, and the low-pass condition produced a greater effect than the high-pass condition. In subsequent experiments, we manipulated the familiarity of scene information during the critical fixation by flipping the filtered scenes upside down or horizontally. Under these conditions, we observed lengthening of fixation durations but no difference between the high-pass and low-pass conditions, suggesting that the filtering effect is related to the mismatch between information extracted within the critical fixation and the ongoing scene representation in memory. We also conducted control experiments that tested the effect of changes to scene orientation (Experiment 2a) and the addition of color to a grayscale scene (Experiment 2b). Fixation distribution analysis suggested two effects on the distribution fixation durations: a fast-acting effect that was sensitive to all transsaccadic changes tested and a later effect in the tail of the distribution that was likely tied to the processing of scene information. These findings are discussed in the context of theories of oculomotor control during scene viewing.     

Delayed memory for visual—haptic exploration of familiar objects

Long-term memory of haptic, visual, and cross-modality information was investigated. In Experiment 1, subjects briefly explored 40 commonplace objects visually or haptically and then received a recognition test with categorically similar foils in the same or the alternative modality both immediately and after 1 week. Recognition was best for visual input and test, with haptic memory still apparent after a week's delay. Recognition was poorest in the cross-modality conditions, with performance on the haptic-visual and visual-haptic cross-modal conditions being nearly identical. Visual and haptic information decayed at similar rates across a week delay. In Experiment 2, subjects simultaneously viewed and handled the same objects, and transfer was tested in a successive cue-modality paradigm. Performance with the visual modality again exceeded that with the haptic modality. Furthermore, initial errors on the haptic test were often corrected when followed by the visual presentation, both immediately and after 1 week. However, visual test errors were corrected by haptic cuing on the immediate test only. These results are discussed in terms of shared information between the haptic and visual modalities, and the ease of transfer between these modalities immediately and after a substantial delay.     

Visual memory for feature bindings: The disruptive effect of responding to new perceptual input

In the current study, we examined how short-term memory for location–identity feature bindings is influenced by subsequent cognitive and perceptual processing demands. Previous work has shown that memory performance for feature bindings can be disrupted by the presentation of subsequent visual information, particularly when this information is similar to that held in memory. The present study demonstrates that memory performance for feature bindings can be profoundly disrupted by also requiring a response to visual information presented subsequent to the visual memory array. Across five experiments, memory for a location–identity binding was substantially impaired following a localization response to a following item that matched the location but mismatched the identity of the memory target. The results point to an important role for action in the episodic integration processes that control short-term visual memory performance.     

The span of the effective stimulus during a fixation in reading

A computer-based eye-movement controlled, display system was developed for the study of perceptual processes in reading. A study was conducted to identify the region from which skilled readers pick up various types of visual information during a fixation while reading. This study involved making display changes, based on eye position, in the text pattern as the subject was in the act of reading from it, and then examining the effects these changes produced on eye behavior. The results indicated that the subjects acquired word-length pattern information at least 12 to 15 character positions to the right of the fixation point, and that this information primarily influenced saccade lengths. Specific letter- and word-shape information were acquired no further than 10 character positions to the right of the fixation point.     

Hemifield asymmetries differentiate VSTM for single- and multiple-feature objects

Visual short-term memory (VSTM) is a capacity-limited system for maintaining visual information across brief durations. Limits in the amount of information held in memory reflect processing constraints in the intraparietal sulcus (IPS), a region of the frontoparietal network also involved in visual attention. During VSTM and visual attention, areas of IPS demonstrate hemispheric asymmetries. Whereas the left hemisphere represents information in only the right hemifield, the right hemisphere represents information across the visual field. In visual attention, hemispheric asymmetries are associated with differences in behavioral performance across the visual field. In order to assess the degree of hemifield asymmetries in VSTM, we measured memory performance across the visual field for both single- and two-feature objects. Consistent with theories of right-hemisphere dominance, there was a memory benefit for single-feature items in the left visual hemifield. However, when the number of features increased, the behavioral bias reversed, demonstrating a benefit for remembering two-feature objects in the right hemifield. On an individual basis, the cost of remembering an additional feature in the hemifields was correlated, suggesting that the shift in hemifield biases reflected a redistribution of resources across the visual field. Furthermore, we demonstrate that these results cannot be explained by differences in perceptual or decision-making load. Our results are consistent with a flexible resource model of VSTM in which attention and/or working memory demands result in representation of items in the right hemifield by both the left and right hemispheres.     

Selective weighting of action-related feature dimensions in visual working memory

Planning an action primes feature dimensions that are relevant for that particular action, increasing the impact of these dimensions on perceptual processing. Here, we investigated whether action planning also affects the short-term maintenance of visual information. In a combined memory and movement task, participants were to memorize items defined by size or color while preparing either a grasping or a pointing movement. Whereas size is a relevant feature dimension for grasping, color can be used to localize the goal object and guide a pointing movement. The results showed that memory for items defined by size was better during the preparation of a grasping movement than during the preparation of a pointing movement. Conversely, memory for color tended to be better when a pointing movement rather than a grasping movement was being planned. This pattern was not only observed when the memory task was embedded within the preparation period of the movement, but also when the movement to be performed was only indicated during the retention interval of the memory task. These findings reveal that a weighting of information in visual working memory according to action relevance can even be implemented at the representational level during maintenance, demonstrating that our actions continue to influence visual processing beyond the perceptual stage.     

Thinking graphically: Connecting vision and cognition during graph comprehension

Task analytic theories of graph comprehension account for the perceptual and conceptual processes required to extract specific information from graphs. Comparatively, the processes underlying information integration have received less attention. We propose a new framework for information integration that highlights visual integration and cognitive integration. During visual integration, pattern recognition processes are used to form visual clusters of information; these visual clusters are then used to reason about the graph during cognitive integration. In 3 experiments, the processes required to extract specific information and to integrate information were examined by collecting verbal protocol and eye movement data. Results supported the task analytic theories for specific information extraction and the processes of visual and cognitive integration for integrative questions. Further, the integrative processes scaled up as graph complexity increased, highlighting the importance of these processes for integration in more complex graphs. Finally, based on this framework, design principles to improve both visual and cognitive integration are described.     

Remapping versus short-term memory in visual stability across saccades

Saccadic eye movements cause displacements of the image of the visual world projected on the retina. Despite the ubiquitous nature of saccades, subjective experience of the world is continuous and stable. In five experiments, we addressed the mechanisms that may support visual stability: matching of pre- and postsaccadic locations of the target by an internal copy of the saccade, or retention of the visual attributes of the target in short-term memory across the saccade. Healthy human adults were instructed to make a saccade to a peripheral Gabor patch. While the saccade was in midflight, the patch could change location, orientation, or both. The change occurred either immediately or following a 250-ms blank during which no visual stimuli were available. In separate experiments, subjects had to report either whether the patch stepped to the left or right or whether the orientation rotated clockwise or counterclockwise. Consistent with previous findings, we found that transsaccadic displacement discrimination was enhanced by the addition of the blank. However, contrary to previous findings reported in the literature, the feature change did not improve performance. Transsaccadic orientation change discrimination did not depend on either an irrelevant temporal blank or a simultaneous irrelevant target displacement. Taken together, these findings suggest that orientation is not a relevant visual feature for transsaccadic correspondence.     

Relating the perception of visual ensemble statistics to individual levels of autistic traits

Integrating information across the visual field into an ensemble (e.g., seeing the forest from the trees) is an effective strategy to efficiently process the visual world, and one that is often impaired in autism spectrum disorder. Individual differences in sensory processing predict ensemble encoding, providing a potential mechanism for differing perceptual strategies across individuals, and possibly across diagnostic groups exhibiting atypical sensory processing. Here, we explore whether ensemble encoding is associated with traits associated with autism spectrum disorder (ASD). Participants (N=68) were presented with an ensemble display consisting of circles of varying sizes and colors, and were asked to remember the size of the red and blue circles, while ignoring the green circles. Participants were then cued to a target location after a brief delay, and instructed to report the remembered size of the circle they had previously viewed in that location, as ensemble information commonly biases memory for individual objects toward the probed mean of a set of similar objects. The Autism-spectrum Quotient (AQ) was completed to measure each individual’s level of autistic traits. We found that an individual’s level of ensemble perception, measured as their bias toward the probed mean, was negatively associated with a higher level of ASD traits. These results suggest that individuals with higher levels of ASD traits are less likely to integrate perceptual information. These findings may shed light on different perceptual processing within the autism spectrum, and provide insight into the relationship between individual differences and ensemble encoding.     

Information persistence in the integration of partial cues for object recognition

A great many studies have shown that the perceptual effects of very brief visual stimuli can persist beyond the duration of the stimulus itself. These effects include sustained perception of the stimulus even though it is no longer present and the integration of information across an interstimulus interval. These two forms of sustained activity can be characterized as visible persistence and information persistence. Iconic memory protocols and a number of discrimination tasks have demonstrated the existence of information persistence that can last up to several hundred milliseconds, but there is little evidence that the cues needed for identification of objects can be transferred across intervals in this range. In the present experiments, a minimal transient discrete cue protocol was used to demonstrate that shape cues, these being provided by subsets of dots that mark the outer boundary of nameable objects, can be integrated over several hundred milliseconds and that the duration is a function of ambient room illumination. The experiments further evaluated whether this information persistence is mediated by visible persistence. Although both perceptual effects have durations that are an inverse function of room illumination, the ability to integrate partial shape cues was not determined by the duration of visible persistence.