Eye movements serialize memory for objects in scenes

A gaze-contingent short-term memory paradigm was used to obtain forgetting functions for realistic objects in scenes. Experiment 1 had observers freely view nine-item scenes. After observers' gaze left a predetermined target, they could fixate from 1-7 intervening nontargets before the scene was replaced by a spatial probe at the target location. The task was then to select the target from four alternatives. A steep recency benefit was found over the 1-2 intervening object range that declined into an above-chance prerecency asymptote over the remainder of the forgetting function. In Experiment 2, we used sequential presentation and variable delays to explore the contributions of decay and extrafoveal processes to these behaviors. We conclude that memory for objects in scenes, when serialized by fixation sequence, shows recency and prerecency effects that are similar to isolated objects presented sequentially over time. We discuss these patterns in the context of the serial order memory literature and object file theory.     

Eye movements are not a prerequisite for learning movement sequence timing through observation

The present experiment examined learning of a three-segment movement sequence using physical or observational practice, and whether permitting eye movements to be made during observation is a prerequisite for learning such a movement sequence. Specifically, participants were required to move a mouse cursor through a three-segment movement sequence in order to satisfy one of three movement time goals (800, 1000, 1200 ms). A yoked-participant design was used in which a physical practice group acted as a learning model, which was viewed simultaneously by two groups that carried out different observational practice procedures. An observation group was permitted to move their eyes whilst observing the model, whereas the fixation group was instructed to maintain fixation on a central target. The difference between pre-test and post-test data indicated that all the three experimental groups significantly altered their timing accuracy, variability and movement kinematics over practice, while the control group’s behaviour was unchanged. These data indicate that movement time as well as the underlying movement control was learned following observation of a movement with or without an explicit contribution from eye movements, albeit to a lesser extent during the final segment of the sequence when compared to the physical practice group. The implication is that while similar processes might normally be involved in physical and observational practice, information afforded by eye movements during observation (e.g., efference copy and eye proprioception) is not necessary for movement sequence learning.     

Eye movements and scene perception: memory for things observed

In this study, we examined the characteristics of on-line scene representations, using a partial-report procedure. Subjects inspected a simple scene containing seven objects for 1, 3, 5, 9, or 15 fixations; shortly after scene offset, a marker cued one scene location for report. Consistent with previous research, the results indicated that scene representations are relatively sparse; even after 15 fixations on a scene, the subjects remembered the position/identity pairings for only about 78% of the objects in the scene, or the equivalent of about five objects-worth of information. Report of the last three objects that were foveated and of the object about to be foveated was very accurate, however, suggesting that recently attended information in a scene is represented quite well. Information about the scene appeared to accumulate over multiple fixations, but the capacity of the on-line scene representation appeared to be limited to about five items. Implications for recent theories of scene representation are discussed.     

Looking for meaning: Eye movements are sensitive to overlapping semantic features, not association

Theories of semantic memory differ in the extent to which relationships among concepts are captured via associative or via semantic relatedness. We examined the contributions of these two factors, using a visual world paradigm in which participants selected the named object from a four-picture display. We controlled for semantic relatedness while manipulating associative strength by using the visual world paradigm’s analogue to presenting asymmetrically associated pairs in either their forward or backward associative direction (e.g., ham—eggs vs. eggs—ham). Semantically related objects were preferentially fixated regardless of the direction of presentation (and the effect size was unchanged by presentation direction). However, when pairs were associated but not semantically related (e.g., iceberg—lettuce), associated objects were not preferentially fixated in either direction. These findings lend support to theories in which semantic memory is organized according to semantic relatedness (e.g., distributed models) and suggest that association by itself has little effect on this organization.     

Bilateral saccadic eye movements and tactile stimulation,but not auditory stimulation,enhance memory retrieval

Recent research has shown superior memory retrieval when participants make a series of horizontal saccadic eye movements between the memory encoding phase and the retrieval phase compared to participants who do not move their eyes or move their eyes vertically. It has been hypothesized that the rapidly alternating activation of the two hemispheres that is associated with the series of left–right eye movements is critical in causing the enhanced retrieval. This hypothesis predicts a beneficial effect on retrieval of alternating left–right stimulation not only of the visuomotor system, but also of the somatosensory system, both of which have a strict contralateral organization. In contrast, this hypothesis does not predict an effect, or a weaker effect, on retrieval of alternating left–right stimulation of the auditory system, which has a much less lateralized organization. Consistent with these predictions, we replicated the horizontal saccade-induced retrieval enhancement (Experiment 1) and showed that a similar retrieval enhancement occurs after alternating left–right tactile stimulation (Experiment 2). Furthermore, retrieval was not enhanced after alternating left–right auditory stimulation compared to simultaneous bilateral auditory stimulation (Experiment 3). We discuss the possibility that alternating bilateral activation of the left and right hemispheres exerts its effects on memory by increasing the functional connectivity between the two hemispheres. We also discuss the findings in the context of clinical practice, in which bilateral eye movements (EMDR) and auditory stimulation are used in the treatment of post-traumatic stress disorder.     

Eye movements, not hypercompatible mappings, are critical for eliminating the cost of task set reconfiguration

Residual switch costs are notoriously difficult to eliminate. Yet Hunt and Klein (2002) eliminated them in a task that required observers to alternate between 8 trials of prosaccades and 8 trials of antisaccades, as long as there was at least 1 sec between the task cue and the onset of the saccade target. It was proposed that the elimination of residual switch costs occurred because prosaccade responses are computed very rapidly. These so-calledhypercompatible responses bypass memory retrieval stages of the response process, thereby eliminating the source of residual switch costs. Here we tested this hypothesis by requiring observers to alternate between responding with the finger that was vibrated (another task that meets the criteria for hypercompatibility) and responding with the finger of the opposite hand. Residual switch costs were not eliminated, suggesting that their elimination in Hunt and Klein (2002) was due to special properties of the prosaccade—antisaccade task.     

Eye movements and the integration of visual memory and visual perception

Because visual perception has temporal extent, temporally discontinuous input must be linked in memory. Recent research has suggested that this may be accomplished by integrating the active contents of visual short-term memory (VSTM) with subsequently perceived information. In the present experiments, we explored the relationship between VSTM consolidation and maintenance and eye movements, in order to discover how attention selects the information that is to be integrated. Specifically, we addressed whether stimuli needed to be overtly attended in order to be included in the memory representation or whether covert attention was sufficient. Results demonstrated that in static displays in which the to-be-integrated information was presented in the same spatial location, VSTM consolidation proceeded independently of the eyes, since subjects made few eye movements. In dynamic displays, however, in which the to-be-integrated information was presented in different spatial locations, eye movements were directly related to task performance. We conclude that these differences are related to different encoding strategies. In the static display case, VSTM was maintained in the same spatial location as that in which it was generated. This could apparently be accomplished with covert deployments of attention. In the dynamic case, however, VSTM was generated in a location that did not overlap with one of the to-be-integrated percepts. In order to "move" the memory trace, overt shifts of attention were required.     

Memory for word location during reading: Eye movements to previously read words are spatially selective but not precise

In two experiments, readers' use of spatial memory was examined by asking them to determine whether an individually shown probe word had appeared in a previously read sentence (Experiment 1) or had occupied a right or left sentence location (Experiment 2). Under these conditions, eye movements during the classification task were generally directed toward the right, irrespective of the location of the relevant target in the previously read sentence. In two additional experiments, readers' knowledge of prior sentence content was examined either without (Experiment 3) or with (Experiment 4) an explicit instruction to move the eyes to a target word in that sentence. Although regressions into the prior sentence were generally directed toward the target, they rarely reached it. In the absence of accurate spatial memories, readers reached previously read target words in two distinct steps--one that moved the eyes in the general vicinity of the target, and one that homed in on it.     

Eye movements and visual memory: detecting changes to saccade targets in scenes

Saccade-contingent change detection provides a powerful tool for investigating scene representation and scene memory. In the present study, critical objects presented within color images of naturalistic scenes were changed during a saccade toward or away from the target. During the saccade,the critical object was changed to another object type, to a visually different token of the same object type, or was deleted from the scene. There were three main results. First, the deletion of a saccade target was special: Detection performance for saccade target deletions was very good, and this level of performance did not decline with the amplitude of the saccade. In contrast, detection of type and token changes at the saccade target, and of all changes including deletions at a location that had just been fixated but was not the saccade target, decreased as the amplitude of the saccade increased. Second, detection performance for type and token changes, both when the changing object was the target of the saccade and when the object had just been fixated but was not the saccade target, was well above chance. Third, mean gaze durations were reliably elevated for those trials in which the change was not overtly detected. The results suggest that the presence of the saccade target plays a special role in trassaccadic integration, and together with other recent findings, suggest more generally that a relatively rich scene representation is retained across saccades and stored in visual memory.     

Mindless reading revisited: Eye movements during reading and scanning are different

In an extension of a study by Vitu, O’Regan, Inhoff, and Topolski (1995), we compared global and local characteristics of eye movements during (1) reading, (2) the scanning of transformed text (in which each letter was replaced with a z), and (3) visual search. Additionally, we examined eye behavior with respect to specific target words of high or low frequency. Globally, the reading condition led to shorter fixations, longer saccades, and less frequent skipping of target strings than did scanning transformed text. Locally, the manipulation of word frequency affected fixation durations on the target word during reading, but not during visual search or z-string scanning. There were also more refixations on target words in reading than in scanning. Contrary to Vitu et al.’s (1995) findings, our results show that eye movements are not guided by a global strategy and local tactics, but by immediate processing demands.     

Eye movements during change detection: implications for search constraints, memory limitations, and scanning strategies

Search, memory, and strategy constraints on change detection were analyzed in terms of oculomotor variables. Observers viewed a repeating sequence of three displays (Scene 1-->Mask-->Scene 2-->Mask...) and indicated the presence-absence of a changing object between Scenes 1 and 2. Scenes depicted real-world objects arranged on a surface. Manipulations included set size (one, three, or nine items) and the orientation of the changing objects (similar or different). Eye movements increased with the number of potentially changing objects in the scene, with this set size effect suggesting a relationship between change detection and search. A preferential fixation analysis determined that memory constraints are better described by the operation comparing the pre- and postchange objects than as a capacity limitation, and a scanpath analysis revealed a change detection strategy relying on the peripheral encoding and comparison of display items. These findings support a signal-in-noise interpretation of change detection in which the signal varies with the similarity of the changing objects and the noise is determined by the distractor objects and scene background.     

Prevalence of dual diagnosis in an institution for individuals who are developmentally handicapped

The prevalence of dual diagnosis (developmental handicap with accompanying mental illness) was examined in a population of 71 persons who are institutionalized and developmentally handicapped. The Reiss Screen for Maladaptive Behavior (Reiss, 1988) revealed that 69% of the popula- tion had a dual diagnosis. The incidence of dual diagnosis was not related to level of functioning, although the type of psychopathology varied considerably across levels of functioning. In contrast to findings from the Reiss Screen, previous psychiatric diagnosis indicated a much lower rate of dual diagnosis (26.8%), as did current psychiatric assessment. There was little agreement between the two psychiatric assessments, or between the Reiss Screen and the psychiatric assessments. The Reiss Screen appears to provide information of more practical value for community placement decisions than is provided by the psychiatric assessments. Limitations to the concept of ?dual diagnosis’? for characterizing the mental health needs of persons with a developmental handicap are discussed.     

Eye movements during search for coded and uncoded targets

Eye movements and search time of four subjects were studied as they searched for a target dial in a 4 х 4 array of dials which were differentiated by (1) color, (2) shape, (3) a combination of color and shape, or (4) were uncoded. Subjects did not exhibit a characteristic scanpath, and method of scanning did not relate to search time. Search time varied reliably among conditions; it was generally shortest in the color condition, followed by color shape, shape, and the uncoded condition. Subjects were capable of using both shape and color simultaneously. Search time was strongly associated with the average number of fixations required for target detection but not with other measures of eye movements. Fixation duration was a particularly inconsistent measure.     

Eye movements as a gatekeeper for memorization: evidence for the persistence of attentional sets in visual memory search

Attention is known to serve multiple goals, including the selection of information for further perceptual analysis (selection for perception) and for goal-directed behavior (selection for action). Here, we study the role of overt attention (i.e., eye movements) as a gatekeeper for memorization processes (selection for memorization). Subjects memorized complex multidimensional stimulus displays and subsequently indicated whether a specific (probe) item was present. In Experiment 1 we utilized an incidental learning setting where in the beginning only a subset of display stimuli was relevant, whereas in a transfer block all stimuli were possible probe items. In Experiment 2, we used an explicit learning setting within a between-group design. Response times and gaze patterns indicated that subjects learned to ignore irrelevant stimuli while forming memory representations. The findings suggest that complex feature binding processes in peripheral vision may serve to guide overt selective attention, which eventually contributes to filtering out irrelevant information even in highly complex environments. Gaze patterns suggested that attentional control settings persisted even when they were no longer required.     

Eye fixations and memory for emotional events.

Subjects watched either an emotional, neutral, or unusual sequence of slides containing 1 critical slide in the middle. Experiments 1 and 2 allowed only a single eye fixation on the critical slide by presenting it for 180 ms (Experiment 1) or 150 ms (Experiment 2). Despite this constraint, memory for a central detail was better for the emotional condition. In Experiment 3, subjects were allowed 2.70 s to view the critical slide while their eye movements were monitored. When subjects who had devoted the same number of fixations were compared, memory for the central detail of the emotional slide was again better. The results suggest that enhanced memory for detail information of an emotional event does not occur solely because more attention is devoted to the emotional information.     

Different states in visual working memory: when it guides attention and when it does not

Recent studies have revealed a strong relationship between visual working memory and selective attention, such that attention is biased by what is currently on our mind. However, other data show that not all memorized items influence the deployment of attention, thus calling for a distinction within working memory: whereas active memory items function as an attentional template and directly affect perception, other, accessory items do not. We review recent evidence that items compete for the status of 'attentional template' that contains only one object at a time. Neurophysiological results provide insight into these different memory states by revealing a more intricate organization of working memory than was previously thought.     

Gesturing during mental problem solving reduces eye movements,especially for individuals with lower visual working memory capacity

Non-communicative hand gestures have been found to benefit problem-solving performance. These gestures seem to compensate for limited internal cognitive capacities, such as visual working memory capacity. Yet, it is not clear how gestures might perform this cognitive function. One hypothesis is that gesturing is a means to spatially index mental simulations, thereby reducing the need for visually projecting the mental simulation onto the visual presentation of the task. If that hypothesis is correct, less eye movements should be made when participants gesture during problem solving than when they do not gesture. We therefore used mobile eye tracking to investigate the effect of co-thought gesturing and visual working memory capacity on eye movements during mental solving of the Tower of Hanoi problem. Results revealed that gesturing indeed reduced the number of eye movements (lower saccade counts), especially for participants with a relatively lower visual working memory capacity. Subsequent problem-solving performance was not affected by having (not) gestured during the mental solving phase. The current findings suggest that our understanding of gestures in problem solving could be improved by taking into account eye movements during gesturing.     

Evaluating a ward staff program for increasing spontaneous and varied communicative gesturing with individuals who are mentally retarded

In the present study we assessed the effect of a multifaceted program for ward staff members, which was intended to have severely/profoundly mentally retarded individuals improve their use of communicative gestures. The program consisted of instruction, verbal and visual (publicly posted) graphic feedback. There were two categories of dependent variables. The first category involved the number of gesture prompts by staff, number of unused opportunities to evoke gestures, and number of consequences given for gestures. The second category involved handicapped individuals' number of prompted gestures, number of spontaneous gestures, and number of different gestures. Results show that the program was effective with regard to all these variables. Changes in the number of spontaneous gestures were especially remarkable, as this variable had not been influenced directly. Follow-up data underline the effectiveness of the program.     

Object appearance and picture-specific viewpoint are not integrated in long-term memory

Previous work has demonstrated that visual long-term memory (VLTM) stores detailed information about object appearance. The current experiments investigate whether object appearance information in VLTM is integrated within representations that contain picture-specific viewpoint information. In three experiments using both incidental and intentional encoding instructions, participants were unable to perform above chance on recognition tests that required recognizing the conjunction of object appearance and viewpoint information (Experiments 1a, 1b, 2, and 3). However, performance was better when object appearance information (Experiments 1a, 1b, and 2) or picture-specific viewpoint information (Experiment 3) alone was sufficient to succeed on the memory test. These results replicate previous work demonstrating good memory for object appearance and viewpoint. However the current results suggest that object appearance and viewpoint are not episodically integrated in VLTM.