Eye movement control during scene viewing: Immediate degradation and enhancement effects of spatial frequency filtering

What controls how long the eyes remain fixated during scene perception? We investigated whether fixation durations are under the immediate control of the quality of the current scene image. Subjects freely viewed photographs of scenes in preparation for a later memory test while their eye movements were recorded. Using the saccade-contingent display change method, scenes were degraded (Experiment 1) or enhanced (Experiment 2) via blurring (low-pass filtering) during predefined saccades. Results showed that fixation durations immediately after a display change were influenced by the degree of blur, with a monotonic relationship between degree of blur and fixation duration. The results also demonstrated that fixation durations can be both increased and decreased by changes in the degree of blur. The results suggest that fixation durations in scene viewing are influenced by the ease of processing of the image currently in view. The results are consistent with models of saccade generation in scenes in which moment-to-moment difficulty in visual and cognitive processing modulates fixation durations.     

Direct control of fixation times in scene viewing: Evidence from analysis of the distribution of first fixation duration

Participants' eye movements were monitored in two scene viewing experiments that manipulated the task-relevance of scene stimuli and their availability for extrafoveal processing. In both experiments, participants viewed arrays containing eight scenes drawn from two categories. The arrays of scenes were either viewed freely (Free Viewing) or in a gaze-contingent viewing mode where extrafoveal preview of the scenes was restricted (No Preview). In Experiment 1a, participants memorized the scenes from one category that was designated as relevant, and in Experiment 1b, participants chose their preferred scene from within the relevant category. We examined first fixations on scenes from the relevant category compared to the irrelevant category (Experiments 1a and 1b), and those on the chosen scene compared to other scenes not chosen within the relevant category (Experiment 1b). A survival analysis was used to estimate the first discernible influence of the task-relevance on the distribution of first-fixation durations. In the free viewing condition in Experiment 1a, the influence of task relevance occurred as early as 81 ms from the start of fixation. In contrast, the corresponding value in the no preview condition was 254 ms, demonstrating the crucial role of extrafoveal processing in enabling direct control of fixation durations in scene viewing. First fixation durations were also influenced by whether or not the scene was eventually chosen (Experiment 1b), but this effect occurred later and affected fewer fixations than the effect of scene category, indicating that the time course of scene processing is an important variable mediating direct control of fixation durations.     

Scene exploration with Fourier-filtered peripheral information.

In a previous moving-window study it was found that scene exploration benefits more from peripheral information of high spatial frequency than of low spatial frequency. In the present study, degraded versions of realistic scenes were presented peripherally during the initial 150 ms of fixations, while the undegraded scene was presented foveally. The undegraded version of the scene was visible both foveally and peripherally during the later part of fixations. During the initial 150 ms, the peripheral part of scenes was low-pass, bandpass, or high-pass filtered, blanked, or decreased in luminance. In a no-change condition, the undegraded scene was presented throughout the whole fixation. Participants freely explored the scenes in the context of an object-decision task. It was found that degrading peripheral information during the initial part of fixations had minimal effect on scene exploration. No reliable differences were found among the three filter types. The results indicate that, in the context of an object-search task, peripheral information is of minor importance during the initial part of fixations.     

Stable individual differences across images in human saccadic eye movements.

Individual differences in eye movements during picture viewing were examined across image format, content, and foveal quality in 3 experiments. Experiment 1 demonstrated that an individual's fixation durations were strongly related across 3 types of scene formats and that saccade amplitudes followed the same pattern. In Experiment 2, a similar relationship was observed for fixation durations across faces and scenes, although the amplitude relationship did not hold as strongly. In Experiment 3, the duration and amplitude relationships were observed when foveal information was degraded and even removed. Eye movement characteristics differ across individuals, but there is a great deal of consistency within individuals when viewing different types of images.     

The role of low-spatial frequencies in lexical decision and masked priming

Spatial frequency filtering was used to test the hypotheses that low-spatial frequency information in printed text can: (1) lead to a rapid lexical decision or (2) facilitate word recognition. Adult proficient readers made lexical decisions in unprimed and masked repetition priming experiments with unfiltered, low-pass, high-pass and notch filtered letter strings. In the unprimed experiments, a filtered target was presented for 105 or 400 ms followed by a pattern mask. Sensitivity (d′) was lowest for the low-pass filtered targets at both durations with a bias towards a ‘non-word’ response. Sensitivity was higher in the high-pass and notch filter conditions. In the priming experiments, a forward mask was followed by a filtered prime then an unfiltered target. Primed words, but not non-words, were identified faster than unprimed words in both the low-pass and high-pass filtered conditions. These results do not support a unique role for low-spatial frequency information in either facilitating or making rapid lexical decisions.     

The effect of spatial frequency content on parameters of eye movements

Two experiments were conducted to examine the influence of the spatial frequency content of natural images on saccadic size and fixation duration. In the first experiment 10 pictures of natural textures were low-pass filtered (0.04-0.76 cycles/deg) and high-pass filtered (1.91-19.56 cycles/deg) and presented with the unfiltered originals in random order, each for 10 s, to 18 participants, with the instruction to inspect them in order to find a suitable name. The participants' eye movements were recorded. It was found that low-pass filtered images resulted in larger saccadic amplitudes compared with high-pass filtered images. A second experiment was conducted with natural stimuli selected for different power spectra which supported the results outlined above. In general, low-spatial frequencies elicit larger saccades associated with shorter fixation durations whereas high-spatial frequencies elicit smaller saccades with longer fixation durations.     

Identification of objects in scenes: the role of scene background in object naming.

Research with brief presentations of scenes has indicated that scene context facilitates object identification. In the present experiments we used a paradigm in which an object in a scene is "wiggled"--drawing both attention and an eye fixation to itself--and then named. Thus the effect of scene context on object identification can be examined in a situation in which the target object is fixated and hence is fully visible. Experiment 1 indicated that a scene background that was episodically consistent with a target object facilitated the speed of naming. In Experiments 2 and 3, we investigated the time course of scene background information acquisition using display changes contingent on eye movements to the target object. The results from Experiment 2 were inconclusive; however, Experiment 3 demonstrated that scene background information present only on either the first or second fixation on a scene significantly affected naming time. Thus background information appears to be both extracted and able to affect object identification continuously during scene viewing.     

Effects of spatial frequency content on classification of face gender and expression

The role of different spatial frequency bands on face gender and expression categorization was studied in three experiments. Accuracy and reaction time were measured for unfiltered, low-pass (cut-off frequency of 1 cycle/deg) and high-pass (cutoff frequency of 3 cycles/deg) filtered faces. Filtered and unfiltered faces were equated in root-mean-squared contrast. For low-pass filtered faces reaction times were higher than unfiltered and high-pass filtered faces in both categorization tasks. In the expression task, these results were obtained with expressive faces presented in isolation (Experiment 1) and also with neutral-expressive dynamic sequences where each expressive face was preceded by a briefly presented neutral version of the same face (Experiment 2). For high-pass filtered faces different effects were observed on gender and expression categorization. While both speed and accuracy of gender categorization were reduced comparing to unfiltered faces, the efficiency of expression classification remained similar. Finally, we found no differences between expressive and non expressive faces in the effects of spatial frequency filtering on gender categorization (Experiment 3). These results show a common role of information from the high spatial frequency band in the categorization of face gender and expression.     

The limits of visual resolution in natural scene viewing

We examined the limits of visual resolution in natural scene viewing, using a gaze-contingent multiresolutional display having a gaze-centred area-of-interest and decreasing resolution with eccentricity. Twelve participants viewed high-resolution scenes in which gaze-contingent multiresolutional versions occasionally appeared for single fixations. Both detection of image degradation (five filtering levels plus a no-area-of-interest control) in the gaze-contingent multiresolutional display, and eye fixation durations, were well predicted by a model of eccentricity-dependent contrast sensitivity. The results also illuminate the time course of detecting image filtering. Detection did not occur for fixations below 100 ms, and reached asymptote for fixations above 200 ms. Detectable filtering lengthened fixation durations by 160 ms, and interference from an imminent manual response occurred by 400-450 ms, often lengthening the next fixation. We provide an estimate of the limits of visual resolution in natural scene viewing useful for theories of scene perception, and help bridge the literature on spatial vision and eye movement control.     

When attractiveness demands longer looks: The effects of situation and gender

We investigated how aesthetics guides our exploration of the environment. We embedded attractive and nonattractive faces into complex, real-world scenes and measured eye movements during scene viewing. We examined whether attractive faces would elicit longer looks, which would suggest that the aesthetic response orients people toward the rewarding and pleasing aspects of the environment. Experiment 1 showed that mean fixation, mean first fixation, and total fixation durations were longer to attractive faces, and fixations were longest to female faces and by female perceivers. In Experiment 2, we examined whether these effects of attractiveness are sensitive to situational factors. When perceivers were subjected to a threat or social approach manipulation prior to viewing the scenes, we confirmed specific hypotheses concerning the two manipulations. In accordance with the hypothesis that males have higher aggression potential than females, there were no differences in fixation durations between attractive and nonattractive male faces in the threat condition. On the other hand, in the social approach condition, both female and male attractive faces received longer looks. These results suggest that the aesthetic response orients people not only to the pleasing aspects of their environment, but also to those features that are adaptively relevant.     

Faces are "spatial"--holistic face perception is supported by low spatial frequencies

Faces are perceived holistically, a phenomenon best illustrated when the processing of a face feature is affected by the other features. Here, the authors tested the hypothesis that the holistic perception of a face mainly relies on its low spatial frequencies. Holistic face perception was tested in two classical paradigms: the whole-part advantage (Experiment 1) and the composite face effect (Experiments 2-4). Holistic effects were equally large or larger for low-pass filtered faces as compared to full-spectrum faces and significantly larger than for high-pass filtered faces. The disproportionate composite effect found for low-pass filtered faces was not observed when holistic perception was disrupted by inversion (Experiment 3). Experiment 4 showed that the composite face effect was enhanced only for low spatial frequencies, but not for intermediate spatial frequencies known be critical for face recognition. These findings indicate that holistic face perception is largely supported by low spatial frequencies. They also suggest that holistic processing precedes the analysis of local features during face perception.     

Top-down attentional modulation of spatial frequency processing in scene perception

Recent evidence suggests that spatial frequency (SF) processing of simple and complex visual patterns is flexible. The use of spatial scale in scene perception seems to be influenced by people's expectations. However as yet there is no direct evidence for top-down attentional effects on flexible scale use in scene perception. In two experiments we provide such evidence. We presented participants with low- and high-pass SF filtered scenes and cued their attention to the relevant scale. In Experiment 1 we subsequently presented them with hybrid scenes (both low- and high-pass scenes present). We observed that participants reported detecting the cued component of hybrids. To explore if this might be due to decision biases, in Experiment 2, we replaced hybrids with images containing meaningful scenes at uncued SFs and noise at the cued SFs (invalid cueing). We found that participants performed poorly on invalid cueing trials. These findings are consistent with top-down attentional modulation of early spatial frequency processing in scene perception.     

Incidental visual memory for objects in scenes

Three experiments were conducted to investigate the existence of incidentally acquired, long-term, detailed visual memory for objects embedded in previously viewed scenes. Participants performed intentional memorization and incidental visual search learning tasks while viewing photographs of real-world scenes. A visual memory test for previously viewed objects from these scenes then followed. Participants were not aware that they would be tested on the scenes following incidental learning in the visual search task. In two types of memory tests for visually specific object information (token discrimination and mirror-image discrimination), performance following both the memorization and visual search conditions was reliably above chance. These results indicate that recent demonstrations of good visual memory during scene viewing are not due to intentional scene memorization. Instead, long-term visual representations are incidentally generated as a natural product of scene perception.     

Fixation durations in scene viewing: Modeling the effects of local image features,oculomotor parameters,and task

Scene perception requires the orchestration of image- and task-related processes with oculomotor constraints. The present study was designed to investigate how these factors influence how long the eyes remain fixated on a given location. Linear mixed models (LMMs) were used to test whether local image statistics (including luminance, luminance contrast, edge density, visual clutter, and the number of homogeneous segments), calculated for 1° circular regions around fixation locations, modulate fixation durations, and how these effects depend on task-related control. Fixation durations and locations were recorded from 72 participants, each viewing 135 scenes under three different viewing instructions (memorization, preference judgment, and search). Along with the image-related predictors, the LMMs simultaneously considered a number of oculomotor and spatiotemporal covariates, including the amplitudes of the previous and next saccades, and viewing time. As a key finding, the local image features around the current fixation predicted this fixation’s duration. For instance, greater luminance was associated with shorter fixation durations. Such immediacy effects were found for all three viewing tasks. Moreover, in the memorization and preference tasks, some evidence for successor effects emerged, such that some image characteristics of the upcoming location influenced how long the eyes stayed at the current location. In contrast, in the search task, scene processing was not distributed across fixation durations within the visual span. The LMM-based framework of analysis, applied to the control of fixation durations in scenes, suggests important constraints for models of scene perception and search, and for visual attention in general.     

The relationship between online visual representation of a scene and long-term scene memory

In 3 experiments the author investigated the relationship between the online visual representation of natural scenes and long-term visual memory. In a change detection task, a target object either changed or remained the same from an initial image of a natural scene to a test image. Two types of changes were possible: rotation in depth, or replacement by another object from the same basic-level category. Change detection during online scene viewing was compared with change detection after delay of 1 trial (Experiments 2A and 2B) until the end of the study session (Experiment 1) or 24 hr (Experiment 3). There was little or no decline in change detection performance from online viewing to a delay of 1 trial or delay until the end of the session, and change detection remained well above chance after 24 hr. These results demonstrate that long-term memory for visual detail in a scene is robust.     

Attentional allocation during the perception of scenes

Semantic influences on attention during the 1st fixation on a scene were explored in 3 experiments. Subjects viewed briefly presented scenes; following scene presentation, a spatial probe was presented at the location of an object whose identity was consistent or inconsistent with the scene category. Responses to the probe served as an index of attention. The results of Experiment 1 suggest that within approximately 150 ms of scene onset, subjects attend preferentially to inconsistent objects. The results of Experiment 2, in which additional scene durations were used, confirm the presence of an inconsistent-object advantage that emerges within approximately 150 ms. Finally, the results of Experiment 3 demonstrate that the inconsistent-object advantage does not reflect strategic allocation of attention to likely probe locations. Implications of the results for scene perception and exploration are discussed.     

Eye movements and picture processing during recognition

Eye movements were monitored during a recognition memory test of previously studied pictures of full-color scenes. The test scenes were identical to the originals, had an object deleted from them, or included a new object substituted for an original object. In contrast to a prior report (Parker, 1978), we found no evidence that object deletions or substitutions could be recognized on the basis of information acquired from the visual periphery. Deletions were difficult to recognize peripherally, and the eyes were not attracted to them. Overall, the amplitude of the average saccade to the critical object during the memory test was less than 4.5 degrees of visual angle in all conditions and averaged 4.1 degrees across conditions. We conclude that information about object presence and identity in a scene is limited to a relatively small region around the current fixation point.     

Spatial asymmetries in viewing and remembering scenes: Consequences of an attentional bias?

Given a single fixation, memory for scenes containing salient objects near both the left and right view boundaries exhibited a rightward bias in boundary extension (Experiment 1). On each trial, a 500-msec picture and 2.5-sec mask were followed by a boundary adjustment task. Observers extended boundaries 5% more on the right than on the left. Might this reflect an asymmetric distribution of attention? In Experiments 2A and 2B, free viewing of pictures revealed that first saccades were more often leftward (62%) than rightward (38%). In Experiment 3, 500-msec pictures were interspersed with 2.5-sec masks. A subsequent object recognition memory test revealed better memory for left-side objects. Scenes were always mirror reversed for half the observers, thus ruling out idiosyncratic scene compositions as the cause of these asymmetries. Results suggest an unexpected leftward bias of attention that selectively enhanced the representations, causing a smaller boundary extension error and better object memory on the views’ left sides.     

Positive emotion can protect against source memory impairment

Despite widespread belief that memory is enhanced by emotion, evidence also suggests that emotion can impair memory. Here we test predictions inspired by object-based binding theory, which states that memory enhancement or impairment depends on the nature of the information to be retrieved. We investigated emotional memory in the context of source retrieval, using images of scenes that were negative, neutral or positive in valence. At study each scene was paired with a colour and during retrieval participants reported the source colour for recognised scenes. Critically, we isolated effects of valence by equating stimulus arousal across conditions. In Experiment 1 colour borders surrounded scenes at study: memory impairment was found for both negative and positive scenes. Experiment 2 used colours superimposed over scenes at study: valence affected source retrieval, with memory impairment for negative scenes only. These findings challenge current theories of emotional memory by showing that emotion can impair memory for both intrinsic and extrinsic source information, even when arousal is equated between emotional and neutral stimuli, and by dissociating the effects of positive and negative emotion on episodic memory retrieval.     

Amnesia is a deficit in relational memory

Eye movements were monitored to assess memory for scenes indirectly (implicitly). Two eye movement–based memory phenomena were observed: (a) the repetition effect , a decrease in sampling of previously viewed scenes compared with new scenes, reflecting memory for those scenes, and (b) the relational manipulation effect , an increase in viewing of the regions where manipulations of relations among scene elements had occurred. In normal control subjects, the relational manipulation effect was expressed only in the absence of explicit awareness of the scene manipulations. Thus, memory representations of scenes contain information about relations among elements of the scenes, at least some of which is not accessible to verbal report. But amnesic patients with severe memory impairment failed to show the relational manipulation effect. Their failure to show any demonstrable memory for relations among the constituent elements of scenes suggests that amnesia involves a fundamental deficit in relational (declarative) memory processing.