Establishing object correspondence across eye movements: Flexible use of spatiotemporal and surface feature information

Visual input is frequently disrupted by eye movements, blinks, and occlusion. The visual system must be able to establish correspondence between objects visible before and after a disruption. Current theories hold that correspondence is established solely on the basis of spatiotemporal information, with no contribution from surface features. In five experiments, we tested the relative contributions of spatiotemporal and surface feature information in establishing object correspondence across saccades. Participants generated a saccade to one of two objects, and the objects were shifted during the saccade so that the eyes landed between them, requiring a corrective saccade to fixate the target. To correct gaze to the appropriate object, correspondence must be established between the remembered saccade target and the target visible after the saccade. Target position and surface feature consistency were manipulated. Contrary to existing theories, surface features and spatiotemporal information both contributed to object correspondence, and the relative weighting of the two sources of information was governed by the demands of the task. These data argue against a special role for spatiotemporal information in object correspondence, indicating instead that the visual system can flexibly use multiple sources of relevant information.     

Eye movements and visual memory: detecting changes to saccade targets in scenes

Saccade-contingent change detection provides a powerful tool for investigating scene representation and scene memory. In the present study, critical objects presented within color images of naturalistic scenes were changed during a saccade toward or away from the target. During the saccade,the critical object was changed to another object type, to a visually different token of the same object type, or was deleted from the scene. There were three main results. First, the deletion of a saccade target was special: Detection performance for saccade target deletions was very good, and this level of performance did not decline with the amplitude of the saccade. In contrast, detection of type and token changes at the saccade target, and of all changes including deletions at a location that had just been fixated but was not the saccade target, decreased as the amplitude of the saccade increased. Second, detection performance for type and token changes, both when the changing object was the target of the saccade and when the object had just been fixated but was not the saccade target, was well above chance. Third, mean gaze durations were reliably elevated for those trials in which the change was not overtly detected. The results suggest that the presence of the saccade target plays a special role in trassaccadic integration, and together with other recent findings, suggest more generally that a relatively rich scene representation is retained across saccades and stored in visual memory.     

Target localization after saccades and at fixation: Nontargets both facilitate and bias responses

ABSTRACT

The image on our retina changes every time we make an eye movement. To maintain visual stability after saccades, specifically to locate visual targets, we may use nontarget objects as “landmarks”. In the current study, we compared how the presence of nontargets affects target localization after saccades and during sustained fixation. Participants fixated a target object, which either maintained its location on the screen (sustained-fixation trials), or displaced to trigger a saccade (saccade trials). After the target disappeared, participants reported the most recent target location with a mouse click. We found that the presence of nontargets decreased response error magnitude and variability. However, this nontarget facilitation effect was not larger for saccade trials than sustained-fixation trials, indicating that nontarget facilitation might be a general effect for target localization, rather than of particular importance to post-saccadic stability. Additionally, participants’ responses were biased towards the nontarget locations, particularly when the nontarget-target relationships were preserved in relative coordinates across the saccade. This nontarget bias interacted with biases from other spatial references, e.g., eye movement paths, possibly in a way that emphasized non-redundant information. In summary, the presence of nontargets is one of several sources of reference that combine to influence (both facilitate and bias) target localization.     

Investigating the parameters of transsaccadic memory: inhibition of return impedes information acquisition near a saccade target

A limited amount of visual information is retained between saccades, which is subsequently stored into a memory system, such as transsaccadic memory. Since the capacity of transsaccadic memory is limited, selection of information is crucial. Selection of relevant information is modulated by attentional processes such as the presaccadic shift of attention. This involuntary shift of attention occurs prior to execution of the saccade and leads to information acquisition at an intended saccade target. The aim of the present study was to investigate the influence that another attentional effect, inhibition of return (IOR), has on the information that gets stored into transsaccadic memory. IOR is the phenomenon where participants are slower to respond to a cue at a previously attended location. To this end, we used a transsaccadic memory paradigm in which stimuli, oriented on a horizontal axis relative to saccade direction, are only visible to the participant before executing a saccade. Previous research showed that items in close proximity to a saccade target are likely to be reported more accurately. In our current study, participants were cued to fixate one of the stimulus locations and subsequently refixated the centre fixation point before executing the transsaccadic memory task. Results indicate that information at a location near a saccade landing point is less likely to be acquired into transsaccadic memory when this location was previously associated with IOR. Furthermore, we found evidence which implicates a reduction of the overall amount of elements retained in transsaccadic memory when a location near a saccade target is associated with IOR. These results suggest that the presaccadic shift of attention may be modulated by IOR and thereby reduces information acquisition by transsaccadic memory.     

When two actions are easier than one: How inhibitory control demands affect response processing

Numerous studies showed that the simultaneous execution of multiple actions is associated with performance costs. Here, we demonstrate that when highly automatic responses are involved, performance in single-response conditions can actually be worse than in dual-response conditions. Participants responded to peripheral visual stimuli with an eye movement (saccade), a manual key press, or both. To manipulate saccade automaticity, a central fixation cross either remained present throughout the trial (overlap condition, lower automaticity) or disappeared 200 ms before visual target onset (gap condition, greater automaticity). Crucially, single-response conditions yielded more performance errors than dual-response conditions (i.e., dual-response benefit), especially in gap trials. This was due to difficulties associated with inhibiting saccades when only manual responses were required, suggesting that response inhibition (remaining fixated) can be even more resource-demanding than overt response execution (saccade to peripheral target).     

Age-group differences in inhibiting an oculomotor response

Age-group differences were examined in the delayed oculomotor response task, which requires that observers delay the execution of a saccade (eye movement) toward an abrupt-onset visual cue. This task differs from antisaccade and attentional capture in that inhibition causes saccades to be postponed, not redirected. Older adults executed more premature saccades than young adults, but there were no age-group differences in latency or accuracy of saccades executed at the proper time. The results suggest that older adults are less capable of inhibiting a prepotent saccadic response, but that other aspects of visual working memory related to the task are preserved.     

Does oculomotor inhibition of return influence fixation probability during scene search?

Oculomotor inhibition of return (IOR) is believed to facilitate scene scanning by decreasing the probability that gaze will return to a previously fixated location. This "foraging" hypothesis was tested during scene search and in response to sudden-onset probes at the immediately previous (one-back) fixation location. The latencies of saccades landing within 1o of the previous fixation location were elevated, consistent with oculomotor IOR. However, there was no decrease in the likelihood that the previous location would be fixated relative to distance-matched controls or an a priori baseline. Saccades exhibit an overall forward bias, but this is due to a general bias to move in the same direction and for the same distance as the last saccade (saccadic momentum) rather than to a spatially specific tendency to avoid previously fixated locations. We find no evidence that oculomotor IOR has a significant impact on return probability during scene search.     

Presaccadic attention allocation and express saccades

Express saccades are visually-guided saccades that are characterized by an extremely short latency of about 100 ms. The present experiments tested the hypothesis that a disengagement of visual attention is necessary for the generation of express saccades. All subjects produced large numbers of express saccades in the gap paradigm, in which the fixation stimulus is removed 200 ms before target onset (Exp. 1), but not in the overlap paradigm, in which the fixation stimulus remains on during the entire trial (Exp. 2). By means of peripheral cues (Exps. 3–5) and central cues (Exps. 6–7), visual attention was directed at the target location for the saccade before the actual appearance of the saccade target. In all experiments, the location cues facilitated rather than abolished express saccades. The generation of express saccades was facilitated even when the currently fixated visual stimulus was not removed before target onset (fixation-overlap; Exps. 5–7). The results are explained by the hypothesis that a disengagement of a separate fixation system is necessary for the generation of express saccades, a hypothesis that is in line with current neurobiological findings.     

Age-group Differences in Inhibiting an Oculomotor Response

ABSTRACT

Age-group differences were examined in the delayed oculomotor response task, which requires that observers delay the execution of a saccade (eye movement) toward an abrupt-onset visual cue. This task differs from antisaccade and attentional capture in that inhibition causes saccades to be postponed, not redirected. Older adults executed more premature saccades than young adults, but there were no age-group differences in latency or accuracy of saccades executed at the proper time. The results suggest that older adults are less capable of inhibiting a prepotent saccadic response, but that other aspects of visual working memory related to the task are preserved.     

Oculomotor capture and Inhibition of Return: evidence for an oculomotor suppression account of IOR

Previous research has shown that when subjects search for a particular target object the sudden appearance of a new object captures the eyes on a large proportion of trials. The present study examined whether the onset affects the oculomotor system even when the eyes move directly towards the target. Using a modified version of the oculomotor paradigm (see Theeuwes, Kramer, Hahn, & Irwin, 1998) we show that when the eyes moved to the target object, subsequent saccades were inhibited from moving to a location at which a new object had previously appeared (inhibition-of-return; IOR). Whether or not a saccade to the onset was executed had no effect on the size of the inhibition. In particular conditions, the trajectories of saccades to the target objects were slightly curved in the opposite direction of the onset. The data are interpreted in the context of a novel hypothesis regarding oculomotor IOR.     

Perceiving stimulus displacements across saccades

The visual world appears stable despite frequent retinal image movements caused by saccades. Many theories of visual stability assume that extraretinal eye position information is used to spatially adjust perceived locations across saccades, whereas others have proposed that visual stability depends upon coding of the relative positions of objects. McConkie and Currie (1996) proposed a refined combination of these views (called the Saccade Target Object Theory) in which the perception of stability across saccades relies on a local evaluation process centred on the saccade target object rather than on a remapping of the entire scene, with some contribution from memory for the relative positions of objects as well. Three experiments investigated the saccade target object theory, along with an alternative hypothesis that proposes that multiple objects are updated across saccades, but with variable resolution, with the saccade target object (by virtue of being the focus of attention before the saccade and residing near the fovea after the saccade) having priority in the perception of displacement. Although support was found for the saccade target object theory in Experiment 1, the results of Experiments 2 and 3 found that multiple objects are updated across saccades and that their positions are evaluated to determine perceived stability. There is an advantage for detecting displacements of the saccade target, most likely because of visual acuity or attentional focus being better near the fovea, but it is not the saccade target alone that determines the perception of stability and of displacements across saccades. Rather, multiple sources of information appear to contribute.     

The effect of repetitive saccade execution on the attention network test: Enhancing executive function with a flick of the eyes

The simple act of repeatedly looking left and right can enhance subsequent cognition, including divergent thinking, detection of matching letters from visual arrays, and memory retrieval. One hypothesis is that saccade execution enhances subsequent cognition by altering attentional control. To test this hypothesis, we compared performance following repetitive bilateral saccades or central fixation on the revised attention network test, which measures the operation of three distinct attentional networks: alerting, orienting, and executive function. The primary finding was that saccade execution increased the subsequent operation of the executive function network, which encompasses attentional control. Specifically, saccade execution decreased response time to target stimuli in the presence of response-incongruent flankers. A secondary finding was that saccade execution decreased response time to targets when an invalid location was cued prior to target onset. These findings suggest that saccades are an effective means of improving attentional control. Of greater theoretical importance, the study establishes attentional enhancement as a potential mechanism by which saccades enhance other aspects of cognition. Although some saccade execution effects have been found to depend on consistency of handedness (i.e., the consistency with which an individual uses one hand over the other), saccade-induced enhancement of attentional control occurred independently of handedness consistency.     

Remapping versus short-term memory in visual stability across saccades

Saccadic eye movements cause displacements of the image of the visual world projected on the retina. Despite the ubiquitous nature of saccades, subjective experience of the world is continuous and stable. In five experiments, we addressed the mechanisms that may support visual stability: matching of pre- and postsaccadic locations of the target by an internal copy of the saccade, or retention of the visual attributes of the target in short-term memory across the saccade. Healthy human adults were instructed to make a saccade to a peripheral Gabor patch. While the saccade was in midflight, the patch could change location, orientation, or both. The change occurred either immediately or following a 250-ms blank during which no visual stimuli were available. In separate experiments, subjects had to report either whether the patch stepped to the left or right or whether the orientation rotated clockwise or counterclockwise. Consistent with previous findings, we found that transsaccadic displacement discrimination was enhanced by the addition of the blank. However, contrary to previous findings reported in the literature, the feature change did not improve performance. Transsaccadic orientation change discrimination did not depend on either an irrelevant temporal blank or a simultaneous irrelevant target displacement. Taken together, these findings suggest that orientation is not a relevant visual feature for transsaccadic correspondence.     

The mechanism underlying inhibition of saccadic return

Human observers take longer to re-direct gaze to a previously fixated location. Although there has been some exploration of the characteristics of inhibition of saccadic return (ISR), the exact mechanisms by which ISR operates are currently unknown. In the framework of accumulation models of response times, in which evidence is integrated over time to a response threshold, ISR could reflect a reduction in the rate of accumulation for saccades to return locations or an increase in the effective criterion for response. In two experiments, participants generated sequences of three saccades, in response to a peripheral or a central cue. ISR occurred across these manipulations: saccade latency was consistently increased for movements to the immediately previously fixated location. Latency distributions from individual observers were fit with a Linear Ballistic Accumulator model. ISR was best accounted for as a change in the accumulation rate. We suggest this parameter represents the overall desirability of a particular course of action, the evidence for which may be derived from a variety of sensory and non-sensory sources.     

Effects of direct and averted gaze on the subsequent saccadic response

The saccadic latency to visual targets is susceptible to the properties of the currently fixated objects. For example, the disappearance of a fixation stimulus prior to presentation of a peripheral target shortens saccadic latencies (the gap effect). In the present study, we investigated the influences of a social signal from a facial fixation stimulus (i.e., gaze direction) on subsequent saccadic responses in the gap paradigm. In Experiment 1, a cartoon face with a direct or averted gaze was used as a fixation stimulus. The pupils of the face were unchanged (overlap), disappeared (gap), or were translated vertically to make or break eye contact (gaze shift). Participants were required to make a saccade toward a target to the left or the right of the fixation stimulus as quickly as possible. The results showed that the gaze direction influenced saccadic latencies only in the gaze shift condition, but not in the gap or overlap condition; the direct-to-averted gaze shift (i.e., breaking eye contact) yielded shorter saccadic latencies than did the averted-to-direct gaze shift (i.e., making eye contact). Further experiments revealed that this effect was eye contact specific (Exp. 2) and that the appearance of an eye gaze immediately before the saccade initiation also influenced the saccadic latency, depending on the gaze direction (Exp. 3). These results suggest that the latency of target-elicited saccades can be modulated not only by physical changes of the fixation stimulus, as has been seen in the conventional gap effect, but also by a social signal from the attended fixation stimulus.     

Quantifying the performance limits of human saccadic targeting during visual search.

In previous studies of saccadic targeting, the issue how visually guided saccades to unambiguous targets are programmed and executed has been examined. These studies have found different degrees of guidance for saccades depending on the task and task difficulty. In this study, we use ideal-observer analysis to estimate the visual information used for the first saccade during a search for a target disk in noise. We quantitatively compare the performance of the first saccadic decision to that of the ideal observer (ie absolute efficiency of the first saccade) and to that of the associated final perceptual decision at the end of the search (ie relative efficiency of the first saccade). Our results show, first, that at all levels of salience tested, the first saccade is based on visual information from the stimulus display, and its highest absolute efficiency is approximately 20%. Second, the efficiency of the first saccade is lower than that of the final perceptual decision after active search (with eye movements) and has a minimum relative efficiency of 19% at the lowest level of saliency investigated. Third, we found that requiring observers to maintain central fixation (no saccades allowed) decreased the absolute efficiency of their perceptual decision by up to a factor of two, but that the magnitude of this effect depended on target salience. Our results demonstrate that ideal-observer analysis can be extended to measure the visual information mediating saccadic target-selection decisions during visual search, which enables direct comparison of saccadic and perceptual efficiencies.     

Eye movements can cause item-specific visual recognition advantages

Prior research suggests that spontaneous saccades localized towards blank regions of space during memory storage and recall improve memory for items at the saccade locations. In the present study, we examined whether a recognition advantage can be observed when a single, exogenously directed saccade occurs during memory maintenance. We manipulated whether participants made a saccade to an item’s previous location or maintained fixation, as well as whether tested items reappeared in their original location or not. The results of three experiments showed that visual recognition was better after a saccade to the location of a probed object than after no saccade or after a saccade to the location of a non-probed object, so long as saccades went to the to-be-tested location more often than chance. Taken together, our findings demonstrate that eye movements can elicit an item-specific recognition advantage in visual working memory.     

Characteristic sounds make you look at target objects more quickly

When you are looking for an object, does hearing its characteristic sound make you find it more quickly? Our recent results supported this possibility by demonstrating that when a cat target, for example, was presented among other objects, a simultaneously presented “meow” sound (containing no spatial information) reduced the manual response time for visual localization of the target. To extend these results, we determined how rapidly an object-specific auditory signal can facilitate target detection in visual search. On each trial, participants fixated a specified target object as quickly as possible. The target’s characteristic sound speeded the saccadic search time within 215–220 msec and also guided the initial saccade toward the target, compared with presentation of a distractor’s sound or with no sound. These results suggest that object-based auditory—visual interactions rapidly increase the target object’s salience in visual search.     

Task-irrelevant emotional faces impair response adjustments in a double-step saccade task

Cognitive control enables us to adjust behaviours according to task demands, and emotion influences the cognitive control. We examined how task-irrelevant emotional stimuli impact the ability to inhibit a prepared response and then programme another appropriate response. In the study, either a single target or two sequential targets appeared after emotional face images (positive, neutral, and negative). Subjects were required to freely viewed the emotional faces and make a saccade quickly upon target onset, but inhibit their initial saccades and redirect gaze to the second target if it appeared. We found that subjects were less successful at inhibiting their initial saccades as the inter-target delay increased. Emotional faces further reduced their inhibition ability with a longer delay, but not with a shorter delay. When subjects failed to inhibit the initial saccade, the longer delay produced longer intersaccadic interval. Especially, positive faces lengthened the intersaccadic interval with a longer delay. These results showed mere presence of emotional stimuli influences gaze redirection by impairing the ability to cancel a prepared saccade and delaying the programming of a corrective saccade. Therefore, we propose that the modulation of response adjustment exerted by emotional faces is related to the stage of initial response programming.