Establishing object correspondence across eye movements: Flexible use of spatiotemporal and surface feature information

Visual input is frequently disrupted by eye movements, blinks, and occlusion. The visual system must be able to establish correspondence between objects visible before and after a disruption. Current theories hold that correspondence is established solely on the basis of spatiotemporal information, with no contribution from surface features. In five experiments, we tested the relative contributions of spatiotemporal and surface feature information in establishing object correspondence across saccades. Participants generated a saccade to one of two objects, and the objects were shifted during the saccade so that the eyes landed between them, requiring a corrective saccade to fixate the target. To correct gaze to the appropriate object, correspondence must be established between the remembered saccade target and the target visible after the saccade. Target position and surface feature consistency were manipulated. Contrary to existing theories, surface features and spatiotemporal information both contributed to object correspondence, and the relative weighting of the two sources of information was governed by the demands of the task. These data argue against a special role for spatiotemporal information in object correspondence, indicating instead that the visual system can flexibly use multiple sources of relevant information.     

Talker adaptation in speech perception: adjusting the signal or the representations?

Past research has established that listeners can accommodate a wide range of talkers in understanding language. How this adjustment operates, however, is a matter of debate. Here, listeners were exposed to spoken words from a speaker of an American English dialect in which the vowel /ae/ is raised before /g/, but not before /k/. Results from two experiments showed that listeners' identification of /k/-final words like back (which are unaffected by the dialect) was facilitated by prior exposure to their dialect-affected /g/-final counterparts, e.g., bag. This facilitation occurred because the competition between interpretations, e.g., bag or back, while hearing the initial portion of the input [bae], was mitigated by the reduced probability for the input to correspond to bag as produced by this talker. Thus, adaptation to an accent is not just a matter of adjusting the speech signal as it is being heard; adaptation involves dynamic adjustment of the representations stored in the lexicon, according to the characteristics of the speaker or the context.     

平面广告中图形与文本加工差异的眼动研究

记录被试在观看不同文本位置的平面广告时的眼动轨迹,探讨了广告中文本和图形加工的差异性、文本在广告中的不同位置(左上、左下、右下、右上)的加工差异、人们对广告图片及文字的评价与其眼动指标是否具有一致性以及与再认成绩的相关程度等。结果表明,(1)瞳孔大小是比较敏感的指标,在观看广告时,文本的位置及评价水平都可以引起瞳孔大小的改变：而不会对注视时间产生影响。(2)人们对文本和图形的加工方式上存在差异,它不仅表现在眼动指标上,同时也表现在再认成绩上。广告中适当的文字有助于广告内容的记忆。     

Sex differences in the effects of visual contact and eye contact in negotiations

Previous research has proposed that the ability to see others would benefit negotiations. We argue that this view is too narrow and that the impact of visual contact on negotiated agreements depends on the meaning individuals ascribe to either its presence or absence. Based on previous research showing that females are more likely to understand others in the presence of visual contact while males understand others better in the absence of visual contact, we explore how visual contact, eye contact, and sex affect the quality of negotiated agreements in a meta-analysis (Study 1) and a laboratory experiment (Study 2). The two studies combined show that because direct communication via the face facilitates a shared understanding for two unacquainted females, their agreements are of higher quality when they have visual contact compared to when they do not (Study 1), and if they have visual contact, their agreements are better when they have eye contact than when they do not (Study 2). Because communication via the face increases discomfort between two unacquainted males, their agreements are of higher quality when they do not have visual contact (Study 1), and if they do have visual contact, their agreements are better when they have no eye contact than when they do (Study 2).     

Pronominal Resolution and Gap Filling in Agrammatic Aphasia: Evidence from Eye Movements

This paper reports the results of three studies examining comprehension and real-time processing of pronominal (Experiment 1) and Wh-movement (Experiments 2 and 3) structures in agrammatic and unimpaired speakers using eyetracking. We asked the following questions: (a) Is off-line comprehension of these constructions impaired in agrammatic listeners?, (b) Do agrammatic, like unimpaired, listeners show eye movement patterns indicative of automatic pronominal reference resolution and/or gap-filling?, and (c) Do eyetracking patterns differ when sentences are correctly versus incorrectly interpreted, or do automatic processes prevail in spite of comprehension failure? Results showed that off-line comprehension of both pronoun and Wh-movement structures was impaired in our agrammatic cohort. However, the aphasic participants showed visual evidence of real-time reference resolution as they processed binding structures, including both pronouns and reflexives, as did our unimpaired control participants. Similarly, both the patients and the control participants showed patterns consistent with successful gap filling during processing of Wh-movement structures. For neither pronominal nor movement structures did we find evidence of delayed processing. Notably, these patterns were found for the aphasic participants even when they incorrectly interpreted target sentences, with the exception of object relative constructions. For incorrectly interpreted sentences, we found end of sentence lexical competition effects. These findings indicate that aberrant lexical integration, rather than representational deficits or generally slowed processing, may underlie agrammatic aphasic listener’s comprehension failure.     

Gaze behaviors of goaltenders under spatial-temporal constraints

It is still not known what underlies successful performance in goaltending. Some studies have reported that advanced cues from the shooter's body (hip, kicking leg or support leg) are most important (Savelsbergh, G. J. P., Williams, A. M., Van der Kamp, J., & Ward, P. (2002). Visual search, anticipation and expertise in soccer goalkeepers. Journal of Sports Sciences, 20, 279-287; Savelsbergh, G. J. P., Williams, A. M., Van der Kamp, J., & Ward, P. (2005). Anticipation and visual search behaviour in expert soccer goalkeepers. Ergonomics, 48, 1686-1697; Williams, A. M., & Burwitz, L. (1993). Advanced cue utilization in soccer. In T. Reilly, J. Clarys, & A. Stibbe (Eds.), Science and football II (pp. 239-243). London, England: E&FN Spon), while others have found that the early tracking of the object prior to and during flight is most critical (Bard, C., & Fleury, M. (1981). Considering eye movement as a predictor of attainment. In: I. M. Cockerill, & W. M. MacGillvary (Eds.), Vision and Sport (pp. 28-41). Cheltenham, England: Stanley Thornes (Publishers) Ltd.). These results are similar to those found in a number of interceptive timing studies (Land, M. F., & McLeod, P. (2000). From eye movements to actions: How batsmen hit the ball. Nature Neuroscience, 3, 1340-1345; Ripoll and Fleurance, 1988; Vickers, J. N., & Adolphe, R. M. (1997). Gaze behaviour during a ball tracking and aiming skill. International Journal of Sports Vision, 4, 18-27). The coupled gaze and motor behavior of elite goaltenders were determined while responding to wrist shots taken from 5 m and 10 m on ice. The results showed that the goalies faced shots that were significantly different in phase durations due to distance (5 versus 10 m), but this was not a factor in making saves. Instead, the ability to stop the puck was dependent on the location, onset and duration of the final fixation/tracking gaze (or quiet eye) prior to initiating the saving action. The relative onset of quiet eye was significantly (p<.001) earlier (8.6%) and the duration was longer on saves (M=80.5%; 952.3 ms) compared to goals (onset 18.86%; M=70.1%, 826.1 ms). The quiet eye was located on the puck/stick during the preparation and execution of the shot in 70.53% of all trials, or on the ice in front of the release point of the puck (25.68%) and rarely on the body of the shooter (2.1%). The results are discussed within the context of current research on goaltending with specific emphasis on the timing of critical cues and the effect of tasks constraints.     

Word meaning and the control of eye fixation: semantic competitor effects and the visual world paradigm

When participants are presented simultaneously with spoken language and a visual display depicting objects to which that language refers, participants spontaneously fixate the visual referents of the words being heard [Cooper, R. M. (1974). The control of eye fixation by the meaning of spoken language: A new methodology for the real-time investigation of speech perception, memory, and language processing. Cognitive Psychology, 6(1), 84-107; Tanenhaus, M. K., Spivey-Knowlton, M. J., Eberhard, K. M., & Sedivy, J. C. (1995). Integration of visual and linguistic information in spoken language comprehension. Science, 268(5217), 1632-1634]. We demonstrate here that such spontaneous fixation can be driven by partial semantic overlap between a word and a visual object. Participants heard the word ‘piano’ when (a) a piano was depicted amongst unrelated distractors; (b) a trumpet was depicted amongst those same distractors; and (c), both the piano and trumpet were depicted. The probability of fixating the piano and the trumpet in the first two conditions rose as the word ‘piano’ unfolded. In the final condition, only fixations to the piano rose, although the trumpet was fixated more than the distractors. We conclude that eye movements are driven by the degree of match, along various dimensions that go beyond simple visual form, between a word and the mental representations of objects in the concurrent visual field.     

Is emotional content obtained from parafoveal words during reading? An eye movement analysis

An eye-movement-contingent display change technique was employed to study whether adult readers extract semantic information from parafoveal words during reading. Three types of parafoveal preview conditions were contrasted: an emotional word, a neutral word, and an identical word condition. To have a maximally effective parafoveal manipulation, high-arousal emotional words (sex- and threat-related and curse words) were used as parafoveal previews. Readers' eye fixation patterns around the target word revealed no evidence for parafoveal semantic processing. Furthermore, the pupil size showed no signs for an emotional response triggered by an emotional word previewed parafoveally. These results are consistent with the view that, as a rule, only the fixated word is processed to a semantic level during reading.     

Learning From Peers’ Eye Movements in the Absence of Expert Guidance: A Proof of Concept Using Laboratory Stock Trading,Eye Tracking,and Machine Learning

Existing research shows that people can improve their decision skills by learning what experts paid attention to when faced with the same problem. However, in domains like financial education, effective instruction requires frequent, personalized feedback given at the point of decision, which makes it time‐consuming for experts to provide and thus, prohibitively costly. We address this by demonstrating an automated feedback mechanism that allows amateur decision‐makers to learn what information to attend to from one another, rather than from an expert. In the first experiment, eye movements of N = 100 subjects were recorded while they repeatedly performed a standard behavioral finance investment task. Consistent with previous studies, we found that a significant proportion of subjects were affected by decision bias. In the second experiment, a different group of N = 100 subjects faced the same task but, after each choice, they received individual, machine learning‐generated feedback on whether their pre‐decision eye movements resembled those made by Experiment 1 subjects prior to good decisions. As a result, Experiment 2 subjects learned to analyze information similarly to their successful peers, which in turn reduced their decision bias. Furthermore, subjects with low Cognitive Reflection Test scores gained more from the proposed form of process feedback than from standard behavioral feedback based on decision outcomes.     

Area activation: a computational model of saccadic selectivity in visual search

The Area Activation Model (Pomplun, Reingold, Shen, & Williams, 2000) is a computational model predicting the statistical distribution of saccadic endpoints in visual search tasks. Its basic assumption is that saccades in visual search tend to foveate display areas that provide a maximum amount of task‐relevant information for processing during the subsequent fixation. In the present study, a counterintuitive prediction by the model is empirically tested, namely that saccadic selectivity towards stimulus features depends on the spatial arrangement of search items. We find good correspondence between simulated and empirically observed selectivity patterns, providing strong support for the Area Activation Model.