Audience gaze while appreciating a multipart musical performance

Visual information has been observed to be crucial for audience members during musical performances. The present study used an eye tracker to investigate audience members’ gazes while appreciating an audiovisual musical ensemble performance, based on evidence of the dominance of musical part in auditory attention when listening to multipart music that contains different melody lines and the joint-attention theory of gaze. We presented singing performances, by a female duo. The main findings were as follows: (1) the melody part (soprano) attracted more visual attention than the accompaniment part (alto) throughout the piece, (2) joint attention emerged when the singers shifted their gazes toward their co-performer, suggesting that inter-performer gazing interactions that play a spotlight role mediated performer-audience visual interaction, and (3) musical part (melody or accompaniment) strongly influenced the total duration of gazes among audiences, while the spotlight effect of gaze was limited to just after the singers’ gaze shifts.     

A Computational Evaluation of Sentence Processing Deficits in Aphasia

Individuals with agrammatic Broca's aphasia experience difficulty when processing reversible non‐canonical sentences. Different accounts have been proposed to explain this phenomenon. The Trace Deletion account (Grodzinsky, 1995, 2000, 2006) attributes this deficit to an impairment in syntactic representations, whereas others (e.g., Caplan, Waters, Dede, Michaud, & Reddy, 2007; Haarmann, Just, & Carpenter, 1997) propose that the underlying structural representations are unimpaired, but sentence comprehension is affected by processing deficits, such as slow lexical activation, reduction in memory resources, slowed processing and/or intermittent deficiency, among others. We test the claims of two processing accounts, slowed processing and intermittent deficiency, and two versions of the Trace Deletion Hypothesis (TDH), in a computational framework for sentence processing (Lewis & Vasishth, 2005) implemented in ACT‐R (Anderson, Byrne, Douglass, Lebiere, & Qin, 2004). The assumption of slowed processing is operationalized as slow procedural memory, so that each processing action is performed slower than normal, and intermittent deficiency as extra noise in the procedural memory, so that the parsing steps are more noisy than normal. We operationalize the TDH as an absence of trace information in the parse tree. To test the predictions of the models implementing these theories, we use the data from a German sentence—picture matching study reported in Hanne, Sekerina, Vasishth, Burchert, and De Bleser (2011). The data consist of offline (sentence‐picture matching accuracies and response times) and online (eye fixation proportions) measures. From among the models considered, the model assuming that both slowed processing and intermittent deficiency are present emerges as the best model of sentence processing difficulty in aphasia. The modeling of individual differences suggests that, if we assume that patients have both slowed processing and intermittent deficiency, they have them in differing degrees.     

The influence of linguistic and cognitive factors on the time course of verb-based implicit causality

In three eye-tracking experiments the influence of the Dutch causal connective “want” (because) and the working memory capacity of readers on the usage of verb-based implicit causality was examined. Experiments 1 and 2 showed that although a causal connective is not required to activate implicit causality information during reading, effects of implicit causality surfaced more rapidly and were more pronounced when a connective was present in the discourse than when it was absent. In addition, Experiment 3 revealed that—in contrast to previous claims—the activation of implicit causality is not a resource-consuming mental operation. Moreover, readers with higher and lower working memory capacities behaved differently in a dual-task situation. Higher span readers were more likely to use implicit causality when they had all their working memory resources at their disposal. Lower span readers showed the opposite pattern as they were more likely to use the implicit causality cue in the case of an additional working memory load. The results emphasize that both linguistic and cognitive factors mediate the impact of implicit causality on text comprehension. The implications of these results are discussed in terms of the ongoing controversies in the literature—that is, the focusing-integration debate and the debates on the source of implicit causality.     

A predictive coding model of gaze shifts and the underlying neurophysiology

A comprehensive model of gaze control must account for a number of empirical observations at both the behavioural and neurophysiological levels. The computational model presented in this article can simulate the coordinated movements of the eye, head, and body required to perform horizontal gaze shifts. In doing so it reproduces the predictable relationships between the movements performed by these different degrees of freedom (DOFs) in the primate. The model also accounts for the saccadic undershoot that accompanies large gaze shifts in the biological visual system. It can also account for our perception of a stable external world despite frequent gaze shifts and the ability to perform accurate memory-guided and double-step saccades. The proposed model also simulates peri-saccadic compression: the mis-localization of a briefly presented visual stimulus towards the location that is the target for a saccade. At the neurophysiological level, the proposed model is consistent with the existence of cortical neurons tuned to the retinal, head-centred, body-centred, and world-centred locations of visual stimuli and cortical neurons that have gain-modulated responses to visual stimuli. Finally, the model also successfully accounts for peri-saccadic receptive field (RF) remapping which results in reduced responses to stimuli in the current RF location and an increased sensitivity to stimuli appearing at the location that will be occupied by the RF after the saccade. The proposed model thus offers a unified explanation for this seemingly diverse range of phenomena. Furthermore, as the proposed model is an implementation of the predictive coding theory, it offers a single computational explanation for these phenomena and relates gaze shifts to a wider framework for understanding cortical function.     

聋人阅读的眼动研究

聋人读者普遍存在阅读困难,通过眼动技术来探索聋人阅读中的基本问题已成为一种新趋势,聋人读者在阅读过程中存在其独特的眼动模式。在回顾以往聋人阅读眼动研究的基础上,提出了对未来研究的展望:(1)眼动技术的广泛应用是聋人阅读研究的一个新趋势;(2)从跨文化研究视角探究中外聋人阅读加工的异同;(3)考察聋人视觉注意的特点与语言加工之间的关系;(4)通过眼动技术考察聋人读者的手语加工效率。     

Why reread? Evidence from garden-path and local coherence structures

Two eye-tracking experiments were conducted to compare the online reading and offline comprehension of main verb/reduced relative garden-path sentences and local coherence sentences. Rereading of early material in garden-path reduced relatives should be revisionary, aimed at reanalysing an earlier misparse; however, rereading of early material in a local coherence reduced relative need only be confirmatory, as the original parse of the earlier portion of these sentences is ultimately correct. Results of online and offline measures showed that local coherence structures elicited signals of reading disruption that arose earlier and lasted longer, and local coherence comprehension was also better than garden path comprehension. Few rereading measures in either sentence type were predicted by structural features of these sentences, nor was rereading related to comprehension accuracy, which was extremely low overall. Results are discussed with respect to selective reanalysis and good-enough processing.     

Spatial But Not Oculomotor Information Biases Perceptual Memory: Evidence From Face Perception and Cognitive Modeling

Recent research put forward the hypothesis that eye movements are integrated in memory representations and are reactivated when later recalled. However, “looking back to nothing” during recall might be a consequence of spatial memory retrieval. Here, we aimed at distinguishing between the effect of spatial and oculomotor information on perceptual memory. Participants’ task was to judge whether a morph looked rather like the first or second previously presented face. Crucially, faces and morphs were presented in a way that the morph reactivated oculomotor and/or spatial information associated with one of the previously encoded faces. Perceptual face memory was largely influenced by these manipulations. We considered a simple computational model with an excellent match (4.3% error) that expresses these biases as a linear combination of recency, saccade, and location. Surprisingly, saccades did not play a role. The results suggest that spatial and temporal rather than oculomotor information biases perceptual face memory.     

Affect in the eyes: explicit and implicit evaluations

The present study investigated whether another individual’s gaze direction influences an observer’s affective responses. In Experiment 1, subjective self-ratings and an affective priming paradigm were employed to examine how participants explicitly and implicitly, respectively, evaluated the affective valence of direct gaze, averted gaze, and closed eyes. The explicit self-ratings showed that participants evaluated closed eyes more positively than direct gaze. However, the implicit priming task showed an inverse pattern of results indicating that direct gaze was automatically evaluated more positively than closed eyes were. Experiment 2 confirmed that the opposite patterns of results between the two tasks were not due to differences in presentation times of the gaze stimuli. The results provide evidence for automatic affective reactions to eye gaze and indicate a dissociation between explicit and implicit affective evaluations of eyes and gaze direction.     

Impulse processing: a dynamical systems model of incremental eye movements in the visual world paradigm

The Visual World Paradigm (VWP) presents listeners with a challenging problem: They must integrate two disparate signals, the spoken language and the visual context, in support of action (e.g., complex movements of the eyes across a scene). We present Impulse Processing, a dynamical systems approach to incremental eye movements in the visual world that suggests a framework for integrating language, vision, and action generally. Our approach assumes that impulses driven by the language and the visual context impinge minutely on a dynamical landscape of attractors corresponding to the potential eye-movement behaviors of the system. We test three unique predictions of our approach in an empirical study in the VWP, and describe an implementation in an artificial neural network. We discuss the Impulse Processing framework in relation to other models of the VWP.     

The time course of anticipatory constraint integration

Several studies have demonstrated that as listeners hear sentences describing events in a scene, their eye movements anticipate upcoming linguistic items predicted by the unfolding relationship between scene and sentence. While this may reflect active prediction based on structural or contextual expectations, the influence of local thematic priming between words has not been fully examined. In Experiment 1, we presented verbs (e.g., arrest) in active (Subject–Verb–Object) sentences with displays containing verb-related patients (e.g., crook) and agents (e.g., policeman). We examined patient and agent fixations following the verb, after the agent role had been filled by another entity, but prior to bottom-up specification of the object. Participants were nearly as likely to fixate agents “anticipatorily” as patients, even though the agent role was already filled. However, the patient advantage suggested simultaneous influences of both local priming and active prediction. In Experiment 2, using passive sentences (Object–Verb–Subject), we found stronger, but still graded influences of role prediction when more time elapsed between verb and target, and more syntactic cues were available. We interpret anticipatory fixations as emerging from constraint-based processes that involve both non-predictive thematic priming and active prediction.