Processing emotion information from both the face and body: An eye-movement study

This study examined the perception of emotional expressions, focusing on the face and the body. Photographs of four actors expressing happiness, sadness, anger, and fear were presented in congruent (e.g., happy face with happy body) and incongruent (e.g., happy face with fearful body) combinations. Participants selected an emotional label using a four-option categorisation task. Reaction times and accuracy for the categorisation judgement, and eye movements were the dependent variables. Two regions of interest were examined: face and body. Results showed better accuracy and faster reaction times for congruent images compared to incongruent images. Eye movements showed an interaction in which there were more fixations and longer dwell times to the face and fewer fixations and shorter dwell times to the body with incongruent images. Thus, conflicting information produced a marked effect on information processing in which participants focused to a greater extent on the face compared to the body.     

Is the glass half empty or half full and does it even matter? Cognition,emotion, and psychopathology

ABSTRACT

This contribution examines the outcomes of recent decades of research on the interaction between cognition and emotion and how it has informed our understanding and treatment of emotional disorders with a special focus on depression. The review identifies important challenges to this work including the dynamic nature of cognitive processes and emotional responding, the bidirectional relation of cognition and emotion, the need for new tasks and for studies conducted outside of the laboratory, and the consideration of context such as interpersonal factors. Examining interactions between cognition and emotion and integrating basic research on cognition into the scientific understanding of emotional difficulties in psychopathology has contributed in many ways to what the field currently knows about the origins and the treatment of emotional disorders. Important challenges lie ahead – especially the integration of this work into the development of novel treatment approaches.     

Tests of the E-Z Reader model: exploring the interface between cognition and eye-movement control

This paper is simultaneously a test and refinement of the E-Z Reader model and an exploration of the interrelationship between visual and language processing and eye-movements in reading. Our modeling indicates that the assumption that words in text are processed serially by skilled readers is a viable and attractive hypothesis, as it accounts not only for "normal" reading data, but also for the pattern of decrements that occur when text is withheld from view during certain periods of time, such as in the boundary paradigm and the disappearing text paradigm. Our analyses also indicate (a) that lexical processing during reading is essentially continuous and (b) that the trigger for eye movements has to be a different (and prior) event to the trigger for shifts of covert spatial attention. In addition, the parameter values in our model are in accordance with what is known about such processes and should be taken as serious hypotheses for how long these processes last. Although a parallel model may be able to duplicate the predictions of the E-Z Reader model, we think that it is likely to be far less parsimonious and not nearly as good a heuristic device for using eye movements to understand language processing in reading.     

Recurrent transient underdetermination and the glass half full

Kyle Stanford’s arguments against scientific realism are assessed, with a focus on the underdetermination of theory by evidence. I argue that discussions of underdetermination have neglected a possible symmetry which may ameliorate the situation.

Assignment of reference to reflexives and pronouns in picture noun phrases: evidence from eye movements

Most structural Binding Theories predict a complementary distribution between reflexives and pronouns in picture noun phrases containing possessors (e.g. "Ken's picture of himself/him"). In two head-mounted eye-tracking experiments, listeners frequently violated Binding Theory predictions for reflexives, often interpreting the reflexives as taking an antecedent outside of the binding domain, and violating complementarity assumptions. Moreover, the pattern and timing of the eye movements showed Binding Theory violations for reflexives during the earliest moments of reference resolution. The results demonstrate that either binding constraints must be reformulated to decouple pronouns and reflexives or all reflexives in picture noun phrases must be treated as logophors, and thus exempt from structural Binding Theory.     

Delayed oculomotor inhibition in patients with lesions to the human frontal oculomotor cortex: Evidence from a study on saccade averaging

The frontal oculomotor cortex is known to play an important role in oculomotor selection. The aim of the current study was to examine whether previously observed findings concerning the role of the frontal oculomotor cortex in the speed of saccade initiation and oculomotor inhibition might be related to a common underlying role of these areas in oculomotor selection. To this end, six patients with lesions to the frontal oculomotor cortex performed a double stimulus paradigm in which two elements were presented simultaneously in close proximity. Patients performed a block in which no specific task instruction was given and a block in which an instruction was provided about which of the two elements was the target. The rationale behind this manipulation was that the introduction of a specific task instruction would require a stronger involvement of top-down factors. In contrast to the block without a specific task instruction, saccade latencies to the contralesional visual field were longer than the ipsilesional visual field when a task instruction was given. This effect was strongest for saccades that landed away from the target and the distractor, reflecting trials in which strong oculomotor inhibition was applied. The observed deficits can be explained in terms of a slowing of the inhibitory signals associated with the rejection of a distractor. Given the known role of the Frontal Eye Fields and the location of the lesions, we attribute these findings to the Frontal Eye Fields, revealing their important role in the voluntary control of eye movements.     

Task effects reveal cognitive flexibility responding to frequency and predictability: Evidence from eye movements in reading and proofreading

It is well-known that word frequency and predictability affect processing time. These effects change magnitude across tasks, but studies testing this use tasks with different response types (e.g., lexical decision, naming, and fixation time during reading; Schilling, Rayner, & Chumbley, 1998), preventing direct comparison. Recently, Kaakinen and Hyönä (2010) overcame this problem, comparing fixation times in reading for comprehension and proofreading, showing that the frequency effect was larger in proofreading than in reading. This result could be explained by readers exhibiting substantial cognitive flexibility, and qualitatively changing how they process words in the proofreading task in a way that magnifies effects of word frequency. Alternatively, readers may not change word processing so dramatically, and instead may perform more careful identification generally, increasing the magnitude of many word processing effects (e.g., both frequency and predictability). We tested these possibilities with two experiments: subjects read for comprehension and then proofread for spelling errors (letter transpositions) that produce nonwords (e.g., trcak for track as in Kaakinen & Hyönä) or that produce real but unintended words (e.g., trial for trail) to compare how the task changes these effects. Replicating Kaakinen and Hyönä, frequency effects increased during proofreading. However, predictability effects only increased when integration with the sentence context was necessary to detect errors (i.e., when spelling errors produced words that were inappropriate in the sentence; trial for trail). The results suggest that readers adopt sophisticated word processing strategies to accommodate task demands.     

Semantic and Syntactic Associations During Word Search Modulate the Relationship Between Attention and Subsequent Memory

Two experiments were conducted to investigate how linguistic information influences attention allocation in visual search and memory for words. In Experiment 1, participants searched for the synonym of a cue word among five words. The distractors included one antonym and three unrelated words. In Experiment 2, participants were asked to judge whether the five words presented on the screen comprise a valid sentence. The relationships among words were sentential, semantically related or unrelated. A memory recognition task followed. Results in both experiments showed that linguistically related words produced better memory performance. We also found that there were significant interactions between linguistic relation conditions and memorization on eye-movement measures, indicating that good memory for words relied on frequent and long fixations during search in the unrelated condition but to a much lesser extent in linguistically related conditions. We conclude that semantic and syntactic associations attenuate the link between overt attention allocation and subsequent memory performance, suggesting that linguistic relatedness can somewhat compensate for a relative lack of attention during word search.     

Word meaning and the control of eye fixation: semantic competitor effects and the visual world paradigm

When participants are presented simultaneously with spoken language and a visual display depicting objects to which that language refers, participants spontaneously fixate the visual referents of the words being heard [Cooper, R. M. (1974). The control of eye fixation by the meaning of spoken language: A new methodology for the real-time investigation of speech perception, memory, and language processing. Cognitive Psychology, 6(1), 84-107; Tanenhaus, M. K., Spivey-Knowlton, M. J., Eberhard, K. M., & Sedivy, J. C. (1995). Integration of visual and linguistic information in spoken language comprehension. Science, 268(5217), 1632-1634]. We demonstrate here that such spontaneous fixation can be driven by partial semantic overlap between a word and a visual object. Participants heard the word ‘piano’ when (a) a piano was depicted amongst unrelated distractors; (b) a trumpet was depicted amongst those same distractors; and (c), both the piano and trumpet were depicted. The probability of fixating the piano and the trumpet in the first two conditions rose as the word ‘piano’ unfolded. In the final condition, only fixations to the piano rose, although the trumpet was fixated more than the distractors. We conclude that eye movements are driven by the degree of match, along various dimensions that go beyond simple visual form, between a word and the mental representations of objects in the concurrent visual field.     

Merging and modifying hypotheses on the emotional and cognitive effects of eye movements: The dopaminergic regulation hypothesis

The integration of hypotheses from Eye-Movement Desensitization and Reprocessing (EMDR) and Saccade Induced Retrieval Enhancement (SIRE), both of which have been met with considerable skepticism, may lead to significant gains in both domains. Cognitive accounts of EMDR, the orienting response (OR) and working memory (WM) hypotheses, and of SIRE, the interhemispheric interaction (IHI) and the top-down attentional control (TDAC) hypotheses, are discussed. The accounts show several blind spots and frictions, for instance, on re-imagining during EMs, on hemispheric lateralization, and on emotional influences of eye movements. The failure to consider these factors across and within domains may well explain the many disparate findings. This perspective aims to remove the artificial separation and seeks a theoretical integration of the domains. It combines elements of OR and TDAC into a new dopaminergic regulation hypothesis while replacing affective (i.e., positive vs. negative) by motivational mechanisms (i.e., fostering approach and recoding). EMs are posited to result in a short-latency, targeted release of dopamine, which is the central neuromodulator in approach tendencies. According to this hypothesis, the largest effects are obtained in individuals with collateralized eye dominance and dopamine dominance. Concrete suggestions are made to improve research conditions and therapeutic efficacy.     

Non-selective lexical access in bilinguals is spontaneous and independent of input monitoring: Evidence from eye tracking

Language non-selective lexical access in bilinguals has been established mainly using tasks requiring explicit language processing. Here, we show that bilinguals activate native language translations even when words presented in their second language are incidentally processed in a nonverbal, visual search task. Chinese–English bilinguals searched for strings of circles or squares presented together with three English words (i.e., distracters) within a 4-item grid. In the experimental trials, all four locations were occupied by English words, including a critical word that phonologically overlapped with the Chinese word for circle or square when translated into Chinese. The eye-tracking results show that, in the experimental trials, bilinguals looked more frequently and longer at critical than control words, a pattern that was absent in English monolingual controls. We conclude that incidental word processing activates lexical representations of both languages of bilinguals, even when the task does not require explicit language processing.     

Watch out! Directional threat-related postures cue attention and the eyes

Previous work indicates that threatening facial expressions with averted eye gaze can act as a signal of imminent danger, enhancing attentional orienting in the gazed-at direction. However, this threat-related gaze-cueing effect is only present in individuals reporting high levels of anxiety. The present study used eye tracking to investigate whether additional directional social cues, such as averted angry and fearful human body postures, not only cue attention, but also the eyes. The data show that although body direction did not predict target location, anxious individuals made faster eye movements when fearful or angry postures were facing towards (congruent condition) rather than away (incongruent condition) from peripheral targets. Our results provide evidence for attentional cueing in response to threat-related directional body postures in those with anxiety. This suggests that for such individuals, attention is guided by threatening social stimuli in ways that can influence and bias eye movement behaviour.     

An Analysis of the Time Course of Lexical Processing During Reading

Reingold, Reichle, Glaholt, and Sheridan (2012) reported a gaze‐contingent eye‐movement experiment in which survival‐curve analyses were used to examine the effects of word frequency, the availability of parafoveal preview, and initial fixation location on the time course of lexical processing. The key results of these analyses suggest that lexical processing begins very rapidly (after approximately 120 ms) and is supported by substantial parafoveal processing (more than 100 ms). Because it is not immediately obvious that these results are congruent with the theoretical assumption that words are processed and identified in a strictly serial manner, we attempted to simulate the experiment using the E‐Z Reader model of eye‐movement control (Reichle, 2011). These simulations were largely consistent with the empirical results, suggesting that parafoveal processing does play an important functional role by allowing lexical processing to occur rapidly enough to mediate direct control over when the eyes move during reading.     

The effects of interword spacing on the eye movements of young and older readers

Recent evidence indicates that older adults (aged 65+) are more disrupted by removing interword spaces than young adults (aged 18–30). However, it is not known whether older readers also show greater sensitivity to the more subtle changes to this spacing that frequently occur during normal reading. In the present study the eye movements of young and older adults were examined when reading texts for which interword spacing was normal, condensed to half its normal size or expanded to 1.5 times its normal size. Although these changes in interword spacing affected eye movement behaviour, this influence did not differ between young and older adults. Furthermore, a word frequency manipulation showed that these changes did not affect word identification for either group. The results indicate that older adults can adapt their eye moment behaviour to accommodate subtle changes in the spatial layout of text equally effectively as young adults.     

Eye movements reveal rapid concurrent access to factual and counterfactual interpretations of the world

Imagining a counterfactual world using conditionals (e.g., If Joanne had remembered her umbrella . . .) is common in everyday language. However, such utterances are likely to involve fairly complex reasoning processes to represent both the explicit hypothetical conjecture and its implied factual meaning. Online research into these mechanisms has so far been limited. The present paper describes two eye movement studies that investigated the time-course with which comprehenders can set up and access factual inferences based on a realistic counterfactual context. Adult participants were eye-tracked while they read short narratives, in which a context sentence set up a counterfactual world (If . . . then . . .), and a subsequent critical sentence described an event that was either consistent or inconsistent with the implied factual world. A factual consistent condition (Because . . . then . . .) was included as a baseline of normal contextual integration. Results showed that within a counterfactual scenario, readers quickly inferred the implied factual meaning of the discourse. However, initial processing of the critical word led to clear, but distinct, anomaly detection responses for both contextually inconsistent and consistent conditions. These results provide evidence that readers can rapidly make a factual inference from a preceding counterfactual context, despite maintaining access to both counterfactual and factual interpretations of events.     

Heuristics and criterion setting during selective encoding in visual decision making: Evidence from eye movements

When making a decision, people spend longer looking at the option they ultimately choose compared to other options—termed the gaze bias effect—even during their first encounter with the options (Glaholt & Reingold, 2009a, 2009b; Schotter, Berry, McKenzie & Rayner, 2010). Schotter et al. (2010) suggested that this is because people selectively encode decision-relevant information about the options, online during the first encounter with them. To extend their findings and test this claim, we recorded subjects' eye movements as they made judgements about pairs of images (i.e., which one was taken more recently or which one was taken longer ago). We manipulated whether both images were presented in the same colour content (e.g., both in colour or both in black-and-white) or whether they differed in colour content and the extent to which colour content was a reliable cue to relative recentness of the images. We found that the magnitude of the gaze bias effect decreased when the colour content cue was not reliable during the first encounter with the images, but no modulation of the gaze bias effect in remaining time on the trial. These data suggest people do selectively encode decision-relevant information online.     

Discourse-mediation of the mapping between language and the visual world: Eye movements and mental representation

Two experiments explored the mapping between language and mental representations of visual scenes. In both experiments, participants viewed, for example, a scene depicting a woman, a wine glass and bottle on the floor, an empty table, and various other objects. In Experiment 1, participants concurrently heard either ‘The woman will put the glass on the table’ or ‘The woman is too lazy to put the glass on the table’. Subsequently, with the scene unchanged, participants heard that the woman ‘will pick up the bottle, and pour the wine carefully into the glass.’ Experiment 2 was identical except that the scene was removed before the onset of the spoken language. In both cases, eye movements after ‘pour’ (anticipating the glass) and at ‘glass’ reflected the language-determined position of the glass, as either on the floor, or moved onto the table, even though the concurrent (Experiment 1) or prior (Experiment 2) scene showed the glass in its unmoved position on the floor. Language-mediated eye movements thus reflect the real-time mapping of language onto dynamically updateable event-based representations of concurrently or previously seen objects (and their locations).