A goal-based perspective on eye movements in visual world studies

There is an emerging literature on visual search in natural tasks suggesting that task-relevant goals account for a remarkably high proportion of saccades, including anticipatory eye movements. Moreover, factors such as “visual saliency” that otherwise affect fixations become less important when they are bound to objects that are not relevant to the task at hand. We briefly review this literature and discuss the implications for task-based variants of the visual world paradigm. We argue that the results and their likely interpretation may profoundly affect the “linking hypothesis” between language processing and the location and timing of fixations in task-based visual world studies. We outline a goal-based linking hypothesis and discuss some of the implications for how we conduct visual world studies, including how we interpret and analyze the data. Finally, we outline some avenues of research, including examples of some classes of experiments that might prove fruitful for evaluating the effects of goals in visual world experiments and the viability of a goal-based linking hypothesis.     

Models of high-dimensional semantic space predict language-mediated eye movements in the visual world

In the visual world paradigm, participants are more likely to fixate a visual referent that has some semantic relationship with a heard word, than they are to fixate an unrelated referent [Cooper, R. M. (1974). The control of eye fixation by the meaning of spoken language. A new methodology for the real-time investigation of speech perception, memory, and language processing. Cognitive Psychology, 6, 813-839]. Here, this method is used to examine the psychological validity of models of high-dimensional semantic space. The data strongly suggest that these corpus-based measures of word semantics predict fixation behavior in the visual world and provide further evidence that language-mediated eye movements to objects in the concurrent visual environment are driven by semantic similarity rather than all-or-none categorical knowledge. The data suggest that the visual world paradigm can, together with other methodologies, converge on the evidence that may help adjudicate between different theoretical accounts of the psychological semantics.     

Anticipatory baseline effects and information integration in visual world studies

Researchers often conduct visual world studies to investigate how listeners integrate linguistic information with prior context. Such studies are likely to generate anticipatory baseline effects (ABEs), differences in listeners' expectations about what a speaker might mention that exist before a critical speech stimulus is presented. ABEs show that listeners have attended to and accessed prior contextual information in time to influence the processing of the critical speech stimulus. However, further evidence is required to show that the information actually did influence subsequent processing. ABEs can compromise the validity of inferences about information integration if they are not appropriately controlled. We discuss four solutions: statistical estimation, experimental control, elimination of “on-target” trials, and neutral gaze. An experiment compares the performance of these solutions, and suggests that the elimination of on-target trials introduces bias in the direction of ABEs, due to the statistical phenomenon of regression toward the mean. We conclude that statistical estimation, possibly coupled with experimental control, offers the most valid and least biased solution.     

Preattentive processing of audio-visual emotional signals

Previous research has shown that redundant information in faces and voices leads to faster emotional categorization compared to incongruent emotional information even when attending to only one modality. The aim of the present study was to test whether these crossmodal effects are predominantly due to a response conflict rather than interference at earlier, e.g. perceptual processing stages. In Experiment 1, participants had to categorize the valence and rate the intensity of happy, sad, angry and neutral unimodal or bimodal face–voice stimuli. They were asked to rate either the facial or vocal expression and ignore the emotion expressed in the other modality. Participants responded faster and more precisely to emotionally congruent compared to incongruent face–voice pairs in both the Attend Face and in the Attend Voice condition. Moreover, when attending to faces, emotionally congruent bimodal stimuli were more efficiently processed than unimodal visual stimuli. To study the role of a possible response conflict, Experiment 2 used a modified paradigm in which emotional and response conflicts were disentangled. Incongruency effects were significant even in the absence of response conflicts. The results suggest that emotional signals available through different sensory channels are automatically combined prior to response selection.     

Using the visual world paradigm to study language processing: a review and critical evaluation

We describe the key features of the visual world paradigm and review the main research areas where it has been used. In our discussion we highlight that the paradigm provides information about the way language users integrate linguistic information with information derived from the visual environment. Therefore the paradigm is well suited to study one of the key issues of current cognitive psychology, namely the interplay between linguistic and visual information processing. However, conclusions about linguistic processing (e.g., about activation, competition, and timing of access of linguistic representations) in the absence of relevant visual information must be drawn with caution.     

Task switching and serial memory: looking into the nature of switches and tasks

Task switching research has so far focused on the impact of switching task-sets between two-choice classification tasks that require little or no memory load. Empirical work is lacking however to determine whether the switching cost can be extended to other cognitive activities and to different types of switches. In the present study, switching between the content – verbal to spatial – of the tasks was contrasted with switching cognitive processes — categorization to serial memory. Our pattern of results revealed the absence of local and general switch costs on serial memory tasks, while substantial costs were observed with two-choice judgement tasks. Such a finding challenges the widely accepted assumption that task alternation comes with a considerable cost in performance regardless of the cognitive tasks undertaken. Our results are discussed in context of the predominant models of task switching.     

Does temporal preparation increase the rate of sensory information accumulation?

Recent studies showed that temporal preparation, i.e., the ability to prepare for an upcoming stimulus, improves perceptual processing. The mechanisms underlying this benefit are still controversial. Based upon the theoretical framework of accumulation models, it has been proposed that the accumulation of sensory stimulus information begins earlier when participants are temporally prepared than when they are unprepared. Alternatively, however, temporal preparation might also affect the accumulation rate of sensory information. In the present study, we examined these possibilities. Specifically, in three experiments, we manipulated participants' decision criterion. This manipulation should interact with any experimental manipulation affecting the rate of information processing, but produce additive effects with any manipulation affecting the onset of information accumulation rather than its rate. We obtained additive effects on RT, irrespective of whether the decision criterion was manipulated by increasing catch trial proportion or nogo trial proportion. These results suggest that temporal preparation improves perceptual processing by operating on the onset of sensory information accumulation rather than the rate of sensory information accumulation.     

Effects of luminance change in preview search: Offsets and onsets can be concurrently prioritized but not in isolation

Two experiments are performed to investigate how luminance change contributes to prioritized selection of new over old elements. Experiment 1 demonstrates that observers prioritize items that undergo a luminance change irrespective of the direction of that change. Experiment 2 shows that foreknowledge concerning the direction of luminance change signaling the target does not allow observers to prioritize the selection of luminance onsets over offsets and vice versa. The results suggest that prioritized selection of new over old elements is mediated by a general mechanism that is sensitive to luminance change, irrespective of its direction.     

Attention on our mind: The role of spatial attention in visual working memory

The current study shows that spatial visual attention is used to retrieve information from visual working memory. Participants had to keep four colored circles in visual working memory. While keeping this information in memory we asked whether one of the colors was present in the array. While retrieving this information, on some trials a probe dot was presented. When this probe dot was presented at the location of the color that had to be retrieved, participants responded faster than when it was presented at another location. Our findings further elaborate the role of visual attention in working memory: not only is attention the mechanism by which information is stored into working memory, it is also the mechanism by which information is retrieved from visual working memory.     

The relation between body semantics and spatial body representations

The present study addressed the relation between body semantics (i.e. semantic knowledge about the human body) and spatial body representations, by presenting participants with word pairs, one below the other, referring to body parts. The spatial position of the word pairs could be congruent (e.g. EYE / MOUTH) or incongruent (MOUTH / EYE) with respect to the spatial position of the words’ referents. In addition, the spatial distance between the words’ referents was varied, resulting in word pairs referring to body parts that are close (e.g. EYE / MOUTH) or far in space (e.g. EYE / FOOT). A spatial congruency effect was observed when subjects made an iconicity judgment (Experiments 2 and 3) but not when making a semantic relatedness judgment (Experiment 1). In addition, when making a semantic relatedness judgment (Experiment 1) reaction times increased with increased distance between the body parts but when making an iconicity judgment (Experiments 2 and 3) reaction times decreased with increased distance. These findings suggest that the processing of body-semantics results in the activation of a detailed visuo-spatial body representation that is modulated by the specific task requirements. We discuss these new data with respect to theories of embodied cognition and body semantics.     

Reading action word affects the visual perception of biological motion

In the present study, we investigate whether reading an action-word can influence subsequent visual perception of biological motion. The participant's task was to perceptually judge whether a human action identifiable in the biological motion of a point-light display embedded in a high density mask was present or not in the visual sequence, which lasted for 633 ms on average. Prior to the judgement task, participants were exposed to an abstract verb or an action verb for 500 ms, which was related to the human action according to a congruent or incongruent semantic relation. Data analysis showed that correct judgements were not affected by action verbs, whereas a facilitation effect on response time (49 ms on average) was observed when a congruent action verb primed the judgement of biological movements. In relation with the existing literature, this finding suggests that the perception, the planning and the linguistic coding of motor action are subtended by common motor representations.     

Domain general mechanisms account for imagined transformations of whole body perspective

Three experiments investigated the contribution of domain general processes to imagined transformations of whole body perspective. In a task where participants make left–right judgements about a schematic human figure, reaction times made from the point of view of the figure are longer when the figure does not share the same spatial orientation as the participant and are substantially longer than those made from the participant's own point of view. These phenomena have been attributed to a specialised mechanism for imagined perspective transformations. In the present experiments, the effect of orientation on performance was influenced by factors that affect spatial stimulus–response (S–R) compatibility; it was attenuated with reduced dimensional overlap and reversed by crossed S–R mappings. Performance in a control experiment in which no figure was present suggests that processes moderating S–R mappings may account for the effect of adopting another's point of view. Our findings demonstrate a role for domain general processes in imagined transformations of whole body perspective, imply that egocentric codes for spatial relationships within visually presented bodies are automatically generated, and undermine evidence for a specialised mechanism for imagined perspective transformations.     

Target-flanker discriminability affects conflict size but not sustained suppression

On successive trials, repetitions of irrelevant information often tend to reduce congruency effects as compared to alternations of irrelevant information. The preferred explanation for this congruency modulation is the sustained-suppression hypothesis, suggesting that suppression of the irrelevant information on a given trial perseveres into the subsequent trial. However, in contrast to the generality of this idea, this modulation is only stable when the irrelevant information contains spatial features, which coincided in the existing research with large conflict sizes and response conflicts. In two arrow flanker experiments, we investigated whether the congruency modulation depends on the size of the conflict, by manipulating the saliency of the target (Experiment 1) and the flankers (Experiment 2). Although these manipulations affected the size of the conflict caused by the flankers, neither experiment showed an influence of conflict size on the reduction of the congruency effect for repetitions as compared to alternation of the irrelevant flankers. We conclude that sustained-suppression is not a consequence of large conflicts, at least if sustained-suppression causes the congruency modulation.     

Processing two tasks with varying task order: central stage duration influences central processing order

In a recent study, Sigman and Dehaene (2006, PLoS Biology, 4, 1227–1238) reported that perceptual processing duration influences processing order of two tasks in the psychological refractory period paradigm (PRP). The present study examines whether the duration of central processes also influences processing order. For this purpose, we employed letter tasks with different central processing durations and varied task order in the PRP. In one part of the experiment, a tone discrimination task was combined with a similar time-consuming letter discrimination task. In the other part, the tone task was combined with a more time-consuming letter task, which required a mental rotation of the letter thereby prolonging central processing. If the duration of central processes influences processing order, participants should perform the tone task first more often when it is presented with the more time-consuming mental rotation task, than when it is presented with the similar time-consuming letter task. The results clearly confirm this prediction and thus show that not only perceptual processing duration but also central processing duration affects processing order in a dual-task situation. We suggest that the reported effect mirrors a tendency of participants to minimize total reaction time to both tasks by reducing slack time.     

Distraction by irrelevant sound during foreperiods selectively impairs temporal preparation

The issue of unidimensionality is dealt with in various research areas in the field of Psychology (e.g. conceptual spaces, semantic modeling, psychometrics) and always involves spatial modeling. An investigation of the dimensionality of opposite spatial scales (even basic) has however not yet been carried out. In this paper we look at whether opposite judgments of height, size, width and length (high/low, large/small, wide/narrow, and long/short) imply underlying unidimensional continua. In three experiments, independent ratings for the 8 above mentioned properties were elicited with participants looking at photographic representations of various objects (Study 1), real life objects (Study 2) and spatial extensions in object-independent conditions (Study 3). Explorative and confirmative factor analysis and Andrich Extended Rating Scale Models were applied in order to determine whether the ratings referred to opposite scales on the same linear continuum. Results from the three studies consistently revealed that this is not the case. A joint analysis of the data showed interesting interactions between the spatial properties analyzed suggesting a possible explanation for the lack of unidimensionality.     

Reinforcement of inhibitory positional priming by spatial working memory contents

This study examined whether positional priming of pop-out is modulated by spatial working memory (WM). In Experiments 1A and 1B, observers were presented with a to-be-memorized item (Experiment 1A: triangle; Experiment 1B: circle) prior to performing a block of search trials, with triangular stimulus configurations (one target, two distractors) on an underlying elliptical frame. Each search block was followed by a memory test (i.e., memory conditions). In Experiment 1C, the presentation of the memory item (triangle) was immediately followed by the memory test, and the search block was performed after this test (i.e., no-memory condition). It was found that targets presented at a previous target were detected faster and targets presented at a previous distractor location slower, relative to a previous empty (‘neutral’) stimulus location, respectively. While facilitatory position priming was uninfluenced by the memory manipulation, inhibitory positional priming was larger in the memory than no-memory condition, but only when the memory stimulus (triangle) matched the (triangle) configuration formed by the three search items. This finding of larger inhibitory priming under memory conditions provides further evidence that positional priming, rather than being purely bottom-up determined, is open to top-down influences.     

Explicit and motoric dependent measures of geographical slant are dissociable: A reassessment of the findings of Durgin, Hajnal, Li, Tonge, and Stigliani (2010)

Durgin et al. (2010) argued that the apparent accuracy of the palmboard measure of geographical slant is accidental and reflects limitations in wrist flexion that reduce palmboard adjustments by just the right amount given the perceptual overestimations upon which they are based. This account is inconsistent with findings that verbal reports and palmboard adjustments are dissociable. In addition to previous evidence found for such dissociation, Durgin et al. also found verbal/palmboard dissociations in Experiment 2. Experiments 1 and 3 of Durgin et al. lacked verbal reports and instead compared palmboard adjustments to free-hand estimates in the context of small wooden surfaces. These experiments are not relevant to the issue of verbal/palmboard dissociability. Across studies, the accuracy of Durgin et al.'s palmboard implementation is far less than that found by others (Feresin & Agostini, 2007). The design of Durgin et al.'s Experiment 5 misrepresented the experimental conditions of Creem and Proffitt (1998), and consequently, the findings of this study have no bearing on the issue at hand.     

Object-based attention: sensory enhancement or scanning prioritization

The three experiments reported here test whether object-modulated attentional spreading can be obtained when the target location is 100% certain. Experiment 1 uses the reaction time (RT)-based flanker task similar to Shomstein and Yantis (2002), and replicates the null result of the object-modulated attentional spreading. RT and accuracy (ACC) have been shown to reflect different processes: postperceptual decision vs. perceptual process (Santee & Egeth, 1982). Experiment 2 adopts the data-limited ACC-based measure and reports that attention could spread within the attended object. To avoid ceiling effects, Experiment 3 adjusts the presentation time based on the trials where the target and flankers were compatible and on the same objects, and provides the convergent evidence supporting the object-modulated attentional spreading. These results suggest that because the RT-based measure is less sensitive in reflecting the quality of perceptual representations, it is not sufficiently a strong evidence to distinguish between sensory enhancement and scanning prioritization accounts.     

Content-context binding in verbal working memory updating: on-line and off-line effects

The role of content–context binding in working memory updating has received only marginal interest, despite its undoubted relevance for the updating process. In the classical updating paradigm, the main focus has been on the process of discarding information when new information is presented. Its efficacy has been determined by measurement of the accuracy of recalled, updated information. In the current study we measured the working memory updating process directly, employing a dynamic memory task composed of alternating sequences of learning, active maintenance and updating phases. Binding was manipulated by changing the type of updating (total or partial) and the strength of the perceptual connection between items. The on-line updating process and the off-line efficacy were separately analysed. Both on-line and off-line measures indicated that partial updating conditions, where the operation of updating content–context binding was required, were more demanding than both total updating conditions and memory and maintenance phases. Our results suggest that working memory updating can be identified not only as a process of substitution of information, but also as inhibition of no longer relevant information and, above all, as a binding updating.     

Target-directed visual attention is a prerequisite for action-specific perception

Witt et al. (2008) have recently shown that golfers who putt with more success perceive the hole to be bigger than golfers who putt with less success. In three experiments, we systematically examined whether this phenomenon, labelled action-specific perception, depends on directing visual attention towards the action target. In Experiment 1 we replicated previously reported action-specific effects on perception in golf putting. In 3 and 4 we directly assessed whether action-specific effects on perception in golf putting are dependent on focusing visual attention on the target. To this end, the participants performed the putting task while visual attention towards the target was either completely withheld (Experiment 2) or divided over the target and other task-relevant objects (Experiment 3). No action-specific effects were found when visual attention towards the action target was occluded or partially diverted from the target. Together, our results provide evidence to suggest that focusing visual attention on the target while performing the action is a prerequisite for the emergence of action-specific perception.