Electrophysiological evidence for the "missing link" in crossmodal attention.

Orienting attention involuntarily to the location of a sensory event influences responses to subsequent stimuli that appear in different modalities with one possible exception: orienting attention involuntarily to a sudden light sometimes fails to affect responses to subsequent sounds (e.g., Spence & Driver, 1997). Here we investigated the effects of involuntary attention to a brief flash on the processing of subsequent sounds in a design that eliminates stimulus-response compatibility effects and criterion shifts as confounding factors. In addition, the neural processes mediating crossmodal attention were studied by recording event-related brain potentials. Our data show that orienting attention to the location of a spatially nonpredictive visual cue modulates behavioural and neural responses to subsequent auditory targets when the stimulus onset asynchrony is short (between 100 and 300 ms). These findings are consistent with the hypothesis that involuntary shifts of attention are controlled by supramodal brain mechanisms rather than by modality-specific ones.     

Cross-modal exogenous attention and distance effects in vision and hearing

In the present study we investigated cross-modal orienting in vision and hearing by using a cueing task with four different horizontal locations. Our main interest concerned cue–target distance effects, which might further our insight in the characteristics of cross-modal spatial attention mechanisms. A very consistent pattern was observed for both the unimodal (cue and target were both visual or auditory) and the cross-modal conditions (cue and target from different modalities). RTs to valid trials were faster than for invalid trials, and, most interestingly, there was a distance effect: RTs increased with greater cue–target distance. This applied to detection of visual targets and to localisation of both visual and auditory targets. The time interval between cue and target was also varied. Surprisingly, there was no indication of inhibition of return even with the longest cue–target intervals. In order to assess the role of endogenous (strategic) factors in exogenous spatial attention we increased in two additional experiments the cue validity from 25% to 80%. This appeared to have no large influence on the cueing pattern in both the detection and localisation tasks. Currently, it is assumed that spatial attention is organised in multiple strongly linked modality-specific systems. The foregoing results are discussed with respect to this supposed organisation.     

跨通道的内源性选择注意

该实验利用空间线索技术研究跨视觉和听觉通道的内源性选择性注意,实验结果表明视觉中央线索能可靠地引导出内源性视觉选择性注意,听觉中央线索在较长的ＳＯＡ（至少５００ｍｓ）条件下也能引导出内源性视觉选择性注意,支持视觉和听觉具有特异性的注意加工通道,但两者之间存在相互连接的假说。     

When cross-modal spatial attention fails

There is now convincing evidence that an involuntary shift of spatial attention to a stimulus in one modality can affect the processing of stimuli in other modalities, but inconsistent findings across different paradigms have led to controversy. Such inconsistencies have important implications for theories of cross-modal attention. The authors investigated why orienting attention to a visual event sometimes influences responses to subsequent sounds and why it sometimes fails to do so. They examined visual-cue-on-auditory-target effects in two paradigms--implicit spatial discrimination (ISD) and orthogonal cuing (OC)--that have yielded conflicting findings in the past. Consistent with previous research, visual cues facilitated responses to same-side auditory targets in the ISD paradigm but not in the OC paradigm. Furthermore, in the ISD paradigm, visual cues facilitated responses to auditory targets only when the targets were presented directly at the cued location, not when they appeared above or below the cued location. This pattern of results confirms recent claims that visual cues fail to influence responses to auditory targets in the OC paradigm because the targets fall outside the focus of attention. (PsycINFO Database Record (c) 2008 APA, all rights reserved).     

内源性注意与外源性注意的跨通道比较

该研究利用空间线索技术的实验模式,考察跨视觉和听觉通道的内源性选择注意与外源性选择性注意的相互关系。实验结果表明：(1)听觉中央线索在较长的SOA(至少500毫秒)条件下,可以引导内源性空间选择性注意;同时外周线索突现也能自动化地吸引部分注意资源。(2)听觉和视觉选择注意是分离的加工通道,但二者之间存在相互联系。     

The cognitive locus of distraction by acoustic novelty in the cross-modal oddball task

Unexpected stimuli are often able to distract us away from a task at hand. The present study seeks to explore some of the mechanisms underpinning this phenomenon. Studies of involuntary attention capture using the oddball task have repeatedly shown that infrequent auditory changes in a series of otherwise repeating sounds trigger an automatic response to the novel or deviant stimulus. This attention capture has been shown to disrupt participants' behavioral performance in a primary task, even when distractors and targets are asynchronous and presented in distinct sensory modalities. This distraction effect is generally considered as a by-product of the capture of attention by the novel or deviant stimulus, but the exact cognitive locus of this effect and the interplay between attention capture and target processing has remained relatively ignored. The present study reports three behavioral experiments using a cross-modal oddball task to examine whether the distraction triggered by auditory novelty affects the processing of the target stimuli. Our results showed that variations in the demands placed on the visual analysis (Experiment 1) or categorical processing of the target (Experiment 2) did not impact on distraction. Instead, the cancellation of distraction by the presentation of an irrelevant visual stimulus presented immediately before the visual target (Experiment 3) suggested that distraction originated in the shifts of attention occurring between attention capture and the onset of the target processing. Possible accounts of these shifts are discussed.     

The neural correlates of orienting to walking direction in 6‐month‐old infants: An ERP study

The ability to detect social signals represents a first step to enter our social world. Behavioral evidence has demonstrated that 6‐month‐old infants are able to orient their attention toward the position indicated by walking direction, showing faster orienting responses toward stimuli cued by the direction of motion than toward uncued stimuli. The present study investigated the neural mechanisms underpinning this attentional priming effect by using a spatial cueing paradigm and recording EEG (Geodesic System 128 channels) from 6‐month‐old infants. Infants were presented with a central point‐light walker followed by a single peripheral target. The target appeared randomly at a position either congruent or incongruent with the walking direction of the cue. We examined infants' target‐locked event‐related potential (ERP) responses and we used cortical source analysis to explore which brain regions gave rise to the ERP responses. The P1 component and saccade latencies toward the peripheral target were modulated by the congruency between the walking direction of the cue and the position of the target. Infants' saccade latencies were faster in response to targets appearing at congruent spatial locations. The P1 component was larger in response to congruent than to incongruent targets and a similar congruency effect was found with cortical source analysis in the parahippocampal gyrus and the anterior fusiform gyrus. Overall, these findings suggest that a type of biological motion like the one of a vertebrate walking on the legs can trigger covert orienting of attention in 6‐month‐old infants, enabling enhancement of neural activity related to visual processing of potentially relevant information as well as a facilitation of oculomotor responses to stimuli appearing at the attended location.     

Multisensory numerosity judgments for visual and tactile stimuli

To date, numerosity judgments have been studied only under conditions of unimodal stimulus presentation. It is therefore unclear whether the same limitations on correctly reporting the number of unimodal visual or tactile stimuli presented in a display might be expected under conditions in which participants have to count stimuli presented simultaneously in two or more different sensory modalities. In Experiment 1, we investigated numerosity judgments using both unimodal and bimodal displays consisting of one to six vibrotactile stimuli (presented over the body surface) and one to six visual stimuli (seen on the body via mirror reflection). Participants had to count the number of stimuli regardless of their modality of presentation. Bimodal numerosity judgments were significantly less accurate than predicted on the basis of an independent modality-specific resources account, thus showing that numerosity judgments might rely on a unitary amodal system instead. The results of a second experiment demonstrated that divided attention costs could not account for the poor performance in the bimodal conditions of Experiment 1. We discuss these results in relation to current theories of cross-modal integration and to the cognitive resources and/or common higher order spatial representations possibly accessed by both visual and tactile stimuli.     

Attention: reaction time and accuracy reveal different mechanisms

The authors propose that there are 2 different mechanisms whereby spatial cues capture attention. The voluntary mechanism is the strategic allocation of perceptual resources to the location most likely to contain the target. The involuntary mechanism is a reflexive orienting response that occurs even when the spatial cue does not indicate the probable target location. Voluntary attention enhances the perceptual representation of the stimulus in the cued location relative to other locations. Hence, voluntary attention affects performance in experiments designed around both accuracy and reaction time. Involuntary attention affects a decision as to which location should be responded to. Because involuntary attention does not change the perceptual representation, it affects performance in reaction time experiments but not accuracy experiments. The authors obtained this pattern of results in 4 different versions of the spatial cuing paradigm.     

Reward expectation influences audiovisual spatial integration

In order to determine the spatial location of an object that is simultaneously seen and heard, the brain assigns higher weights to the sensory inputs that provide the most reliable information. For example, in the well-known ventriloquism effect, the perceived location of a sound is shifted toward the location of a concurrent but spatially misaligned visual stimulus. This perceptual illusion can be explained by the usually much higher spatial resolution of the visual system as compared to the auditory system. Recently, it has been demonstrated that this cross-modal binding process is not fully automatic, but can be modulated by emotional learning. Here we tested whether cross-modal binding is similarly affected by motivational factors, as exemplified by reward expectancy. Participants received a monetary reward for precise and accurate localization of brief auditory stimuli. Auditory stimuli were accompanied by task-irrelevant, spatially misaligned visual stimuli. Thus, the participants’ motivational goal of maximizing their reward was put in conflict with the spatial bias of auditory localization induced by the ventriloquist situation. Crucially, the amounts of expected reward differed between the two hemifields. As compared to the hemifield associated with a low reward, the ventriloquism effect was reduced in the high-reward hemifield. This finding suggests that reward expectations modulate cross-modal binding processes, possibly mediated via cognitive control mechanisms. The motivational significance of the stimulus material, thus, constitutes an important factor that needs to be considered in the study of top-down influences on multisensory integration.     

视听刺激突显性对视觉主导效应的影响

本研究使用空间任务-转换范式,控制视、听刺激的突显性,探讨自下而上注意对视觉主导效应的影响。结果表明视、听刺激突显性显著地影响视觉主导效应,实验一中当听觉刺激为高突显性时,视觉主导效应显著减弱。实验二中当听觉刺激为高突显性并且视觉刺激为低突显性时,视觉主导效应进一步减弱但依然存在。结果支持偏向竞争理论,在跨通道视听交互过程中视觉刺激更具突显性,在多感觉整合过程中更具加工优势。     

The time course of intended and unintended allocation of attention

According to the contingent involuntary orienting hypothesis, only stimuli that match the attentional control settings based on intentions capture attention. In contrast, the surprise-capture hypothesis states that expectancy-discrepant stimuli can capture attention even if they do not match the control settings, implying unintended capture. The purpose of this study is to investigate whether unintended and intended attentional shifts are characterized by different time courses, indicating different underlying mechanisms. An unintended attentional shift was tested by the first, unannounced presentation of a color singleton at the location of a visual search target, and intended shifts by the following repeated presentations of a predictive singleton. Differences in time course were revealed by varying the stimulus onset asynchrony (SOA) between singleton and target. Results showed that accuracy with expected singletons was barely affected by SOA, whereas SOA strongly affected accuracy with the unexpected singleton. The results are interpreted as supporting the surprise-capture hypothesis. It is furthermore argued that a division of labor between contingent capture and surprise in the control of attention supports adaptive behavior.     

Out with the new,in with the old: Exogenous orienting to locations with physically constant stimulation

Dominant methods of investigating exogenous orienting presume that attention is captured most effectively at locations containing new events. This is evidenced by the ubiquitous use of transient stimuli as cues in the literature on exogenous orienting. In the present study, we showed that attention can be oriented exogenously toward a location containing a completely unchanging stimulus by modifying Posner’s landmark exogenous spatial-cueing paradigm. Observers searched a six-element array of placeholder stimuli for an onset target. The target was preceded by a decrement in luminance to five of the six placeholders, such that one location remained physically constant. This “nonset” stimulus (so named to distinguish it from a traditional onsetting transient) acted as an exogenous cue, eliciting patterns of facilitation and inhibition at the nonset location and demonstrating that exogenous orienting is not always evident at the location of a visual transient. This method eliminates the decades-long confounding of orienting to a location with the processing of new events at that location, permitting alternative considerations of the nature of attentional selection.     

返回抑制时程的三因素理论及其发展

综述了返回抑制(Inhibition of return,IOR)出现时程机制的三因素理论内容,支持该理论的实验证据,以及对理论的质疑和发展。该理论认为空间定向收益、出现检测缺失和空间选择收益三个因素共同决定了IOR出现的时程及机制。研究者通过操控任务加工水平或将实验范式相结合的方式对三因素理论提出了质疑和发展。许多关键问题如三个因素存在的认知神经科学证据、出现检测缺失和空间选择收益发挥作用的方式等是未来研究的重点和方向。     

Cross-modality correspondence between pitch and spatial location modulates attentional orienting

The brain constantly integrates incoming signals across the senses to form a cohesive view of the world. Most studies on multisensory integration concern the roles of spatial and temporal parameters. However, recent findings suggest cross-modal correspondences (eg high-pitched sounds associated with bright, small objects located high up) also affect multisensory integration. Here, we focus on the association between auditory pitch and spatial location. Surprisingly little is known about the cognitive and perceptual roots of this phenomenon, despite its long use in ergonomic design. In a series of experiments, we explore how this cross-modal mapping affects the allocation of attention with an attentional cuing paradigm. Our results demonstrate that high and low tones induce attention shifts to upper or lower locations, depending on pitch height. Furthermore, this pitch-induced cuing effect is susceptible to contextual manipulations and volitional control. These findings suggest the cross-modal interaction between pitch and location originates from an attentional level rather than from response mapping alone. The flexible contextual mapping between pitch and location, as well as its susceptibility to top-down control, suggests the pitch-induced cuing effect is primarily mediated by cognitive processes after initial sensory encoding and occurs at a relatively late stage of voluntary attention orienting.     

Timing is everything: Changes in presentation rate have opposite effects on auditory and visual implicit statistical learning

Implicit statistical learning (ISL) is exclusive to neither a particular sensory modality nor a single domain of processing. Even so, differences in perceptual processing may substantially affect learning across modalities. In three experiments, statistically equivalent auditory and visual familiarizations were presented under different timing conditions that either facilitated or disrupted temporal processing (fast or slow presentation rates). We find an interaction of rate and modality of presentation: At fast rates, auditory ISL was superior to visual. However, at slow presentation rates, the opposite pattern of results was found: Visual ISL was superior to auditory. Thus, we find that changes to presentation rate differentially affect ISL across sensory modalities. Additional experiments confirmed that this modality-specific effect was not due to cross-modal interference or attentional manipulations. These findings suggest that ISL is rooted in modality-specific, perceptually based processes.     

Timing is everything: changes in presentation rate have opposite effects on auditory and visual implicit statistical learning

Implicit statistical learning (ISL) is exclusive to neither a particular sensory modality nor a single domain of processing. Even so, differences in perceptual processing may substantially affect learning across modalities. In three experiments, statistically equivalent auditory and visual familiarizations were presented under different timing conditions that either facilitated or disrupted temporal processing (fast or slow presentation rates). We find an interaction of rate and modality of presentation: At fast rates, auditory ISL was superior to visual. However, at slow presentation rates, the opposite pattern of results was found: Visual ISL was superior to auditory. Thus, we find that changes to presentation rate differentially affect ISL across sensory modalities. Additional experiments confirmed that this modality-specific effect was not due to cross-modal interference or attentional manipulations. These findings suggest that ISL is rooted in modality-specific, perceptually based processes.     

The ventriloquist effect does not depend on the direction of deliberate visual attention

It is well known that discrepancies in the location of synchronized auditory and visual events can lead to mislocalizations of the auditory source, so-called ventriloquism. In two experiments, we tested whether such cross-modal influences on auditory localization depend on deliberate visual attention to the biasing visual event. In Experiment 1, subjects pointed to the apparent source of sounds in the presence or absence of a synchronous peripheral flash. They also monitored for target visual events, either at the location of the peripheral flash or in a central location. Auditory localization was attracted toward the synchronous peripheral flash, but this was unaffected by where deliberate visual attention was directed in the monitoring task. In Experiment 2, bilateral flashes were presented in synchrony with each sound, to provide competing visual attractors. When these visual events were equally salient on the two sides, auditory localization was unaffected by which side subjects monitored for visual targets. When one flash was larger than the other, auditory localization was slightly but reliably attracted toward it, but again regardless of where visual monitoring was required. We conclude that ventriloquism largely reflects automatic sensory interactions, with little or no role for deliberate spatial attention.     

Is “Inhibition of Return” due to the inhibition of the return of attention?

Inhibition of Return (IOR) is usually explained in terms of orienting–reorienting of attention, emphasizing an underlying mechanism that inhibits the return of attention to previously selected locations. Recent data challenge this explanation to the extent that the IOR effect is observed at the location where attention is oriented to, where no reorienting of attention is needed. To date, these studies have involved endogenous attentional selection of attention and thus indicate a dissociation between the voluntary attention of spatial attention and the IOR effect. The present work demonstrates a dissociation between the involuntary orienting of spatial attention and the IOR effect. We combined nonpredictive peripheral cues with nonpredictive central orienting cues (either arrows or gaze). The IOR effect was observed to operate independent of involuntary spatial orienting. These data speak against the “reorienting hypothesis” of IOR. We suggest an alternative explanation whereby the IOR effect reflects a cost in detecting a new event (the target) at the location where another event (a cue) was coded before.     

Behold the voice of wrath: cross-modal modulation of visual attention by anger prosody

Emotionally relevant stimuli are prioritized in human information processing. It has repeatedly been shown that selective spatial attention is modulated by the emotional content of a stimulus. Until now, studies investigating this phenomenon have only examined within-modality effects, most frequently using pictures of emotional stimuli to modulate visual attention. In this study, we used simultaneously presented utterances with emotional and neutral prosody as cues for a visually presented target in a cross-modal dot probe task. Response times towards targets were faster when they appeared at the location of the source of the emotional prosody. Our results show for the first time a cross-modal attentional modulation of visual attention by auditory affective prosody.