On the role of eye movements and saccade preparation in generating auditory inhibition of return.

In three experiments, listeners were required to either localize or identify the second of two successive sounds. The first sound (the cue) and the second sound (the target) could originate from either the same or different locations, and the interval between the onsets of the two sounds (Stimulus Onset Asynchrony, SOA) was varied. Sounds were presented out of visual range at 135 azimuth left or right. In Experiment 1, localization responses were made more quickly at 100 ms SOA when the target sounded from the same location as the cue (i.e., a facilitative effect), and at 700 ms SOA when the target and cue sounded from different locations (i.e., an inhibitory effect). In Experiments 2 and 3, listeners were required to monitor visual information presented directly in front of them at the same time as the auditory cue and target were presented behind them. These two experiments differed in that in order to perform the visual task accurately in Experiment 3, eye movements to visual stimuli were required. In both experiments, a transition from facilitation at a brief SOA to inhibition at a longer SOA was observed for the auditory task. Taken together these results suggest that location-based auditory IOR is not dependent on either eye movements or saccade programming to sound locations.     

试次历史对跨通道非空间返回抑制的影响

个体对刺激的反应不仅受刺激本身的影响, 还会受到先前刺激的影响, 表现为对当前试次中刺激的反应会受到前一试次的影响, 即试次历史。本研究采用“线索-中性线索-靶子”范式探讨前一试次有效性对跨通道的非空间返回抑制的影响。实验1通过连续两个试次间的线索有效性考察在跨通道非空间返回抑制中试次历史的影响。为了在跨通道非空间返回抑制中减小试次历史的影响, 实验2通过延长试次间时间间隔考察跨通道非空间返回抑制中试次历史的作用是否减小。结果发现, 前一试次线索无效时, 当前试次中的返回抑制效应量显著小于前一试次有效时, 这种影响会根据试次中线索和靶子通道的不同而不同。并且当延长试次间的时间间隔可以有效地减少前一试次对当前试次的影响。因此本研究表明, 试次历史能够对跨通道非空间返回抑制产生影响, 并且这种影响可以通过增大试次间时间间隔来减小。     

偏差声音干扰刺激对视觉注意网络系统的影响

结合注意网络测量工具和听视ODDBALL干扰范式,考察偏差声音干扰刺激的出现对注意网络系统不同功能的影响。结果发现,声音刺激序列下,视觉线索唤醒效应均消失; 标准声音刺激下,不存在线索的脱离和转移效应; 偏差声音刺激下,出现线索脱离和转移效应。研究表明,注意网络中的警觉功能具有超通道加工机制,偏差刺激下视觉注意系统中注意定向和线索转移效应的出现表明偏差干扰效应主要由偏差刺激引起的注意脱离和注意转移损耗部分引起。     

时间结构和速度线索对碰撞时间估计的影响

估计被遮挡的运动物体何时到达某一特定位置称为碰撞时间估计。本研究利用遮挡范式,通过控制时间结构、物理速度及运动过程是否可见设计两个（2物理速度×3时间结构）实验来探究三者对碰撞时间估计的影响。结果：当时间结构一致时,不论物理速度快慢及是否存在视觉速度线索,被试的绩效都没有显著差异;当时间结构不一致时,不论物理速度快慢,存在视觉速度条件下被试的绝对偏差小于无视觉速度条件。结论：时间结构一致时,被试主要使用时间线索进行碰撞时间估计;时间结构不一致时,被试通过整合时间结构和视觉速度线索进行估计。     

Age-related changes in event-cued visual and auditory prospective memory proper

We rely upon prospective memory proper (ProMP) to bring back to awareness previously formed plans and intentions at the right place and time, and to enable us to act upon those plans and intentions. To examine age-related changes in ProMP, younger and older participants made decisions about simple stimuli (ongoing task) and at the same time were required to respond to a ProM cue, either a picture (visually cued ProM test) or a sound (auditorily cued ProM test), embedded in a simultaneously presented series of similar stimuli (either pictures or sounds). The cue display size or loudness increased across trials until a response was made. The cue size and cue loudness at the time of response indexed ProMP. The main results showed that both visual and auditory ProMP declined with age, and that such declines were mediated by age declines in sensory functions (visual acuity and hearing level), processing resources, working memory, intelligence, and ongoing task resource allocation.     

Visual inertia of rotating 3-D objects

Five experiments were designed to determine whether a rotating, transparent 3-D cloud of dots (simulated sphere) could influence the perceived direction of rotation of a subsequent sphere. Experiment 1 established conditions under which the direction of rotation of a virtual sphere was perceived unambiguously. When a near-far luminance difference and perspective depth cues were present, observers consistently saw the sphere rotate in the intended direction. In Experiment 2, a near-far luminance difference was used to create an unambiguous rotation sequence that was followed by a directionally ambiguous rotation sequence that lacked both the near-far luminance cue and the perspective cue. Observers consistently saw the second sequence as rotating in the same direction as the first, indicating the presence of 3-D visual inertia. Experiment 3 showed that 3-D visual inertia was sufficiently powerful to bias the perceived direction of a rotation sequence made unambiguous by a near-far luminance cue. Experiment 5 showed that 3-D visual inertia could be obtained using an occlusion depth cue to create an unambiguous inertia-inducing sequence. Finally, Experiments 2, 4, and 5 all revealed a fast-decay phase of inertia that lasted for approximately 800 msec, followed by an asymptotic phase that lasted for periods as long as 1,600 msec. The implications of these findings are examined with respect to motion mechanisms of 3-D visual inertia.     

High-intensity sound increases the size of visually perceived objects

The effect of audiovisual interactions on size perception has yet to be examined, despite its fundamental importance in daily life. Previous studies have reported that object length can be estimated solely on the basis of the sounds produced when an object is dropped. Moreover, it has been shown that people typically and easily perceive the correspondence between object sizes and sound intensities. It is therefore possible that auditory stimuli may act as cues for object size, thereby altering the visual perception of size. Thus, in the present study we examined the effects of auditory stimuli on the visual perception of size. Specifically, we investigated the effects of the sound intensity of auditory stimuli, the temporal window of audiovisual interactions, and the effects of the retinal eccentricity of visual stimuli. The results indicated that high-intensity auditory stimuli increased visually perceived object size, and that this effect was especially strong in the peripheral visual field. Additional consideration indicated that this effect on the visual perception of size is induced when the cue reliability is relatively higher for the auditory than for the visual stimuli. In addition, we further suggest that the cue reliabilities of visual and auditory stimuli relate to retinal eccentricity and sound intensity, respectively.     

Estimating the relative weights of visual and auditory tau versus heuristic-based cues for time-to-contact judgments in realistic,familiar scenes by older and younger adults

Estimating time to contact (TTC) involves multiple sensory systems, including vision and audition. Previous findings suggested that the ratio of an object’s instantaneous optical size/sound intensity to its instantaneous rate of change in optical size/sound intensity (τ) drives TTC judgments. Other evidence has shown that heuristic-based cues are used, including final optical size or final sound pressure level. Most previous studies have used decontextualized and unfamiliar stimuli (e.g., geometric shapes on a blank background). Here we evaluated TTC estimates by using a traffic scene with an approaching vehicle to evaluate the weights of visual and auditory TTC cues under more realistic conditions. Younger (18–39 years) and older (65+ years) participants made TTC estimates in three sensory conditions: visual-only, auditory-only, and audio–visual. Stimuli were presented within an immersive virtual-reality environment, and cue weights were calculated for both visual cues (e.g., visual τ, final optical size) and auditory cues (e.g., auditory τ, final sound pressure level). The results demonstrated the use of visual τ as well as heuristic cues in the visual-only condition. TTC estimates in the auditory-only condition, however, were primarily based on an auditory heuristic cue (final sound pressure level), rather than on auditory τ. In the audio–visual condition, the visual cues dominated overall, with the highest weight being assigned to visual τ by younger adults, and a more equal weighting of visual τ and heuristic cues in older adults. Overall, better characterizing the effects of combined sensory inputs, stimulus characteristics, and age on the cues used to estimate TTC will provide important insights into how these factors may affect everyday behavior.     

A Study of Happiness and Unhappiness in the Childhood and Adolescence of Girls in Different Cultures

Experiments were reviewed which dealt with the effect of an auditory cue stimulus on sensitivity to a visual detection stimulus, and vice versa. In most cases the heteromodal cue was temporally varied around the time of occurrence of the detection stimulus.

Different results were reported when a prolonged cue was used (which overlapped the detection stimulus when the onsets of both were not simultaneous) than when a brief-duration cue was used: e.g., in the former case, but not in the latter, sensitivity to the detection stimulus was affected by the intensity of a clearly perceptible heteromodal cue. Es tended to explain the effects of prolonged cue in “neurophysiological-interaction” terms and of brief-duration cue in “attentional” terms. Delta t (difference between times of detection-stimulus and cue onset) was most likely to affect sensitivity to the detection stimulus when marker stimuli reliably informed S when to attend for the detection stimulus, but did not exactly define its time of possible occurrence. A consistent delta-t effect was that a light-cued sound stimulus was best detected when the flash preceded it by 0 to 500 msec.     

Evidence for the automatic processing of prelexical codes in an orthographic but not a phonological task

The automatic activation of phonological and orthographic information in auditory and visual word processing was examined using a task-set procedure. Participants engaged in a phonological task (i.e., determining whether the letter “a” in a word sounded like /e/ or /æ/) or an orthographic task (i.e., determining whether the sound /s/ in a word was spelled with an “s” or a “c”). Participants were cued regarding which task to perform simultaneously with, or 750 ms before, a clear or degraded target. The stimulus clarity effect (i.e., clear words responded to faster than degraded words) was absorbed into the time that it took participants to identify the task on the basis of the cue in a simultaneous cue–target as compared to a delayed cue–target condition, but only for the orthographic task. These data are consistent with the claim that prelexical processing occurs in a capacity-free manner upon stimulus presentation when participants are trying to extract orthographic codes from words presented in the visual and auditory modalities. Such affirmative data were not obtained when participants attempted to extract phonological codes from words, since here the effects of stimulus clarity and cue delay were additive.     

Multisensory-mediated auditory localization

Multisensory integration is a powerful mechanism for maximizing sensitivity to sensory events. We examined its effects on auditory localization in healthy human subjects. The specific objective was to test whether the relative intensity and location of a seemingly irrelevant visual stimulus would influence auditory localization in accordance with the inverse effectiveness and spatial rules of multisensory integration that have been developed from neurophysiological studies with animals [Stein and Meredith, 1993 The Merging of the Senses (Cambridge, MA: MIT Press)]. Subjects were asked to localize a sound in one condition in which a neutral visual stimulus was either above threshold (supra-threshold) or at threshold. In both cases the spatial disparity of the visual and auditory stimuli was systematically varied. The results reveal that stimulus salience is a critical factor in determining the effect of a neutral visual cue on auditory localization. Visual bias and, hence, perceptual translocation of the auditory stimulus appeared when the visual stimulus was supra-threshold, regardless of its location. However, this was not the case when the visual stimulus was at threshold. In this case, the influence of the visual cue was apparent only when the two cues were spatially coincident and resulted in an enhancement of stimulus localization. These data suggest that the brain uses multiple strategies to integrate multisensory information.     

Crossmodal spatial attention shift produced by centrally presented gaze cues1

Abstract: Despite previous failures to identify visual‐upon‐auditory spatial‐cuing effects, recent studies have demonstrated that the abrupt onset of a lateralized visual stimulus triggers a shift of spatial attention in response to auditory judgment. Nevertheless, whether a centrally presented visual stimulus orients auditory attention remained unclear. The present study investigated whether centrally presented gaze cues trigger a reflexive shift of attention in response to auditory judgment. Participants fixated on a schematic face in which the eyes looked left or right (the cue). A target sound was then presented to the left or right of the cue. Participants judged the direction of the target as quickly as possible. Even though participants were told that the gaze direction did not predict the direction of the target, the response time was significantly faster when the gaze was in the target direction than when it was in the non‐target direction. These findings provide initial evidence for visual‐upon‐auditory spatial‐cuing effects produced by centrally presented cues, suggesting that a reflexive crossmodal shift of attention does occur with a centrally presented visual stimulus.     

The importance of "what": Infants use featural information to index events

Dynamic spatial indexing is the ability to encode, remember, and track the location of complex events. For example, in a previous study, 6-month-old infants were familiarized to a toy making a particular sound in a particular location, and later they fixated that empty location when they heard the sound presented alone (Journal of Experimental Psychology: General, 2004, Vol. 133, pp. 46-62). The basis and developmental trajectory of this ability are currently unclear. We investigated dynamic spatial indexing across the first year after birth and tested the hypothesis that the structure of visual cues supports infants' learning of sound and location associations. In our study, 3-, 6-, and 10-month-olds were tested in a dynamic spatial indexing eye movement paradigm that paired two sounds with two locations. In one condition, these were reliably paired with two sets of visual features (two toys condition), replicating the original studies. We also presented a single set of visual cues in both locations (one toy condition) and multiple sets of visual features in both locations (six toys condition). Infants from 3months of age onward showed evidence of dynamic spatial indexing in the two toys condition, but only the 10-month-olds succeeded in the one toy and six toys conditions. We argue that this may reflect a broader developmental trajectory, whereby infants first make use of multiple cue integration but with age are able to learn from a narrow set of cues.     

Fast temporal dynamics of visual cue integration

The integration of information from different sensors, cues, or modalities lies at the very heart of perception. We are studying adaptive phenomena in visual cue integration. To this end, we have designed a visual tracking task, where subjects track a target object among distractors and try to identify the target after an occlusion. Objects are defined by three different attributes (color, shape, size) which change randomly within a single trial. When the attributes differ in their reliability (two change frequently, one is stable), our results show that subjects dynamically adapt their processing. The results are consistent with the hypothesis that subjects rapidly re-weight the information provided by the different cues by emphasizing the information from the stable cue. This effect seems to be automatic, ie not requiring subjects' awareness of the differential reliabilities of the cues. The hypothesized re-weighting seems to take place in about 1 s. Our results suggest that cue integration can exhibit adaptive phenomena on a very fast time scale. We propose a probabilistic model with temporal dynamics that accounts for the observed effect.     

Multidimensional processing of dynamic sounds: more than meets the ear

Strong cross-modal interactions exist between visual and auditory processing. The relative contributions of perceptual versus decision-related processes to such interactions are only beginning to be understood. We used methodological and statistical approaches to control for potential decision-related contributions such as response interference, decisional criterion shift, and strategy selection. Participants were presented with rising-, falling-, and constant-amplitude sounds and were asked to detect change (increase or decrease) in sound amplitude while ignoring an irrelevant visual cue of a disk that grew, shrank, or stayed constant in size. Across two experiments, testing context was manipulated by varying the grouping of visual cues during testing, and cross-modal congruency showed independent perceptual and decision-related effects. Whereas a change in testing context greatly affected criterion shifts, cross-modal effects on perceptual sensitivity remained relatively consistent. In general, participants were more sensitive to increases in sound amplitude and less sensitive to sounds paired with dynamic visual cues. As compared with incongruent visual cues, congruent cues enhanced detection of amplitude decreases, but not increases. These findings suggest that the relative contributions of perceptual and decisional processing and the impacts of these processes on cross-modal interactions can vary significantly depending on asymmetries in within-modal processing, as well as consistencies in cross-modal dynamics.     

Peak velocity as a cue in audiovisual synchrony perception of rhythmic stimuli

This study investigated audiovisual synchrony perception in a rhythmic context, where the sound was not consequent upon the observed movement. Participants judged synchrony between a bouncing point-light figure and an auditory rhythm in two experiments. Two questions were of interest: (1) whether the reference in the visual movement, with which the auditory beat should coincide, relies on a position or a velocity cue; (2) whether the figure form and motion profile affect synchrony perception. Experiment 1 required synchrony judgment with regard to the same (lowest) position of the movement in four visual conditions: two figure forms (human or non-human) combined with two motion profiles (human or ball trajectory). Whereas figure form did not affect synchrony perception, the point of subjective simultaneity differed between the two motions, suggesting that participants adopted the peak velocity in each downward trajectory as their visual reference. Experiment 2 further demonstrated that, when judgment was required with regard to the highest position, the maximal synchrony response was considerably low for ball motion, which lacked a peak velocity in the upward trajectory. The finding of peak velocity as a cue parallels results of visuomotor synchronization tasks employing biological stimuli, suggesting that synchrony judgment with rhythmic motions relies on the perceived visual beat.     

Learning foreign sounds in an alien world: videogame training improves non-native speech categorization

Although speech categories are defined by multiple acoustic dimensions, some are perceptually weighted more than others and there are residual effects of native-language weightings in non-native speech perception. Recent research on nonlinguistic sound category learning suggests that the distribution characteristics of experienced sounds influence perceptual cue weights: Increasing variability across a dimension leads listeners to rely upon it less in subsequent category learning (Holt & Lotto, 2006). The present experiment investigated the implications of this among native Japanese learning English /r/-/l/ categories. Training was accomplished using a videogame paradigm that emphasizes associations among sound categories, visual information, and players' responses to videogame characters rather than overt categorization or explicit feedback. Subjects who played the game for 2.5h across 5 days exhibited improvements in /r/-/l/ perception on par with 2-4 weeks of explicit categorization training in previous research and exhibited a shift toward more native-like perceptual cue weights.     

Sound localization in an Old-World fruit bat (Rousettus aegyptiacus): acuity, use of binaural cues, and relationship to vision.

The passive sound-localization acuity of Egyptian fruit bats (Rousettus aegyptiacus) was determined using a conditioned-avoidance procedure. The mean minimum audible angle for left-right discrimination for 3 bats was 11.6 degrees--very near the mean for terrestrial mammals. The bats also were able to localize low- and high-frequency pure tones, indicating that they can use both binaural phase-difference and binaural intensity-difference cues to localize sound. Moreover, they were able to use the binaural phase-difference cue up to at least 5.6 kHz, which is higher than other mammals yet tested. The width of the Egyptian fruit bats' field of best vision was 27 degrees. This value is consistent with the hypothesis that the role of passive sound localization is to direct the eyes for visual scrutiny of sound sources. Thus, the passive localization abilities of these echolocating megachiropteran fruit bats do not deviate from the patterns established for nonecholocating mammals.     

Encoding specificity and modality effects in episodic memory

Summary A modified Brown-Peterson paradigm was employed to determine the extent to which proactive interference effects of a sensory nature would influence the occurrence of sensory encoding specificity effects in episodic memory. Subjects presented with an auditory or a visual study list of five to-be-remembered (TBR) words engaged in either an auditory or a visual arithmetic distractor task for 30 s, and then received the first word in the study list as an auditory or visual intralist retrieval cue or received no cue. Presentation of the intralist retrieval cue in the modality of the study list enhanced the effectiveness of the retrieval cue, whereas presentation of the retrieval cue in the modality of the distractor task decreased the effectiveness of the retrieval cue. Sensory encoding specificity effects were largest when the distractor task occurred in the modality opposite to the modality in which both the study list and the retrieval cue were presented.     

固定位置区域提示下视觉注意范围等级的ERP研究

研究视觉空间注意中注意范围的脑内时程的动态变化。被试为 14名青年人 ,使用固定位置的、3种不同直径的线圈作为注意范围的区域性提示 ,祛除空间定位因素的影响 ,记录反应时和事件相关电位 (ERP)数据。结果显示 :提示范围由小到中等时 ,反应时延长 ,而由中等到大范围时 ,反应时缩短 ;提示物和靶刺激诱发ERP的早成分 (N1、P1)不受提示范围大小的调节 ,而其P2、N2的波幅与潜伏期均明显受到提示范围大小的影响。这些结果说明 :①视觉注意诱发的P1、N1成分 ,主要与空间定位信息的加工相关 ;②提示物诱发的P2、N2成分与注意范围相关 ;③在视觉信息加工过程中 ,空间位置的信息要早于其它信息被加工 ,支持视觉信息串行加工观点