Perception of the duration of auditory and visual stimuli in children and adults

This experiment investigated the effect of modality on temporal discrimination in children aged 5 and 8 years and adults using a bisection task with visual and auditory stimuli ranging from 200 to 800 ms. In the first session, participants were required to compare stimulus durations with standard durations presented in the same modality (within-modality session), and in the second session in different modalities (cross-modal session). Psychophysical functions were orderly in all age groups, with the proportion of long responses (judgement that a duration was more similar to the long than to the short standard) increasing with the stimulus duration, although functions were flatter in the 5-year-olds than in the 8-year-olds and adults. Auditory stimuli were judged to be longer than visual stimuli in all age groups. The statistical results and a theoretical model suggested that this modality effect was due to differences in the pacemaker speed of the internal clock. The 5-year-olds also judged visual stimuli as more variable than auditory ones, indicating that their temporal sensitivity was lower in the visual than in the auditory modality.     

Why 'Sounds Are Judged Longer Than Lights': Application of a Model of the Internal Clock in Humans

Three experiments, using temporal generalization and verbal estimation methods, studied judgements of duration of auditory (500-Hz tone) and visual (14-cm blue square) stimuli. With both methods, auditory stimuli were judged longer, and less variable, than visual ones. The verbal estimation experiments used stimuli from 77 to 1183 msec in length, and the slope of the function relating mean estimate to real length differed between modalities (but the intercept did not), consistent with the idea that a pacemaker generating duration representations ran faster for auditory than for visual stimuli. The different variability of auditory and visual stimuli was attributed to differential variability in the operation of a switch of a pacemaker-accumulator clock, and experimental datasuggested that such switch effects were separable from changes in pacemaker speed. Overall, the work showed how a clock model consistent with scalar timing theory, the leading account of animal timing, can address an issue derived from the classical literature on human time perception.     

Auditory,visual, and cross-modal negative priming

Auditory, visual, and cross-modal negative priming was investigated using a task in which participants judged whether stimuli were animals or musical instruments. Negative priming was observed, but only if the attended and the ignored primes evoked different responses. This pattern—negative priming after conflict, but not after nonconflict, primes—was demonstrated with visual stimuli and replicated with auditory stimuli, as well as across modalities, both auditory to visual and visual to auditory. Implications for theories of negative priming are discussed.     

Crossmodal attention switching: Auditory dominance in temporal discrimination tasks

Visual stimuli are often processed more efficiently than accompanying stimuli in another modality. In line with this “visual dominance”, earlier studies on attentional switching showed a clear benefit for visual stimuli in a bimodal visual–auditory modality-switch paradigm that required spatial stimulus localization in the relevant modality. The present study aimed to examine the generality of this visual dominance effect. The modality appropriateness hypothesis proposes that stimuli in different modalities are differentially effectively processed depending on the task dimension, so that processing of visual stimuli is favored in the dimension of space, whereas processing auditory stimuli is favored in the dimension of time. In the present study, we examined this proposition by using a temporal duration judgment in a bimodal visual–auditory switching paradigm. Two experiments demonstrated that crossmodal interference (i.e., temporal stimulus congruence) was larger for visual stimuli than for auditory stimuli, suggesting auditory dominance when performing temporal judgment tasks. However, attention switch costs were larger for the auditory modality than for visual modality, indicating a dissociation of the mechanisms underlying crossmodal competition in stimulus processing and modality-specific biasing of attentional set.     

Differential effects of auditory and visual signals on clock speed and temporal memory

The effects of signal modality on duration classification in college students were studied with the duration bisection task. When auditory and visual signals were presented in the same test session and shared common anchor durations, visual signals were classified as shorter than equivalent duration auditory signals. This occurred when auditory and visual signals were presented sequentially in the same test session and when presented simultaneously but asynchronously. Presentation of a single modality signal within a test session, or both modalities but with different anchor durations did not result in classification differences. The authors posit a model in which auditory and visual signals drive an internal clock at different rates. The clock rate difference is due to an attentional effect on the mode switch and is revealed only when the memories for the short and long anchor durations consist of a mix of contributions from accumulations generated by both the fast auditory and slower visual clock rates. When this occurs auditory signals seem longer than visual signals relative to the composite memory representation.     

视听刺激突显性对视觉主导效应的影响

本研究使用空间任务-转换范式,控制视、听刺激的突显性,探讨自下而上注意对视觉主导效应的影响。结果表明视、听刺激突显性显著地影响视觉主导效应,实验一中当听觉刺激为高突显性时,视觉主导效应显著减弱。实验二中当听觉刺激为高突显性并且视觉刺激为低突显性时,视觉主导效应进一步减弱但依然存在。结果支持偏向竞争理论,在跨通道视听交互过程中视觉刺激更具突显性,在多感觉整合过程中更具加工优势。     

Investigating the modality specificity of response selection using a temporal flanker task

The neurocognitive architecture for response selection is uncertain. Some theorists suggest that it is mediated by an amodal central mechanism, whereas others propose a set of independent control mechanisms. In a functional neuroimaging experiment, we investigated the nature of response selection by examining how its underlying brain mechanisms are affected by stimulus modality. To do this, we used a modified flanker task, in which the target and flanker (distractor) stimuli differed in time rather than space, making it accessible for both visual and auditory stimuli. As in the traditional flanker task, larger reaction times were observed for incongruent than congruent trials (i.e., a congruency effect) for both modalities. Congruency affected brain activation for both modalities in prefrontal cortex, parietal cortex, and the putamen. Modality-dependent activation was found in additional prefrontal and parietal regions for the visual modality and in left inferior prefrontal cortex for the auditory modality. Modality-dependent activity specifically related to response congruency was also found in sensory cortical regions. These data suggest that modality affects the brain regions throughout the cortex mediating response selection even for conceptually identical stimuli and tasks. They are consistent with the hypothesis that (at least partially) independent brain networks mediate response selection and that input modality may be a powerful factor for organizing neural activity to support task performance.     

On the interplay of visuospatial and audiotemporal dominance: Evidence from a multimodal kappa effect

When participants judge multimodal audiovisual stimuli, the auditory information strongly dominates temporal judgments, whereas the visual information dominates spatial judgments. However, temporal judgments are not independent of spatial features. For example, in the kappa effect, the time interval between two marker stimuli appears longer when they originate from spatially distant sources rather than from the same source. We investigated the kappa effect for auditory markers presented with accompanying irrelevant visual stimuli. The spatial sources of the markers were varied such that they were either congruent or incongruent across modalities. In two experiments, we demonstrated that the spatial layout of the visual stimuli affected perceived auditory interval duration. This effect occurred although the visual stimuli were designated to be task-irrelevant for the duration reproduction task in Experiment 1, and even when the visual stimuli did not contain sufficient temporal information to perform a two-interval comparison task in Experiment 2. We conclude that the visual and auditory marker stimuli were integrated into a combined multisensory percept containing temporal as well as task-irrelevant spatial aspects of the stimulation. Through this multisensory integration process, visuospatial information affected even temporal judgments, which are typically dominated by the auditory modality.     

Task integration as a factor in secondary-task effects on sequence learning

In several studies it has been found that implicit sequence learning is impaired by the concurrent performance of a secondary task. In most studies the task was to count high-pitched tones when high-pitched and low-pitched tones were presented in a random sequence. In this paper the hypothesis is examined that dual-task interference in the particular combination of tasks results from task integration, in particular from the learning of an integrated visual-auditory sequence in which every second (auditory) element is random. Instead of the tone-counting task a similar go/no-go task was used in which foot responses to high-pitched tones were performed. In Exp. 1 the sequence of tones was random in one condition, but in two other conditions repeated tone sequences of 5 and 6 elements were combined with visual sequences of 6 elements. Under dual-task test conditions, implicit learning of the visual as well as the auditory sequences was better with the auditory sequence of 6 elements than of 5 elements, while under single-task test conditions the nature of the tone sequence had no effect. In Exp. 2 the superior implicit learning with the 6-element sequence was replicated with different test procedures in which either the visual or auditory sequence was changed to random or in which the two sequences remained intact but were shifted by one element relative to each other. Randomization of the visual or auditory sequences not only impaired visual or auditory RT, respectively, but also impaired RT to stimuli in the other modality, and this cross-modal effect was almost as strong as the intra-modal effect of randomization. Finally, in Exp. 3 it was shown that integrated visual-auditory sequences are learned only when responses to both of them are required, but not when the tones can be neglected. These results are consistent with a conception of implicit learning as (at least partly) a basic and nonselective type of learning of all potentially behaviorally relevant relations between stimuli in the environment and one's own actions.     

Good vibrations: Human interval timing in the vibrotactile modality

This article reports a detailed examination of timing in the vibrotactile modality and comparison with that of visual and auditory modalities. Three experiments investigated human timing in the vibrotactile modality. In Experiment 1, a staircase threshold procedure with a standard duration of 1,000 ms revealed a difference threshold of 160.35 ms for vibrotactile stimuli, which was significantly higher than that for auditory stimuli (103.25 ms) but not significantly lower than that obtained for visual stimuli (196.76 ms). In Experiment 2, verbal estimation revealed a significant slope difference between vibrotactile and auditory timing, but not between vibrotactile and visual timing. That is, both vibrations and lights were judged as shorter than sounds, and this comparative difference was greater at longer durations than at shorter ones. In Experiment 3, performance on a temporal generalization task showed characteristics consistent with the predications of scalar expectancy theory (SET: Gibbon, 1977) with both mean accuracy and scalar variance exhibited. The results were modelled using the modified Church and Gibbon model (MCG; derived by Wearden, 1992, from Church & Gibbon 1982). The model was found to give an excellent fit to the data, and the parameter values obtained were compared with those for visual and auditory temporal generalization. The pattern of results suggest that timing in the vibrotactile modality conforms to SET and that the internal clock speed for vibrotactile stimuli is significantly slower than that for auditory stimuli, which is logically consistent with the significant differences in difference threshold that were obtained.     

视-听跨通道汉语词汇信息加工中的抑制机制

采用选择性再认的方法考察在汉语词汇加工过程中 ,视 -听跨通道信息与视觉单通道信息加工过程中的抑制机制。结果表明 :对于视觉词汇的总体再认“否”反应 ,单通道干扰条件下的成绩优于跨通道干扰条件下的成绩。在视觉词汇加工过程中 ,对外在干扰材料的抑制效率不受输入干扰刺激的通道的影响。抑制效率受干扰材料语义关系性的影响 ,与目标材料属于同一语义范畴的比异范畴的干扰材料更难以被抑制。     

The influence of stimulus properties on multisensory processing in the awake primate superior colliculus.

Multisensory integration is a process whereby information converges from different sensory modalities to produce a response that is different from that elicited by the individual modalities presented alone. A neural basis for multisensory integration has been identified within a variety of brain regions, but the most thoroughly examined model has been that of the superior colliculus (SC). Multisensory processing in the SC of anaesthetized animals has been shown to be dependent on the physical parameters of the individual stimuli presented (e.g., intensity, direction, velocity) as well as their spatial relationship. However, it is unknown whether these stimulus features are important, or evident, in the awake behaving animal. To address this question, we evaluated the influence of physical properties of sensory stimuli (visual intensity, direction, and velocity; auditory intensity and location) on sensory activity and multisensory integration of SC neurons in awake, behaving primates. Monkeys were trained to fixate a central visual fixation point while visual and/or auditory stimuli were presented in the periphery. Visual stimuli were always presented within the contralateral receptive field of the neuron whereas auditory stimuli were presented at either ipsi- or contralateral locations. Many of the SC neurons responsive to these sensory stimuli (n = 66/84; 76%) had stronger responses when the visual and auditory stimuli were combined at contralateral locations than when the auditory stimulus was located on the ipsilateral side. This trend was significant across the population of auditory-responsive neurons. In addition, some SC neurons (n = 31) were presented a battery of tests in which the quality of one stimulus of a pair was systematically manipulated. A small proportion of these neurons (n = 8/31; 26%) showed preferential responses to stimuli with specific physical properties, and these preferences were not significantly altered when multisensory stimulus combinations were presented. These data demonstrate that multisensory processing in the awake behaving primate is influenced by the spatial congruency of the stimuli as well as their individual physical properties.     

Something in the way she moves

A recent study using a crossmodal matching task showed that the identity of a talker could be recognized even when the auditory and visual stimuli that were being matched were different sentences spoken by the talker. This finding implies that general temporal features of a person's speech are shared across the auditory and visual modalities.     

Nonconcurrently presented auditory tones reduce distraction

Recent research has demonstrated that task-irrelevant stimuli presented simultaneously with a target-distractor stimulus reduce distraction and improve selective attention. In studies examining this reduced interference effect, visual selective attention tasks and concurrently presented task-irrelevant stimuli are used. We report first evidence for a similar effect in the auditory domain and with nonconcurrent stimuli (i.e., the task-irrelevant stimuli are presented before the target). The effect of nonconcurrently presented auditory tones on an auditory Stroop task developed by Leboe and Mondor (Psychological Research, 71, 568--575, 2007) was investigated. Stroop interference was reduced when task-irrelevant tones were presented before the Stroop stimulus. We conclude that task-irrelevant stimuli can improve selective attention not only when presented concurrently, but also when presented before the selective attention task. Our results are consistent with the hypothesis that interference reduction is due to perceptual dilution caused by task-irrelevant stimuli.     

听觉通道下负数的低水平加工引起注意的SNARC效应

采用刺激探测任务,探讨了听觉通道下负数的低水平加工能否引起注意的SNARC效应。结果表明:(1)负数的低水平加工能够产生注意的SNARC效应,即线索为绝对值小的负数时,被试对左耳目标反应更快;线索为绝对值大的负数时,被试对右耳目标反应更快。(2)负数的加工所引起听觉空间注意的转移并不依赖于负数所在的情境,而仅通过其绝对值大小与空间表征发生关联。这一结论与视觉条件下所得到的结论不一致,并支持了系统发生论假说。     

The role of modality: Auditory and visual distractors in Stroop interference

As a commonly used measure of selective attention, it is important to understand the factors contributing to interference in the Stroop task. The current research examined distracting stimuli in the auditory and visual modalities to determine whether the use of auditory distractors would create additional interference, beyond what is typically observed in the print-based Stroop task. Research by Cowan and Barron supported the additive effects of auditory and visual distractors; however, there is only one empirical demonstration of this finding to date. Using different versions of the Stroop colour-naming task, behavioural analyses of reaction times (RT) were conducted, along with distributional RT analyses. The results indicated that a combination of visual and auditory distraction did not lead to a larger interference effect than visually based distraction alone. These findings suggest that methodological issues may have influenced the prior finding of additive effects of the two modalities, and are discussed in relation to the word production architecture account of Stroop effects.     

Shared and distinct factors driving attention and temporal processing across modalities

In addition to the classic finding that “sounds are judged longer than lights,” the timing of auditory stimuli is often more precise and accurate than is the timing of visual stimuli. In cognitive models of temporal processing, these modality differences are explained by positing that auditory stimuli more automatically capture and hold attention, more efficiently closing an attentional switch that allows the accumulation of pulses marking the passage of time (Penney, Gibbon, & Meck, 2000). However, attention is a multifaceted construct, and there has been little attempt to determine which aspects of attention may be related to modality effects. We used visual and auditory versions of the Continuous Temporal Expectancy Task (CTET; O'Connell et al., 2009) a timing task previously linked to behavioral and electrophysiological measures of mind-wandering and attention lapses, and tested participants with or without the presence of a video distractor. Performance in the auditory condition was generally superior to that in the visual condition, replicating standard results in the timing literature. The auditory modality was also less affected by declines in sustained attention indexed by declines in performance over time. In contrast, distraction had an equivalent impact on performance in the two modalities. Analysis of individual differences in performance revealed further differences between the two modalities: Poor performance in the auditory condition was primarily related to boredom whereas poor performance in the visual condition was primarily related to distractibility. These results suggest that: 1) challenges to different aspects of attention reveal both modality-specific and nonspecific effects on temporal processing, and 2) different factors drive individual differences when testing across modalities.     

Age-related changes in processing auditory stimuli during visual attention: evidence for deficits in inhibitory control and sensory memory.

Age-related declines in attention and cognition have been associated with a difficulty in inhibiting the processing of task-irrelevant information (i.e., the inhibitory deficit hypothesis). However, evidence supporting the inhibitory deficit hypothesis remains equivocal, in part because of complexities in examining the processing of irrelevant stimuli using purely behavioral techniques. The effects of age on the processing of task-irrelevant stimuli were examined using scalp-recorded event-related brain potentials. Participants performed a visual discrimination task while standard and deviant auditory stimuli were presented in the background. Deviant auditory stimuli generated a mismatch negativity (MMN) wave that decreased with age, in part because of an age-related enhancement in sensory-evoked responses. The age-related changes in processing task-irrelevant auditory stimuli are consistent with the inhibitory deficit hypothesis and suggest that impaired inhibitory control of sensory input may play a role in the age-related declines in performance during selective attention tasks.     

0 + 1 > 1: How adding noninformative sound improves performance on a visual task

It is well known that the nervous system combines information from different cues within and across sensory modalities to improve performance on perceptual tasks. In this article, we present results showing that in a visual motion-detection task, concurrent auditory motion stimuli improve accuracy even when they do not provide any useful information for the task. When participants judged which of two stimulus intervals contained visual coherent motion, the addition of identical moving sounds to both intervals improved accuracy. However, this enhancement occurred only with sounds that moved in the same direction as the visual motion. Therefore, it appears that the observed benefit of auditory stimulation is due to auditory-visual interactions at a sensory level. Thus, auditory and visual motion-processing pathways interact at a sensory-representation level in addition to the level at which perceptual estimates are combined.